'guix shell' skips profile collision checks

  • Open
  • quality assurance status badge
Details
6 participants
  • Greg Hogan
  • Konrad Hinsen
  • Liliana Marie Prikler
  • Ludovic Courtès
  • Maxime Devos
  • Suhail
Owner
unassigned
Submitted by
Maxime Devos
Severity
important
M
M
Maxime Devos wrote on 12 Mar 2022 10:45
Profile collisions are ignored, installing multiple versions of the same package is silently broken
(address . bug-guix@gnu.org)
afde4231155f7fe32e709dc0d3a4c12d71ea8011.camel@telenet.be
Hi guix,

Try

$ guix shell --pure bash pango@1.42 pango@1.48
$ ~/.guix-profile/bin/ls -l $GUIX_ENVIRONMENT/lib/libpango*.so

Result:
[...]/lib/libpango-1.0.so -> [...]-pango-1.42.4/lib/libpango-1.0.so
[...]/lib/libpangocairo-1.0.so -> [...]-pango-1.42.4/lib/libpangocairo-1.0.so
[...]/lib/libpangoft2-1.0.so -> [...]-1.42.4/lib/libpangoft2-1.0.so
[...]/lib/libpangoxft-1.0.so -> [...]-pango-1.42.4/lib/libpangoxft-1.0.so

Question: how does this even work? Aren't error messages about profile
collisions a thing? Another question: if this use case was supported,
shouldn't it point to the newer pango instead of the older pango?

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYixryhccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7scMAP4m3Lc0IrXnN283dLcFjqCQLEKj
kOFxW92uoNb34Izj0wEA5SlU+JT7Dmb2aklTtFWmYyjRmErfv3nSeiS5J6Adsgc=
=9KfV
-----END PGP SIGNATURE-----


L
L
Liliana Marie Prikler wrote on 12 Mar 2022 15:34
80793191d3eea8c53cec9a61515905327844c73e.camel@gmail.com
Am Samstag, dem 12.03.2022 um 10:45 +0100 schrieb Maxime Devos:
Toggle quote (17 lines)
> Hi guix,
>
> Try
>
> $ guix shell --pure bash pango@1.42 pango@1.48
> $ ~/.guix-profile/bin/ls -l $GUIX_ENVIRONMENT/lib/libpango*.so
>
> Result:
> [...]/lib/libpango-1.0.so -> [...]-pango-1.42.4/lib/libpango-1.0.so
> [...]/lib/libpangocairo-1.0.so -> [...]-pango-
> 1.42.4/lib/libpangocairo-1.0.so
> [...]/lib/libpangoft2-1.0.so -> [...]-1.42.4/lib/libpangoft2-1.0.so
> [...]/lib/libpangoxft-1.0.so -> [...]-pango-1.42.4/lib/libpangoxft-
> 1.0.so
>
> Question: how does this even work?  Aren't error messages about
> profile collisions a thing? 
They ought to be, but perhaps these are only checked for propagated
packages, not for explicitly specified ones. Would still be weird, if
you e.g. put pango@1.48 and something propagating pango@1.42 together,
it ought to error. Perhaps an oversight?

Toggle quote (2 lines)
> Another question: if this use case was supported, shouldn't it point
> to the newer pango instead of the older pango?
union-build implicitly takes the first file it finds -- this is
actually sane, but sadly undocumented behaviour. In other words, if
you specify pango@1.48 first, it should be the one being picked.

Cheers
L
L
Ludovic Courtès wrote on 15 Mar 2022 14:50
Re: bug#54350: Profile collisions are ignored, installing multiple versions of the same package is silently broken
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 54350@debbugs.gnu.org)
87zglrjpef.fsf@gnu.org
Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

Toggle quote (12 lines)
> $ guix shell --pure bash pango@1.42 pango@1.48
> $ ~/.guix-profile/bin/ls -l $GUIX_ENVIRONMENT/lib/libpango*.so
>
> Result:
> [...]/lib/libpango-1.0.so -> [...]-pango-1.42.4/lib/libpango-1.0.so
> [...]/lib/libpangocairo-1.0.so -> [...]-pango-1.42.4/lib/libpangocairo-1.0.so
> [...]/lib/libpangoft2-1.0.so -> [...]-1.42.4/lib/libpangoft2-1.0.so
> [...]/lib/libpangoxft-1.0.so -> [...]-pango-1.42.4/lib/libpangoxft-1.0.so
>
> Question: how does this even work? Aren't error messages about profile
> collisions a thing?

Unlike ‘guix package’, ‘guix environment/shell’ explicitly disables
collision checks—see #:allow-collisions? in environment.scm.

The rationale back then was that if you do:

guix shell -D guile

then you likely end up with profile collisions, like so (I modified the
code to do #:allow-collisions? #f):

Toggle snippet (7 lines)
$ ./pre-inst-env guix shell -D guile -n
guix shell: error: profile contains conflicting entries for bash-minimal
guix shell: error: first entry: bash-minimal@5.1.8 /gnu/store/chfwin3a4qp1znnpsjbmydr2jbzk0d6y-bash-minimal-5.1.8
guix shell: error: second entry: bash-minimal@5.1.8 /gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8
hint: You cannot have two different versions or variants of `bash-minimal' in the same profile.

These collisions are harmless but frequent in package development
environments, which is why ‘guix environment’ had to turn off those
checks (commit afd06f605bf88a796acefc7ed598b43879346a6b).

That’s a bit of a hack though.

What we could do, as a mitigation, is to enable collision checks unless
there’s at least one ‘-D’ flag. We’d also need to introduce
‘--allow-collisions’ for ‘guix shell’.

WDYT?

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 15 Mar 2022 14:51
control message for bug #54350
(address . control@debbugs.gnu.org)
87y21bjpdj.fsf@gnu.org
retitle 54350 'guix shell' skips profile collision checks
quit
M
M
Maxime Devos wrote on 15 Mar 2022 16:58
Re: bug#54350: Profile collisions are ignored, installing multiple versions of the same package is silently broken
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 54350@debbugs.gnu.org)
b1c608f460465b7c4421aed44677f3a2f94302ec.camel@telenet.be
Ludovic Courtès schreef op di 15-03-2022 om 14:50 [+0100]:
Toggle quote (13 lines)
> $ ./pre-inst-env guix shell -D guile -n
> guix shell: error: profile contains conflicting entries for bash-minimal
> guix shell: error:   first entry: bash-minimal@5.1.8 /gnu/store/chfwin3a4qp1znnpsjbmydr2jbzk0d6y-bash-minimal-5.1.8
> guix shell: error:   second entry: bash-minimal@5.1.8 /gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8
> hint: You cannot have two different versions or variants of `bash-minimal' in the same profile.
> --8<---------------cut here---------------end--------------->8---
>
> These collisions are harmless but frequent in package development
> environments, which is why ‘guix environment’ had to turn off those
> checks (commit afd06f605bf88a796acefc7ed598b43879346a6b).
>
> That’s a bit of a hack though.

These collisions happen because the bag of a package contains of multiple
variants of bash: the default canonical bash-minimal, from (gnu packages
commencement), not directly visible (i.e., with "guix show" and
'specification->package'), which is overidden by the bash-minimal from
native-inputs, which is visible from specification->package and as a public
variable, and as I understand it, the bash-minimal supposed to end up
in the references of non-bootstrap packages.

For these kind of collisions, I think the non-canonical package (that's
actually visible with "guix show FOO") is preferred here, so perhaps
the canonical variants can be filtered out whenever that would resolve
a collision?

That seems also a bit of a hack though.

Toggle quote (4 lines)
> What we could do, as a mitigation, is to enable collision checks
> unless there’s at least one ‘-D’ flag. We’d also need to introduce
> ‘--allow-collisions’ for ‘guix shell’.

Special-casing -D like this also seems like a hack to me. I'm not sure
which hack is better, though I prefer this one and the ‘filter out
canonical packages’ above the original ‘just disable collision
checking’.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYjC3sBccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7oDxAP9eipXUqNPc36a8Dy1OE/61ePW0
3vstSYCI6hH+RrCgogD+KNGMRFaS7UfPk/yFkCeLhPgYijpJiexQnaec0xxELgU=
=MzrT
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 16 Mar 2022 11:18
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 54350@debbugs.gnu.org)
87czimgpz9.fsf@gnu.org
Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

Toggle quote (8 lines)
> Ludovic Courtès schreef op di 15-03-2022 om 14:50 [+0100]:
>> $ ./pre-inst-env guix shell -D guile -n
>> guix shell: error: profile contains conflicting entries for bash-minimal
>> guix shell: error:   first entry: bash-minimal@5.1.8 /gnu/store/chfwin3a4qp1znnpsjbmydr2jbzk0d6y-bash-minimal-5.1.8
>> guix shell: error:   second entry: bash-minimal@5.1.8 /gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8
>> hint: You cannot have two different versions or variants of `bash-minimal' in the same profile.
>> --8<---------------cut here---------------end--------------->8---

I wonder whether this is the exception rather than the rule. Here’s an
unscientific not-quite-random sample:

Toggle snippet (49 lines)
$ ./pre-inst-env guix shell -D guix
guix shell: error: profile contains conflicting entries for bzip2
guix shell: error: first entry: bzip2@1.0.8 /gnu/store/gzmwrl6cb8nxmc68hpgqzblng2kamkgc-bzip2-1.0.8
guix shell: error: second entry: bzip2@1.0.8 /gnu/store/s3hl12jxz9ybs7nsy7kq7ybzz7qnzmsg-bzip2-1.0.8
hint: You cannot have two different versions or variants of `bzip2' in the same profile.

$ ./pre-inst-env guix shell -D inkscape --no-grafts -n
guix shell: error: profile contains conflicting entries for libsigc++
guix shell: error: first entry: libsigc++@3.0.6 /gnu/store/ng4k2yl94d758p5vnashd4nvyb1aw8s1-libsigc++-3.0.6
guix shell: error: ... propagated from cairomm@1.14.2
guix shell: error: ... propagated from gtkmm@3.24.5
guix shell: error: second entry: libsigc++@2.9.3 /gnu/store/iad8jg1fm7jsq0pqj547f3n5s2jn9rp0-libsigc++-2.9.3
hint: Try upgrading both `gtkmm' and `libsigc++', or remove one of them from the profile.

$ ./pre-inst-env guix shell -D gimp --no-grafts -n
guix shell: error: profile contains conflicting entries for librsvg
guix shell: error: first entry: librsvg@2.50.7 /gnu/store/wj3sigq6pwjc2z68qgbpv8pxcv8haklf-librsvg-2.50.7
guix shell: error: second entry: librsvg@2.50.7 /gnu/store/q3vymxinnp6cxqq1cpz1rdqmnwq5x1ni-librsvg-2.50.7
hint: You cannot have two different versions or variants of `librsvg' in the same profile.

$ ./pre-inst-env guix shell -D openmpi --no-grafts -n
The following derivation would be built:
/gnu/store/fpj74z159bf6v7cxsxs5rxm3xm4w2amz-profile.drv

$ ./pre-inst-env guix shell -D hwloc --no-grafts -n
The following derivation would be built:
/gnu/store/ikf40sfmk3y7r3frxm9fbkzmma4ym8ay-profile.drv

$ ./pre-inst-env guix shell -D guile-gcrypt --no-grafts -n
The following derivation would be built:
/gnu/store/3ip5kq50b97bsm6zhygz0fjx54w0fqa0-profile.drv

$ ./pre-inst-env guix shell -D libgit2 --no-grafts -n
The following derivation would be built:
/gnu/store/8f83l9p88g40mrnl5l4wrzqg5zwgn6cm-profile.drv

$ ./pre-inst-env guix shell -D python-pytorch --no-grafts -n
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
The following derivation would be built:
/gnu/store/aly6xd3vi5j0qimjzj2hj0wry5zma376-profile.drv

37.2 MB would be downloaded
$ ./pre-inst-env guix shell -D python-itsdangerous --no-grafts -n
The following derivation would be built:
/gnu/store/cgxgvj73wgx263lzd3f82vvjl39vn4pn-profile.drv


We should check automatically the fraction of packages for which
‘package->development-manifest’ contains colliding entries.

If it’s rather rare, we might as well have ‘guix shell’ default to
#:allow-collisions? #f.

(The issue for GIMP comes from librsvg vs. librsvg-bootstrap. The one
for inkscape is a genuine problem.)

If we take that route, we could also add a linter that checks for
collisions in package dependencies.

Toggle quote (19 lines)
>> These collisions are harmless but frequent in package development
>> environments, which is why ‘guix environment’ had to turn off those
>> checks (commit afd06f605bf88a796acefc7ed598b43879346a6b).
>>
>> That’s a bit of a hack though.
>
> These collisions happen because the bag of a package contains of multiple
> variants of bash: the default canonical bash-minimal, from (gnu packages
> commencement), not directly visible (i.e., with "guix show" and
> 'specification->package'), which is overidden by the bash-minimal from
> native-inputs, which is visible from specification->package and as a public
> variable, and as I understand it, the bash-minimal supposed to end up
> in the references of non-bootstrap packages.
>
> For these kind of collisions, I think the non-canonical package (that's
> actually visible with "guix show FOO") is preferred here, so perhaps
> the canonical variants can be filtered out whenever that would resolve
> a collision?

I’m not fond of filtering out, because that can make it hard to
understand what the inputs of a package really are.

However, we could tweak the collision detection code to ignore
collisions with hidden packages or something along these lines.

Thoughts?

Ludo’.
L
L
Ludovic Courtès wrote on 16 Mar 2022 14:49
control message for bug #54350
(address . control@debbugs.gnu.org)
877d8uf1ns.fsf@gnu.org
severity 54350 important
quit
G
G
Greg Hogan wrote on 16 Mar 2022 21:31
Re: bug#54350: Profile collisions are ignored, installing multiple versions of the same package is silently broken
(name . Ludovic Courtès)(address . ludo@gnu.org)
CA+3U0Z=JbVMq1PLqZrFh8-ZBW=emtdRa0XvVEEJZPpG_r0HmEw@mail.gmail.com
On Tue, Mar 15, 2022 at 9:51 AM Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (5 lines)
> What we could do, as a mitigation, is to enable collision checks unless
> there’s at least one ‘-D’ flag. We’d also need to introduce
> ‘--allow-collisions’ for ‘guix shell’.
>

Also 'guix pack'.
Attachment: file
K
K
Konrad Hinsen wrote on 12 Dec 2023 14:10
Profile collisions in "guix shell"
(address . 54350@debbugs.gnu.org)
m1jzpjzhux.fsf@fastmail.net
Hi everyone,

I ran into this issue when I was trying to turn a much-used shell
environment into a profile for persisting the binaries in the store.
I had been using it for several months, believing it to be OK.
Fortunately it was easy to fix.

Background: as part of a reproducible computation workflow (see my talk
at the ten-year meeting,
I run everything that is part of a research project in containers,
meaning "guix shell -C". Since I rely on Python 2 software, I end up
using package transformations frequently to get old software to work. As
it turns out now, package transformations are also an excellent way to
create profile collisions: three out of five of my commonly used
containers have collisions, which I wasn't aware of until a few days
ago.

From that perspective, the current behavior of "guix shell" is... let's
say "suboptimal".

I do understand the motivation, as explained in this thread, but I don't
quite understand (1) why profile collisions are so frequent in
development environments and (2) why ignoring them in that use case
doesn't cause any trouble.

When I run "guix shell -D inkscape", I expect to have, as much as
possible, the same environment as the one the daemon uses when build
inkscape. Does the daemon also switch off collision detection?

Cheers,
Konrad.
K
K
Konrad Hinsen wrote on 15 Dec 2023 07:51
(address . 54350@debbugs.gnu.org)
m1v890x8ju.fsf@fastmail.net
Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

Toggle quote (3 lines)
> I run everything that is part of a research project in containers,
> meaning "guix shell -C".

Related: HPC users on clusters using NFS are also advised to use
"guix shell" rather than profiles:


Cheers,
Konrad.
S
S
Suhail wrote on 16 Jan 19:36 +0100
bug#54350: Profile collisions in "guix shell"
(address . 54350@debbugs.gnu.org)
<87wms9nn1p.fsf@>
Hello Guix,

I came across this issue the other day when attempting to create a
profile that could act as a development environment for Guix. In
addition to noting my observations below, I'd like to understand why
there are collisions in a *new* profile tasked with installing the
dependencies of guix? Is this a bug?

The [documentation] notes the below command to start "a new shell where
all the dependencies and appropriate environment variables are set up to
hack on Guix":

#+begin_src sh
guix shell -D guix -CPW
#+end_src

Using guix-shell to export the manifest corresponding to the above one
gets:

#+header: :tangle ./manifest.scm
#+begin_src scheme
;; What follows is a "manifest" equivalent to the command line you gave.
;; You can store it in a file that you may then pass to any 'guix' command
;; that accepts a '--manifest' (or '-m') option.
(package->development-manifest
(specification->package "guix"))
#+end_src

Saving the above to manifest.scm and evaluating the below:

#+begin_src sh
mkdir -p ${GUIX_EXTRA_PROFILES}/guix-build-2
guix package -p ${GUIX_EXTRA_PROFILES}/guix-build-2/guix-build-2 -m ./manifest.scm
#+end_src

Results in the following error:

#+begin_example
...
guix package: error: profile contains conflicting entries for bzip2
guix package: error: first entry: bzip2@1.0.8 /gnu/store/4jkvfh9hp9zf9plrcw490ci6vpvc033d-bzip2-1.0.8
guix package: error: second entry: bzip2@1.0.8 /gnu/store/0ngy7fmrp9g3q35wwccnjxzaaqq9f8wi-bzip2-1.0.8
hint: You cannot have two different versions or variants of `bzip2' in the same profile.
#+end_example

Adding --allow-collisions to match guix-shell behaviour makes the
installation succeed, as expected.


--
Suhail

This email is not an offer capable of acceptance, does not evidence an
intention to enter into an agreement, has no operative effect until a
definitive agreement is signed in writing by both parties, and that no
party should act in reliance on the email or any representations of the
sender until a definitive agreement is signed in writing by both
parties.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 54350@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 54350
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch