‘guix shell’ computes different package derivation than ‘guix build’

  • Done
  • quality assurance status badge
Details
3 participants
  • Josselin Poiret
  • Ludovic Courtès
  • Maxim Cournoyer
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
important
L
L
Ludovic Courtès wrote on 27 Feb 2023 14:36
‘guix shell’ computes different package derivation than ‘guix build’
(address . bug-guix@gnu.org)
87zg8zkz5z.fsf@inria.fr
Look at this weird phenomenon.

First, with ‘guix build’, everything works as expected:

Toggle snippet (20 lines)
$ guix describe
Generation 247 Feb 27 2023 08:47:41 (current)
guix 17bd024
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 17bd0243310754045cdc6cc362c804db4a8f9317
$ guix build ungoo
ungoogled-chromium ungoogled-chromium-wayland
$ guix build ungoogled-chromium -n
The following grafts would be made:
/gnu/store/sb9bva92ycw40jiwvklvdwpr5pqs5y14-ungoogled-chromium-109.0.5414.119-1.drv
/gnu/store/88bvz2ch3wsiz66qcmhhpbz2i60ms14j-harfbuzz-5.3.1.drv
/gnu/store/pvrhcbpajl9cf2jjy8p01p5fh6kjf3fi-pipewire-0.3.63.drv
/gnu/store/mjm6k8l2d0j1j3j3p7rqrrrj2pla4cwa-jack2-1.9.21.drv
$ guix build ungoogled-chromium -n --no-grafts
/gnu/store/p984s3dna89qw3j1s9w1jpz3wjw1jmfg-ungoogled-chromium-109.0.5414.119-1
$ guix build ungoogled-chromium -n --no-grafts -d
/gnu/store/i8f4qri399l1r2k7hrwpdxxgc3q77izw-ungoogled-chromium-109.0.5414.119-1.drv

But now ‘guix shell’ (same revision) wants to build ungoogled-chromium:

Toggle snippet (21 lines)
$ guix shell ungoogled-chromium -n
The following derivations would be built:
/gnu/store/6kgc8cd8wn010ba3jnywj2qsjsjk511s-ungoogled-chromium-109.0.5414.119-1.drv
/gnu/store/xfiqj5qnnld9g8hdh879aa7gp7wprhaf-chromium-109.0.5414.119.tar.xz.drv
/gnu/store/4ic70ax830gng6fbaqk3mf3rf2x6vf1j-jsoncpp.patch.drv
/gnu/store/9sbl6km93sgb7wc2b14sc1fgf7alsb79-openjpeg.patch.drv
/gnu/store/jcx82l35b070dz91aiz6gffgkngcf1gl-zlib.patch.drv

1,930.3 MB would be downloaded
$ guix shell ungoogled-chromium -n --no-grafts
The following derivations would be built:
/gnu/store/mpfvrssy125chnxsy4qg61kk1a2s235d-profile.drv
/gnu/store/kyncllwl57l2pk63rc154zdfmbx8759v-ungoogled-chromium-109.0.5414.119-1.drv
/gnu/store/xfiqj5qnnld9g8hdh879aa7gp7wprhaf-chromium-109.0.5414.119.tar.xz.drv
/gnu/store/4ic70ax830gng6fbaqk3mf3rf2x6vf1j-jsoncpp.patch.drv
/gnu/store/9sbl6km93sgb7wc2b14sc1fgf7alsb79-openjpeg.patch.drv
/gnu/store/jcx82l35b070dz91aiz6gffgkngcf1gl-zlib.patch.drv

1,930.3 MB would be downloaded

Why does ‘guix shell’ want to build a different derivation for the same
package?

The derivation ‘guix shell’ computed depends on different *.patch.drv,
compared to the “right” one:

Toggle snippet (6 lines)
$ guix gc -R $(guix build ungoogled-chromium -d --no-grafts) |grep -E '(jsoncpp|openjpeg|zlib)\.patch\.drv'
/gnu/store/ysmmqzva7j49x7sswr5kdgdj59dsh8ip-zlib.patch.drv
/gnu/store/iamxhd6d0jqvmjfh458qwhm62bq2adb7-jsoncpp.patch.drv
/gnu/store/03hqmvhl97b8pxrcqahc9xk1bil2pbs3-openjpeg.patch.drv

To be continued…

Ludo’.
L
L
Ludovic Courtès wrote on 27 Feb 2023 16:02
control message for bug #61841
(address . control@debbugs.gnu.org)
871qmbjgmp.fsf@gnu.org
severity 61841 important
quit
L
L
Ludovic Courtès wrote on 27 Feb 2023 16:10
Re: bug#61255: [PATCH 0/5] Add support for the RPM format to "guix pack"
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87k003i1pu.fsf_-_@gnu.org
Hi Maxim,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (14 lines)
> I’m really not sure what the impact of
> 68775338a510f84e63657ab09242d79e726fa457 is, nor whether it was the only
> solution to the problem.
>
> One thing that probably happens is that (default-guile) is now never
> used for <computed-file>, contrary to what was happening before. The
> spirit is that (default-guile) would be used as the default for all the
> declarative file-like objects; gexp compilers refer to (default-guile),
> not (%guile-for-build).
>
> Importantly, (%guile-for-build) is a derivation, possibly built for
> another system, whereas (default-guile) is a package, which allows
> ‘lower-object’ to return the derivation for the right system type.

Commit 68775338a510f84e63657ab09242d79e726fa457 turned out to have
unintended side effects:


I fixed it with:

a516a0ba93 gexp: computed-file: Do not honor %guile-for-build.
fee1d08f0d pack: Make sure tests can run without a world rebuild.

Please take a look.

We should think about how to improve our processes to avoid such issues
in the future. I did raise concerns about this very patch late at night
during FOSDEM, 24h after submission, and reaffirmed my viewpoint days
later. I understand that delaying a nice patch series like this one is
unpleasant, but I think those concerns should have been taken into
account.

Ludo’.
L
L
Ludovic Courtès wrote on 27 Feb 2023 16:39
control message for bug #61841
(address . control@debbugs.gnu.org)
87bklfi0c5.fsf@gnu.org
close 61841
quit
M
M
Maxim Cournoyer wrote on 27 Feb 2023 17:41
Re: bug#61255: [PATCH 0/5] Add support for the RPM format to "guix pack"
(name . Ludovic Courtès)(address . ludo@gnu.org)
87bklfkqmb.fsf@gmail.com
Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (23 lines)
> Hi Maxim,
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> I’m really not sure what the impact of
>> 68775338a510f84e63657ab09242d79e726fa457 is, nor whether it was the only
>> solution to the problem.
>>
>> One thing that probably happens is that (default-guile) is now never
>> used for <computed-file>, contrary to what was happening before. The
>> spirit is that (default-guile) would be used as the default for all the
>> declarative file-like objects; gexp compilers refer to (default-guile),
>> not (%guile-for-build).
>>
>> Importantly, (%guile-for-build) is a derivation, possibly built for
>> another system, whereas (default-guile) is a package, which allows
>> ‘lower-object’ to return the derivation for the right system type.
>
> Commit 68775338a510f84e63657ab09242d79e726fa457 turned out to have
> unintended side effects:
>
> https://issues.guix.gnu.org/61841

Ugh.

Toggle quote (7 lines)
> I fixed it with:
>
> a516a0ba93 gexp: computed-file: Do not honor %guile-for-build.
> fee1d08f0d pack: Make sure tests can run without a world rebuild.
>
> Please take a look.

Thank you. I still think it'd be nicer if computed-file had a means to
honor %guile-for-build rather than having to accommodate it specially as
you did in fee1d08f0d, so that it'd be symmetrical to gexp->derivation
in that regard. Why can't they?

Toggle quote (7 lines)
> We should think about how to improve our processes to avoid such issues
> in the future. I did raise concerns about this very patch late at night
> during FOSDEM, 24h after submission, and reaffirmed my viewpoint days
> later. I understand that delaying a nice patch series like this one is
> unpleasant, but I think those concerns should have been taken into
> account.

You are right, I should have delayed this submission passed its 2 weeks,
to let some extra time to look at alternatives w.r.t. the
%guile-for-build patch. Apologies for being too eager!

--
Thanks,
Maxim
L
L
Ludovic Courtès wrote on 27 Feb 2023 22:08
Re: bug#61841: ‘guix shell’ computes different package derivation than ‘guix build’
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87r0uahl4a.fsf_-_@gnu.org
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (2 lines)
> Ludovic Courtès <ludo@gnu.org> writes:

[...]

Toggle quote (19 lines)
>> Commit 68775338a510f84e63657ab09242d79e726fa457 turned out to have
>> unintended side effects:
>>
>> https://issues.guix.gnu.org/61841
>
> Ugh.
>
>> I fixed it with:
>>
>> a516a0ba93 gexp: computed-file: Do not honor %guile-for-build.
>> fee1d08f0d pack: Make sure tests can run without a world rebuild.
>>
>> Please take a look.
>
> Thank you. I still think it'd be nicer if computed-file had a means to
> honor %guile-for-build rather than having to accommodate it specially as
> you did in fee1d08f0d, so that it'd be symmetrical to gexp->derivation
> in that regard. Why can't they?

Like I wrote, ‘default-guile’ returns a package whereas
‘%guile-for-build’ returns a derivation.

The latter is inherently lower-level: it’s used together with the
monadic interface or with plain ‘derivation’, when we know which system
we’re targeting. The former is higher-level, system-independent; it
must be used for <computed-file> and similar forms, which are
system-independent.

Ludo’.
M
M
Maxim Cournoyer wrote on 28 Feb 2023 03:25
(name . Ludovic Courtès)(address . ludo@gnu.org)
87r0uaikzz.fsf@gmail.com
Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (34 lines)
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>>> Commit 68775338a510f84e63657ab09242d79e726fa457 turned out to have
>>> unintended side effects:
>>>
>>> https://issues.guix.gnu.org/61841
>>
>> Ugh.
>>
>>> I fixed it with:
>>>
>>> a516a0ba93 gexp: computed-file: Do not honor %guile-for-build.
>>> fee1d08f0d pack: Make sure tests can run without a world rebuild.
>>>
>>> Please take a look.
>>
>> Thank you. I still think it'd be nicer if computed-file had a means to
>> honor %guile-for-build rather than having to accommodate it specially as
>> you did in fee1d08f0d, so that it'd be symmetrical to gexp->derivation
>> in that regard. Why can't they?
>
> Like I wrote, ‘default-guile’ returns a package whereas
> ‘%guile-for-build’ returns a derivation.
>
> The latter is inherently lower-level: it’s used together with the
> monadic interface or with plain ‘derivation’, when we know which system
> we’re targeting. The former is higher-level, system-independent; it
> must be used for <computed-file> and similar forms, which are
> system-independent.

I see, it's starting to make sense. I'll sleep on it :-).

--
Thanks,
Maxim
J
J
Josselin Poiret wrote on 1 Mar 2023 10:05
878rggon93.fsf@jpoiret.xyz
Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (9 lines)
> Look at this weird phenomenon.
>
> First, with ‘guix build’, everything works as expected:
> [...]
> But now ‘guix shell’ (same revision) wants to build ungoogled-chromium:
> [...]
> Why does ‘guix shell’ want to build a different derivation for the same
> package?

Funnily enough, I don't have that problem locally: the `guix shell`
invocation only tells me it's gonna build a profile derivation (I don't
have ungoogled-chromium in my store btw), and the input derivation for
that profile is the same as for `guix build`.

Best,
--
Josselin Poiret
-----BEGIN PGP SIGNATURE-----

iQHEBAEBCAAuFiEEOSSM2EHGPMM23K8vUF5AuRYXGooFAmP/FUgQHGRldkBqcG9p
cmV0Lnh5egAKCRBQXkC5FhcaikUDDAC7lVebDZdySQs5chJzCihg7GcQy6f8SJVk
26srwpGHcdFt4vxPrs/aZqrPOHr0qE0l5VsA+Uu7szr3WUqB9mD+FZ7HwGsdWSZm
ij1A67HZ2WEGjv7BuaTuEfxDnGbDWIBPAJFrGWNaKCwn9jWU564wckYY3WlTe66/
uB4+jDYitc6/Gmi0/FBHyTAEu3liPHLf8RtpUUYZei281VSQbHPhkaMiXPbAciJ6
kUT7XA3A9xujICeNfH41IqDSdpnUrjbjC+EE/Aaeyvo3MRI0I4GIWU4EFuYzVH95
J47Wjw3trI9tLh6T9hEFH/Z2EIV6r0HRUfh/SwJ/TI6OcLtIBxMXdgstgy7xMpzx
CkXG8lpaGJchf8Ad3ur8r9H+FX/JjZQ+3nPL2xxVAiAivrVufbGYxNjIqWnm6jPX
v57Rjppir9N8OB1IHkq4vS+N/1dTMVzmuUYkz3LuKu6o59VxM141X3hsIx4aOr6p
tULj1LimSZpEA4lklUU9zwVC4dpxbt8=
=JB9C
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 1 Mar 2023 22:34
(name . Josselin Poiret)(address . dev@jpoiret.xyz)(address . 61841@debbugs.gnu.org)
874jr49mwe.fsf@gnu.org
Howdy,

Josselin Poiret <dev@jpoiret.xyz> skribis:

Toggle quote (16 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Look at this weird phenomenon.
>>
>> First, with ‘guix build’, everything works as expected:
>> [...]
>> But now ‘guix shell’ (same revision) wants to build ungoogled-chromium:
>> [...]
>> Why does ‘guix shell’ want to build a different derivation for the same
>> package?
>
> Funnily enough, I don't have that problem locally: the `guix shell`
> invocation only tells me it's gonna build a profile derivation (I don't
> have ungoogled-chromium in my store btw), and the input derivation for
> that profile is the same as for `guix build`.

It was fixed on Monday though, so perhaps you’re using a known-good
revision?

Anyway there are still people complaining about “lack of
ungoogled-chromium substitutes” (most likely: wrong ungoogled-chromium
derivation) right now on IRC.

Ludo’.
J
J
Josselin Poiret wrote on 2 Mar 2023 10:23
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 61841@debbugs.gnu.org)
87bklbld6m.fsf@jpoiret.xyz
Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (3 lines)
> It was fixed on Monday though, so perhaps you’re using a known-good
> revision?

That's probably the piece of info I was missing! The discussion about
the issue didn't show up in the same thread in my MUA so I thought this
was simply unanswered, my bad.

Best,
--
Josselin Poiret
-----BEGIN PGP SIGNATURE-----

iQHEBAEBCAAuFiEEOSSM2EHGPMM23K8vUF5AuRYXGooFAmQAawEQHGRldkBqcG9p
cmV0Lnh5egAKCRBQXkC5Fhcaij+DC/9JlOXVbJWK1uEBeVnT44NESmeY28qfYMEm
zuUf1MDsFm0/LjbPhJll2C9iP+V/jmtBtPWm3IvuWBsbimyfReC4N1eNDzJgDdzv
GeRu4+YtXFy4bveQFVc8Vaib7tDcYSFUuAfVQVDNhlYYWV7N1xHYvId3VW3I8+wv
MLgRyLSn7yuhZuNoM6rVOoRP86Q04R7bb9hJZJ2o/3CAiT1C+VmD0imNRifegwVX
dhw7e8K8zPsIKgu2nUnInE/2Wal/yMPpCKAJgmeVJQPet5eD6i/Xcr08dR/GGkxn
bLYSgNcCj/DSnMP5NWa4WpD/rzOhx0XJIcEP3LUmXCvAts6Xg2pSjIrQQSNh2UEb
QLU6rnzjCLA1wEHxc+cnCgLOeBYxXRL92muTaJA5GM1Gsbbwu8JcQoLpNi0Yzcb+
4UljpHRnkfTup78PNQI7nGmlDYBrIm+avX2dPsMX2x4da6GqB1sdz/bxa4rBN/eC
581FrgnYF5ZtY3OoHgBmdBTpJUIlB8c=
=id11
-----END PGP SIGNATURE-----

?