‘guix shell’ computes different package derivation than ‘guix build’

Done

Details

3 participants

Josselin Poiret
Ludovic Courtès
Maxim Cournoyer

Owner: unassigned

Submitted by: Ludovic Courtès

Severity: important

Ludovic Courtès wrote on 27 Feb 2023 14:36

‘guix shell’ computes different package derivation than ‘guix build’

Recipients:(address . bug-guix@gnu.org)

Message-ID:87zg8zkz5z.fsf@inria.fr

Look at this weird phenomenon.

First, with ‘guix build’, everything works as expected:

Toggle snippet (20 lines)$ guix describe
Generation 247  Feb 27 2023 08:47:41    (current)
  guix 17bd024
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 17bd0243310754045cdc6cc362c804db4a8f9317
$ guix build ungoo
ungoogled-chromium          ungoogled-chromium-wayland  
$ guix build ungoogled-chromium -n
The following grafts would be made:
   /gnu/store/sb9bva92ycw40jiwvklvdwpr5pqs5y14-ungoogled-chromium-109.0.5414.119-1.drv
   /gnu/store/88bvz2ch3wsiz66qcmhhpbz2i60ms14j-harfbuzz-5.3.1.drv
   /gnu/store/pvrhcbpajl9cf2jjy8p01p5fh6kjf3fi-pipewire-0.3.63.drv
   /gnu/store/mjm6k8l2d0j1j3j3p7rqrrrj2pla4cwa-jack2-1.9.21.drv
$ guix build ungoogled-chromium -n --no-grafts
/gnu/store/p984s3dna89qw3j1s9w1jpz3wjw1jmfg-ungoogled-chromium-109.0.5414.119-1
$ guix build ungoogled-chromium -n --no-grafts -d
/gnu/store/i8f4qri399l1r2k7hrwpdxxgc3q77izw-ungoogled-chromium-109.0.5414.119-1.drv

But now ‘guix shell’ (same revision) wants to build ungoogled-chromium:

Toggle snippet (21 lines)

$ guix shell ungoogled-chromium -n

The following derivations would be built:

/gnu/store/6kgc8cd8wn010ba3jnywj2qsjsjk511s-ungoogled-chromium-109.0.5414.119-1.drv

/gnu/store/xfiqj5qnnld9g8hdh879aa7gp7wprhaf-chromium-109.0.5414.119.tar.xz.drv

/gnu/store/4ic70ax830gng6fbaqk3mf3rf2x6vf1j-jsoncpp.patch.drv

/gnu/store/9sbl6km93sgb7wc2b14sc1fgf7alsb79-openjpeg.patch.drv

/gnu/store/jcx82l35b070dz91aiz6gffgkngcf1gl-zlib.patch.drv

1,930.3 MB would be downloaded

$ guix shell ungoogled-chromium -n --no-grafts

The following derivations would be built:

/gnu/store/mpfvrssy125chnxsy4qg61kk1a2s235d-profile.drv

/gnu/store/kyncllwl57l2pk63rc154zdfmbx8759v-ungoogled-chromium-109.0.5414.119-1.drv

/gnu/store/xfiqj5qnnld9g8hdh879aa7gp7wprhaf-chromium-109.0.5414.119.tar.xz.drv

/gnu/store/4ic70ax830gng6fbaqk3mf3rf2x6vf1j-jsoncpp.patch.drv

/gnu/store/9sbl6km93sgb7wc2b14sc1fgf7alsb79-openjpeg.patch.drv

/gnu/store/jcx82l35b070dz91aiz6gffgkngcf1gl-zlib.patch.drv

1,930.3 MB would be downloaded

Why does ‘guix shell’ want to build a different derivation for the same

package?

The derivation ‘guix shell’ computed depends on different *.patch.drv,

compared to the “right” one:

Toggle snippet (6 lines)

$ guix gc -R $(guix build ungoogled-chromium -d --no-grafts) |grep -E '(jsoncpp|openjpeg|zlib)\.patch\.drv'

/gnu/store/ysmmqzva7j49x7sswr5kdgdj59dsh8ip-zlib.patch.drv

/gnu/store/iamxhd6d0jqvmjfh458qwhm62bq2adb7-jsoncpp.patch.drv

/gnu/store/03hqmvhl97b8pxrcqahc9xk1bil2pbs3-openjpeg.patch.drv

To be continued…

Ludo’.

Ludovic Courtès wrote on 27 Feb 2023 16:02

control message for bug #61841

Recipients:(address . control@debbugs.gnu.org)

Message-ID:871qmbjgmp.fsf@gnu.org

severity 61841 important

quit

Ludovic Courtès wrote on 27 Feb 2023 16:10

Re: bug#61255: [PATCH 0/5] Add support for the RPM format to "guix pack"

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:87k003i1pu.fsf_-_@gnu.org

Hi Maxim,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (14 lines)> I’m really not sure what the impact of
> 68775338a510f84e63657ab09242d79e726fa457 is, nor whether it was the only
> solution to the problem.
>
> One thing that probably happens is that (default-guile) is now never
> used for <computed-file>, contrary to what was happening before.  The
> spirit is that (default-guile) would be used as the default for all the
> declarative file-like objects; gexp compilers refer to (default-guile),
> not (%guile-for-build).
>
> Importantly, (%guile-for-build) is a derivation, possibly built for
> another system, whereas (default-guile) is a package, which allows
> ‘lower-object’ to return the derivation for the right system type.

Commit 68775338a510f84e63657ab09242d79e726fa457 turned out to have

unintended side effects:

https://issues.guix.gnu.org/61841

I fixed it with:

a516a0ba93 gexp: computed-file: Do not honor %guile-for-build.

fee1d08f0d pack: Make sure tests can run without a world rebuild.

Please take a look.

We should think about how to improve our processes to avoid such issues

in the future. I did raise concerns about this very patch late at night

during FOSDEM, 24h after submission, and reaffirmed my viewpoint days

later. I understand that delaying a nice patch series like this one is

unpleasant, but I think those concerns should have been taken into

account.

Ludo’.

Ludovic Courtès wrote on 27 Feb 2023 16:39

control message for bug #61841

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87bklfi0c5.fsf@gnu.org

close 61841

quit

Maxim Cournoyer wrote on 27 Feb 2023 17:41

Re: bug#61255: [PATCH 0/5] Add support for the RPM format to "guix pack"

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87bklfkqmb.fsf@gmail.com

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (23 lines)> Hi Maxim,
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> I’m really not sure what the impact of
>> 68775338a510f84e63657ab09242d79e726fa457 is, nor whether it was the only
>> solution to the problem.
>>
>> One thing that probably happens is that (default-guile) is now never
>> used for <computed-file>, contrary to what was happening before.  The
>> spirit is that (default-guile) would be used as the default for all the
>> declarative file-like objects; gexp compilers refer to (default-guile),
>> not (%guile-for-build).
>>
>> Importantly, (%guile-for-build) is a derivation, possibly built for
>> another system, whereas (default-guile) is a package, which allows
>> ‘lower-object’ to return the derivation for the right system type.
>
> Commit 68775338a510f84e63657ab09242d79e726fa457 turned out to have
> unintended side effects:
>
>   https://issues.guix.gnu.org/61841

Ugh.

Toggle quote (7 lines)

> I fixed it with:

> a516a0ba93 gexp: computed-file: Do not honor %guile-for-build.

> fee1d08f0d pack: Make sure tests can run without a world rebuild.

> Please take a look.

Thank you. I still think it'd be nicer if computed-file had a means to

honor %guile-for-build rather than having to accommodate it specially as

you did in fee1d08f0d, so that it'd be symmetrical to gexp->derivation

in that regard. Why can't they?

Toggle quote (7 lines)

> We should think about how to improve our processes to avoid such issues

> in the future. I did raise concerns about this very patch late at night

> during FOSDEM, 24h after submission, and reaffirmed my viewpoint days

> later. I understand that delaying a nice patch series like this one is

> unpleasant, but I think those concerns should have been taken into

> account.

You are right, I should have delayed this submission passed its 2 weeks,
to let some extra time to look at alternatives w.r.t. the
%guile-for-build patch.  Apologies for being too eager!

-- 
Thanks,
Maxim

Ludovic Courtès wrote on 27 Feb 2023 22:08

Re: bug#61841: ‘guix shell’ computes different package derivation than ‘guix build’

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:87r0uahl4a.fsf_-_@gnu.org

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (2 lines)

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

Toggle quote (19 lines)>> Commit 68775338a510f84e63657ab09242d79e726fa457 turned out to have
>> unintended side effects:
>>
>>   https://issues.guix.gnu.org/61841
>
> Ugh.
>
>> I fixed it with:
>>
>>   a516a0ba93 gexp: computed-file: Do not honor %guile-for-build.
>>   fee1d08f0d pack: Make sure tests can run without a world rebuild.
>>
>> Please take a look.
>
> Thank you.  I still think it'd be nicer if computed-file had a means to
> honor %guile-for-build rather than having to accommodate it specially as
> you did in fee1d08f0d, so that it'd be symmetrical to gexp->derivation
> in that regard.  Why can't they?

Like I wrote, ‘default-guile’ returns a package whereas

‘%guile-for-build’ returns a derivation.

The latter is inherently lower-level: it’s used together with the

monadic interface or with plain ‘derivation’, when we know which system

we’re targeting. The former is higher-level, system-independent; it

must be used for <computed-file> and similar forms, which are

system-independent.

Ludo’.

Maxim Cournoyer wrote on 28 Feb 2023 03:25

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87r0uaikzz.fsf@gmail.com

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (34 lines)> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>>> Commit 68775338a510f84e63657ab09242d79e726fa457 turned out to have
>>> unintended side effects:
>>>
>>>   https://issues.guix.gnu.org/61841
>>
>> Ugh.
>>
>>> I fixed it with:
>>>
>>>   a516a0ba93 gexp: computed-file: Do not honor %guile-for-build.
>>>   fee1d08f0d pack: Make sure tests can run without a world rebuild.
>>>
>>> Please take a look.
>>
>> Thank you.  I still think it'd be nicer if computed-file had a means to
>> honor %guile-for-build rather than having to accommodate it specially as
>> you did in fee1d08f0d, so that it'd be symmetrical to gexp->derivation
>> in that regard.  Why can't they?
>
> Like I wrote, ‘default-guile’ returns a package whereas
> ‘%guile-for-build’ returns a derivation.
>
> The latter is inherently lower-level: it’s used together with the
> monadic interface or with plain ‘derivation’, when we know which system
> we’re targeting.  The former is higher-level, system-independent; it
> must be used for <computed-file> and similar forms, which are
> system-independent.

I see, it's starting to make sense. I'll sleep on it :-).

Thanks,

Maxim

Josselin Poiret wrote on 1 Mar 2023 10:05

Recipients:

Message-ID:878rggon93.fsf@jpoiret.xyz

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (9 lines)

> Look at this weird phenomenon.

> First, with ‘guix build’, everything works as expected:

> [...]

> But now ‘guix shell’ (same revision) wants to build ungoogled-chromium:

> [...]

> Why does ‘guix shell’ want to build a different derivation for the same

> package?

Funnily enough, I don't have that problem locally: the `guix shell`
invocation only tells me it's gonna build a profile derivation (I don't
have ungoogled-chromium in my store btw), and the input derivation for
that profile is the same as for `guix build`.

Best,
-- 
Josselin Poiret

Ludovic Courtès wrote on 1 Mar 2023 22:34

Recipients:(name . Josselin Poiret)(address . dev@jpoiret.xyz)(address . 61841@debbugs.gnu.org)

Message-ID:874jr49mwe.fsf@gnu.org

Howdy,

Josselin Poiret <dev@jpoiret.xyz> skribis:

Toggle quote (16 lines)> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Look at this weird phenomenon.
>>
>> First, with ‘guix build’, everything works as expected:
>> [...]
>> But now ‘guix shell’ (same revision) wants to build ungoogled-chromium:
>> [...]
>> Why does ‘guix shell’ want to build a different derivation for the same
>> package?
>
> Funnily enough, I don't have that problem locally: the `guix shell`
> invocation only tells me it's gonna build a profile derivation (I don't
> have ungoogled-chromium in my store btw), and the input derivation for
> that profile is the same as for `guix build`.

It was fixed on Monday though, so perhaps you’re using a known-good

revision?

Anyway there are still people complaining about “lack of

ungoogled-chromium substitutes” (most likely: wrong ungoogled-chromium

derivation) right now on IRC.

Ludo’.

Josselin Poiret wrote on 2 Mar 2023 10:23

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 61841@debbugs.gnu.org)

Message-ID:87bklbld6m.fsf@jpoiret.xyz

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (3 lines)

> It was fixed on Monday though, so perhaps you’re using a known-good

> revision?

That's probably the piece of info I was missing!  The discussion about
the issue didn't show up in the same thread in my MUA so I thought this
was simply unanswered, my bad.

Best,
-- 
Josselin Poiret

Your comment

This issue is archived.

To comment on this conversation send an email to 61841@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 61841

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date