Installing, then removing, a package yields a different profile

Done

Details

4 participants

Chris Marusich
Jelle Licht
Ludovic Courtès
Ricardo Wurmus

Owner: unassigned

Submitted by: Chris Marusich

Severity: important

Chris Marusich wrote on 6 Nov 2018 07:25

Recipients:(address . bug-guix@gnu.org)

Message-ID:8736seyb6d.fsf@gmail.com

Hi,

I've noticed that if I do the following...

guix package -p .guix-profile -i hello

guix package -p .guix-profile -i coreutils

guix package -p .guix-profile -r coreutils

...I get profile generations like the following:

.guix-profile-1-link -> /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile

.guix-profile-2-link -> /gnu/store/m6cwnnxcccnd8bga52jxn74vm67n7ry1-profile

.guix-profile-3-link -> /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile

However, the 1st and 3rd generations are practically the same:

Toggle snippet (10 lines)

$ diff -r /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile

diff -r /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile/etc/profile /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile/etc/profile

11c11

< export PATH="${GUIX_PROFILE:-/gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile}/bin${PATH:+:}$PATH"

---

> export PATH="${GUIX_PROFILE:-/gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile}/bin${PATH:+:}$PATH"

[1] marusich@garuda.local:/tmp/tmp.ZOTdkBr4rD

Now, this isn't really a problem, since it isn't wasting space, and it
doesn't take long to build.  It's more of a curiosity.  Why is it that
profile generations 1 and 3 don't both point to the same profile in the
store?  Is this expected behavior?

I wasn't sure what would happen when I tried this.  I had guessed
(incorrectly) that generation 3 would point to the same profile that had
been built for generation 1, so I was a little surprised by this.

-- 
Chris

Ludovic Courtès wrote on 6 Nov 2018 15:25

Recipients:(name . Chris Marusich)(address . cmmarusich@gmail.com)(address . 33285@debbugs.gnu.org)

Message-ID:87r2fyz3ik.fsf@gnu.org

Hi,

Chris Marusich <cmmarusich@gmail.com> skribis:

Toggle quote (28 lines)> I've noticed that if I do the following...
>
> guix package -p .guix-profile -i hello
> guix package -p .guix-profile -i coreutils
> guix package -p .guix-profile -r coreutils
>
> ...I get profile generations like the following:
>
> .guix-profile-1-link  ->  /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile
> .guix-profile-2-link  ->  /gnu/store/m6cwnnxcccnd8bga52jxn74vm67n7ry1-profile
> .guix-profile-3-link  ->  /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile
>
> However, the 1st and 3rd generations are practically the same:
>
> $ diff -r /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile
> diff -r /gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile/etc/profile /gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile/etc/profile
> 11c11
> < export PATH="${GUIX_PROFILE:-/gnu/store/mbvwb1pv8ps0l39gil41g4zj3khx3r8p-profile}/bin${PATH:+:}$PATH"
> ---
>> export PATH="${GUIX_PROFILE:-/gnu/store/qfpmx9rz7kxa8w8r2pz4vj8ilbi70dcp-profile}/bin${PATH:+:}$PATH"
> [1] marusich@garuda.local:/tmp/tmp.ZOTdkBr4rD
> $ 
>
> Now, this isn't really a problem, since it isn't wasting space, and it
> doesn't take long to build.  It's more of a curiosity.  Why is it that
> profile generations 1 and 3 don't both point to the same profile in the
> store?  Is this expected behavior?

It’s not! Could you look at the derivers of these two profiles (use

‘guix gc --derivers’) and see what differs?

Thanks,

Ludo’.

Chris Marusich wrote on 7 Nov 2018 08:08

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33285@debbugs.gnu.org)

Message-ID:87woppl5zy.fsf@gmail.com

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (12 lines)> Hi,
>
> Chris Marusich <cmmarusich@gmail.com> skribis:
>
>> [...]
>>
>> Why is it that profile generations 1 and 3 don't both point to the
>> same profile in the store?  Is this expected behavior?
>
> It’s not!  Could you look at the derivers of these two profiles (use
> ‘guix gc --derivers’) and see what differs?

Due to preparations for SeaGL and other matters, I'm afraid I won't have

much extra time until the end of this month. However, I've peeked at

the derivers, and I may have a hint for now.

The derivers' builders differ as follows:

Toggle snippet (20 lines)$ diff -u  <(cat /gnu/store/nxjngfdrnbid45vibaa3rwyahs8mxdy8-profile-builder | tr ' ' '\n') <(cat /gnu/store/ibgc1gmwmqpvq3shls19b6w7x96r773m-profile-builder | tr ' ' '\n')
--- /dev/fd/63	2018-11-06 22:57:06.113873035 -0800
+++ /dev/fd/62	2018-11-06 22:57:06.113873035 -0800
@@ -38,10 +38,10 @@
 getenv)
 "out")
 (quote
-("/gnu/store/cb6cxb0k18a648qclh7zd7pvpsh1imny-info-dir"
-"/gnu/store/s616mj87kgz41g89b46fgr12y85gldrv-manual-database"
-"/gnu/store/1mj3mlqc28p68q0lnplv217vv4v9fslc-fonts-dir"
-"/gnu/store/g2q9r93pylmfqx4sw3k5gxixg8q2ksb1-ca-certificate-bundle"
+("/gnu/store/vh0ngq20a6x7zshjpv4qnd39p6sfh892-info-dir"
+"/gnu/store/ha6an8rfxadrk3slg9rxwdirv3a64kbz-manual-database"
+"/gnu/store/6cn1mrlyp1xlvln6msx25h822zjk62c3-fonts-dir"
+"/gnu/store/r5vlx8c33xs9g041hi7fyjc0fy2yndrn-ca-certificate-bundle"
 "/gnu/store/bihfrh609gkxb9dp7n96wlpigiv3krfy-hello-2.10"))
 #:symlink
 symlink

Notably, they DO both seem to use the same store path for hello.  The
only differing paths look like they were produced by profile hooks.
Perhaps this issue has something to do with profile hooks?

-- 
Chris

Ludovic Courtès wrote on 7 Nov 2018 22:37

control message for bug #33285

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87lg64sh6e.fsf@gnu.org

severity 33285 important

Ricardo Wurmus wrote on 4 Feb 2019 00:14

Re: bug#33285: Installing, then removing, a package yields a different profile

Recipients:(name . Chris Marusich)(address . cmmarusich@gmail.com)

Message-ID:87sgx4wjlv.fsf@elephly.net

Hi,

Chris Marusich <cmmarusich@gmail.com> writes:

Toggle quote (3 lines)

> The only differing paths look like they were produced by profile

> hooks. Perhaps this issue has something to do with profile hooks?

This seems to be the case. Here are the derivations for the info-dir

hook for the first and then the third profile:

Toggle snippet (16 lines)Derive
([("out","/gnu/store/vq1frsx6jkvk957mi2sx0rl629r31vi0-info-dir","","")]
 ,[("/gnu/store/32dcpfx9chyf1z3jhv41r7hx2a6vklw2-glibc-utf8-locales-2.28.drv",["out"])
   ,("/gnu/store/h0905d946s3xarcfakan8jnblcdqxs3a-hello-2.10.drv",["out"])
   ,("/gnu/store/jbnidakwvhr84rp8p8icdg1lnczjvi6w-gzip-1.9.drv",["out"])
   ,("/gnu/store/n9zlvq692s12aarbrfppz7ayzggxfkqx-guile-2.2.4.drv",["out"])
   ,("/gnu/store/p6ml6z1d3y387y06mbcvpnf19ik2qywh-module-import-compiled.drv",["out"])
   ,("/gnu/store/x62102cgam20b3g8vfdvw06snvznx12k-texinfo-6.5.drv",["out"])]
 ,["/gnu/store/n6gp00cayxm4lc5c17lrwhwx93fzg8hr-info-dir-builder","/gnu/store/p5q59xqmhwg60y2437fnny76nw7l6gbz-module-import"]
 ,"x86_64-linux","/gnu/store/r658y3cgpnf99nxjxqgjiaizx20ac4k0-guile-2.2.4/bin/guile",["--no-auto-compile","-L","/gnu/store/p5q59xqmhwg60y2437fnny76nw7l6gbz-module-import","-C","/gnu/store/qkqypgkam1fncl32l5qklplqh3a6k3fz-module-import-compiled","/gnu/store/n6gp00cayxm4lc5c17lrwhwx93fzg8hr-info-dir-builder"]
 ,[("allowSubstitutes","0")
   ,("guix properties","((type . profile-hook) (hook . info-dir))")
   ,("out","/gnu/store/vq1frsx6jkvk957mi2sx0rl629r31vi0-info-dir")
   ,("preferLocalBuild","1")])

Toggle snippet (15 lines)Derive
([("out","/gnu/store/2ivmnbir288p7w7xyj1bb338h30q00ma-info-dir","","")]
 ,[("/gnu/store/32dcpfx9chyf1z3jhv41r7hx2a6vklw2-glibc-utf8-locales-2.28.drv",["out"])
   ,("/gnu/store/jbnidakwvhr84rp8p8icdg1lnczjvi6w-gzip-1.9.drv",["out"])
   ,("/gnu/store/n9zlvq692s12aarbrfppz7ayzggxfkqx-guile-2.2.4.drv",["out"])
   ,("/gnu/store/p6ml6z1d3y387y06mbcvpnf19ik2qywh-module-import-compiled.drv",["out"])
   ,("/gnu/store/x62102cgam20b3g8vfdvw06snvznx12k-texinfo-6.5.drv",["out"])]
 ,["/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10","/gnu/store/n6gp00cayxm4lc5c17lrwhwx93fzg8hr-info-dir-builder","/gnu/store/p5q59xqmhwg60y2437fnny76nw7l6gbz-module-import"]
 ,"x86_64-linux","/gnu/store/r658y3cgpnf99nxjxqgjiaizx20ac4k0-guile-2.2.4/bin/guile",["--no-auto-compile","-L","/gnu/store/p5q59xqmhwg60y2437fnny76nw7l6gbz-module-import","-C","/gnu/store/qkqypgkam1fncl32l5qklplqh3a6k3fz-module-import-compiled","/gnu/store/n6gp00cayxm4lc5c17lrwhwx93fzg8hr-info-dir-builder"]
 ,[("allowSubstitutes","0")
   ,("guix properties","((type . profile-hook) (hook . info-dir))")
   ,("out","/gnu/store/2ivmnbir288p7w7xyj1bb338h30q00ma-info-dir")
   ,("preferLocalBuild","1")])

Note that they differ in the inclusion of /gnu/store/h0905d946s3xarcfakan8jnblcdqxs3a-hello-2.10.drv as

a derivation input.

I guess that the problem might be in “profile-derivation”, which defines

“inputs” like this:

(define inputs

(append (filter-map (lambda (drv)

(and (derivation? drv)

(gexp-input drv)))

extras)

(manifest-inputs manifest)))

I pk’d at the value returned by manifest-inputs and found this:

Toggle snippet (32 lines)

$ guix package -p .guix-profile -i hello

The following package will be installed:

hello 2.10 /gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10

;;; ((#<gexp-input #<package hello@2.10 gnu/packages/base.scm:68 31e4d80>:out>))

1 package in profile

The following environment variable definitions may be needed:

export PATH=".guix-profile/bin${PATH:+:}$PATH"

$ guix package -p .guix-profile -i coreutils

The following package will be installed:

coreutils 8.30 /gnu/store/xzgfh8kdxilyymi1a8i97b1nci1iybhi-coreutils-8.30

;;; ((#<gexp-input #<package coreutils@8.30 gnu/packages/base.scm:338 39a7000>:out> #<gexp-input "/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10":out>))

2 packages in profile

The following environment variable definitions may be needed:

export PATH=".guix-profile/bin${PATH:+:}$PATH"

$ guix package -p .guix-profile -r coreutils

The following package will be removed:

coreutils 8.30 /gnu/store/xzgfh8kdxilyymi1a8i97b1nci1iybhi-coreutils-8.30

;;; ((#<gexp-input "/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10":out>))

1 package in profile

The following environment variable definitions may be needed:

export PATH=".guix-profile/bin${PATH:+:}$PATH"

Should manifest-inputs distinguish between a package entry and a file

entry and paper over the differences?

Ricardo

Ludovic Courtès wrote on 4 Feb 2019 19:02

Recipients:(name . Ricardo Wurmus)(address . rekado@elephly.net)

Message-ID:877eefsa8n.fsf@gnu.org

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

Toggle quote (32 lines)> Note that they differ in the inclusion of /gnu/store/h0905d946s3xarcfakan8jnblcdqxs3a-hello-2.10.drv as
> a derivation input.
>
> I guess that the problem might be in “profile-derivation”, which defines
> “inputs” like this:
>
>     (define inputs
>       (append (filter-map (lambda (drv)
>                             (and (derivation? drv)
>                                  (gexp-input drv)))
>                           extras)
>               (manifest-inputs manifest)))
>
> I pk’d at the value returned by manifest-inputs and found this:
>
> $ guix package -p .guix-profile -i hello
> The following package will be installed:
>    hello	2.10	/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10
>
>
> ;;; ((#<gexp-input #<package hello@2.10 gnu/packages/base.scm:68 31e4d80>:out>))
> 1 package in profile
> The following environment variable definitions may be needed:
>    export PATH=".guix-profile/bin${PATH:+:}$PATH"
>
> $ guix package -p .guix-profile -i coreutils
> The following package will be installed:
>    coreutils	8.30	/gnu/store/xzgfh8kdxilyymi1a8i97b1nci1iybhi-coreutils-8.30
>
>
> ;;; ((#<gexp-input #<package coreutils@8.30 gnu/packages/base.scm:338 39a7000>:out> #<gexp-input "/gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10":out>))

Good catch!

Toggle quote (3 lines)

> Should manifest-inputs distinguish between a package entry and a file

> entry and paper over the differences?

Actually, to do things correctly, we should really store the .drv in the

‘manifest’ file. That way, manifest entries would always contains a

valid “lowerable” object (a package or a derivation record), as opposed

to a store reference that happens to be valid.

I’ll see if I can come up with a patch for this.

Thanks,

Ludo’.

Ludovic Courtès wrote on 6 Feb 2019 14:19

Recipients:(name . Ricardo Wurmus)(address . rekado@elephly.net)

Message-ID:878sytawbl.fsf@gnu.org

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (5 lines)

> Actually, to do things correctly, we should really store the .drv in the

> ‘manifest’ file. That way, manifest entries would always contains a

> valid “lowerable” object (a package or a derivation record), as opposed

> to a store reference that happens to be valid.

On second thought that’s probably not a great idea: we’d retain

references to the .drv themselves, which can end up taking a bit of

space, especially if --gc-keep-outputs is in use.

So… the more I think about it the more I’m tempted to punt on this one.

The discrepancy we see here does not happen when using --manifest, which

is probably what matters most.

Thoughts?

Ludo’.

Ricardo Wurmus wrote on 6 Feb 2019 15:29

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:871s4lgfcz.fsf@elephly.net

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (16 lines)> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Actually, to do things correctly, we should really store the .drv in the
>> ‘manifest’ file.  That way, manifest entries would always contains a
>> valid “lowerable” object (a package or a derivation record), as opposed
>> to a store reference that happens to be valid.
>
> On second thought that’s probably not a great idea: we’d retain
> references to the .drv themselves, which can end up taking a bit of
> space, especially if --gc-keep-outputs is in use.
>
> So… the more I think about it the more I’m tempted to punt on this one.
>
> The discrepancy we see here does not happen when using --manifest, which
> is probably what matters most.

I think it’s fine to leave it as it is, then. “--install” and

“--remove” are stateful and should be expected to have quirks like this.

(E.g. upgrading Guix in between two “--install”s can lead to

a mosaic of a profile that could not be produced any other way.)

“--manifest” is the recommended way to get reproducible profile

generations, so if that’s working fine I’m okay with closing this as

wontfix. Chris, what do you think?

Thanks for investigating!

Ricardo

Jelle Licht wrote on 7 Feb 2019 12:11

Recipients:(name . Ricardo Wurmus)(address . rekado@elephly.net)

Message-ID:87mun7onte.fsf@fsfe.org

Ricardo Wurmus <rekado@elephly.net> writes:

Toggle quote (7 lines)

> [snip]

> I think it’s fine to leave it as it is, then. “--install” and

> “--remove” are stateful and should be expected to have quirks like this.

> (E.g. upgrading Guix in between two “--install”s can lead to

> a mosaic of a profile that could not be produced any other way.)

This seems like a very reasonable perspective on this issue, so I

concur. Should we state this explicitly somewhere in the manual?

Chris Marusich wrote on 9 Feb 2019 04:44

Recipients:(name . Jelle Licht)(address . jlicht@fsfe.org)

Message-ID:87a7j5sk0u.fsf@gmail.com

Jelle Licht <jlicht@fsfe.org> writes:

Toggle quote (12 lines)> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> [snip]
>>
>> I think it’s fine to leave it as it is, then.  “--install” and
>> “--remove” are stateful and should be expected to have quirks like this.
>> (E.g. upgrading Guix in between two “--install”s can lead to
>> a mosaic of a profile that could not be produced any other way.)
>
> This seems like a very reasonable perspective on this issue, so I
> concur. Should we state this explicitly somewhere in the manual?

Thinking about it that way, I agree it makes sense as is.  I think it's
valid to say this isn't a bug, then.  I've confirmed that "adding" and
then "removing" a package by using a stateless method (e.g., the "guix
package -m" method) results in the expected behavior: the first and
third profile generations refer to exactly the same profile:

Toggle snippet (7 lines)

$ ls -al /tmp/test-profile*

lrwxrwxrwx 1 marusich users 19 Feb 8 19:39 /tmp/test-profile -> test-profile-3-link

lrwxrwxrwx 1 marusich users 51 Feb 8 19:39 /tmp/test-profile-1-link -> /gnu/store/7j6gy2xvyz7zzpccwsssbxxzykv6q0zg-profile

lrwxrwxrwx 1 marusich users 51 Feb 8 19:39 /tmp/test-profile-2-link -> /gnu/store/yz9qskdl926mdy1i63yzsqwhcciswv90-profile

lrwxrwxrwx 1 marusich users 51 Feb 8 19:39 /tmp/test-profile-3-link -> /gnu/store/7j6gy2xvyz7zzpccwsssbxxzykv6q0zg-profile

I'm satisfied.  I'm honestly not sure it's worth mentioning in the
manual.  I feel like explaining it would add little value and just
clutter up the documentation.  This was mainly a curiosity for me.

-- 
Chris

Ricardo Wurmus wrote on 12 Mar 2019 12:58

Recipients:(name . Chris Marusich)(address . cmmarusich@gmail.com)(address . 33285-done@debbugs.gnu.org)

Message-ID:87a7i0tibg.fsf@elephly.net

Chris Marusich <cmmarusich@gmail.com> writes:

Toggle quote (3 lines)

> Thinking about it that way, I agree it makes sense as is. I think it's

> valid to say this isn't a bug, then.

Okay, I’m closing this bug.

Thanks for bringing this up!

Ricardo

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 33285@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date