Failed to produce output path for guix-package-cache

  • Open
  • quality assurance status badge
Details
5 participants
  • Ludovic Courtès
  • Maxime Devos
  • Roel Janssen
  • Vagrant Cascadian
  • zimoun
Owner
unassigned
Submitted by
Roel Janssen
Severity
normal
R
R
Roel Janssen wrote on 22 Apr 2021 13:38
(address . bug-guix@gnu.org)
86b58d34-b9ae-b4bb-f64c-9250e2c109cd@gnu.org
Dear Guix,

I'm running into the following problem:

----
$ guix pull
Updating channel 'guix-science' from Git repository at
Updating channel 'guix' from Git repository at
Building from these channels:
Computing Guix derivation for 'x86_64-linux'... |
The following derivation will be built:
   /gnu/store/xy2q5rk35sxn07xjq89l3ga95igjmmsi-profile.drv

building package cache...
/builder for
`/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-cache.drv'
failed to produce output path
`/gnu/store/xgjiqmh9qlvnq1701zb5dsbbwnjx76qq-guix-package-cache'
build of
/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-cache.drv failed
Could not find build log for
'/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-cache.drv'.
cannot build derivation
`/gnu/store/xy2q5rk35sxn07xjq89l3ga95igjmmsi-profile.drv': 1
dependencies couldn't be built
guix pull: error: build of
`/gnu/store/xy2q5rk35sxn07xjq89l3ga95igjmmsi-profile.drv' failed
----

Is this a known problem? I assume this may be a problem with the
"guix-science" channel (which would be my own fault of course).
If it is, then perhaps the UI could be improved to show why the package
cache couldn't be built.
Perhaps, as a matter of diagnostics, it could build package caches
separately for each channel to pinpoint which channel is causing the
problem.

Thanks.

Kind regards,
Roel Janssen
L
L
Ludovic Courtès wrote on 28 Apr 2021 23:38
(name . Roel Janssen)(address . roel@gnu.org)(address . 47949@debbugs.gnu.org)
87h7jqozsv.fsf@gnu.org
Hi Roel,

Roel Janssen <roel@gnu.org> skribis:

Toggle quote (10 lines)
> /builder for
> `/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-cache.drv'
> failed to produce output path
> `/gnu/store/xgjiqmh9qlvnq1701zb5dsbbwnjx76qq-guix-package-cache'
> build of
> /gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-cache.drv
> failed
> Could not find build log for
> '/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-cache.drv'.

Could you find the log of this derivation? :-)

It’ll tell us what’s wrong.

Toggle quote (4 lines)
> Perhaps, as a matter of diagnostics, it could build package caches
> separately for each channel to pinpoint which channel is causing the
> problem.

Yeah, we’ll have to see what’s doable, but I agree there’s room for
improvement here.

Ludo’.
R
R
Roel Janssen wrote on 29 Apr 2021 09:01
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 47949@debbugs.gnu.org)
26aef3de604e17d9199edff5f4b6eff170df8a1c.camel@gnu.org
On Wed, 2021-04-28 at 23:38 +0200, Ludovic Courtès wrote:
Toggle quote (21 lines)
> Hi Roel,
>
> Roel Janssen <roel@gnu.org> skribis:
>
> > /builder for
> > `/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-
> > cache.drv'
> > failed to produce output path
> > `/gnu/store/xgjiqmh9qlvnq1701zb5dsbbwnjx76qq-guix-package-cache'
> > build of
> > /gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-cache.drv
> > failed
> > Could not find build log for
> > '/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-
> > cache.drv'.
>
> Could you find the log of this derivation?  :-)
>
> It’ll tell us what’s wrong.
>

I'm confused.. The message says "Could not find build log for ...". Is
there any other place I can look?

Thank you!

Kind regards,
Roel Janssen
L
L
Ludovic Courtès wrote on 29 Apr 2021 09:56
(name . Roel Janssen)(address . roel@gnu.org)(address . 47949@debbugs.gnu.org)
87zgxhmslv.fsf@gnu.org
Hi,

Roel Janssen <roel@gnu.org> skribis:

Toggle quote (25 lines)
> On Wed, 2021-04-28 at 23:38 +0200, Ludovic Courtès wrote:
>> Hi Roel,
>>
>> Roel Janssen <roel@gnu.org> skribis:
>>
>> > /builder for
>> > `/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-
>> > cache.drv'
>> > failed to produce output path
>> > `/gnu/store/xgjiqmh9qlvnq1701zb5dsbbwnjx76qq-guix-package-cache'
>> > build of
>> > /gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-cache.drv
>> > failed
>> > Could not find build log for
>> > '/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-
>> > cache.drv'.
>>
>> Could you find the log of this derivation?  :-)
>>
>> It’ll tell us what’s wrong.
>>
>
> I'm confused.. The message says "Could not find build log for ...". Is
> there any other place I can look?

“Could not find build log” typically happens if you’re talking to a
remote daemon, via GUIX_DAEMON_SOCKET. In that case, the build log is
in /var/log/guix/drvs (or similar) on the machine where the daemon is
running.

HTH!

Ludo’.
V
V
Vagrant Cascadian wrote on 25 Oct 2022 21:50
(address . 47949@debbugs.gnu.org)
87fsfbznrk.fsf@contorta
On 2021-04-29, Ludovic Courtès wrote:
Toggle quote (29 lines)
> Roel Janssen <roel@gnu.org> skribis:
>> On Wed, 2021-04-28 at 23:38 +0200, Ludovic Courtès wrote:
>>> Roel Janssen <roel@gnu.org> skribis:
>>>
>>> > /builder for
>>> > `/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-
>>> > cache.drv'
>>> > failed to produce output path
>>> > `/gnu/store/xgjiqmh9qlvnq1701zb5dsbbwnjx76qq-guix-package-cache'
>>> > build of
>>> > /gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-cache.drv
>>> > failed
>>> > Could not find build log for
>>> > '/gnu/store/4q9aprxi2rr1i6yjk1y7d76nbavwp4fy-guix-package-
>>> > cache.drv'.
>>>
>>> Could you find the log of this derivation?  :-)
>>>
>>> It’ll tell us what’s wrong.
>>>
>>
>> I'm confused.. The message says "Could not find build log for ...". Is
>> there any other place I can look?
>
> “Could not find build log” typically happens if you’re talking to a
> remote daemon, via GUIX_DAEMON_SOCKET. In that case, the build log is
> in /var/log/guix/drvs (or similar) on the machine where the daemon is
> running.

I just started getting hit by what appears to be the same issue on a
Debian system running guix whenever i guix pull...

originally stared for
me with guix successfully pulled to
bb2701b9111a3d82a82ceaaf2b22b51ecd8ac21f, and then trying to guix pull
to anything newer would trigger the issue...

I managed to workaround it by using an older guix pull at commit
e61660c78f1190c578dd6f202bc5529cbdcff84e, and then guix pull to
8663be6da7f13a8eeea71dc1f493f7adc5b7672a was successful ... but now with
8663be6da7f13a8eeea71dc1f493f7adc5b7672a it appears to be happening
again. Wow, that's confusing...


builder for `/gnu/store/r3y5jir2dv5inrfkqhjgzkvdh7lmqh5m-guix-package-cache.drv' failed to produce output path `/gnu/store/p8926hbpnrrdb9d3awzrnp75i4x9y7v1-guix-package-cache'
build of /gnu/store/r3y5jir2dv5inrfkqhjgzkvdh7lmqh5m-guix-package-cache.drv failed
View build log at '/var/log/guix/drvs/r3/y5jir2dv5inrfkqhjgzkvdh7lmqh5m-guix-package-cache.drv.bz2'.
cannot build derivation `/gnu/store/z2859cbabpsdz7wcy7iq5fmgkwjphhqk-profile.drv': 1 dependencies couldn't be built
guix pull: error: build of `/gnu/store/z2859cbabpsdz7wcy7iq5fmgkwjphhqk-profile.drv' failed

/var/log/guix/drvs/r3/y5jir2dv5inrfkqhjgzkvdh7lmqh5m-guix-package-cache.drv.bz2 says:

(repl-version 0 1 1)
Generating package cache for '/gnu/store/fwjz2hfj9kizx1xpimq1a13p2rinfzvh-profile'...

Backtrace:
In guix/repl.scm:
141:4 19 (machine-repl _ _)
126:7 18 (_)
In ice-9/boot-9.scm:
1747:15 17 (with-exception-handler #<procedure 7ffff050f540 at ic?> ?)
1752:10 16 (with-exception-handler _ _ #:unwind? _ # _)
In guix/repl.scm:
99:21 15 (_)
In unknown file:
14 (_ #<procedure 7ffff052d240 at guix/repl.scm:100:25 ()> ?)
13 (primitive-load "/gnu/store/ia3qygs61fk0zg18x4il6lb28vx?")
In ice-9/boot-9.scm:
1752:10 12 (with-exception-handler _ _ #:unwind? _ # _)
In gnu/packages.scm:
438:11 11 (generate-package-cache _)
In srfi/srfi-1.scm:
460:18 10 (fold #<procedure expand-cache expr> _ _)
In guix/packages.scm:
570:21 9 (expand-cache . _)
1317:17 8 (supported-package? #<package wicd@1.7.4 gnu/packages/?> ?)
In guix/memoization.scm:
101:0 7 (_ #<hash-table 7fffeef54ee0 23100/28099> #<package wi?> ?)
In guix/packages.scm:
1295:37 6 (_)
1555:16 5 (package->bag _ _ _ #:graft? _)
1656:48 4 (thunk)
In gnu/packages/wicd.scm:
59:32 3 (inputs #<package wicd@1.7.4 gnu/packages/wicd.scm:39 7?>)
In ice-9/boot-9.scm:
1685:16 2 (raise-exception _ #:continuable? _)
1780:13 1 (_ #<&compound-exception components: (#<&undefined-vari?>)
In unknown file:
0 (backtrace #<undefined>)

(exception unbound-variable (value #f) (value "Unbound variable: ~S") (value (python2-pygtk)) (value #f))

If it helps,
/gnu/store/r3y5jir2dv5inrfkqhjgzkvdh7lmqh5m-guix-package-cache.drv
contains:

Derive([("out","/gnu/store/p8926hbpnrrdb9d3awzrnp75i4x9y7v1-guix-package-cache","","")],[("/gnu/store/1s1izmdn2xnznrj5mrfil6ibmcb7ishh-guile-3.0.7.drv",["out"]),("/gnu/store/nc5yrjbj78f490mjc8s622g2v0v516gb-inferior-script.scm.drv",["out"]),("/gnu/store/zjnd5hvfr2b4q299if7g508r1ilnwyp5-profile.drv",["out"])],["/gnu/store/ck9dnn4n5ljhbswydf23aaymanm8xsa2-guix-package-cache-builder"],"x86_64-linux","/gnu/store/1kws5vkl0glvpxg7arabsv6q9vazp0hx-guile-3.0.7/bin/guile",["--no-auto-compile","/gnu/store/ck9dnn4n5ljhbswydf23aaymanm8xsa2-guix-package-cache-builder"],[("guix properties","((type . profile-hook) (hook . package-cache))"),("out","/gnu/store/p8926hbpnrrdb9d3awzrnp75i4x9y7v1-guix-package-cache"),("preferLocalBuild","1")])


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCY1g98AAKCRDcUY/If5cW
qkrHAQC24doZ1EpVL09Od9Qi6CsACXncZHSQGWOV3vY7qGUeugD/T94w5tkbRxUR
JUUEy/eHQ3ojoYKEB1EblDYiFf7L5QY=
=NFCW
-----END PGP SIGNATURE-----

Z
Z
zimoun wrote on 26 Oct 2022 10:10
(address . 47949@debbugs.gnu.org)
86fsfb0zvj.fsf@gmail.com
Hi,

On Tue, 25 Oct 2022 at 12:50, Vagrant Cascadian <vagrant@debian.org> wrote:

Toggle quote (11 lines)
> originally stared for
> me with guix successfully pulled to
> bb2701b9111a3d82a82ceaaf2b22b51ecd8ac21f, and then trying to guix pull
> to anything newer would trigger the issue...
>
> I managed to workaround it by using an older guix pull at commit
> e61660c78f1190c578dd6f202bc5529cbdcff84e, and then guix pull to
> 8663be6da7f13a8eeea71dc1f493f7adc5b7672a was successful ... but now with
> 8663be6da7f13a8eeea71dc1f493f7adc5b7672a it appears to be happening
> again. Wow, that's confusing...

These commits are from:

bb2701b9111a3d82a82ceaaf2b22b51ecd8ac21f Sat Oct 22 18:37:02 2022 +0200
e61660c78f1190c578dd6f202bc5529cbdcff84e Sun Oct 16 02:00:43 2022 +0200
8663be6da7f13a8eeea71dc1f493f7adc5b7672a Mon Oct 24 20:31:34 2022 +0200

when Guix complains about

Toggle quote (2 lines)
> (exception unbound-variable (value #f) (value "Unbound variable: ~S") (value (python2-pygtk)) (value #f))

removed by

1c09ed37211d983d04e626d736df8f69f504630d Tue May 31 14:53:45 2022 -0400


Is the ’guix pull’ failure consistent? Is it always because this or
does it vary?


Cheers,
simon
V
V
Vagrant Cascadian wrote on 26 Oct 2022 18:57
(address . 47949@debbugs.gnu.org)
87eduule00.fsf@contorta
On 2022-10-26, zimoun wrote:
Toggle quote (31 lines)
> On Tue, 25 Oct 2022 at 12:50, Vagrant Cascadian <vagrant@debian.org> wrote:
>
>> originally stared for
>> me with guix successfully pulled to
>> bb2701b9111a3d82a82ceaaf2b22b51ecd8ac21f, and then trying to guix pull
>> to anything newer would trigger the issue...
>>
>> I managed to workaround it by using an older guix pull at commit
>> e61660c78f1190c578dd6f202bc5529cbdcff84e, and then guix pull to
>> 8663be6da7f13a8eeea71dc1f493f7adc5b7672a was successful ... but now with
>> 8663be6da7f13a8eeea71dc1f493f7adc5b7672a it appears to be happening
>> again. Wow, that's confusing...
>
> These commits are from:
>
> bb2701b9111a3d82a82ceaaf2b22b51ecd8ac21f Sat Oct 22 18:37:02 2022 +0200
> e61660c78f1190c578dd6f202bc5529cbdcff84e Sun Oct 16 02:00:43 2022 +0200
> 8663be6da7f13a8eeea71dc1f493f7adc5b7672a Mon Oct 24 20:31:34 2022 +0200
>
> when Guix complains about
>
>> (exception unbound-variable (value #f) (value "Unbound variable: ~S") (value (python2-pygtk)) (value #f))
>
> removed by
>
> 1c09ed37211d983d04e626d736df8f69f504630d Tue May 31 14:53:45 2022 -0400
>
>
> Is the ’guix pull’ failure consistent? Is it always because this or
> does it vary?

It is consistently the same errors in the log, though on further looking
i discovered a branch in ~/.cache/guix/checkouts/ that had old removed
files in it (including wicd.scm/wicd.go) ... *maybe* that was somehow
related. I did remove all the evidence, so so if stale checkouts is
somehow the issue, it will be hard to reproduce again... oops.

I did manage again to use an old commit to pull up to a more recent
master (a0751e3250dfea7e52468c8090e18c3118d93a60), and see there are new
commits now so will try again. Will see.


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCY1lm4AAKCRDcUY/If5cW
qnEPAP0XvYz/RTgyWV9U69DR7cCGfl3Frgi7FIXfJv53HCb+ZwEAlksIjtBMupEO
HKNDlX2l2cw4PG4CnPKWyQL6r4a9kwo=
=6ky6
-----END PGP SIGNATURE-----

Z
Z
zimoun wrote on 26 Oct 2022 21:58
(address . 47949@debbugs.gnu.org)
86o7tybbmq.fsf@gmail.com
Hi Vagrant,

On Wed, 26 Oct 2022 at 09:57, Vagrant Cascadian <vagrant@debian.org> wrote:

Toggle quote (6 lines)
> It is consistently the same errors in the log, though on further looking
> i discovered a branch in ~/.cache/guix/checkouts/ that had old removed
> files in it (including wicd.scm/wicd.go) ... *maybe* that was somehow
> related. I did remove all the evidence, so so if stale checkouts is
> somehow the issue, it will be hard to reproduce again... oops.

Hum, what does “guix describe” say?


Toggle quote (4 lines)
> I did manage again to use an old commit to pull up to a more recent
> master (a0751e3250dfea7e52468c8090e18c3118d93a60), and see there are new
> commits now so will try again. Will see.

What is your hackish workflow? You do,

guix pull --commit=<old> && guix pull

right? From your recent pull, does this

guix time-machine --commit=<new> -- help

work? where <new> is newer than your current revision.


Cheers,
simon
V
V
Vagrant Cascadian wrote on 27 Oct 2022 01:25
(address . 47949@debbugs.gnu.org)
878rl2kw0l.fsf@contorta
On 2022-10-26, zimoun wrote:
Toggle quote (10 lines)
> On Wed, 26 Oct 2022 at 09:57, Vagrant Cascadian <vagrant@debian.org> wrote:
>
>> It is consistently the same errors in the log, though on further looking
>> i discovered a branch in ~/.cache/guix/checkouts/ that had old removed
>> files in it (including wicd.scm/wicd.go) ... *maybe* that was somehow
>> related. I did remove all the evidence, so so if stale checkouts is
>> somehow the issue, it will be hard to reproduce again... oops.
>
> Hum, what does “guix describe” say?

I'll try this instead, with some annotation:

$ guix pull -l 1w
# Version used when 139 failed:
Generation 138 Oct 19 2022 19:04:26
guix e61660c
repository URL: /home/vagrant/src/guix
branch: master
commit: e61660c78f1190c578dd6f202bc5529cbdcff84e

# First version where I noticed failures:
Generation 139 Oct 22 2022 13:07:08
guix bb2701b
repository URL: /home/vagrant/src/guix
branch: master
commit: bb2701b9111a3d82a82ceaaf2b22b51ecd8ac21f

# Failed to pull from 139, successfully pulled from 138:
Generation 140 Oct 24 2022 13:19:21
guix 8663be6
repository URL: /home/vagrant/src/guix-master
branch: master
commit: 8663be6da7f13a8eeea71dc1f493f7adc5b7672a

# failed to pull from 139, successfully pulled from 138:
Generation 141 Oct 25 2022 13:08:12
guix a0751e3
repository URL: /home/vagrant/src/guix-master
branch: master
commit: a0751e3250dfea7e52468c8090e18c3118d93a60

# Finally noticed the ~/.cache/guix/... leftover cruft and removed
# cached checkouts, successfully pulled from 141:
Generation 142 Oct 26 2022 10:03:18 (current)
guix c07b55e
repository URL: /home/vagrant/src/guix-master
branch: master
commit: c07b55eb94f8cfa9d0f56cfd97a16f2f7d842652

Apparently I sometimes used:

guix pull --url=/home/vagrant/src/guix --branch=master

and sometimes:

guix pull --url=/home/vagrant/src/guix-master --branch=master

Which end up using different directories in ~/.cache/guix/checkouts/
... and one of the directories has some cruft leftover in it.

After cleaning out the cruft, so far so good...


Toggle quote (8 lines)
>> I did manage again to use an old commit to pull up to a more recent
>> master (a0751e3250dfea7e52468c8090e18c3118d93a60), and see there are new
>> commits now so will try again. Will see.
>
> What is your hackish workflow? You do,
>
> guix pull --commit=<old> && guix pull

To use the older generations I used:

/var/guix/profiles/per-user/vagrant/current-guix-138-link/bin/guix pull ...


Toggle quote (6 lines)
> right? From your recent pull, does this
>
> guix time-machine --commit=<new> -- help
>
> work? where <new> is newer than your current revision.

Oh yeah, that reminds me to add to the confusion, "guix time-machine
--commit=SOMECOMMIT" worked fine, even where "SOMECOMMIT" didn't
successfully work with guix pull.

Maybe that's a clue pointing to the crufty .cache directories?


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCY1nB6wAKCRDcUY/If5cW
qhaIAQDyLVsG00sutZ863s7cx6+Xyn/g+WQDXNHjEOx7xjGpOgEAj0iGYQ3/RmVR
M0ybixx/IisD5zwVVeGl8IV32qdCwwE=
=I3sP
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 28 Oct 2022 22:23
(address . 47949@debbugs.gnu.org)
87zgdfd7ee.fsf@contorta
On 2022-10-26, Vagrant Cascadian wrote:
Toggle quote (8 lines)
> On 2022-10-26, zimoun wrote:
>> On Wed, 26 Oct 2022 at 09:57, Vagrant Cascadian <vagrant@debian.org> wrote:
>>
>>> It is consistently the same errors in the log, though on further looking
>>> i discovered a branch in ~/.cache/guix/checkouts/ that had old removed
>>> files in it (including wicd.scm/wicd.go) ... *maybe* that was somehow
>>> related. I did remove all the evidence, so so if stale checkouts is
>>> somehow the issue, it will be hard to reproduce again... oops.
...
Toggle quote (25 lines)
>>> I did manage again to use an old commit to pull up to a more recent
>>> master (a0751e3250dfea7e52468c8090e18c3118d93a60), and see there are new
>>> commits now so will try again. Will see.
>>
>> What is your hackish workflow? You do,
>>
>> guix pull --commit=<old> && guix pull
>
> To use the older generations I used:
>
> /var/guix/profiles/per-user/vagrant/current-guix-138-link/bin/guix pull ...
>
>
>> right? From your recent pull, does this
>>
>> guix time-machine --commit=<new> -- help
>>
>> work? where <new> is newer than your current revision.
>
> Oh yeah, that reminds me to add to the confusion, "guix time-machine
> --commit=SOMECOMMIT" worked fine, even where "SOMECOMMIT" didn't
> successfully work with guix pull.
>
> Maybe that's a clue pointing to the crufty .cache directories?

Well, after removing ~/.cache/guix/checkouts/ I haven't had the problem
again, with several successful pulls.

This suggests to me that guix should make sure to not use a dirty
checkout to prevent this in the future ... either exporting the guix
tree it's working with to a temporary location, or something like
running "git clean -dfx" before using the checkout...


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCY1w6SgAKCRDcUY/If5cW
qmIoAP4xpfVt0USPBv39W7scCx1Nhcwc7lSYZtB4NymSNo2ecwEAqvtWPGWgB11Z
pzIrds7Z+eCtX5HnZmElhB/bd4s69Qw=
=/gZl
-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 29 Oct 2022 16:42
(address . 47949@debbugs.gnu.org)
c4d12e61-a64f-0baf-f9e4-9e092de65165@telenet.be
On 28-10-2022 22:23, Vagrant Cascadian wrote:
Toggle quote (11 lines)
>[...]
>> Maybe that's a clue pointing to the crufty .cache directories?
>
> Well, after removing ~/.cache/guix/checkouts/ I haven't had the problem
> again, with several successful pulls.
>
> This suggests to me that guix should make sure to not use a dirty
> checkout to prevent this in the future ... either exporting the guix
> tree it's working with to a temporary location, or something like
> running "git clean -dfx" before using the checkout...
>
A (similar) problem was reported previously but I couldn't find it
anymore. A proposed solution was to use temporary git worktrees ("git
clean -dfx" wouldn't be sufficient in context of concurrent "guix
time-machine").
Greetings,
Maxime.
Attachment: OpenPGP_signature
Z
Z
zimoun wrote on 2 Nov 2022 12:02
(address . 47949@debbugs.gnu.org)
87iljxhb5a.fsf@gmail.com
Hi Vagrant,

On ven., 28 oct. 2022 at 13:23, Vagrant Cascadian <vagrant@debian.org> wrote:

Toggle quote (9 lines)
>> Oh yeah, that reminds me to add to the confusion, "guix time-machine
>> --commit=SOMECOMMIT" worked fine, even where "SOMECOMMIT" didn't
>> successfully work with guix pull.
>>
>> Maybe that's a clue pointing to the crufty .cache directories?
>
> Well, after removing ~/.cache/guix/checkouts/ I haven't had the problem
> again, with several successful pulls.

Well, “guix time-machine --commit=<some-commit>” uses,

Toggle snippet (16 lines)
(define %default-channel-url
;; URL of the default 'guix' channel.
"https://git.savannah.gnu.org/git/guix.git")

(define %default-guix-channel
(channel
(name 'guix)
(branch "master")
(url %default-channel-url)
(introduction %guix-channel-introduction)))

(define %default-channels
;; Default list of channels.
(list %default-guix-channel))

which is another directory checkout under ~/.cache/guix/checkouts. :-)


Toggle quote (5 lines)
> This suggests to me that guix should make sure to not use a dirty
> checkout to prevent this in the future ... either exporting the guix
> tree it's working with to a temporary location, or something like
> running "git clean -dfx" before using the checkout...

From my understanding, you have 3 similar checkouts of Guix; the default one:

Toggle snippet (14 lines)
$ cat ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/config
[core]
bare = false
repositoryformatversion = 0
filemode = true
logallrefupdates = true
[remote "origin"]
url = https://git.savannah.gnu.org/git/guix.git
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
remote = origin
merge = refs/heads/master

and then 2 others, probably located at these hash names:

Toggle snippet (21 lines)
$ guix repl
GNU Guile 3.0.8
Copyright (C) 1995-2021 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guix-user)> ,use(guix git)
scheme@(guix-user)> ,pp (map url-cache-directory (list
"https://git.savannah.gnu.org/git/guix.git"
"file:///home/vagrant/src/guix"
"file:///home/vagrant/src/guix-master"))
$1 = (
"/home/simon/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq"
"/home/simon/.cache/guix/checkouts/cbek2jy4zeoea2wj4xppwabceeayfzzuap6iw6uk7kylh4hdolfa"
"/home/simon/.cache/guix/checkouts/duxgcjge5pc2tzexf4qwjra3uzu3t4htsxojtxralegek3bqkisa"
)

then you pulled,

guix pull --url=/home/vagrant/src/guix --branch=master

and sometimes:

guix pull --url=/home/vagrant/src/guix-master --branch=master

How clean are these 2 local repositories?

I do not know if “git clean -dfx” would help because here the error is
probably between the repositories src/guix and src/guix-master and Guix
manages them under 2 unrelated directory checkouts.

When you cleaned ~/.cache/guix/checkouts and pulled again, you created a
new directory checkout. Based on which origin? Default? Or src/guix?
Or src/guix-master?

Cheers,
simon
V
V
Vagrant Cascadian wrote on 2 Nov 2022 19:40
(address . 47949@debbugs.gnu.org)
87sfj1ci97.fsf@contorta
On 2022-11-02, zimoun wrote:
Toggle quote (32 lines)
> On ven., 28 oct. 2022 at 13:23, Vagrant Cascadian <vagrant@debian.org> wrote:
>
>>> Oh yeah, that reminds me to add to the confusion, "guix time-machine
>>> --commit=SOMECOMMIT" worked fine, even where "SOMECOMMIT" didn't
>>> successfully work with guix pull.
>>>
>>> Maybe that's a clue pointing to the crufty .cache directories?
>>
>> Well, after removing ~/.cache/guix/checkouts/ I haven't had the problem
>> again, with several successful pulls.
>
> Well, “guix time-machine --commit=<some-commit>” uses,
>
> --8<---------------cut here---------------start------------->8---
> (define %default-channel-url
> ;; URL of the default 'guix' channel.
> "https://git.savannah.gnu.org/git/guix.git")
>
> (define %default-guix-channel
> (channel
> (name 'guix)
> (branch "master")
> (url %default-channel-url)
> (introduction %guix-channel-introduction)))
>
> (define %default-channels
> ;; Default list of channels.
> (list %default-guix-channel))
> --8<---------------cut here---------------end--------------->8---
>
> which is another directory checkout under ~/.cache/guix/checkouts. :-)

Ok, that's promising. :)


Toggle quote (46 lines)
>> This suggests to me that guix should make sure to not use a dirty
>> checkout to prevent this in the future ... either exporting the guix
>> tree it's working with to a temporary location, or something like
>> running "git clean -dfx" before using the checkout...
>
> From my understanding, you have 3 similar checkouts of Guix; the default one:
>
> --8<---------------cut here---------------start------------->8---
> $ cat ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/config
> [core]
> bare = false
> repositoryformatversion = 0
> filemode = true
> logallrefupdates = true
> [remote "origin"]
> url = https://git.savannah.gnu.org/git/guix.git
> fetch = +refs/heads/*:refs/remotes/origin/*
> [branch "master"]
> remote = origin
> merge = refs/heads/master
> --8<---------------cut here---------------end--------------->8---
>
> and then 2 others, probably located at these hash names:
>
> --8<---------------cut here---------------start------------->8---
> $ guix repl
> GNU Guile 3.0.8
> Copyright (C) 1995-2021 Free Software Foundation, Inc.
>
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
>
> Enter `,help' for help.
> scheme@(guix-user)> ,use(guix git)
> scheme@(guix-user)> ,pp (map url-cache-directory (list
> "https://git.savannah.gnu.org/git/guix.git"
> "file:///home/vagrant/src/guix"
> "file:///home/vagrant/src/guix-master"))
> $1 = (
> "/home/simon/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq"
> "/home/simon/.cache/guix/checkouts/cbek2jy4zeoea2wj4xppwabceeayfzzuap6iw6uk7kylh4hdolfa"
> "/home/simon/.cache/guix/checkouts/duxgcjge5pc2tzexf4qwjra3uzu3t4htsxojtxralegek3bqkisa"
> )
> --8<---------------cut here---------------end--------------->8---

Hashes are coming out different, maybe without the file:// prefix?


Toggle quote (10 lines)
> then you pulled,
>
> guix pull --url=/home/vagrant/src/guix --branch=master
>
> and sometimes:
>
> guix pull --url=/home/vagrant/src/guix-master --branch=master
>
> How clean are these 2 local repositories?

src/guix tends to have cruft laying around, but src/guix-master I tend
to keep clean, barring accidents. I *usually* just pull from
src/guix-master, which is essentially a local mirror of upstream
guix.git. Though I suspect the state of the local branch does not or at
least should not matter, since it's pulling from --branch=master.


Toggle quote (4 lines)
> I do not know if “git clean -dfx” would help because here the error is
> probably between the repositories src/guix and src/guix-master and Guix
> manages them under 2 unrelated directory checkouts.

I had the impression that it uses git to copy the branches (rather than
cp -r or something), otherwise the --branch=master argument would be
meaningless (e.g. --branch=master does not mean whatever happens to be
in the working directory), so I would assume the state of the local
working directory wouldn't matter, only what's in the specified branch
as git sees it.

So My suspicion here is somehow guix is working off of dirty checkouts
in ~/.cache/guix/checkouts ... which, maybe would not be so hard to
reproduce ... just drop some files from an ancient checkout or
something. Not sure what the conditions are necessary to make it
actually leave behind cruft in a way that triggers the issue. It
definitely seemed related to an old copy of wicd.scm that has since been
removed.


FWIW, I use my git checkout to avoid re-downloading the git history
multiple times... since I already typically have a full copy of the git
history.


Toggle quote (4 lines)
> When you cleaned ~/.cache/guix/checkouts and pulled again, you created a
> new directory checkout. Based on which origin? Default? Or src/guix?
> Or src/guix-master?

I think so far only src/guix-master.

live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCY2K5lAAKCRDcUY/If5cW
qnV2AQDt0Ddrh0L28GF+hOfu18Hw5P77/JQPgjDPXDnvK+HyrwEAyw/UvUAehi6m
EcEbqTfgXxGydphWviZLZqXJKVH5TgU=
=YSyP
-----END PGP SIGNATURE-----

Z
Z
zimoun wrote on 3 Nov 2022 09:48
(address . 47949@debbugs.gnu.org)
861qqke84q.fsf@gmail.com
Hi Vagrant,

On Wed, 02 Nov 2022 at 11:40, Vagrant Cascadian <vagrant@debian.org> wrote:

Toggle quote (11 lines)
>> I do not know if “git clean -dfx” would help because here the error is
>> probably between the repositories src/guix and src/guix-master and Guix
>> manages them under 2 unrelated directory checkouts.
>
> I had the impression that it uses git to copy the branches (rather than
> cp -r or something), otherwise the --branch=master argument would be
> meaningless (e.g. --branch=master does not mean whatever happens to be
> in the working directory), so I would assume the state of the local
> working directory wouldn't matter, only what's in the specified branch
> as git sees it.

From my understanding, this

guix pull --url=/home/vagrant/src/guix --branch=master

creates one directory under ~/.cache/guix/checkouts/ and then this,

guix pull --url=/home/vagrant/src/guix-master --branch=master

creates another directory under ~/.cache/guix/checkouts/ and these both
directory does not have the same name because their url is different.

IIUC, you were at generation 139 (using a clone of
/home/vagrant/src/guix living under say
~/.cache/guix/checkouts/maboosgggz6g6va54rikfxrsdsxhe363jri7g5dhcqb57odwklaa)

Toggle snippet (15 lines)
# First version where I noticed failures:
Generation 139 Oct 22 2022 13:07:08
guix bb2701b
repository URL: /home/vagrant/src/guix
branch: master
commit: bb2701b9111a3d82a82ceaaf2b22b51ecd8ac21f

# Failed to pull from 139, successfully pulled from 138:
Generation 140 Oct 24 2022 13:19:21
guix 8663be6
repository URL: /home/vagrant/src/guix-master
branch: master
commit: 8663be6da7f13a8eeea71dc1f493f7adc5b7672a

and then you had difficulties, but note that “guix pull” generating 140
created another clone from /home/vagrant/src/guix-master living under
~/.cache/guix/checkouts/ie7tn3i564wpt5i2dqqm7ixzqfxwpp2ns6rqkyqn4z55d4kqujuq.

Generation 140 did not updated the checkout used by generation 139.

Somehow, the state of
~/.cache/guix/checkouts/maboosgggz6g6va54rikfxrsdsxhe363jri7g5dhcqb57odwklaa
and the state of
~/.cache/guix/checkouts/ie7tn3i564wpt5i2dqqm7ixzqfxwpp2ns6rqkyqn4z55d4kqujuq
require some compatibility, no?


Well, I have lost the topic of the initial bug report. :-)

BTW, I agree that “guix pull” and “guix time-machine” are requiring too
much resource (especially bandwidth) when we could imagine more resource
shares between various accounts and root. Well, I do no know if
guile-git provides the git-worktree feature. Or we could also imagine a
more automated workflow using only one local clone as you are doing.


Cheers,
simon
V
V
Vagrant Cascadian wrote on 3 Nov 2022 19:35
(address . 47949@debbugs.gnu.org)
87edujevii.fsf@contorta
On 2022-11-03, zimoun wrote:
Toggle quote (23 lines)
> On Wed, 02 Nov 2022 at 11:40, Vagrant Cascadian <vagrant@debian.org> wrote:
>>> I do not know if “git clean -dfx” would help because here the error is
>>> probably between the repositories src/guix and src/guix-master and Guix
>>> manages them under 2 unrelated directory checkouts.
>>
>> I had the impression that it uses git to copy the branches (rather than
>> cp -r or something), otherwise the --branch=master argument would be
>> meaningless (e.g. --branch=master does not mean whatever happens to be
>> in the working directory), so I would assume the state of the local
>> working directory wouldn't matter, only what's in the specified branch
>> as git sees it.
>
> From my understanding, this
>
> guix pull --url=/home/vagrant/src/guix --branch=master
>
> creates one directory under ~/.cache/guix/checkouts/ and then this,
>
> guix pull --url=/home/vagrant/src/guix-master --branch=master
>
> creates another directory under ~/.cache/guix/checkouts/ and these both
> directory does not have the same name because their url is different.

Yes, this seems consistent with my understanding too.

I should also add that src/guix and src/guix-master are using the same
git repository, src/guix-master is just another worktree.


Toggle quote (32 lines)
> IIUC, you were at generation 139 (using a clone of
> /home/vagrant/src/guix living under say
> ~/.cache/guix/checkouts/maboosgggz6g6va54rikfxrsdsxhe363jri7g5dhcqb57odwklaa)
>
> --8<---------------cut here---------------start------------->8---
> # First version where I noticed failures:
> Generation 139 Oct 22 2022 13:07:08
> guix bb2701b
> repository URL: /home/vagrant/src/guix
> branch: master
> commit: bb2701b9111a3d82a82ceaaf2b22b51ecd8ac21f
>
> # Failed to pull from 139, successfully pulled from 138:
> Generation 140 Oct 24 2022 13:19:21
> guix 8663be6
> repository URL: /home/vagrant/src/guix-master
> branch: master
> commit: 8663be6da7f13a8eeea71dc1f493f7adc5b7672a
> --8<---------------cut here---------------end--------------->8---
>
> and then you had difficulties, but note that “guix pull” generating 140
> created another clone from /home/vagrant/src/guix-master living under
> ~/.cache/guix/checkouts/ie7tn3i564wpt5i2dqqm7ixzqfxwpp2ns6rqkyqn4z55d4kqujuq.
>
> Generation 140 did not updated the checkout used by generation 139.
>
> Somehow, the state of
> ~/.cache/guix/checkouts/maboosgggz6g6va54rikfxrsdsxhe363jri7g5dhcqb57odwklaa
> and the state of
> ~/.cache/guix/checkouts/ie7tn3i564wpt5i2dqqm7ixzqfxwpp2ns6rqkyqn4z55d4kqujuq
> require some compatibility, no?

Well, with the same inputs, they ought to produce the same outputs... :)


Toggle quote (2 lines)
> Well, I have lost the topic of the initial bug report. :-)

While I think somewhat of a digression, it explained the confusing
situation where "guix pull" appeared to behave differently (as I was
calling it with diffrent --url), as one of the ~/.cache/guix/checkouts/
was dirty and that mattered (as there were old removed files present in
the tree) ... which I suspect is the crux of the bug report, though I
don't know with absolutely certainty.

I'm guessing one could dirty the tree I'm guessing reverting:

edac21bfc78ffea85f4dac7206e5e7cd621bba19 gnu: Remove wicd.

Something like this, maybe?

cd ~/.cache/guix/checkouts/SOMECHECKOUT
git revert edac21bfc78ffea85f4dac7206e5e7cd621bba19
git reset origin/master
cd ~
guix pull ...


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHQEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCY2QJ9gAKCRDcUY/If5cW
qvDlAQCw7FyC9BPRJOEZVH3E80TZKIMk7ddLHROa+uMKJNG7ogD2MsL3dtyiIbnR
gVdhfEITrLfLJa3QY8yUZeI1z1lXCw==
=ckWT
-----END PGP SIGNATURE-----

?