Possibly found a bug while doing a "guix pull"

DoneSubmitted by Léon Lain Delysid.
Details
5 participants
  • Danny Milosavljevic
  • Julien Lepiller
  • Léon Lain Delysid
  • Ludovic Courtès
  • zimoun
Owner
unassigned
Severity
normal
L
L
Léon Lain Delysid wrote on 4 Jun 16:38 +0200
(address . bug-guix@gnu.org)
CAHecTjeq=mk7BGcnGhi089srDbGAvkU0OqdYUtG2-joD_dn_yg@mail.gmail.com
Hello!I just tried to "guix pull" on my Debian Buster GNU/Linux system. Twice itmade the system completely freeze for a few seconds and then reboot. Andthat third time it didn't crash the system but the guix pull failed andreturned this error:https://pastebin.com/vhDR8gDC
Can somebody help me please?
(Before doing the third "pull", I did an "apt update" and "upgrade", but Iwasn't that much behind in updates, I had done it less than a week agolast. My last "guix pull" was made at the same time as the "apt update", afew days ago.)
Hoping I can resolve this soon or that my feedback was positive for thedevelopment of Guix. =)I can provide log debug info (if there is any) upon request.
Best regards!
Attachment: file
J
J
Julien Lepiller wrote on 4 Jun 17:41 +0200
DE1A9277-D737-45E4-908A-A63E6DF6BE1F@lepiller.eu
Le 4 juin 2020 10:38:09 GMT-04:00, "Léon Lain Delysid" <leon.lain.delysid@gmail.com> a écrit :
Toggle quote (24 lines)>Hello!>I just tried to "guix pull" on my Debian Buster GNU/Linux system. Twice>it>made the system completely freeze for a few seconds and then reboot.>And>that third time it didn't crash the system but the guix pull failed and>returned this error:>https://pastebin.com/vhDR8gDC>>Can somebody help me please?>>(Before doing the third "pull", I did an "apt update" and "upgrade",>but I>wasn't that much behind in updates, I had done it less than a week ago>last. My last "guix pull" was made at the same time as the "apt>update", a>few days ago.)>>Hoping I can resolve this soon or that my feedback was positive for the>development of Guix. =)>I can provide log debug info (if there is any) upon request.>>Best regards!
Sorry pastebin is not tor friendly, I couldn't look at it.
For the freezing part, could it be that you have relatively low free ram (<2GB) and no swap? Or maybe /tmp is mounted as a tmpfs and for some reason it takes a lot of space?
You might need to use substitutes when runnning guix pull. I don't have that much ram either on some computers, so I always take a look at https://ci.guix.gnu.org/jobset/guix-modular-masterand choose the latest commit that was built for my architeeture. Then eg. guix pull --commit=c987b72
This will allow guix to download substitutes for guix pull instead of building locally, which doesn't require any ram usage.
HTH!
Z
Z
zimoun wrote on 4 Jun 20:11 +0200
(name . Julien Lepiller)(address . julien@lepiller.eu)
CAJ3okZ26+qJP-9vDFUdKtKUxM-iaTgGd50Rrd1x=efAUf18qbQ@mail.gmail.com
Dear,
On Thu, 4 Jun 2020 at 17:42, Julien Lepiller <julien@lepiller.eu> wrote:
Toggle quote (6 lines)> Le 4 juin 2020 10:38:09 GMT-04:00, "Léon Lain Delysid" <leon.lain.delysid@gmail.com> a écrit :
> >I just tried to "guix pull" on my Debian Buster GNU/Linux system. Twice> >it> >made the system completely freeze for a few seconds and then reboot.
What do you mean by "then reboot"?Does it reboot by itself or do you manually force the reboot?
Toggle quote (5 lines)> >And> >that third time it didn't crash the system but the guix pull failed and> >returned this error:> >https://pastebin.com/vhDR8gDC
How do you start guix-daemon?

Toggle quote (7 lines)> >(Before doing the third "pull", I did an "apt update" and "upgrade",> >but I> >wasn't that much behind in updates, I had done it less than a week ago> >last. My last "guix pull" was made at the same time as the "apt> >update", a> >few days ago.)
What is your "guix describe"?
Toggle quote (9 lines)>> >Hoping I can resolve this soon or that my feedback was positive for the> >development of Guix. =)> >I can provide log debug info (if there is any) upon request.> >> >Best regards!>> Sorry pastebin is not tor friendly, I couldn't look at it.
Julien, here the paste:
Toggle snippet (14 lines)./guix/store.scm:1035:9: ERROR: 1. &store-protocol-error: message: "error parsing derivation`/gnu/store/m5djg8zwgl7nh0rm1av4nnlbwgwi0fwg-guix-daemon-1.1.0-4.bdc801e.drv':expected string `Derive(['" status: 1guix pull: error: You found a bug: the program'/gnu/store/kpxami25fi3mrxb37sfbbx2s366chpk5-compute-guix-derivation'failed to compute the derivation for Guix (version:"415b90f24ee83059102f5fb0ce74a2bc0954fe58"; system: "x86_64-linux";host version: "018cffc9c9e5a5855733f5f45a1c4d396bb6a321"; pull-version: 1).Please report it by email to <bug-guix@gnu.org>.
First time, I see that.
All the best,simon
L
L
Ludovic Courtès wrote on 6 Jun 15:22 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
87a71guy5m.fsf@gnu.org
Hi,
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (13 lines)> ./guix/store.scm:1035:9: ERROR:> 1. &store-protocol-error:> message: "error parsing derivation> `/gnu/store/m5djg8zwgl7nh0rm1av4nnlbwgwi0fwg-guix-daemon-1.1.0-4.bdc801e.drv':> expected string `Derive(['"> status: 1> guix pull: error: You found a bug: the program> '/gnu/store/kpxami25fi3mrxb37sfbbx2s366chpk5-compute-guix-derivation'> failed to compute the derivation for Guix (version:> "415b90f24ee83059102f5fb0ce74a2bc0954fe58"; system: "x86_64-linux";> host version: "018cffc9c9e5a5855733f5f45a1c4d396bb6a321"; pull-version: 1).> Please report it by email to <bug-guix@gnu.org>.
Léon, is this .drv file empty by any chance?
If the answer is yes, that suggests a file system corruption. We seereports like this sometimes. It could maybe happen if the file systemwas not cleanly unmounted.
The solution may be to run:
guix gc -D /gnu/store/m5djg8zwgl7nh0rm1av4nnlbwgwi0fwg-guix-daemon-1.1.0-4.bdc801e.drv
before re-running ‘guix pull’.
Could you try that and report back?
Thanks,Ludo’.
L
L
Léon Lain Delysid wrote on 7 Jun 03:12 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAHecTjezH8uuU-+y7k_H+jSvf3oA3jr4u+AQCJ_oxSC7wTDF=A@mail.gmail.com
Hello, here's the result of the command you asked me to run. I executed it,then did a "guix pull" again and got the same result. But the corruptedfilesystem idea might be that one. Can you give me the best command to runa repair of the filesystem?[image: Screenshot from 2020-06-07 03-05-33.jpg]
[image: Screenshot from 2020-06-07 02-50-54.jpg]
On Sat, Jun 6, 2020 at 3:22 PM Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (37 lines)> Hi,>> zimoun <zimon.toutoune@gmail.com> skribis:>> > ./guix/store.scm:1035:9: ERROR:> > 1. &store-protocol-error:> > message: "error parsing derivation> >> `/gnu/store/m5djg8zwgl7nh0rm1av4nnlbwgwi0fwg-guix-daemon-1.1.0-4.bdc801e.drv':> > expected string `Derive(['"> > status: 1> > guix pull: error: You found a bug: the program> > '/gnu/store/kpxami25fi3mrxb37sfbbx2s366chpk5-compute-guix-derivation'> > failed to compute the derivation for Guix (version:> > "415b90f24ee83059102f5fb0ce74a2bc0954fe58"; system: "x86_64-linux";> > host version: "018cffc9c9e5a5855733f5f45a1c4d396bb6a321"; pull-version:> 1).> > Please report it by email to <bug-guix@gnu.org>.>> Léon, is this .drv file empty by any chance?>> If the answer is yes, that suggests a file system corruption. We see> reports like this sometimes. It could maybe happen if the file system> was not cleanly unmounted.>> The solution may be to run:>> guix gc -D> /gnu/store/m5djg8zwgl7nh0rm1av4nnlbwgwi0fwg-guix-daemon-1.1.0-4.bdc801e.drv>> before re-running ‘guix pull’.>> Could you try that and report back?>> Thanks,> Ludo’.>
Attachment: file
L
L
Léon Lain Delysid wrote on 7 Jun 03:15 +0200
(name . Julien Lepiller)(address . julien@lepiller.eu)(address . 41710@debbugs.gnu.org)
CAHecTjfDLmc0CrYfi__VqkRVYjwtLRj5fTRPSSGioO2qbqFrDA@mail.gmail.com
Hello,Sorry for responding so late, I was busy otherwise.I have more than enough RAM, 16 GB. And also lots of swap space. And my/tmp takes 104KB of space.


[image: Screenshot from 2020-06-07 03-05-33.jpg]
On Thu, Jun 4, 2020 at 5:41 PM Julien Lepiller <julien@lepiller.eu> wrote:
Toggle quote (43 lines)> Le 4 juin 2020 10:38:09 GMT-04:00, "Léon Lain Delysid" <> leon.lain.delysid@gmail.com> a écrit :> >Hello!> >I just tried to "guix pull" on my Debian Buster GNU/Linux system. Twice> >it> >made the system completely freeze for a few seconds and then reboot.> >And> >that third time it didn't crash the system but the guix pull failed and> >returned this error:> >https://pastebin.com/vhDR8gDC> >> >Can somebody help me please?> >> >(Before doing the third "pull", I did an "apt update" and "upgrade",> >but I> >wasn't that much behind in updates, I had done it less than a week ago> >last. My last "guix pull" was made at the same time as the "apt> >update", a> >few days ago.)> >> >Hoping I can resolve this soon or that my feedback was positive for the> >development of Guix. =)> >I can provide log debug info (if there is any) upon request.> >> >Best regards!>> Sorry pastebin is not tor friendly, I couldn't look at it.>> For the freezing part, could it be that you have relatively low free ram> (<2GB) and no swap? Or maybe /tmp is mounted as a tmpfs and for some reason> it takes a lot of space?>> You might need to use substitutes when runnning guix pull. I don't have> that much ram either on some computers, so I always take a look at> https://ci.guix.gnu.org/jobset/guix-modular-master and choose the latest> commit that was built for my architeeture. Then eg. guix pull> --commit=c987b72>> This will allow guix to download substitutes for guix pull instead of> building locally, which doesn't require any ram usage.>> HTH!>
Attachment: file
L
L
Léon Lain Delysid wrote on 7 Jun 03:19 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
CAHecTjd3SyMEYSzVt0br0m3QAcYhRyLXfHEiVNbL2uYi__4R_A@mail.gmail.com
Hi!Sorry for the late response, I was busy otherwise.By "then reboot", I meant it rebooted by itself. The first two times Itried the "guix pull".
"How do you start guix-daemon?"I didn't change any of the defaults regarding this. So I guess the daemonmust start at startup with the other daemons.
"guix describe" gives this output:$ guix describeGeneration 12 mai 31 2020 11:28:03 (current) guix 018cffc repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 018cffc9c9e5a5855733f5f45a1c4d396bb6a321

[image: Screenshot from 2020-06-07 03-05-33.jpg]
On Thu, Jun 4, 2020 at 8:11 PM zimoun <zimon.toutoune@gmail.com> wrote:
Toggle quote (62 lines)> Dear,>> On Thu, 4 Jun 2020 at 17:42, Julien Lepiller <julien@lepiller.eu> wrote:> > Le 4 juin 2020 10:38:09 GMT-04:00, "Léon Lain Delysid" <> leon.lain.delysid@gmail.com> a écrit :>> > >I just tried to "guix pull" on my Debian Buster GNU/Linux system. Twice> > >it> > >made the system completely freeze for a few seconds and then reboot.>> What do you mean by "then reboot"?> Does it reboot by itself or do you manually force the reboot?>> > >And> > >that third time it didn't crash the system but the guix pull failed and> > >returned this error:> > >https://pastebin.com/vhDR8gDC>> How do you start guix-daemon?>>> > >(Before doing the third "pull", I did an "apt update" and "upgrade",> > >but I> > >wasn't that much behind in updates, I had done it less than a week ago> > >last. My last "guix pull" was made at the same time as the "apt> > >update", a> > >few days ago.)>> What is your "guix describe"?>> >> > >Hoping I can resolve this soon or that my feedback was positive for the> > >development of Guix. =)> > >I can provide log debug info (if there is any) upon request.> > >> > >Best regards!> >> > Sorry pastebin is not tor friendly, I couldn't look at it.>> Julien, here the paste:>> --8<---------------cut here---------------start------------->8---> ./guix/store.scm:1035:9: ERROR:> 1. &store-protocol-error:> message: "error parsing derivation>> `/gnu/store/m5djg8zwgl7nh0rm1av4nnlbwgwi0fwg-guix-daemon-1.1.0-4.bdc801e.drv':> expected string `Derive(['"> status: 1> guix pull: error: You found a bug: the program> '/gnu/store/kpxami25fi3mrxb37sfbbx2s366chpk5-compute-guix-derivation'> failed to compute the derivation for Guix (version:> "415b90f24ee83059102f5fb0ce74a2bc0954fe58"; system: "x86_64-linux";> host version: "018cffc9c9e5a5855733f5f45a1c4d396bb6a321"; pull-version: 1).> Please report it by email to <bug-guix@gnu.org>.> --8<---------------cut here---------------end--------------->8--->> First time, I see that.>> All the best,> simon>
Attachment: file
L
L
Ludovic Courtès wrote on 7 Jun 22:06 +0200
(name . Léon Lain Delysid)(address . leon.lain.delysid@gmail.com)
87wo4ir67g.fsf@gnu.org
Hi Léon,
Léon Lain Delysid <leon.lain.delysid@gmail.com> skribis:
Toggle quote (3 lines)> Hello, here's the result of the command you asked me to run. I executed it,> then did a "guix pull" again and got the same result.
Did you try re-running ‘guix pull’ after the ‘guix gc -D’ command? (Idon’t see it on the screenshot.)
Toggle quote (3 lines)> But the corrupted filesystem idea might be that one. Can you give me> the best command to run a repair of the filesystem?
If the suggestion above is not above, then you could try running this asroot:
guix build --repair \ /gnu/store/m5djg8zwgl7nh0rm1av4nnlbwgwi0fwg-guix-daemon-1.1.0-4.bdc801e.drv
Let me know how it goes.
HTH,Ludo’.
L
L
Léon Lain Delysid wrote on 8 Jun 01:15 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAHecTjeZecr8HguqBnkLT=MQbmQJF3ea0FO1JRUWuHxO1d07tQ@mail.gmail.com
Actually, the biggest screenshot of the two was the end of the output ofthe "guix pull" command after doing the "guix gc -D" command.
And the "build --repair" command resulted in this output:guix build: error: error parsing derivation`/gnu/store/8ablj2a1ihn32ilnfw1c3brpv9l7akzc-guile-json-4.0.1.drv':expected string `Derive(['
So, that file was empty aswell: /gnu/store/8ablj2a1ihn32ilnfw1c3brpv9l7akzc-guile-json-4.0.1.drvSo I ran the "guix gc -D" on it, then "guix pull" again, and it found otherempty .drv's. I repeated this scheme until the "guix pull" command wassuccessful.
Thank you people so very much for your kind help!My system is now bug free again. I don't know if I discovered real bugs, orif my system crashing twice during a pull command somehow broke it, but Ihope this feedback helped. Maybe one day I will find out why my systemcrashed and rebooted by itself a minute later. After all, it could havebeen caused by something else on my system at that moment…
Have a good day! =)

On Sun, Jun 7, 2020 at 10:06 PM Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (26 lines)> Hi Léon,>> Léon Lain Delysid <leon.lain.delysid@gmail.com> skribis:>> > Hello, here's the result of the command you asked me to run. I executed> it,> > then did a "guix pull" again and got the same result.>> Did you try re-running ‘guix pull’ after the ‘guix gc -D’ command? (I> don’t see it on the screenshot.)>> > But the corrupted filesystem idea might be that one. Can you give me> > the best command to run a repair of the filesystem?>> If the suggestion above is not above, then you could try running this as> root:>> guix build --repair \>> /gnu/store/m5djg8zwgl7nh0rm1av4nnlbwgwi0fwg-guix-daemon-1.1.0-4.bdc801e.drv>> Let me know how it goes.>> HTH,> Ludo’.>
Attachment: file
D
D
Danny Milosavljevic wrote on 8 Jun 13:02 +0200
(name . Léon Lain Delysid)(address . leon.lain.delysid@gmail.com)
20200608130257.66286fbd@scratchpost.org
Hi,
On Mon, 8 Jun 2020 01:15:46 +0200Léon Lain Delysid <leon.lain.delysid@gmail.com> wrote:
Toggle quote (3 lines)>or> if my system crashing twice during a pull command somehow broke it,
Probably.
Toggle quote (3 lines)>but I> hope this feedback helped.
It sure helped. It's good to know that that can happen.
I remember the first time I used Guix, I picked some file system that wouldkeep doing that: leave empty files if the system crashed (among lots of otherthings). And that system crashed a lot. I had the same result as you,and a lot of additional problems.
Back then we already improved a lot of places that were really reallyimportant (added fsync calls), so the remaining places should be quiteharmless--like this one. Because of Guix, you can always rebuild/gnu/store just as it was--after a long build time maybe, but it's possible(could be made a LOT more usable, though).
(fsync degrades performance, so it makes no sense to fsync for /gnu/store)
I think we can't really do more without imposing undue mainentance burden onus (for something the file system shouldn't be doing in the first place),or we could recommend another file system or different file system optionsin the manual. What would the latter be?
Also, how it the world didn't the file system checker fsck
(1) automatically run and(2) fix this
in your case?
I mean what if those were user documents, which *couldn't* be restored soeasily? Weird...
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl7eGuEACgkQ5xo1VCwwuqUtDggAikSYqEsLCaFjTnGRIa2cXdMNi892N5hm+2K1DeSbRGyspG9mH5bBDB//hW1hTWEZhYE8y7FeHk0OFITroYEZzR2cUoblmcLeSkE0mqPy0Hct+Y4QHbSWwC95JHQdtZy3DVmTuW6BfSDmK8BlLt67ZYbC0hnZYLbwqKMDX+J3P+mWezwd1g3PLibrmJrTvEhE+h3ehU49eiDZukVSKUagg+3ZgR0BmPL3ZVXa5bSNvKjhqVde5OB6kojagMxng6VkVfoKIrkFYuI2YkGCvrxdjOPWBie9LlfyMktJlvgcULXwyPC/WKRTJbeKpgC03ANfarYa/C4dRtHv9BMbO5Tgqw===NeyH-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 10 Jun 15:25 +0200
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87v9jzdpdh.fsf@gnu.org
Hello,
Danny Milosavljevic <dannym@scratchpost.org> skribis:
Toggle quote (38 lines)> On Mon, 8 Jun 2020 01:15:46 +0200> Léon Lain Delysid <leon.lain.delysid@gmail.com> wrote:>>>or>> if my system crashing twice during a pull command somehow broke it, >> Probably.>>>but I>> hope this feedback helped. >> It sure helped. It's good to know that that can happen.>> I remember the first time I used Guix, I picked some file system that would> keep doing that: leave empty files if the system crashed (among lots of other> things). And that system crashed a lot. I had the same result as you,> and a lot of additional problems.>> Back then we already improved a lot of places that were really really> important (added fsync calls), so the remaining places should be quite> harmless--like this one. Because of Guix, you can always rebuild> /gnu/store just as it was--after a long build time maybe, but it's possible> (could be made a LOT more usable, though).>> (fsync degrades performance, so it makes no sense to fsync for /gnu/store)>> I think we can't really do more without imposing undue mainentance burden on> us (for something the file system shouldn't be doing in the first place),> or we could recommend another file system or different file system options> in the manual. What would the latter be?>> Also, how it the world didn't the file system checker fsck>> (1) automatically run and> (2) fix this>> in your case?
Yeah, that’s really weird. I never experienced it first-hand, but it’snot the first time we have such a report.
Ext4 & co. reportedly can leave empty files upon crashes; perhaps that’sa problem with those file systems (though I’ve always used ext2/3/4 andnever had this problem myself, but that’s not statically significant).
Anyway, closing.
Thank you, Léon!
Ludo’.
Closed
L
L
Léon Lain Delysid wrote on 12 Jun 11:55 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAHecTjdJzU7uDw1weZoQ8zEgV9wi9AECAU=-qJeekU9xRqW2vA@mail.gmail.com
That's really weird. I thought EXT3/4 was the best of the best? If thefiles were user documents, I would be pissed. Well, we can't have perfectcomputer programs out of the box.And then again, I did say my system crashed in a very violent way (twice).I had only one window open: the terminal with the "guix pull" running, butit might have been some other program running in the background that frozemy system. The command was running, and then all of a sudden, the wholesystem froze. The screen was frozen, the mouse, and even the keyboard. Thenafter a minute, it rebooted by itself. That's a pretty violent crash.
On Wed, Jun 10, 2020, 15:25 Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (61 lines)> Hello,>> Danny Milosavljevic <dannym@scratchpost.org> skribis:>> > On Mon, 8 Jun 2020 01:15:46 +0200> > Léon Lain Delysid <leon.lain.delysid@gmail.com> wrote:> >> >>or> >> if my system crashing twice during a pull command somehow broke it,> >> > Probably.> >> >>but I> >> hope this feedback helped.> >> > It sure helped. It's good to know that that can happen.> >> > I remember the first time I used Guix, I picked some file system that> would> > keep doing that: leave empty files if the system crashed (among lots of> other> > things). And that system crashed a lot. I had the same result as you,> > and a lot of additional problems.> >> > Back then we already improved a lot of places that were really really> > important (added fsync calls), so the remaining places should be quite> > harmless--like this one. Because of Guix, you can always rebuild> > /gnu/store just as it was--after a long build time maybe, but it's> possible> > (could be made a LOT more usable, though).> >> > (fsync degrades performance, so it makes no sense to fsync for> /gnu/store)> >> > I think we can't really do more without imposing undue mainentance> burden on> > us (for something the file system shouldn't be doing in the first place),> > or we could recommend another file system or different file system> options> > in the manual. What would the latter be?> >> > Also, how it the world didn't the file system checker fsck> >> > (1) automatically run and> > (2) fix this> >> > in your case?>> Yeah, that’s really weird. I never experienced it first-hand, but it’s> not the first time we have such a report.>> Ext4 & co. reportedly can leave empty files upon crashes; perhaps that’s> a problem with those file systems (though I’ve always used ext2/3/4 and> never had this problem myself, but that’s not statically significant).>> Anyway, closing.>> Thank you, Léon!>> Ludo’.>
Attachment: file
Closed
?