Corrupt, un-repairable store after power cut during upgrade

  • Open
  • quality assurance status badge
Details
2 participants
  • elaexuotee
  • Liliana Marie Prikler
Owner
unassigned
Submitted by
elaexuotee
Severity
normal
E
E
elaexuotee wrote on 16 May 2021 10:59
(address . bug-guix@gnu.org)
3OE5JSDGIIX2D.3SOQ0QIKI7XZR@wilsonb.com
Hey Guix,

Recently, during in the middle of a `guix package -m .guix/manifest.scm`, my
machine had power failure. After rebotting, I simply reran the above and
it completed without error.

However, it appears that some of files in the store items are corrupt:

$ vim
vim: error while loading shared libraries: /gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32/lib/libgdk-x11-2.0.so.0: file too short

$ file -L /gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32/lib/libgdk-x11-2.0.so.0
/gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32/lib/libgdk-x11-2.0.so.0: empty

As an extra bit of annoyance, the handy verify and repair tools fail to see a
problem:

$ command -v vim | xargs readlink -f
/gnu/store/h9q2fik60axj9sai5b66cjk2sfl37ij1-vim-full-8.2.2689/bin/vim

$ sudo guix build --repair vim-full
/gnu/store/h9q2fik60axj9sai5b66cjk2sfl37ij1-vim-full-8.2.2689

$ sudo guix build --repair --no-grafts vim-full
path `/gnu/store/73p5rzka1cb9dx9fy059lmkf1jg6sv95-gtk+-2.24.32' is corrupted or missing!
found corrupted or missing path `/gnu/store/73p5rzka1cb9dx9fy059lmkf1jg6sv95-gtk+-2.24.32' in the output closure of `/gnu/store/qyp10sss1ascp70yshs87zdzr1lg71f8-vim-full-8.2.2689.drv'
substituting /gnu/store/73p5rzka1cb9dx9fy059lmkf1jg6sv95-gtk+-2.24.32...
gtk%2B-2.24.32 6.4MiB 152KiB/s 00:43 [##################] 100.0%

/gnu/store/d21m63am0pxw77883h0y1jmy9iiazvvx-vim-full-8.2.2689

$ sudo guix gc --verify=repair,contents
...

$ vim
vim: error while loading shared libraries: /gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32/lib/libgdk-x11-2.0.so.0: file too short

Even going in with a crude hammer doesn't fix things:

$ sudo mount -o rw,remount /gnu/store

$ sudo mv /gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32 .

$ sudo guix build --repair vim-full
path `/gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32' is corrupted or missing!
found corrupted or missing path `/gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32' in the output closure of `/gnu/store/z2q3zna20cn32if3v271x4xavd70ml5z-vim-full-8.2.2689.drv'
path `/gnu/store/73p5rzka1cb9dx9fy059lmkf1jg6sv95-gtk+-2.24.32' is corrupted or missing!
substituting /gnu/store/73p5rzka1cb9dx9fy059lmkf1jg6sv95-gtk+-2.24.32...
gtk%2B-2.24.32 6.4MiB 134KiB/s 00:49 [##################] 100.0%

applying 17 grafts for /gnu/store/lm9xqz82q4cwlqs79i0pq77z6cfsx3rw-gtk+-2.24.32.drv ...
grafting '/gnu/store/73p5rzka1cb9dx9fy059lmkf1jg6sv95-gtk+-2.24.32' -> '/gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32'...
successfully built /gnu/store/lm9xqz82q4cwlqs79i0pq77z6cfsx3rw-gtk+-2.24.32.drv
gnu/store/h9q2fik60axj9sai5b66cjk2sfl37ij1-vim-full-8.2.2689

$ vim
vim: error while loading shared libraries: /gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32/lib/libgdk-x11-2.0.so.0: file too short

It seems like the above graft is going badly:

$ file -L /gnu/store/73p5rzka1cb9dx9fy059lmkf1jg6sv95-gtk+-2.24.32/lib/libgtk-x11-2.0.so.0
/gnu/store/73p5rzka1cb9dx9fy059lmkf1jg6sv95-gtk+-2.24.32/lib/libgtk-x11-2.0.so.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), statically linked, not stripped

$ file -L /gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32/lib/libgdk-x11-2.0.so.0
/gnu/store/6z2g9ybl3fkx6bdfg226mz8rpc0g53jx-gtk+-2.24.32/lib/libgdk-x11-2.0.so.0: empty

Some questions:

1) Am I just missing something obvious?
2) How could grafting end up producing empty files in the output?
3) Is the store supposed to be robust against power failures?
L
L
Liliana Marie Prikler wrote on 23 Oct 2021 09:49
182fe7e6dbcdfedce51c77efc56e35e45da817a6.camel@gmail.com
Hi Guix,

Am Sonntag, den 16.05.2021, 17:59 +0900 schrieb elaexuotee@wilsonb.com:
Toggle quote (21 lines)
> Hey Guix,
>
> Recently, during in the middle of a `guix package -m
> .guix/manifest.scm`, my
> machine had power failure. After rebotting, I simply reran the above
> and it completed without error.
>
> However, it appears that some of files in the store items are
> corrupt:
>
> [...]
> As an extra bit of annoyance, the handy verify and repair tools fail
> to see a problem:
>
> [...]
>
> Some questions:
>
> 1) Am I just missing something obvious?
> 2) How could grafting end up producing empty files in the output?
> 3) Is the store supposed to be robust against power failures?
After dealing with some softer issues related to power outage during
`guix package', I think I'm now able to say that Guix does not process
grafts correctly during repair.

What happened for me, was that certain store files were empty and when
they were later grafted, those empty files were carried over. Guix,
upon repairing the store, did then not regraft those, but rather kept
them as-is. I have since fixed said issue by manually deleting the
graft and rebuilding (this was possible, because nothing referred to it
– I'm not sure why `guix gc' didn't pick it up as dead earlier,
though). Stuff in my profiles seem to also have been rebuilt after
roll-back+gc.

So I think content repair needs to have at least two steps:
1. checking and repairing all "normal" derivations.
2. fixing grafts of packages that were repaired in step 1.

WDYT?
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 48462@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 48462
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch