'gexp-modules' fails to delete duplicates when code is evaluated

  • Done
  • quality assurance status badge
Details
2 participants
  • Clément Lassieur
  • Ludovic Courtès
Owner
unassigned
Submitted by
Clément Lassieur
Severity
important
C
C
Clément Lassieur wrote on 7 Oct 2018 14:23
tests/install.scm tests all fail
(address . bug-guix@gnu.org)
87murq0wtn.fsf@lassieur.org
Hi,

Those tests are run at each evaluation and they all fail.

test.btrfs-root-os.x86_64-linux
test.encrypted-root-os.x86_64-linux
test.installed-extlinux-os.x86_64-linux
test.separate-home-os.x86_64-linux
test.raid-root-os.x86_64-linux
test.separate-store-os.x86_64-linux
test.iso-image-installer.x86_64-linux
test.installed-os.x86_64-linux
test.btrfs-root-os.i686-linux
test.encrypted-root-os.i686-linux
test.installed-extlinux-os.i686-linux
test.separate-home-os.i686-linux
test.raid-root-os.i686-linux
test.separate-store-os.i686-linux
test.iso-image-installer.i686-linux
test.installed-os.i686-linux

This is the output of one of them when run manually:

Toggle snippet (47 lines)
$ guix build /gnu/store/0l77fgdgp4c3yq82mwd1kq1rzckn5x9b-btrfs-root-os.drv
substitute: updating substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bayfront.guixsd.org'... 100.0%
substitute: updating substitutes from 'https://berlin.guixsd.org'... 100.0%
The following derivations will be built:
/gnu/store/0l77fgdgp4c3yq82mwd1kq1rzckn5x9b-btrfs-root-os.drv
/gnu/store/hk27hcjkrcsjacpj0f6bjdr48glj73jv-module-import-compiled.drv
/gnu/store/l6vzjb50ywb3xh7m0jh67b1y1l8mddc5-module-import.drv
/gnu/store/kg33b56kayrqy6a12rmcg2icwchxaik8-builder-in-linux-vm.drv
/gnu/store/a8cwqky87qjwi1wrfm38wv8ivhvim3dr-shepherd-cow-store.scm.drv
/gnu/store/7nkmah5qy0176zx1hv9szifc4hqf7xwj-shepherd-term-tty3.scm.drv
/gnu/store/9gqb8lgmlmmpfrvlqyzzzygknvdq0j77-shepherd.conf.drv
/gnu/store/8z3nfh5psn6gf16yn744bkd1b6k9s958-boot.drv
/gnu/store/wv6h79wl3iwwkkxw1lj482ccig8vvg0j-manual-database.drv
/gnu/store/68fdr40vb3k7hnjxhagd5i13vlp7irvn-profile.drv
/gnu/store/5zwawq90pdibkjwirfm6ag4cf9zp344h-system.drv
/gnu/store/7b054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv
/gnu/store/ip8h5xsn6cw2xh7l16sqhm4f41nqgjdd-grub.cfg.drv
/gnu/store/w3i1s5k3d01wwn8xyc2m4v6aqk2fw4wx-module-import-compiled.drv
/gnu/store/x7ap44jargq6kq2kwvgf6h81dz4j15ln-linux-vm-loader.drv
/gnu/store/wi23bh8017dxq4ljsvv7m3gqjkynbbdn-disk-image.drv
/gnu/store/qn1302dzn8f1k2b8v928hbm9ln86qx13-installation.drv
0.0 MB will be downloaded:
/gnu/store/54j2a2bfzfd8gdxlan0v6r8zjfb6ywj2-gc-roots
/gnu/store/fzbxdfy357zmbs3f5333kg3l5b5h20jp-system
building /gnu/store/7b054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv...
substitute: updating substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bayfront.guixsd.org'... 100.0%
substitute: updating substitutes from 'https://berlin.guixsd.org'... 100.0%
Backtrace:
2 (primitive-load "/gnu/store/fcybzbgns1nsglrkdj833mk53si?")
In srfi/srfi-1.scm:
640:9 1 (for-each #<procedure c487a0 at ice-9/eval.scm:333:13 ?> ?)
In unknown file:
0 (copy-file "/gnu/store/sqacjhyfz32diqys3pp7j6x8bs2zx7p?" ?)

ERROR: In procedure copy-file:
In procedure copy-file: Permission denied
builder for `/gnu/store/7b054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv' failed with exit code 1
build of /gnu/store/7b054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv failed
View build log at '/var/log/guix/drvs/7b/054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv.gz'.
cannot build derivation `/gnu/store/wi23bh8017dxq4ljsvv7m3gqjkynbbdn-disk-image.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/qn1302dzn8f1k2b8v928hbm9ln86qx13-installation.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/0l77fgdgp4c3yq82mwd1kq1rzckn5x9b-btrfs-root-os.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/0l77fgdgp4c3yq82mwd1kq1rzckn5x9b-btrfs-root-os.drv' failed

Clément
L
L
Ludovic Courtès wrote on 8 Oct 2018 15:46
(name . Clément Lassieur)(address . clement@lassieur.org)(address . 32966@debbugs.gnu.org)
87pnwky2j5.fsf@gnu.org
Hello!

Clément Lassieur <clement@lassieur.org> skribis:

Toggle quote (20 lines)
> building /gnu/store/7b054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv...
> substitute: updating substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
> substitute: updating substitutes from 'https://bayfront.guixsd.org'... 100.0%
> substitute: updating substitutes from 'https://berlin.guixsd.org'... 100.0%
> Backtrace:
> 2 (primitive-load "/gnu/store/fcybzbgns1nsglrkdj833mk53si?")
> In srfi/srfi-1.scm:
> 640:9 1 (for-each #<procedure c487a0 at ice-9/eval.scm:333:13 ?> ?)
> In unknown file:
> 0 (copy-file "/gnu/store/sqacjhyfz32diqys3pp7j6x8bs2zx7p?" ?)
>
> ERROR: In procedure copy-file:
> In procedure copy-file: Permission denied
> builder for `/gnu/store/7b054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv' failed with exit code 1
> build of /gnu/store/7b054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv failed
> View build log at '/var/log/guix/drvs/7b/054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv.gz'.
> cannot build derivation `/gnu/store/wi23bh8017dxq4ljsvv7m3gqjkynbbdn-disk-image.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/qn1302dzn8f1k2b8v928hbm9ln86qx13-installation.drv': 1 dependencies couldn't be built
> cannot build derivation `/gnu/store/0l77fgdgp4c3yq82mwd1kq1rzckn5x9b-btrfs-root-os.drv': 1 dependencies couldn't be built

I have better results here:

Toggle snippet (12 lines)
$ make check-system TESTS=btrfs-root-os

[...]

# of expected passes 20
@ build-succeeded /gnu/store/0ph6bsnc4armmm2wmsm2z5jcjzk0dhnn-btrfs-root-os.drv -
TOTAL: 1
PASS: /gnu/store/qvyg5j81p32hgbgfkkjqyxhqa9a9sz1i-btrfs-root-os
$ git describe
v0.15.0-2480-g1deab8cbf

Could you try from the same commit?

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 27 Oct 2018 15:39
control message for bug #32966
(address . control@debbugs.gnu.org)
871s8bbjak.fsf@gnu.org
severity 32966 important
L
L
Ludovic Courtès wrote on 27 Oct 2018 15:47
(address . control@debbugs.gnu.org)
87zhuza4cn.fsf@gnu.org
retitle 32966 'gexp-modules' fails to delete duplicates when code is evaluated
L
L
Ludovic Courtès wrote on 27 Oct 2018 17:18
Re: bug#32966: tests/install.scm tests all fail
(name . Clément Lassieur)(address . clement@lassieur.org)(address . 32966-done@debbugs.gnu.org)
87d0rv770z.fsf@gnu.org
Hi Clément,

Clément Lassieur <clement@lassieur.org> skribis:

Toggle quote (4 lines)
> This is the output of one of them when run manually:
>
> $ guix build /gnu/store/0l77fgdgp4c3yq82mwd1kq1rzckn5x9b-btrfs-root-os.drv

[...]

Toggle quote (11 lines)
> Backtrace:
> 2 (primitive-load "/gnu/store/fcybzbgns1nsglrkdj833mk53si?")
> In srfi/srfi-1.scm:
> 640:9 1 (for-each #<procedure c487a0 at ice-9/eval.scm:333:13 ?> ?)
> In unknown file:
> 0 (copy-file "/gnu/store/sqacjhyfz32diqys3pp7j6x8bs2zx7p?" ?)
>
> ERROR: In procedure copy-file:
> In procedure copy-file: Permission denied
> builder for `/gnu/store/7b054w5p8870k2rdc7jfimqk2k6n163d-module-import.drv' failed with exit code 1

This is fixed by 932d1600564cbf359a6ccd1086b968a934bef8e5, which
contains an explanation.

To reproduce it from a checkout, all you had to do is:

make
touch guix/self.scm
./pre-inst-env guix system disk-image gnu/system/examples/bare-bones.tmpl

The effect of ‘touch’ is that (guix self) was being evaluated and as a
consequence, ‘make-config.scm’, which is called by ‘qemu-image’ in (gnu
system vm), would always return a fresh object.

As a result, the ‘delete-duplicates’ call in ‘gexp-attribute’ would
leave two occurrences of:

((guix config) => ,(scheme-file …))

Thus, the builder of module-import.drv would look like this:

Toggle snippet (12 lines)
(for-each (match-lambda
((final-path store-path)
(mkdir-p (dirname final-path))
(copy-file store-path final-path)))
(quote (("guix/build/utils.scm" "/gnu/store/bp7pms1s0563kcyjyla0f7c89jimra9k-utils.scm")
("gnu/build/vm.scm" "/gnu/store/jnfmvx6lay9iqz7fy25myvvf1n3imac1-vm.scm")
;; …
("gnu/build/install.scm" "/gnu/store/wfrh0xm36r7bbqfq07s91ymcinpc79sa-install.scm")
("guix/config.scm" "/gnu/store/sqacjhyfz32diqys3pp7j6x8bs2zx7pv-config.scm")
("guix/config.scm" "/gnu/store/sqacjhyfz32diqys3pp7j6x8bs2zx7pv-config.scm")))) ;<- twice!

The module-import.drv build would then fail with “Permission denied”
while attempting to copy guix/config.scm for the second time.


As to why (guix self) is evaluated in the context of Cuirass… it’s
probably because of the ‘guix offload’ issue fixed in
e4752118691e41ae8307649d1abfd4739b3e4bfa. Are you using offloading on
your build farm?

Thanks,
Ludo’.
Closed
C
C
Clément Lassieur wrote on 22 Nov 2018 10:12
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 32966-done@debbugs.gnu.org)
87efbdpjb8.fsf@lassieur.org
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (32 lines)
> Hi Clément,
>
> Clément Lassieur <clement@lassieur.org> skribis:
>
>> For some time after your message, only 8 of those 16 tests seemed to
>> fail. But now they all fail again[1].
>
> These GuixSD installation tests occasionally fail for several reasons:
>
> • Sometimes I mess up with ‘guix system’ and the tests catch the
> regression (e.g., commit a29ce1f97dc8336e5a063f14d98d673b33b4bff4).
>
> • Sometimes there are missing files in the Guix tarball, which breaks
> ‘guix’ usage in the installation VM (commit
> 1e5b79e5d0e6629dce838dbb10febad676eea773).
>
> • Sometimes installing bare-bones takes slightly more than 1G and the
> tests assume 1G is enough (which is a good thing IMO, because it
> ensures the image size doesn’t grow unreasonably.)
>
> On this last point I expect we’ll do better when ‘core-updates’ is
> merged because currently on ‘master’ there’s both Guile 2.2.4 and 2.2.3
> in the system closure (the latter coming from the ‘guix’ package).
>
> Overall I think these tests are doing a good job a finding problems. :-)
>
> The problem is that these issues lead to dependency-failed failures on
> Cuirass/Hydra, and Cuirass currently doesn’t allow you to see which
> dependency failed and to view its build log (I can view that by logging
> in on berlin but that’s obviously suboptimal.) We should fix Cuirass to
> display that info though I’m not entirely sure how.


Toggle quote (2 lines)
> Thoughts?

Thank you for the explanation. So they are different failures. It's
probably not worth keeping a bug open for them, so I'm closing it again.

Cheers,
Clément
Closed
?