Test of guix-1.1.0-4.bdc801e is failing on aarch64

OpenSubmitted by Stefan.
Details
One participant
  • Stefan
Owner
unassigned
Severity
normal
S
S
Stefan wrote on 2 Jun 2020 01:14
Download
(address . bug-guix@gnu.org)
E46D3F13-D3E8-473E-AFA7-689A33053BBA@vodafonemail.de
Download
Hi!
Building guix-1.1.0-4.bdc801e is failing on aarch64 since about two weeks on ci.guix.gnu.org.
http://ci.guix.gnu.org/search?query=guix-1.1.0-4.bdc801e+system%3Aaarch64-linux
There is always the same failure during the test. See for example the raw log from http://ci.guix.gnu.org/build/2794788/details:
test-name: network-interface-names…actual-error:+ (wrong-type-arg+ "list-tail"+ "Wrong type argument in position ~A (expecting ~A): ~S"+ (1 "pair" ())+ (()))result: FAIL

When building locally this particular test is passing (after about three days on an SBC with 1 GB RAM).
But unfortunately building locally I get two different failures instead:
FAIL: tests/store.scmFAIL: tests/guix-package.sh
Here are some more details:

test-name: verify-store + check-contentslocation: /tmp/guix-build-guix-1.1.0-4.bdc801e.drv-0/source/tests/store.scm:972…actual-error:+ (%exception+ #<&store-protocol-error message: "path `dtmp/guix-tests/store/bpl866rgsjyzdczlngfw8ss2lkld6bim-mirrors' is not in the store" status: 1>)result: FAIL
Note the “dtmp/” instead of “/tmp/”. I saw the same error some weeks ago already, but shortly after that was happening, there was a substitute available. But it seems this error is reproducible in my case.

guix install: error: profile t-guix-package-541/profile is locked by another process+ true+ kill 1387+ rm -f t-profile-541 t-profile-541.lock t-profile-541-1-link t-guix-package-file-541+ rm -rf t-guix-package-541 t-home-541rm: cannot remove 't-guix-package-541': Directory not emptyFAIL tests/guix-package.sh (exit status: 1)

This may be due to a deleted but still opened file on an NFS share. There may be an intermediate hidden .nfs… file, which may get created in such a case (“delete on last close”, “silly rename”), However, the RFC-5661 for NFS demands even if OPEN4_RESULT_PRESERVE_UNLINKED is supported, that the directory entry of an open file must not be removed in this case, thus preventing a directory removal.
Attachment: 54pswxwm35kq47zk2dswvsq3nm55kb-guix-1.1.0-4.bdc801e.drv.bz2
S
S
Stefan wrote on 23 Nov 2020 09:35
Download
Re: bug#41654: Acknowledgement (Test of guix-1.1.0-4.bdc801e is failing on aarch64)
(address . 41654@debbugs.gnu.org)
0DF2D52E-2544-45E3-BB0B-F2B441EBED03@vodafonemail.de
Download
Hi!
The two errors when testing guix are still reproducible, now with guix-1.2.0rc2-1.0d4b1af, see the log for details.
I’m really wondering about the “dtmp/” instead of “/tmp/” bug. May this be a bug in guile itself?

Bye
Stefan
Attachment: fwpdy5zrhd8i1h8k7qh7a2kh3wawrh-guix-1.2.0rc2-1.0d4b1af.drv.bz2
S
S
Stefan wrote on 23 Nov 2020 09:38
Download
[PATCH] tests: Ignoring incomplete clean-up on NFS share.
(address . 41654@debbugs.gnu.org)
AC42C906-A019-4DF9-964B-973657E1FAFA@vodafonemail.de
Download
* tests/guix-package.sh: The 'rm -rf' for clean-up inside the trap may notsucceed on an NFS share, but this should not fail the test.--- tests/guix-package.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Toggle diff (15 lines)diff --git a/tests/guix-package.sh b/tests/guix-package.shindex 3e5fa71d20..b1c6eeffb8 100644--- a/tests/guix-package.sh+++ b/tests/guix-package.sh@@ -33,7 +33,7 @@ profile="t-profile-$$" tmpfile="t-guix-package-file-$$" rm -f "$profile" "$tmpfile" -trap 'rm -f "$profile" "$profile.lock" "$profile-"[0-9]* "$tmpfile"; rm -rf "$module_dir" t-home-'"$$" EXIT+trap 'rm -f "$profile" "$profile.lock" "$profile-"[0-9]* "$tmpfile"; rm -rf "$module_dir" t-home-'"$$"' || echo "incomplete clean-up ignored"' EXIT # Use `-e' with a non-package expression. ! guix package --bootstrap -e +-- 2.29.2
S
S
Stefan wrote on 23 Nov 2020 10:20
Download
test-name: verify-store + check-contents failing on guix-1.2.0rc2-1.0d4b1af
(address . 41654@debbugs.gnu.org)
E447536D-56DB-4B72-BBDE-0E78A835B6F5@vodafonemail.de
Download
Hi!
The “dtmp/” issue is not new. It appeared already in these bugs:
https://issues.guix.gnu.org/27032https://issues.guix.gnu.org/27034https://issues.guix.gnu.org/29676
In all cases it happens in tests/store.scm:
test-name: verify-store + check-contents
This bug happened on x86_64 and aarch64. The issues 27032 and 27034 where reported when the installer used a unionfs. Probably it was hidden when switching to an overlayfs.
This issue 41654 and 29676 show the “dtmp/” error when running this test on an NFS share.

Bye
Stefan
S
S
Stefan wrote on 30 Dec 2020 14:50
Download
708455FD-2A6C-4E6C-8F2C-D2E5549F5886@vodafonemail.de
Download
Hi Ludo’!
Could you please take a look at the bug report https://issues.guix.gnu.org/41654?
There are two reproducible problems when building the guix-1.2.0 package.
There is a patch already for the first of the two problems (an incomplete cleanup) at https://issues.guix.gnu.org/41654#2.
For the other issue with a wrong path "dtmp/guix-tests/store/", there is a summary at https://issues.guix.gnu.org/41654#3. This problem is existing since years and seems to be depending on the underlying file-system. It seems to appear with a tmp-store on a unionfs-fuse or NFS. In the past it got hidden by switching to overlayfs or changes on the NFS server side.
As the CI for aarch64 is always lagging behind these issues are kind of blockers to me. Today they block the build of guix-1.2.0-8.7624ebb.

Bye
Stefan
S
S
Stefan wrote on 4 Jan 02:05 +0100
Download
B8A36638-AF42-4F6A-BF33-FDED72854552@vodafonemail.de
Download
Hi!
Today I thought that it should be possible to work around this bug on my aarch64 system by mounting /tmp as a tmpfs, backed by swap on a small SD card, to prevent any issues due to my NFS root. This worked for the trap-cleanup-issue. But to my surprise this didn’t work out for the “dtmp/” error.
I was wondering, if using /tmp as a tmpfs on my virtual x86_64 system would make the test fail there as well. So I did
sudo mount -t tmpfs -o size=768M tmpfs /tmpguix build --check --no-grafts guix@1.2.0-8.7624ebb
But there it was passing.
This is the interesting part from the log of the aarch64 system:
test-name: verify-store + check-contentslocation: /tmp/guix-build-guix-1.2.0-8.7624ebb.drv-0/source/tests/store.scm:1156source:+ (test-assert+ "verify-store + check-contents"+ (with-store+ s+ (let* ((text (random-text))+ (drv (build-expression->derivation+ s+ "corrupt"+ `(let ((out (assoc-ref %outputs "out")))+ (call-with-output-file+ out+ (lambda (port) (display ,text port)))+ #t)+ #:guile-for-build+ (package-derivation+ s+ %bootstrap-guile+ (%current-system))))+ (file (derivation->output-path drv)))+ (with-derivation-substitute+ drv+ text+ (and (build-derivations s (list drv))+ (verify-store s #:check-contents? #t)+ (begin+ (chmod file 420)+ (call-with-output-file+ file+ (lambda (port) (display "corrupt!" port)))+ #t)+ (not (verify-store s #:check-contents? #t))+ (delete-paths s (list file)))))))actual-value: #factual-error:+ (%exception+ #<&store-protocol-error message: "path `dtmp/guix-tests/store/rn6xq5kaipp8aanhcnz2hvyfmr3y2laa-mirrors' is not in the store" status: 1>)result: FAIL

Bye
Stefan
?
Your may also send email to 41654@debbugs.gnu.org to comment.