base32.scm is miscompiled at -O1

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Marius Bakke
Owner
unassigned
Submitted by
Marius Bakke
Severity
important
M
M
Marius Bakke wrote on 20 Sep 2021 11:37
[core-updates-frozen] Wrong output hashes computed since cb06f7c61e4b839
(address . bug-guix@gnu.org)
87a6k78u4z.fsf@gnu.org
Hello,

On the 'core-updates-frozen' branch, attempting to pull 'master' gives a
backtrace along the lines of ...

Computing Guix derivation for 'x86_64-linux'... |Backtrace:
In ice-9/boot-9.scm:
152:2 19 (with-fluid* _ _ _)
152:2 18 (with-fluid* _ _ _)
In ./guix/store.scm:
2108:24 17 (run-with-store #<store-connection 256.99 7fbe31aac140> #<procedure 7fbe291da870 at ./guix/self.scm:13?> ?)
In ./guix/self.scm:
1313:2 16 (_ #<store-connection 256.99 7fbe31aac140>)
In ./guix/packages.scm:
1548:17 15 (_ #<store-connection 256.99 7fbe31aac140>)
1177:16 14 (cache! #<weak-table 0/113> #<package guile@3.0.7 7fbe22baf8c0> ("x86_64-linux" . #t) #<procedure thunk?>)
1495:22 13 (thunk)
1428:25 12 (bag->derivation #<store-connection 256.99 7fbe31aac140> #<<bag> name: "guile-3.0.7" system: "x86_64-l?> ?)
In srfi/srfi-1.scm:
586:17 11 (map1 (("source" #<origin "mirror://gnu/guile/guile-3.0.7.tar.xz" #<content-hash sha256:0h0i0040l0?>) ?))
In ./guix/packages.scm:
1239:5 10 (failure)
In ice-9/boot-9.scm:
1752:10 9 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In ./guix/packages.scm:
1244:18 8 (_)
In ./guix/store.scm:
2108:24 7 (run-with-store #<store-connection 256.99 7fbe31aac140> #<procedure 7fbe222a72d0 at ./guix/gexp.scm:26?> ?)
1945:8 6 (_ #<store-connection 256.99 7fbe31aac140>)
/In ./guix/gexp.scm:
275:18 5 (_ #<store-connection 256.99 7fbe31aac140>)
In ./guix/packages.scm:
1603:5 4 (_ #<store-connection 256.99 7fbe31aac140>)
In ./guix/store.scm:
1980:38 3 (_ #<store-connection 256.99 7fbe221cf690>)
In ./guix/derivations.scm:
955:17 2 (derivation #<store-connection 256.99 7fbe221cf690> "guile-3.0.7.tar.xz" "builtin:download" () #:system ?)
In ./guix/store.scm:
1058:9 1 (_ #<store-connection 256.99 7fbe221cf690> "guile-3.0.7.tar.xz.drv" #vu8(68 101 114 105 118 101 40 91 ?) ?)
In ice-9/boot-9.scm:
1685:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
ERROR:
1. &store-protocol-error:
message: "derivation `/gnu/store/szv5fh043rrqq01xpycp7xci4y7myqfk-guile-3.0.7.tar.xz.drv' has incorrect output `/gnu/store/qwjhql8lqw40fqhaqw40jwqs0hc0248w-guile-3.0.7.tar.xz', should be `/gnu/store/0zliqv8lcw13f9iasy14kmwsqj3d309w-guile-3.0.7.tar.xz'"
status: 1

I bisected it down to cb06f7c61e4b:

commit cb06f7c61e4b8393abf38f1f5891e03c33d53b9b
Author: Ludovic Courtès <ludo@gnu.org>
Date: Thu Sep 9 23:22:10 2021 +0200

base32: Provide an open-coded 'bit-field'.

This improves the throughput of 'bytevector->base32-string' a bit.

* guix/base32.scm (bit-field): New macro.

It can be reproduced with a two-dimensional time machine:

$ guix time-machine --branch=core-updates-frozen -- time-machine \
--commit=cb06f7c61e4b8393abf38f1f5891e03c33d53b9b -- build hello

...or by pulling 'core-updates-frozen' and attempting to pull 'master'.

The commit was merged to 'core-updates-frozen' in 5c3cb22c9b281066, so
pulling core-updates-frozen -> core-updates-frozen also fails in a
similar manner (albeit on a different derivation).
-----BEGIN PGP SIGNATURE-----

iIUEARYKAC0WIQRNTknu3zbaMQ2ddzTocYulkRQQdwUCYUhWbA8cbWFyaXVzQGdu
dS5vcmcACgkQ6HGLpZEUEHfo2wEA15kQ+Xmp3vl7zTDiwy8CoCBQW2IxNEYD+sdL
sMgL04ABAMXKMW232089qcppaf6X8bf8MvuxsUMVvcU53CXl7LAM
=yvI8
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 20 Sep 2021 16:22
control message for bug #50696
(address . control@debbugs.gnu.org)
874kafpbsn.fsf@gnu.org
severity 50696 important
quit
L
L
Ludovic Courtès wrote on 20 Sep 2021 23:38
Re: bug#50696: [core-updates-frozen] Wrong output hashes computed since cb06f7c61e4b839
(name . Marius Bakke)(address . marius@gnu.org)(address . 50696@debbugs.gnu.org)
87wnnbnd15.fsf@gnu.org
Hi Marius,

Marius Bakke <marius@gnu.org> skribis:

Toggle quote (5 lines)
> On the 'core-updates-frozen' branch, attempting to pull 'master' gives a
> backtrace along the lines of ...
>
> Computing Guix derivation for 'x86_64-linux'... |Backtrace:

[...]

Toggle quote (16 lines)
> 1. &store-protocol-error:
> message: "derivation `/gnu/store/szv5fh043rrqq01xpycp7xci4y7myqfk-guile-3.0.7.tar.xz.drv' has incorrect output `/gnu/store/qwjhql8lqw40fqhaqw40jwqs0hc0248w-guile-3.0.7.tar.xz', should be `/gnu/store/0zliqv8lcw13f9iasy14kmwsqj3d309w-guile-3.0.7.tar.xz'"
> status: 1
>
> I bisected it down to cb06f7c61e4b:
>
> commit cb06f7c61e4b8393abf38f1f5891e03c33d53b9b
> Author: Ludovic Courtès <ludo@gnu.org>
> Date: Thu Sep 9 23:22:10 2021 +0200
>
> base32: Provide an open-coded 'bit-field'.
>
> This improves the throughput of 'bytevector->base32-string' a bit.
>
> * guix/base32.scm (bit-field): New macro.

The problem is that at -O1 with Guile 3.0.7 (which is the default for
imported modules on ‘core-updates-frozen’ since
331a09654eb7e9f6212b7e8469077fa7393e8b11), that code gets miscompiled:

Toggle snippet (19 lines)
$ rm guix/base32.go
$ guild compile -O1 guix/base32.scm -o guix/base32.go
wrote `guix/base32.go'
$ ./pre-inst-env guile
GNU Guile 3.0.7
Copyright (C) 1995-2021 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,use(guix base32)
scheme@(guile-user)> (nix-base32-string->bytevector "0wqd8sjmxfskrflaxywc7gqw7sfawrfvdxd9skxawzfgyy0pzdz6")
$1 = #vu8(230 180 126 128 240 207 124 168 250 212 169 244 186 92 228 202 232 206 240 60 140 248 170 168 200 83 184 90 164 64 13 112)
scheme@(guile-user)> (bytevector->nix-base32-string $1)
$2 = "000d0080l00k00h0l00c0h00w00a0hq0lh090hl0hh0g0000wh06"

Long story short: the “baseline compiler” (-O0 and -O1) would
erroneously translate (ash x N) to (ash x (- N)) when N is a literal:

Toggle snippet (10 lines)
scheme@(guix base32)> (compile '(lambda (x) (ash x -2)) #:to 'value #:env (current-module) #:optimization-level 1)
$46 = #<procedure 36aeed8 (x)>
scheme@(guix base32)> (compile '(lambda (x) (ash x -2)) #:to 'value #:env (current-module) #:optimization-level 2)
$47 = #<procedure 2e4d7a8 (x)>
scheme@(guix base32)> ($47 4)
$48 = 1
scheme@(guix base32)> ($46 4)
$49 = 16

Fixed upstream:


However I’m not sure how to work around the bug, other than by reverting
the change. I’ll think a bit more about it.

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 21 Sep 2021 10:14
control message for bug #50696
(address . control@debbugs.gnu.org)
87v92uny5z.fsf@gnu.org
retitle 50696 base32.scm is miscompiled at -O1
quit
L
L
Ludovic Courtès wrote on 21 Sep 2021 10:15
Re: bug#50696: [core-updates-frozen] Wrong output hashes computed since cb06f7c61e4b839
(name . Marius Bakke)(address . marius@gnu.org)(address . 50696-done@debbugs.gnu.org)
87r1diny33.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (4 lines)
> Fixed upstream:
>
> https://git.savannah.gnu.org/cgit/guile.git/commit/?id=74abae04aa68dc18676bd84c4a73324b0613475d

And here’s the clever workaroud, in all modesty:


:-)

Ludo’.
Closed
?