[PATCH, core-updates-frozen, RFC] Fix static guile on aarch64

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Pierre Langlois
Owner
unassigned
Submitted by
Pierre Langlois
Severity
normal
P
P
Pierre Langlois wrote on 23 Nov 2021 22:22
(address . guix-patches@gnu.org)
878rxeebh6.fsf@gmx.com
Hi Guix!

With the recent call for testing on core-updates-frozen I thought I'd
test my router configuration on the RockPro64 and I saw that building
the stripped static guile for initrd didn't build. I found a fix but
I'm not 100% sure it's correct, so beware, here's my debugging session!

%guile-static-stripped fails to build as such:

Toggle snippet (17 lines)
$ ./pre-inst-env guix build -e '(@ (gnu packages make-bootstrap) %guile-static-stripped)'
...
Backtrace:
1 (primitive-load "/gnu/store/gz31w23fjaipnm9c9hzw6p8s619?")
In guix/build/utils.scm:
761:6 0 (invoke "/gnu/store/9rwp8z98xdllih5rzf2ncq25xhnkgcvq-g?" ?)

guix/build/utils.scm:761:6: In procedure invoke:
ERROR:
1. &invoke-error:
program: "/gnu/store/9rwp8z98xdllih5rzf2ncq25xhnkgcvq-guile-static-stripped-3.0.7/bin/guile"
arguments: ("--version")
exit-status: 127
term-signal: #f
stop-signal: #f

The stripped guile crashes, and if we check it, it's actually not a
static binary! So when stripping out store references we break it of
course:

Toggle snippet (8 lines)
$ file /gnu/store/9rwp8z98xdllih5rzf2ncq25xhnkgcvq-guile-static-stripped-3.0.7/bin/guile
/gnu/store/9rwp8z98xdllih5rzf2ncq25xhnkgcvq-guile-static-stripped-3.0.7/bin/guile:
ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV),
dynamically linked, interpreter /gnu/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-glibc-2.33/lib/ld-linux-aarch64.so.1, for GNU/Linux 2.6.32, stripped
^^^^^^^^
Oups :-)

The root of the issue seems to be that the make-guile-static procedure
isn't actually producing a static binary on aarch64. Digging into it
more, we do pass the -all-static flag to libtool, but the gcc command
doesn't actually do any static linking:

Toggle snippet (6 lines)
/tmp/guix-build-guile-static-3.0.7.drv-0/guile-3.0.7$ make V=1
...
../libtool --tag=CC --mode=link gcc -std=gnu11 -pthread -Wall -Wmissing-prototypes -Wpointer-arith -fno-strict-aliasing -fwrapv -fvisibility=hidden -g -O2 -all-static -pthread -ldl -o guile guile-guile.o libguile-3.0.la -ldl -lcrypt -lm
libtool: link: gcc -std=gnu11 -pthread -Wall -Wmissing-prototypes -Wpointer-arith -fno-strict-aliasing -fwrapv -fvisibility=hidden -g -O2 -pthread -o guile guile-guile.o ./.libs/libguile-3.0.a -L/gnu/store/3539zsmc939g1r9g3r02bpy0m1b9v9c8-libgc-8.0.4/lib -L/gnu/store/sn95w7yk9qwxhw74l590606y53pf5mkf-libffi-3.3/lib /gnu/store/3539zsmc939g1r9g3r02bpy0m1b9v9c8-libgc-8.0.4/lib/libgc.a -lpthread /gnu/store/sn95w7yk9qwxhw74l590606y53pf5mkf-libffi-3.3/lib/libffi.a -lunistring -ldl -lcrypt -lm -pthread

And if we try and do a static link anyway, it doens't work because of
missing references related to atomic ops. My guess is that libtool tried
to statically link, and fell back to dynamic linking when that didn't
work, I'm not so sure.

Toggle snippet (17 lines)
/tmp/guix-build-guile-static-3.0.7.drv-0/guile-3.0.7/libguile$ gcc -std=gnu11 -pthread -Wall -Wmissing-prototypes -Wpointer-arith -fno-strict-aliasing -fwrapv -fvisibility=hidden -g -O2 -pthread -o guile guile-guile.o ./.libs/libguile-3.0.a -L/gnu/store/3539zsmc939g1
r9g3r02bpy0m1b9v9c8-libgc-8.0.4/lib -L/gnu/store/sn95w7yk9qwxhw74l590606y53pf5mkf-libffi-3.3/lib /gnu/store/3539zsmc939g1r9g3r02bpy0m1b9v9c8-libgc-8.0.4/lib/libgc.a -lpthread /gnu/store/sn95w7yk9qwxhw74l590606y53pf5mkf-libffi-3.3/lib/libffi.a -lunistring -ldl -lcrypt -lm -p
thread -static
ld: ./.libs/libguile-3.0.a(libguile_3.0_la-dynl.o): in function `scm_dlopen':
/tmp/guix-build-guile-static-3.0.7.drv-0/guile-3.0.7/libguile/dynl.c:78: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
ld: ./.libs/libguile-3.0.a(libguile_3.0_la-posix.o): in function `scm_tmpnam':
/tmp/guix-build-guile-static-3.0.7.drv-0/guile-3.0.7/libguile/posix.c:1611: warning: the use of `tmpnam' is dangerous, better use `mkstemp'
ld: /gnu/store/b05hb686wamq78d5jzpisns9xdx6nkp2-glibc-2.33-static/lib/libc.a(abort.o): in function `abort':
(.text.unlikely+0x34): undefined reference to `__aarch64_cas4_acq'
ld: (.text.unlikely+0xa0): undefined reference to `__aarch64_swp4_rel'
ld: (.text.unlikely+0xe4): undefined reference to `__aarch64_cas4_acq'
ld: /gnu/store/b05hb686wamq78d5jzpisns9xdx6nkp2-glibc-2.33-static/lib/libpthread.a(pthread_create.o):(.text+0x5c): undefined reference to `__aarch64_cas4_acq'
ld: /gnu/store/b05hb686wamq78d5jzpisns9xdx6nkp2-glibc-2.33-static/lib/libpthread.a(pthread_create.o):(.text+0x1fc): undefined reference to `__aarch64_cas4_acq'
ld: /gnu/store/b05hb686wamq78d5jzpisns9xdx6nkp2-glibc-2.33-static/lib/libpthread.a(pthread_create.o):(.text+0x29c): undefined reference to `__aarch64_cas4_acq'
...

Doing more digging, I found a reference to a similar issue here:
related to the -moutline-atomics option that's enabled by default with
GCC 10. But we're building guile with GCC 7 so it was a bit confusing.

However, glibc here is actually built with the default GCC 10, and if we
change that then it works!

Toggle snippet (14 lines)
--- a/gnu/packages/make-bootstrap.scm
+++ b/gnu/packages/make-bootstrap.scm
@@ -85,6 +85,9 @@ (define glibc-for-bootstrap
`(cons* "--disable-nscd" "--disable-build-nscd"
"--enable-static-nss"
,flags))))
+ (native-inputs
+ `(("gcc" ,gcc-7)
+ ,@(package-native-inputs base)))
;; Remove the 'debug' output to allow bit-reproducible builds (when the
;; 'debug' output is used, ELF files end up with a .gnu_debuglink, which

I'm not 100% sure what's going on, there seems to be an incompatiblity
between GCC 7 and 10, but I don't know if this is a bug or if it's
working as intended. I have a feeling statically linking the libc with
different compiler version might just not be supported, but it works in
practise a lot of the time.

Does this make sense? Do you think the following patch would be the
right way to fix this?

Thanks!
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmGdb2UYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UQGQH/iLPGyFADQ5f3fCC/A8waA2Z
N0kDe1PCST69FGjwOX8QZSgL25dl+QmEduiZ2/Zz+sKQRMc7cBkfHkW/HtUvII6i
KwwCUjR52Z2g3OxPosnbFo9EZkhDgXFGsg8BtmPmjmY+hHKw9rJPIWw7yUmZrk0a
+YL1tAYu/eFvORZSLrEP+95wZqqS8kSNGSEN7iC6kMaGeb1xCuJOH2Cv5bA1D0He
7QidpDr9HVe47sqk6PYOhmCxm8/wrnIL6Hj0T3J7WrByhWLWFmb1hexuCrBEmz+G
FB11vS38ZGPMe3RN+ojrfaDxXmUJPlT7X6ax9n4QNChw4ljBLrS3rHQQYiM+Llc=
=aeET
-----END PGP SIGNATURE-----

From c1f61c161b3f5f7e8430c4925b0785d3819e255e Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Tue, 23 Nov 2021 22:41:20 +0000
Subject: [PATCH] gnu: glibc-for-bootstrap: Build with GCC 7.

* gnu/packages/make-bootstrap.scm (glibc-for-bootstrap)[native-inputs]:
Add gcc-7.
---
gnu/packages/make-bootstrap.scm | 6 ++++++
1 file changed, 6 insertions(+)

Toggle diff (17 lines)
diff --git a/gnu/packages/make-bootstrap.scm b/gnu/packages/make-bootstrap.scm
index 267adde48c..d8f1f0eb1e 100644
--- a/gnu/packages/make-bootstrap.scm
+++ b/gnu/packages/make-bootstrap.scm
@@ -86,6 +86,12 @@ (define glibc-for-bootstrap
"--enable-static-nss"
,flags))))

+ ;; Make sure to build glibc with the same compiler version with the rest
+ ;; of the bootstrap. Otherwise it fails to statically link on aarch64.
+ (native-inputs
+ `(("gcc" ,gcc-7)
+ ,@(package-native-inputs base)))
+
;; Remove the 'debug' output to allow bit-reproducible builds (when the
;; 'debug' output is used, ELF files end up with a .gnu_debuglink, which
;; includes a CRC of the corresponding debugging symbols; those symbols
--
2.34.0
L
L
Ludovic Courtès wrote on 26 Nov 2021 11:00
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)(address . 52066@debbugs.gnu.org)
874k7zb5ju.fsf@gnu.org
Hi Pierre,

Pierre Langlois <pierre.langlois@gmx.com> skribis:

Toggle quote (10 lines)
> The root of the issue seems to be that the make-guile-static procedure
> isn't actually producing a static binary on aarch64. Digging into it
> more, we do pass the -all-static flag to libtool, but the gcc command
> doesn't actually do any static linking:
>
> /tmp/guix-build-guile-static-3.0.7.drv-0/guile-3.0.7$ make V=1
> ...
> ../libtool --tag=CC --mode=link gcc -std=gnu11 -pthread -Wall -Wmissing-prototypes -Wpointer-arith -fno-strict-aliasing -fwrapv -fvisibility=hidden -g -O2 -all-static -pthread -ldl -o guile guile-guile.o libguile-3.0.la -ldl -lcrypt -lm
> libtool: link: gcc -std=gnu11 -pthread -Wall -Wmissing-prototypes -Wpointer-arith -fno-strict-aliasing -fwrapv -fvisibility=hidden -g -O2 -pthread -o guile guile-guile.o ./.libs/libguile-3.0.a -L/gnu/store/3539zsmc939g1r9g3r02bpy0m1b9v9c8-libgc-8.0.4/lib -L/gnu/store/sn95w7yk9qwxhw74l590606y53pf5mkf-libffi-3.3/lib /gnu/store/3539zsmc939g1r9g3r02bpy0m1b9v9c8-libgc-8.0.4/lib/libgc.a -lpthread /gnu/store/sn95w7yk9qwxhw74l590606y53pf5mkf-libffi-3.3/lib/libffi.a -lunistring -ldl -lcrypt -lm -pthread

So ‘libtool’ is not passing ‘-static’, right? Why is that? (It does
pick .a libs though.)

[...]

Toggle quote (21 lines)
> Doing more digging, I found a reference to a similar issue here:
> https://bugzilla.redhat.com/show_bug.cgi?id=1830472#c1, this looks
> related to the -moutline-atomics option that's enabled by default with
> GCC 10. But we're building guile with GCC 7 so it was a bit confusing.
>
> However, glibc here is actually built with the default GCC 10, and if we
> change that then it works!
>
> --- a/gnu/packages/make-bootstrap.scm
> +++ b/gnu/packages/make-bootstrap.scm
> @@ -85,6 +85,9 @@ (define glibc-for-bootstrap
> `(cons* "--disable-nscd" "--disable-build-nscd"
> "--enable-static-nss"
> ,flags))))
> + (native-inputs
> + `(("gcc" ,gcc-7)
> + ,@(package-native-inputs base)))
>
> ;; Remove the 'debug' output to allow bit-reproducible builds (when the
> ;; 'debug' output is used, ELF files end up with a .gnu_debuglink, which

I think that’s a reasonable fix for now; you can push the patch.

However, we should eventually switch to GCC 10 in make-bootstrap.scm; I
don’t think there’s any good justification for sticking to GCC 7.

Thank you!

Ludo’.
P
P
Pierre Langlois wrote on 27 Nov 2021 15:34
(name . Ludovic Courtès)(address . ludo@gnu.org)
87h7bxd5jp.fsf@gmx.com
Hi Ludo',

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (24 lines)
> Hi Pierre,
>
> Pierre Langlois <pierre.langlois@gmx.com> skribis:
>
>> The root of the issue seems to be that the make-guile-static procedure
>> isn't actually producing a static binary on aarch64. Digging into it
>> more, we do pass the -all-static flag to libtool, but the gcc command
>> doesn't actually do any static linking:
>>
>> /tmp/guix-build-guile-static-3.0.7.drv-0/guile-3.0.7$ make V=1
>> ...
>> ../libtool --tag=CC --mode=link gcc -std=gnu11 -pthread -Wall -Wmissing-prototypes -Wpointer-arith -fno-strict-aliasing -fwrapv -fvisibility=hidden -g -O2 -all-static -pthread -ldl -o guile guile-guile.o libguile-3.0.la -ldl -lcrypt -lm
>> libtool: link: gcc -std=gnu11 -pthread -Wall -Wmissing-prototypes
>> -Wpointer-arith -fno-strict-aliasing -fwrapv -fvisibility=hidden -g -O2
>> -pthread -o guile guile-guile.o ./.libs/libguile-3.0.a
>> -L/gnu/store/3539zsmc939g1r9g3r02bpy0m1b9v9c8-libgc-8.0.4/lib
>> -L/gnu/store/sn95w7yk9qwxhw74l590606y53pf5mkf-libffi-3.3/lib
>> /gnu/store/3539zsmc939g1r9g3r02bpy0m1b9v9c8-libgc-8.0.4/lib/libgc.a -lpthread
>> /gnu/store/sn95w7yk9qwxhw74l590606y53pf5mkf-libffi-3.3/lib/libffi.a
>> -lunistring -ldl -lcrypt -lm -pthread
>
> So ‘libtool’ is not passing ‘-static’, right? Why is that? (It does
> pick .a libs though.)

Yeah, I'm also confused about that. I did take a quick look at the
libtool sources but I couldn't work out what was happening. My best
guess is that it tries to use the -static flag and fails, but it prefer
to drop the flag rather than issue an error. It could be for backwards
compatiblity reasons.

Toggle quote (31 lines)
>
> [...]
>
>> Doing more digging, I found a reference to a similar issue here:
>> https://bugzilla.redhat.com/show_bug.cgi?id=1830472#c1, this looks
>> related to the -moutline-atomics option that's enabled by default with
>> GCC 10. But we're building guile with GCC 7 so it was a bit confusing.
>>
>> However, glibc here is actually built with the default GCC 10, and if we
>> change that then it works!
>>
>> --- a/gnu/packages/make-bootstrap.scm
>> +++ b/gnu/packages/make-bootstrap.scm
>> @@ -85,6 +85,9 @@ (define glibc-for-bootstrap
>> `(cons* "--disable-nscd" "--disable-build-nscd"
>> "--enable-static-nss"
>> ,flags))))
>> + (native-inputs
>> + `(("gcc" ,gcc-7)
>> + ,@(package-native-inputs base)))
>>
>> ;; Remove the 'debug' output to allow bit-reproducible builds (when the
>> ;; 'debug' output is used, ELF files end up with a .gnu_debuglink, which
>
> I think that’s a reasonable fix for now; you can push the patch.
>
> However, we should eventually switch to GCC 10 in make-bootstrap.scm; I
> don’t think there’s any good justification for sticking to GCC 7.
>
> Thank you!

Thanks! I just pushed it with as
48b754553aeb98660fe145633616a9d965fbd60e.

Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmGiQ6sYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UzoEH/A+/ulDPKetjRHYUMTXP6hNm
lX/Xc8Reb1bKr1w77RUI2fuMezasuK3ctIlZT8PYQ8xG0I0cPlQPSeyufUIGe+4Z
8FJdahtt87cW0v/oE0ksbBtXNzT/RjzapzXyVICErF1fz3nDPcdTDgByD4LWAMYn
XX07UU8cFdZFMArmhAdF+BHi0/MWLU8lmqrkIQHOnYKYleVP5Q4JezMJNw5H0uPS
Nyu1Uu0GcSjTlOmB/+2ujWHyxwOn1pMsNpyXI6ZhVi1yeRgf4xyfjcep916M1BdC
NhF7zgWq7K0ncLSmIdP0wW4BjQRsvRrEMSRULu2T6eORG6vMUKPvrMB9nJKXduA=
=42hE
-----END PGP SIGNATURE-----

Closed
?