[patch] update glibc to 2.35

  • Done
  • quality assurance status badge
Details
5 participants
  • Greg Hogan
  • Ludovic Courtès
  • Marius Bakke
  • Maxime Devos
  • zamfofex
Owner
unassigned
Submitted by
zamfofex
Severity
normal
Z
Z
zamfofex wrote on 10 Apr 2022 03:16
(name . guix-patches@gnu.org)(address . guix-patches@gnu.org)
935236349.13698.1649553391613@privateemail.com
Hello! I have decided I wanted to work on updating glibc. I tested the updated glibc with the packages ‘hello’, ‘coreutils’, ‘grep’, ‘sed’ and ‘guile’, and they all built successfully!

I have attached the generated ‘git diff’ to this message, and I hope that is fine. If there are any issues, please feel free to let me know!
Attachment: glibc.diff
M
M
Maxime Devos wrote on 10 Apr 2022 12:16
9cef0f8b9f2e3a847bb8faa5e6b48f688f74841a.camel@telenet.be
zamfofex schreef op za 09-04-2022 om 22:16 [-0300]:
+                     (define empty-static-libraries '("libpthread.a"
"libdl.a" "libutil.a" "libanl.a"))
+                     (define (empty-static-library? file)
+                       (any (lambda (s) (string=? file s)) empty-
static-libraries))
+
                      (define (static-library? file)
                        ;; Return true if FILE is a static library. 
The
                        ;; "_nonshared.a" files are referred to by
libc.so,
                        ;; libpthread.so, etc., which are in fact
linker
                        ;; scripts.
                        (and (string-suffix? ".a" file)
-                            (not (string-contains file
"_nonshared"))))
+                            (not (string-contains file
"_nonshared"))
+                            (not (empty-static-library? file))))

Why are empty static libraries skipped? Would "gcc -static -lpthread main.c" (*)
(in an environment that includes glibc:static) still work, or does it now fail?

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYlKufhccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7pV4AP9hScZcBwwBOmyT1AD3rua81iyx
Do3H6V8ZfQRm1uvi/wD/XOhTB5Kiiqm82MfrZsvaI7qH904HSVnKbnFHSSa8dQ0=
=55a7
-----END PGP SIGNATURE-----


M
M
Maxime Devos wrote on 10 Apr 2022 12:17
c658e0f05ba73845b65d3aff0fd9f79d4904d9fb.camel@telenet.be
zamfofex schreef op za 09-04-2022 om 22:16 [-0300]:
Toggle quote (16 lines)
> --- /dev/null
> +++ b/gnu/packages/patches/m4-failing-test-bug.patch
> @@ -0,0 +1,7 @@
> +--- a/tests/test-execute.sh
> ++++ b/tests/test-execute.sh
> +@@ -0,1 +0,3 @@
> +-for i in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 ;
> do
> ++# Test case 5 is disabled because of a bug in Guix whereby
> ++# raising 'SIGINT' does not work as expected.
> ++for i in 0 1 2 3 4 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 ; do
> diff --git a/gnu/packages/patches/m4-shell.patch
> b/gnu/packages/patches/m4-shell.patch
> new file mode 100644
> index 0000000..f10e99f

An explanation and a link to an upstream report is mising.
(apparently upstream=Guix here?).

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYlKuzxccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7h6ZAP9dlL9fTJWkMlKU57LD48CbPSXy
52c8qKyDWUF0OPKB9QEAlIqAfImmfdIiZUyz2tRp+ZiJjigJOxps1eeNJSU0OgA=
=pwBk
-----END PGP SIGNATURE-----


M
M
Maxime Devos wrote on 10 Apr 2022 12:18
0f62cbe519a54e1eb4e1fa7b3ba079dea5920f61.camel@telenet.be
zamfofex schreef op za 09-04-2022 om 22:16 [-0300]:
Toggle quote (28 lines)
> diff --git a/gnu/packages/patches/m4-shell.patch
> b/gnu/packages/patches/m4-shell.patch
> new file mode 100644
> index 0000000..f10e99f
> --- /dev/null
> +++ b/gnu/packages/patches/m4-shell.patch
> @@ -0,0 +1,16 @@
> +--- a/lib/config.hin
> ++++ b/lib/config.hin
> +@@ -0,12 +0,1 @@
> +-/* File name of the Bourne shell.  */
> +-#if (defined _WIN32 && !defined __CYGWIN__) || defined __CYGWIN__
> || defined __ANDROID__
> +-/* Omit the directory part because
> +-   - For native Windows programs in a Cygwin environment, the
> Cygwin mounts
> +-     are not visible.
> +-   - For 32-bit Cygwin programs in a 64-bit Cygwin environment, the
> Cygwin
> +-     mounts are not visible.
> +-   - On Android, /bin/sh does not exist. It's /system/bin/sh
> instead.  */
> +-# define BOURNE_SHELL "sh"
> +-#else
> +-# define BOURNE_SHELL "/bin/sh"
> +-#endif
> ++# define BOURNE_SHELL "sh"

What's the reason for this patch?

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYlKu9BccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7oFWAQDZcyBG8bm1w7BH5nm1uBL0bEID
m62DhzHjsQCkZaRHEwD7BgmVO3qJcCUos9Ld4Of75Mdg2Mbf6ronzzZ9JnJfPQM=
=lG8x
-----END PGP SIGNATURE-----


Z
Z
zamfofex wrote on 24 Apr 2022 16:19
(name . 54832@debbugs.gnu.org)(address . 54832@debbugs.gnu.org)(name . maximedevos@telenet.be)(address . maximedevos@telenet.be)
600444931.762626.1650809966586@privateemail.com
Toggle quote (3 lines)
> Why are empty static libraries skipped? Would "gcc -static -lpthread main.c" (*)
> (in an environment that includes glibc:static) still work, or does it now fail?

They are skipped from being moved, but then should be copied in the loop immediately below it.

Toggle quote (3 lines)
> An explanation and a link to an upstream report is mising.
> (apparently upstream=Guix here?).

Toggle quote (2 lines)
> What's the reason for this patch?

‘/bin/sh’ doesn’t seem available in the build environment, so tests were failing. From what I was able to see, that macro is only used in tests, so I didn’t think it would be worthwhile to make it refer to ‘/gnu/store/.../bash’.

- - - - - - -

Apologies to Maxime for the repeated messages, and to others for taking so long to reply. Apparently I hadn’t replied to the debbugs address, so my initial messages were sent only to Maxime.

I hope this can be looked into soon enough!
L
L
Ludovic Courtès wrote on 15 May 2022 23:31
Re: bug#54832: [patch] update glibc to 2.35
(name . zamfofex)(address . zamfofex@twdb.moe)
87r14ubhyu.fsf@gnu.org
Hello!

zamfofex <zamfofex@twdb.moe> skribis:

Toggle quote (2 lines)
> Hello! I have decided I wanted to work on updating glibc. I tested the updated glibc with the packages ‘hello’, ‘coreutils’, ‘grep’, ‘sed’ and ‘guile’, and they all built successfully!

Yay!

Toggle quote (6 lines)
> (base32
> - "1zvp0qdfbdyqrzydz18d9zg3n5ygy8ps7cmny1bvsp8h1q05c99f"))
> - (patches (search-patches "glibc-ldd-powerpc.patch"
> - "glibc-ldd-x86_64.patch"
> - "glibc-dl-cache.patch"

Could you confirm that these patches are no longer needed?

Toggle quote (4 lines)
> + (define empty-static-libraries '("libpthread.a" "libdl.a" "libutil.a" "libanl.a"))
> + (define (empty-static-library? file)
> + (any (lambda (s) (string=? file s)) empty-static-libraries))

[...]

Toggle quote (14 lines)
> (files (scandir lib static-library?))
> + (files2 (scandir lib empty-static-library?))
> (static (assoc-ref outputs "static"))
> (slib (string-append static "/lib")))
> (mkdir-p slib)
> @@ -876,6 +883,10 @@ (define (linker-script? file)
> (rename-file (string-append lib "/" base)
> (string-append slib "/" base)))
> files)
> + (for-each (lambda (base)
> + (copy-file (string-append lib "/" base)
> + (string-append slib "/" base)))
> + files2)

Like Maxime wrote, this needs comments in the code.

Could you explain why the empty .a files need special treatment? In the
end, it seems we’re copying them to the “static” output anyway, so why
not let them in ‘files’?

Toggle quote (6 lines)
> (define-public m4
> (package
> (name "m4")
> - (version "1.4.18")
> + (version "1.4.19")

This should be done in a separate patch.

Toggle quote (5 lines)
> +++ b/gnu/packages/patches/m4-failing-test-bug.patch
> @@ -0,0 +1,7 @@
> +--- a/tests/test-execute.sh
> ++++ b/tests/test-execute.sh

Like Maxime wrote, please start the patch with a short comment
explaining what it does, and with a link to the upstream commit or bug
report.

One last thing: could you use ‘git format-patch’ and (optionally) ‘git
send-email’ to send a revised patch?


Thanks in advance, and apologies for the delay!

Ludo’.
Z
Z
zamfofex wrote on 6 Jun 2022 14:06
(name . Ludovic Courtès)(address . ludo@gnu.org)
818784149.855838.1654517187059@privateemail.com
Toggle quote (2 lines)
> Could you confirm that these patches are no longer needed?

I don’t remember exactly what my thought process was for removing the ‘glibc-dl-cache’ patch except that it wasn’t applicable anymore. At any rate, I don’t fully understand what the patch is actually doing, so it’s a bit difficult to assess whether it’s still necessary to me. (Help would be appreciated!)

Other than that, I did verify the other two patches, and it seems the regexes they were patching have already been fixed upstream!

Toggle quote (2 lines)
> Could you explain why the empty .a files need special treatment? In the end, it seems we’re copying them to the “static” output anyway, so why not let them in ‘files’?

Those files need to be present in both the ‘static’ and ‘out’ outputs, whereas without considering them specially, they would be moved to the ‘static’ output (with ‘rename-file’ as opposed to ‘copy-file’). Is a comment worthwhile? What should I write? I could explain the change in glibc that made those into empty ‘.a’ files and link to the changelog. Is that enough?

Toggle quote (2 lines)
> This should be done in a separate patch.

That is fine. Though I will note that the previous version of m4 did not work with the updated glibc, so I think it would make sense to updated it *before* updating glibc in the commit history. Do I need to verify whether the newer version works with the previous glibc too?

Toggle quote (2 lines)
> Like Maxime wrote, please start the patch with a short comment explaining what it does, and with a link to the upstream commit or bug report.

I’m still confused about what I should link to. I can write a comment explaining the issues and link to the IRC conversation we held, or maybe even to this thread. But I don’t think there actually is “an upstream commit or bug report” that I could link to.

Toggle quote (2 lines)
> One last thing: could you use ‘git format-patch’ and (optionally) ‘git send-email’ to send a revised patch?

I definitely don’t mind investigating using those tools more carefully! I think I can prepare and send another patch once my questions are clarified.

- - -

Apologies to Maxime and Ludovic for the repeated messages once again and to others for taking so long to respond. I’m still getting familiar with sending emails properly!

At any rate, thank you for looking into it!
L
L
Ludovic Courtès wrote on 17 Jul 2022 12:06
(name . zamfofex)(address . zamfofex@twdb.moe)
874jzg59zc.fsf_-_@gnu.org
Hi!

Sorry for the long delay.

zamfofex <zamfofex@twdb.moe> skribis:

Toggle quote (4 lines)
>> Could you confirm that these patches are no longer needed?
>
> I don’t remember exactly what my thought process was for removing the ‘glibc-dl-cache’ patch except that it wasn’t applicable anymore. At any rate, I don’t fully understand what the patch is actually doing, so it’s a bit difficult to assess whether it’s still necessary to me. (Help would be appreciated!)

There’s a comment at the top of ‘glibc-dl-cache.patch’ that explains
what it does, but see
for details. I can take a look and update it.

Toggle quote (2 lines)
> Other than that, I did verify the other two patches, and it seems the regexes they were patching have already been fixed upstream!

Great.

Toggle quote (4 lines)
>> Could you explain why the empty .a files need special treatment? In the end, it seems we’re copying them to the “static” output anyway, so why not let them in ‘files’?
>
> Those files need to be present in both the ‘static’ and ‘out’ outputs,

Why?

Toggle quote (6 lines)
> whereas without considering them specially, they would be moved to the
> ‘static’ output (with ‘rename-file’ as opposed to ‘copy-file’). Is a
> comment worthwhile? What should I write? I could explain the change in
> glibc that made those into empty ‘.a’ files and link to the
> changelog. Is that enough?

We’d need a comment like “Keep empty .a files in OUT in addition to
STATIC because …”.

Toggle quote (4 lines)
>> This should be done in a separate patch.
>
> That is fine. Though I will note that the previous version of m4 did not work with the updated glibc, so I think it would make sense to updated it *before* updating glibc in the commit history. Do I need to verify whether the newer version works with the previous glibc too?

Ideally yes.

Toggle quote (4 lines)
>> Like Maxime wrote, please start the patch with a short comment explaining what it does, and with a link to the upstream commit or bug report.
>
> I’m still confused about what I should link to. I can write a comment explaining the issues and link to the IRC conversation we held, or maybe even to this thread. But I don’t think there actually is “an upstream commit or bug report” that I could link to.

When applying a patch to a package, we seek to document the reason why
we’re doing it, to ease maintenance; usually, patches come from a bug
report or from a change upstream, so we would like to it.

Toggle quote (4 lines)
>> One last thing: could you use ‘git format-patch’ and (optionally) ‘git send-email’ to send a revised patch?
>
> I definitely don’t mind investigating using those tools more carefully! I think I can prepare and send another patch once my questions are clarified.

Perfect. I realize upgrading glibc is a rather tricky task, so thanks
for giving it a try! Surely we can team up to get it past the finish
line.

Thanks!

Ludo’.
Z
Z
zamfofex wrote on 19 Jul 2022 16:12
(name . Ludovic Courtès)(address . ludo@gnu.org)
1143838129.1375777.1658239971665@privateemail.com
Toggle quote (5 lines)
> There’s a comment at the top of ‘glibc-dl-cache.patch’ that explains
> what it does, but see
> <https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/>
> for details. I can take a look and update it.

It seems we both misinterpreted the diff I had sent originally, as that patch has not actually been removed, but rather its line was changed (it was moved upwards). So good news, no need for changes or further investigation!

Toggle quote (3 lines)
> We’d need a comment like “Keep empty .a files in OUT in addition to
> STATIC because …”.

I added a comment explaining the change! I hope it is enough.

Toggle quote (2 lines)
> [conversation about M4 changes]

Apparently M4 has already been updated in core-updates by now! So that all of that has already been resolved.

Toggle quote (4 lines)
> Perfect. I realize upgrading glibc is a rather tricky task, so thanks
> for giving it a try! Surely we can team up to get it past the finish
> line.

Thanks for the encouragement! However, since glibc 2.36 releases a few weeks from now, do you feel like it would make sense to wait until then to update it? I suppose it could always be updated again later, but I don’t know if that’s ideal.

- - -

Also, I could not figure out how to use ‘git prepare-patch’ or ‘git send-email’ properly, so I hope a ‘git diff’ attachment is enough.
Attachment: glibc.diff
G
G
Greg Hogan wrote on 19 Jul 2022 21:16
Re: [bug#54832] [patch] update glibc to 2.35
(name . zamfofex)(address . zamfofex@twdb.moe)
CA+3U0ZnVfd16RArnSVzHUeEf2Zku-sOGeZ6YA+2DpbS-_F106w@mail.gmail.com
On Tue, Jul 19, 2022 at 12:14 PM zamfofex <zamfofex@twdb.moe> wrote:
Toggle quote (8 lines)
>
> > There’s a comment at the top of ‘glibc-dl-cache.patch’ that explains
> > what it does, but see
> > <https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/>
> > for details. I can take a look and update it.
>
> It seems we both misinterpreted the diff I had sent originally, as that patch has not actually been removed, but rather its line was changed (it was moved upwards). So good news, no need for changes or further investigation!

I have used your patch to bootstrap with GCC 12 (in order to use both
gcc-12 and llvm-14 in the same profile) and I have found that without
"glibc-ldd-x86_64.patch", sometimes ldd will throw a "not a dynamic
executable" error. I assume the powerpc patch is likewise still
required.

Toggle quote (2 lines)
> Thanks for the encouragement! However, since glibc 2.36 releases a few weeks from now, do you feel like it would make sense to wait until then to update it? I suppose it could always be updated again later, but I don’t know if that’s ideal.

I would like to see the 2.35 upgrade since others may benefit before
the 2.36 release and in case of a core-updates merge to master.

Greg
M
M
Marius Bakke wrote on 2 Sep 2022 00:32
87czcebu93.fsf@gnu.org
Greg Hogan <code@greghogan.com> skriver:

Toggle quote (15 lines)
> On Tue, Jul 19, 2022 at 12:14 PM zamfofex <zamfofex@twdb.moe> wrote:
>>
>> > There’s a comment at the top of ‘glibc-dl-cache.patch’ that explains
>> > what it does, but see
>> > <https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/>
>> > for details. I can take a look and update it.
>>
>> It seems we both misinterpreted the diff I had sent originally, as that patch has not actually been removed, but rather its line was changed (it was moved upwards). So good news, no need for changes or further investigation!
>
> I have used your patch to bootstrap with GCC 12 (in order to use both
> gcc-12 and llvm-14 in the same profile) and I have found that without
> "glibc-ldd-x86_64.patch", sometimes ldd will throw a "not a dynamic
> executable" error. I assume the powerpc patch is likewise still
> required.

I submitted a variant of this patch to #57533 that preserves
glibc-ldd-x86_64.patch.

Closing this issue, as it has been superseded by #57533.

Thanks zamfofex!
-----BEGIN PGP SIGNATURE-----

iIUEARYKAC0WIQRNTknu3zbaMQ2ddzTocYulkRQQdwUCYxEzGA8cbWFyaXVzQGdu
dS5vcmcACgkQ6HGLpZEUEHd0JQEAm55WU1PFhWLYG8aub2Cnj8vMMRiBhTKk1e5t
7QwBERwA/3j/w3rPD6rxe8DKJ5HMjg3bE2+DX8haU9q7PJaKeNYD
=uW9+
-----END PGP SIGNATURE-----

?