‘binutils-mesboot0’ includes non-zero timestamps in ar archives

OpenSubmitted by Ludovic Courtès.
Details
3 participants
  • Jan Nieuwenhuizen
  • Ludovic Courtès
  • Maxim Cournoyer
Owner
unassigned
Severity
important
L
L
Ludovic Courtès wrote on 18 Jan 18:29 +0100
(address . bug-guix@gnu.org)
87im7ujgqz.fsf@inria.fr
Hi!
On #bootstrappable, mid-kid reported that ‘binutils-mesboot0’ incommencement.scm lacks ‘--enable-deterministic-archives’. So I checkedif this had an effect by running:
guix build -e '(@@ (gnu packages commencement) gcc-core-mesboot0)' \ --check -K
and yes, it does:
Toggle snippet (7 lines)$ diff -ru /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3{,-check}Binary files /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3/lib/gcc-lib/i686-unknown-linux-gnu/2.95.3/libc.a and /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3-check/lib/gcc-lib/i686-unknown-linux-gnu/2.95.3/libc.a differBinary files /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3/lib/gcc-lib/i686-unknown-linux-gnu/2.95.3/libgcc.a and /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3-check/lib/gcc-lib/i686-unknown-linux-gnu/2.95.3/libgcc.a differBinary files /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3/lib/libgcc2.a and /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3-check/lib/libgcc2.a differBinary files /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3/lib/libiberty.a and /gnu/store/ri28kdl41bb76qjr4cyarylw7kxpvfxy-gcc-core-mesboot0-2.95.3-check/lib/libiberty.a differ
Apparently Binutils 2.14 didn’t have ‘--enable-deterministic-archives’so we’ll have to patch it.
There are a few other Binutils variants in commencement.scm that we’llhave to check.
Ludo’.
L
L
Ludovic Courtès wrote on 20 Jan 09:45 +0100
control message for bug #45962
(address . control@debbugs.gnu.org)
87pn20f13k.fsf@gnu.org
severity 45962 importantquit
L
L
Ludovic Courtès wrote on 3 Mar 14:48 +0100
Re: bug#45962: ‘binutils-mesboot0’ includes non-zero timestamps in ar archives
(address . 45962@debbugs.gnu.org)
87lfb4s5z7.fsf@gnu.org
Hi,
Ludovic Courtès <ludo@gnu.org> skribis:
Toggle quote (4 lines)> On #bootstrappable, mid-kid reported that ‘binutils-mesboot0’ in> commencement.scm lacks ‘--enable-deterministic-archives’. So I checked> if this had an effect by running:
[...]
Toggle quote (3 lines)> Apparently Binutils 2.14 didn’t have ‘--enable-deterministic-archives’> so we’ll have to patch it.
Sikonas on #bootstrappable provided a patch for that (thanks!) so I wentahead and gave it a try on ‘core-updates’ (Guix patch attached).
The binutils source gets patched and repacked, but then decompressing itfails:
Toggle snippet (37 lines)building /gnu/store/fwz150xjaqbh8n02z6gsmpm9w8lxckak-binutils-mesboot0-2.14.drv...starting phase `set-SOURCE-DATE-EPOCH'phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 secondsstarting phase `set-paths'environment variable `PATH' set to `/gnu/store/70w6gnl3vzv2rsjhyjg0wsxl3pbnpjb5-bash-mesboot0-2.05b/bin:/gnu/store/2wkjilvkdyb2wrb20ba82dhdpnx2732n-bzip2-mesboot-1.0.8/bin:/gnu/store/4nxicfzj8a390f4scxcgglhdqgbyhlkw-gzip-mesboot-1.2.4/bin:/gnu/store/6fgjdcl2qrrs7fvdznkx1gcbw6wjfrn0-patch-mesboot-2.5.9/bin:/gnu/store/4lpagdav2gm022v1fgzcs8323xrggz4b-sed-mesboot0-1.18/bin:/gnu/store/y70rava2vqf1nilr8smg54qqxdclvrz6-gash-utils-boot-0.1.0/bin:/gnu/store/y9fy4r99q6pqisrw81fn92gk8mzbyzn1-tcc-boot-0.9.27/bin:/gnu/store/krnwfrki71dj6zicl2qwv6bdyqvsxgcg-make-mesboot0-3.80/bin:/gnu/store/xxnxdnjdlav4s8v3lfhg7x7amrqcrv57-gash-boot-0.2.0/bin:/gnu/store/aglbpf6bihv35pwpvdy6chxv318hsafq-bootar-1a/bin:/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin'environment variable `BASH_LOADABLES_PATH' unsetenvironment variable `C_INCLUDE_PATH' set to `/gnu/store/2wkjilvkdyb2wrb20ba82dhdpnx2732n-bzip2-mesboot-1.0.8/include:/gnu/store/y9fy4r99q6pqisrw81fn92gk8mzbyzn1-tcc-boot-0.9.27/include'environment variable `LIBRARY_PATH' set to `/gnu/store/2wkjilvkdyb2wrb20ba82dhdpnx2732n-bzip2-mesboot-1.0.8/lib:/gnu/store/y70rava2vqf1nilr8smg54qqxdclvrz6-gash-utils-boot-0.1.0/lib:/gnu/store/y9fy4r99q6pqisrw81fn92gk8mzbyzn1-tcc-boot-0.9.27/lib:/gnu/store/xxnxdnjdlav4s8v3lfhg7x7amrqcrv57-gash-boot-0.2.0/lib:/gnu/store/aglbpf6bihv35pwpvdy6chxv318hsafq-bootar-1a/lib:/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/lib'phase `set-paths' succeeded after 0.0 secondsstarting phase `install-locale'warning: failed to install 'en_US.utf8' locale: Invalid argumentphase `install-locale' succeeded after 0.0 secondsstarting phase `unpack'xz: cannot decompress from stdinBacktrace:In ice-9/boot-9.scm: 157: 5 [catch #t #<catch-closure c928e0> ...]In unknown file: ?: 4 [apply-smob/1 #<catch-closure c928e0>]In ice-9/boot-9.scm: 63: 3 [call-with-prompt prompt0 ...]In ice-9/eval.scm: 432: 2 [eval # #]In gash/guix-utils.scm: 138: 1 [call-with-decompressed-port xz ...]In unknown file: ?: 0 [scm-error misc-error #f "~A ~S" ("decompressed-port failure" (7)) #f]
ERROR: In procedure scm-error:ERROR: decompressed-port failure (7)error: in phase 'unpack': uncaught exception:srfi-34 #<condition &invoke-error [program: "tar" arguments: ("xvf" "/gnu/store/ywf5j03423yiawix3z21xa14hyyvd83z-binutils-2.14.tar.xz") exit-status: 1 term-signal: #f stop-signal: #f] 1215400> phase `unpack' failed after 0.3 secondscommand "tar" "xvf" "/gnu/store/ywf5j03423yiawix3z21xa14hyyvd83z-binutils-2.14.tar.xz" failed with status 1builder for `/gnu/store/fwz150xjaqbh8n02z6gsmpm9w8lxckak-binutils-mesboot0-2.14.drv' failed with exit code 1
Maxime, does that ring a bell? Could it be that this bootstrap ‘xz’ isless capable, or could it be a Gash-Utils bug?
Thanks,Ludo’.
Toggle diff (99 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scmindex 890d57941f..5c523ae7ad 100644--- a/gnu/packages/commencement.scm+++ b/gnu/packages/commencement.scm@@ -997,13 +997,15 @@ $MES -e '(mescc)' module/mescc.scm -- \"$@\" (inherit binutils) (name "binutils-mesboot0") (version "2.14")- (source (origin- (method url-fetch)- (uri (string-append "mirror://gnu/binutils/binutils-"- version ".tar.gz"))- (sha256- (base32- "1w8xp7k44bkijr974x9918i4p1sw4g2fcd5mxvspkjpg38m214ds"))))+ (source (bootstrap-origin+ (origin+ (method url-fetch)+ (uri (string-append "mirror://gnu/binutils/binutils-"+ version ".tar.gz"))+ (sha256+ (base32+ "1w8xp7k44bkijr974x9918i4p1sw4g2fcd5mxvspkjpg38m214ds"))+ (patches (search-patches "binutils-2.14-deterministic-ar.patch"))))) (inputs '()) (propagated-inputs '()) (native-inputs (%boot-tcc-inputs))diff --git a/gnu/packages/patches/binutils-2.14-deterministic-ar.patch b/gnu/packages/patches/binutils-2.14-deterministic-ar.patchnew file mode 100644index 0000000000..7f9310e994--- /dev/null+++ b/gnu/packages/patches/binutils-2.14-deterministic-ar.patch@@ -0,0 +1,66 @@+Old binutils do not have support for creating deterministic archives.+Backported from upstream commit 36e4dce69dd23bea9ea2258dea35f034b6d6351c++Patch by Chris Demetriou <cgd@google.com> (2009), adapted by+Andrius Štikonas <andrius@stikonas.eu> (2021).++--- a/bfd/archive.c 2021-03-01 00:05:54.888301655 +0000++++ b/bfd/archive.c 2021-03-02 21:53:51.001617689 +0000+@@ -1396,10 +1396,6 @@+ {+ /* Assume we just "made" the member, and fake it. */+ struct bfd_in_memory *bim = (struct bfd_in_memory *) member->iostream;+- time (&status.st_mtime);+- status.st_uid = getuid ();+- status.st_gid = getgid ();+- status.st_mode = 0644;+ status.st_size = bim->size;+ }+ else if (stat (filename, &status) != 0)+@@ -1408,6 +1404,11 @@+ return NULL;+ }+ ++ status.st_mtime = 0;++ status.st_uid = 0;++ status.st_gid = 0;++ status.st_mode = 0644;+++ amt = sizeof (struct ar_hdr) + sizeof (struct areltdata);+ ared = (struct areltdata *) bfd_zalloc (abfd, amt);+ if (ared == NULL)+@@ -2003,13 +2004,11 @@+ stat (arch->filename, &statbuf);+ memset ((char *) (&hdr), 0, sizeof (struct ar_hdr));+ sprintf (hdr.ar_name, RANLIBMAG);+- /* Remember the timestamp, to keep it holy. But fudge it a little. */+- bfd_ardata (arch)->armap_timestamp = statbuf.st_mtime + ARMAP_TIME_OFFSET;+ bfd_ardata (arch)->armap_datepos = (SARMAG+ + offsetof (struct ar_hdr, ar_date[0]));+- sprintf (hdr.ar_date, "%ld", bfd_ardata (arch)->armap_timestamp);+- sprintf (hdr.ar_uid, "%ld", (long) getuid ());+- sprintf (hdr.ar_gid, "%ld", (long) getgid ());++ sprintf (hdr.ar_date, "%ld", 0);++ sprintf (hdr.ar_uid, "%ld", 0);++ sprintf (hdr.ar_gid, "%ld", 0);+ sprintf (hdr.ar_size, "%-10d", (int) mapsize);+ strncpy (hdr.ar_fmag, ARFMAG, 2);+ for (i = 0; i < sizeof (struct ar_hdr); i++)+@@ -2082,6 +2081,8 @@+ struct ar_hdr hdr;+ unsigned int i;+ ++ return TRUE;+++ /* Flush writes, get last-write timestamp from file, and compare it+ to the timestamp IN the file. */+ bfd_flush (arch);+@@ -2169,7 +2170,7 @@+ memset ((char *) (&hdr), 0, sizeof (struct ar_hdr));+ hdr.ar_name[0] = '/';+ sprintf (hdr.ar_size, "%-10d", (int) mapsize);+- sprintf (hdr.ar_date, "%ld", (long) time (NULL));++ sprintf (hdr.ar_date, "%ld", 0);+ /* This, at least, is what Intel coff sets the values to. */+ sprintf ((hdr.ar_uid), "%d", 0);+ sprintf ((hdr.ar_gid), "%d", 0);
J
J
Jan Nieuwenhuizen wrote on 3 Mar 18:54 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)
8735xc86ne.fsf@gnu.org
Ludovic Courtès writes:
Hello!
Toggle quote (14 lines)> Ludovic Courtès <ludo@gnu.org> skribis:>>> On #bootstrappable, mid-kid reported that ‘binutils-mesboot0’ in>> commencement.scm lacks ‘--enable-deterministic-archives’. So I checked>> if this had an effect by running:
> [...]>>> Apparently Binutils 2.14 didn’t have ‘--enable-deterministic-archives’>> so we’ll have to patch it.>> Sikonas on #bootstrappable provided a patch for that (thanks!) so I went> ahead and gave it a try on ‘core-updates’ (Guix patch attached).
Great!
Toggle quote (3 lines)> The binutils source gets patched and repacked, but then decompressing it> fails:
[..]
Toggle quote (3 lines)> Maxime, does that ring a bell? Could it be that this bootstrap ‘xz’ is> less capable, or could it be a Gash-Utils bug?
Currently, we avoid using non-bootstrapped binaries in the bootstrap,that includes 'xz'; earlier in the bootstrap that includes also 'patch'.
See also gcc-core-mesboot0: it applies the patch in a manual phase. SoI'm not sure if we want to start depending on 'xz' an this stage?
Greetings,Janneke
-- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.orgFreelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
M
M
Maxim Cournoyer wrote on 4 Mar 17:14 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)
87k0qmq4ka.fsf@gmail.com
Hi,
Ludovic Courtès <ludo@gnu.org> writes:
[...]
Toggle quote (11 lines)> ERROR: In procedure scm-error:> ERROR: decompressed-port failure (7)> error: in phase 'unpack': uncaught exception:> srfi-34 #<condition &invoke-error [program: "tar" arguments: ("xvf" "/gnu/store/ywf5j03423yiawix3z21xa14hyyvd83z-binutils-2.14.tar.xz") exit-status: 1 term-signal: #f stop-signal: #f] 1215400> > phase `unpack' failed after 0.3 seconds> command "tar" "xvf" "/gnu/store/ywf5j03423yiawix3z21xa14hyyvd83z-binutils-2.14.tar.xz" failed with status 1> builder for `/gnu/store/fwz150xjaqbh8n02z6gsmpm9w8lxckak-binutils-mesboot0-2.14.drv' failed with exit code 1>> Maxime, does that ring a bell? Could it be that this bootstrap ‘xz’ is> less capable, or could it be a Gash-Utils bug?
From my IRC logs:
2020-09-20 22:32:01 apteryx janneke: haha! the xz-bootstrap supports--memlimit with % after all, my mistake was really silly... forgetting aspace between the args passed as XZ_DEFAULTS
I recall a similar error I had hit when working on adding multi-corecompression support to xz, but it ended up being just a mistake on mypart; the xz-bootstrap supported the required arguments just fine afterall.
HTH,
Maxim
M
M
Maxim Cournoyer wrote on 4 Mar 17:39 +0100
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
87ft1aq3fb.fsf@gmail.com
Jan Nieuwenhuizen <janneke@gnu.org> writes:
Toggle quote (34 lines)> Ludovic Courtès writes:>> Hello!>>> Ludovic Courtès <ludo@gnu.org> skribis:>>>>> On #bootstrappable, mid-kid reported that ‘binutils-mesboot0’ in>>> commencement.scm lacks ‘--enable-deterministic-archives’. So I checked>>> if this had an effect by running:>>> [...]>>>>> Apparently Binutils 2.14 didn’t have ‘--enable-deterministic-archives’>>> so we’ll have to patch it.>>>> Sikonas on #bootstrappable provided a patch for that (thanks!) so I went>> ahead and gave it a try on ‘core-updates’ (Guix patch attached).>> Great!>>> The binutils source gets patched and repacked, but then decompressing it>> fails:>> [..]>>> Maxime, does that ring a bell? Could it be that this bootstrap ‘xz’ is>> less capable, or could it be a Gash-Utils bug?>> Currently, we avoid using non-bootstrapped binaries in the bootstrap,> that includes 'xz'; earlier in the bootstrap that includes also 'patch'.>> See also gcc-core-mesboot0: it applies the patch in a manual phase. So> I'm not sure if we want to start depending on 'xz' an this stage?
I see; so what Ludovic got surprised by is the fact that when addingpatches or a snippet to an origin it gets repacked as an xz tarball.That's nothing new (it's how it is on the master branch too); but it canindeed be surprising.
Maxim
L
L
Ludovic Courtès wrote on 8 Mar 15:13 +0100
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
87ft15d98v.fsf@gnu.org
Howdy Janneke!
Jan Nieuwenhuizen <janneke@gnu.org> skribis:
Toggle quote (6 lines)> Currently, we avoid using non-bootstrapped binaries in the bootstrap,> that includes 'xz'; earlier in the bootstrap that includes also 'patch'.>> See also gcc-core-mesboot0: it applies the patch in a manual phase. So> I'm not sure if we want to start depending on 'xz' an this stage?
Probably not, good point!
I can think of two possibilities, then: (1) apply the patch in a phaserather than via the ‘patches’ field, and (2) arrange so that‘patch-and-repack’ does not compress the patched code or compresses itwith the bootstrap gzip.
My understanding is that #2 may be doable now thanks to Maxim’s recentchanges. I’ll take a look!
Thanks,Ludo’.
J
J
Jan Nieuwenhuizen wrote on 10 Mar 06:52 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)
87blbrsggs.fsf@gnu.org
Ludovic Courtès writes:
Hey Ludo!
Toggle quote (7 lines)> Jan Nieuwenhuizen <janneke@gnu.org> skribis:>> I can think of two possibilities, then: (1) apply the patch in a phase> rather than via the ‘patches’ field, and (2) arrange so that> ‘patch-and-repack’ does not compress the patched code or compresses it> with the bootstrap gzip.
Oh, that would be nice: we could remove all these clumsy manual patchingstages. Also, we may be able to remove XZ from the bootstrap chain, ifno XZ-repackaging happens and only build a final version.
Toggle quote (3 lines)> My understanding is that #2 may be doable now thanks to Maxim’s recent> changes. I’ll take a look!
Great!Greetings,Janneke
-- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.orgFreelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
?