Mark H Weaver writes: > Hi Marius, > > Marius Bakke writes: > >> Marius Bakke writes: >> >>> Jan Nieuwenhuizen writes: >>> >>>> Mark H Weaver writes: >>>> >>>> Hi Mark, >>>> >>>>>> I called that `wip-binaries', @master from three weeks ago. >>>>> >>>>> Thank you, that was a good start. I found that some additional patches >>>>> were needed to match the bootstrap binaries that 'core-updates' is >>>>> currently based on. >>>>> >>>>> I ended up deleting and repushing a revised 'wip-binaries' to Savannah. >>>>> It includes slightly modified versions of the two commits you had >>>>> included, as well as some additional cherry-picked commits of yours to >>>>> update mescc-tools and add linux-libre-headers-bootstrap-tarball, and a >>>>> few of my own. >>>> >>>> Very nice. >>>> >>>>> I built the new bootstrap tarballs at the new 'wip-binaries', commit >>>>> c67becb31c30a5cd7685f166970ac4793e3a34a9, and here's what I got: >>>>> >>>>> mhw@jojen ~/guix-wip-binaries$ git describe >>>>> v1.0.1-2404-gc67becb31c >>>>> mhw@jojen ~/guix-wip-binaries$ ./pre-inst-env guix build --system=i686-linux bootstrap-tarballs >>>>> /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0 >>>>> mhw@jojen ~/guix-wip-binaries$ cd /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0 >>>>> mhw@jojen /gnu/store/bg086i2qw1fn2jgbd15d9v91hyjrjsb2-bootstrap-tarballs-0$ sha256sum * >>>>> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile-static-stripped-2.2.4-i686-linux.tar.xz >>>>> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz >>>>> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz >>>>> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-minimal-stripped-0.19-i686-linux.tar.xz >>>>> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 static-binaries-0-i686-linux.tar.xz >>>> >>>>> Can you try "guix build --system=i686-linux bootstrap-tarballs" at the >>>>> new 'wip-binaries' branch and see if you get the same results? >>>> >>>> Yes, on c67becb31c30a5cd7685f166970ac4793e3a34a9 running >>>> "./pre-inst-env guix build --system=i686-linux bootstrap-tarballs" gives me exactly this, >>>> also for guile-static-stripped! \o/ >>>> >>>>> Also, I have a question: One of the changes I made to 'wip-binaries' was >>>>> to update mescc-tools to 0.5.2-0.bb062b0, to match the >>>>> %bootstrap-mescc-tools that's currently being used in 'core-updates'. >>>>> >>>>> However, I noticed that you have also apparently built the official >>>>> release of mescc-tools-0.5.2, which is on your site: >>>>> >>>>> http://lilypond.org/janneke/guix/20190722/mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz >>>>> >>>>> and that this tarball is identical to the build output of the later git >>>>> commit: mescc-tools-static-stripped-0.5.2-0.bb062b0-i686-linux.tar.xz. >>>>> >>>>> With this in mind, could we just use 0.5.2? What changed between 0.5.2 >>>>> and 0.5.2-0.bb062b0, and what was the rationale for updating to bb062b0? >>>> >>>> Good catch. We probably can, we might try that. >>>> >>>> I think the need for updating to bb062b0 has been removed during the >>>> review of the integration of the reduced binary seed bootstrap into >>>> core-updates by Ludovic. >>>> >>>> For historical reasons, I think this mescc-tools commit >>>> >>>> --8<---------------cut here---------------start------------->8--- >>>> commit c184e95096881a13f29ebd7fc507fe305d3d8de5 (gitlab/janneke, janneke) >>>> Author: Jan Nieuwenhuizen >>>> Date: Thu Oct 4 22:03:31 2018 +0200 >>>> >>>> build.sh: Update for mes 0.18. >>>> --8<---------------cut here---------------end--------------->8--- >>>> >>>> was needed at a time that we did not have mescc-tools or mes in >>>> bootstrap tarballs. We built bootstrap variants of mescc-tools and mes >>>> using a externally (outside fo Guix) built mescc-tools-seed and >>>> (an almost pure ASCII) mes-seed. >>> >>> I tried building the i686 bootstrap tarballs from wip-binaries with this >>> additional patch: >>> >>> diff --git a/gnu/packages/mes.scm b/gnu/packages/mes.scm >>> index e298cb05c1..380cac6c88 100644 >>> --- a/gnu/packages/mes.scm >>> +++ b/gnu/packages/mes.scm >>> @@ -139,33 +139,31 @@ Guile.") >>> (license gpl3+))) >>> >>> (define-public mescc-tools >>> - (let ((commit "bb062b0da7bf2724ca40f9002b121579898d4ef7") >>> - (revision "0") >>> - (version "0.5.2")) >>> - (package >>> - (name "mescc-tools") >>> - (version (string-append version "-" revision "." (string-take commit 7))) >>> - (source (origin >>> - (method url-fetch) >>> - (uri (string-append >>> - "https://git.savannah.nongnu.org/cgit/mescc-tools.git/snapshot/" >>> - name "-" commit >>> - ".tar.gz")) >>> - (sha256 >>> - (base32 >>> - "1h6j57wyf91i42b26f8msbv6451cw3nm4nmpl1fckp9c7vi8mwkh")))) >>> - (build-system gnu-build-system) >>> - (supported-systems '("i686-linux" "x86_64-linux")) >>> - (arguments >>> - `(#:make-flags (list (string-append "PREFIX=" (assoc-ref %outputs "out"))) >>> - #:test-target "test" >>> - #:phases (modify-phases %standard-phases >>> - (delete 'configure)))) >>> - (synopsis "Tools for the full source bootstrapping process") >>> - (description >>> - "Mescc-tools is a collection of tools for use in a full source >>> + (package >>> + (name "mescc-tools") >>> + (version "0.5.2") >>> + (source (origin >>> + (method url-fetch) >>> + (uri (string-append >>> + "https://git.savannah.nongnu.org/cgit/mescc-tools.git/snapshot/" >>> + name "-Release_" version >>> + ".tar.gz")) >>> + (file-name (string-append name "-" version ".tar.gz")) >>> + (sha256 >>> + (base32 >>> + "01x7bhmgwyf6mc2g1hcvibhps98nllacqm4f0j5l51b1mbi18pc2")))) >>> + (build-system gnu-build-system) >>> + (supported-systems '("i686-linux" "x86_64-linux")) >>> + (arguments >>> + `(#:make-flags (list (string-append "PREFIX=" (assoc-ref %outputs "out"))) >>> + #:test-target "test" >>> + #:phases (modify-phases %standard-phases >>> + (delete 'configure)))) >>> + (synopsis "Tools for the full source bootstrapping process") >>> + (description >>> + "Mescc-tools is a collection of tools for use in a full source >>> bootstrapping process. It consists of the M1 macro assembler, the hex2 >>> linker, the blood-elf symbol table generator, the kaem shell, exec_enable and >>> get_machine.") >>> (home-page "https://savannah.nongnu.org/projects/mescc-tools") >>> - (license gpl3+)))) >>> + (license gpl3+))) > > I guess this is equivalent to reverting commit > 78ced7975b0665e810834391d826c9f0ef7277e1 on the 'wip-binaries' branch, > yes? Indeed. >>> And got this result: >>> >>> $ cd /gnu/store/rdwyr8mh7dvhfkb5g4cws6q40hp23rbi-bootstrap-tarballs-0 >>> $ sha256sum * >>> 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile-static-stripped-2.2.4-i686-linux.tar.xz >>> 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz >>> 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz >>> fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-minimal-stripped-0.19-i686-linux.tar.xz >>> c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 static-binaries-0-i686-linux.tar.xz > > Great, looks good to me! > >>> I also merged the branch to core-updates and reverted the bash patch, > > I think that 'bash-4.4-linux-pgrp-pipe.patch', adapted to 5.0, should be > applied to all 'bash' packages in 'core-updates'. To be more precise, > the patch should be applied to the main 'bash' package in bash.scm, > inherited from all other bash packages, so that PGRP_PIPE is set > unconditionally set on systems based on Linux (the kernel), regardless > of the kernel version running on the build machine. Got it. >>> which produced this derivation for "guix build -d -s i686-linux >>> bootstrap-tarballs": >>> >>> /gnu/store/ld82vli1msfrlimjaryznrqcwm0jc5ii-bootstrap-tarballs-0.drv >> >> Here are the hashes from this derivation: >> >> $ cd /gnu/store/srsqilb3g70r8c7vma0gpam21z1zmg2w-bootstrap-tarballs-0 >> $ sha256sum * >> 4e5b219be4d9ad4d125f17b8a8a991e78be3908aadc8d22d1a115e96ec856279 guile-static-stripped-2.2.6-i686-linux.tar.xz >> 9e34b26526b184912b0d97df96e316151fd97ede24aec2f22dd13ed33438b2e4 linux-libre-headers-stripped-4.19.56-i686-linux.tar.xz >> 7db07a7097a7920e17f6f1794098b9f29bc3522c36a33cf797123f64581cdc34 mescc-tools-static-0.5.2-i686-linux.tar.xz >> 3a9b050c1a2b61bb1cbed16ee07aecdc83e9097ce4a914623aa1b9808abe57f4 mes-minimal-stripped-0.19-i686-linux.tar.xz >> 1ccb4f39656eb977706db7a4aa811461785200b0a11e847035bb89fef91b85ab static-binaries-0-i686-linux.tar.xz > > What was the purpose of building these? 'core-updates' is built upon on > the earlier, unverified, reduced binary seed bootstrap binaries. I wanted to check that the bootstrap-tarballs machinery still worked after merging the branch, since it was non-trivial. Mainly to make the commit that created them reachable forever, but maybe we don't need it. >> @Mark: do you think we are ready to merge this now? Can you do it? > > I think what needs to be done is the following: > > (1) commit 78ced7975b0665e810834391d826c9f0ef7277e1 on 'wip-binaries' > should be reverted, to downgrade mescc-tools to the 0.5.2 release. > > (2) The 'wip-binaries' tarballs should be uploaded to a new subdirectory > of , along > with digital signatures, of course. I'm talking about these in > particular: > > 3e50c070a100b6bcf84c4bf5c868f9cd0a9fd1570f5d82fbfb78f8411959091b guile-static-stripped-2.2.4-i686-linux.tar.xz > 1acd8f83e27d2fac311a5ca78e9bf11a9a1638b82469870d5c854c4e7afaa26a linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz > 021543d9bb6af55f39e68d69692e3cb74646ced2cad0bb9ac0047ef81e9d7330 mescc-tools-static-stripped-0.5.2-i686-linux.tar.xz > fb32090071b39fc804fb9a7fba96f0bc5eb844a0efd268fb24c42e6bfa959de0 mes-minimal-stripped-0.19-i686-linux.tar.xz > c80cdd17b0a24eebdd75570ff72c4ec06e129bd702ac008186b57f6301c448e7 static-binaries-0-i686-linux.tar.xz > > (3) The following bootstrap packages in 'core-updates' bootstrap.scm > should be updated to use the new binaries above: > > (a) %bootstrap-linux-libre-headers > (b) %bootstrap-mescc-tools > (c) %bootstrap-mes > > (4) Berlin should start rebuilding 'core-updates'. > > If desired, steps (3) and (4) could come before (2) if someone > temporarily uploads the new binaries somewhere else, and adjusts > '%bootstrap-base-urls' accordingly. The key is for the hashes and file > names to match what we've agreed on here, as I listed in (2) above. > > What do you think? Thank you for the excellent summary. I can look into adjusting the bash fix for 5.0, and updating the bootstrap binary URLs and hashes. I will do this in a 'core-updates-next' branch. I would also like to merge wip-binaries into it as a final step, unless someone has objections. Ludovic should be back in a couple of days and can hopefully take care of the uploads. Ricardo: can you instruct Cuirass to add a reduced jobset for 'core-updates-next', that only builds builds the "core" package subset?