[PATCH 0/8] Fix cross build and native build for the Hurd.

Done

Details

3 participants

Josselin Poiret
Janneke Nieuwenhuizen
Ludovic Courtès

Owner: unassigned

Submitted by: Janneke Nieuwenhuizen

Severity: normal

Janneke Nieuwenhuizen wrote on 18 Jul 2023 15:11

Recipients:(address . guix-patches@gnu.org)

Message-ID:cover.1689685374.git.janneke@gnu.org

Hi!

So...updating mig using git-fetch created a circular dependency because

mig-boot0 re-used mig's source, and we cannot use git in commencement.

This patch series now uses a hack by Ludo (thanks!) as suggested on IRC

https://logs.guix.gnu.org/guix/2023-07-17.log#233051 to not directly use

generated cgit tarballs from savannah.

The versions of gnumach-headers-boot0, mig-boot0, and hurd-headers-boot0 are

updated so that native builds work again. Because upstream does not produce

release tarballs we now also need autoconf and automake in commencement.

Note that building natively won't get very far, as for many packages (guile,

coreutils, grep, findutils, diffutils, gettext-minimal, m4, openssl, python,

...) the `check' phase will fail. See the `hurd-team' branch for updates on

this.

Now that mig-boot0 uses its own source definition, we can safely re-apply the

mig update.

Greetings,

Janneke

Janneke Nieuwenhuizen (6):

gnu: commencement: Add autoconf-boot0.

gnu: commencement: Add automake-boot0.

gnu: commencement: gnumach-headers-boot0: Update to 1.8+git20221224.

gnu: commencement: mig-boot0: Update to 1.8+git20230520.

gnu: commencement: hurd-headers-boot0: Update to 0.9.git20230216.

gnu: commencement: hurd-minimal-boot0: Update to 0.9.git20230216.

Josselin Poiret (1):

gnu: mig: Update to 1.8+git20230520.

Ludovic Courtès (1):

gnu: commencement: Add git-fetch-from-tarball utility.

gnu/packages/commencement.scm | 200 ++++++++++++++++++++++++++--------

gnu/packages/hurd.scm | 20 ++--

2 files changed, 160 insertions(+), 60 deletions(-)

base-commit: 94f83088d237ecd032c411bb94e2d647a2931990

2.40.1

Janneke Nieuwenhuizen wrote on 18 Jul 2023 15:14

[PATCH 1/8] gnu: commencement: Add autoconf-boot0.

Recipients:(address . 64708@debbugs.gnu.org)

Message-ID:3401eff21a200de7e6571ecb6123558c54c0ec4e.1689685374.git.janneke@gnu.org

* gnu/packages/commencement.scm (autoconf-boot0): New variable.

---

gnu/packages/commencement.scm | 10 ++++++++++

1 file changed, 10 insertions(+)

Toggle diff (30 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 9bd0c113c2..3ccd51ac52 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -33,6 +33,7 @@
 (define-module (gnu packages commencement)
   #:use-module (gnu packages)
   #:use-module (gnu packages bootstrap)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages c)
@@ -2602,6 +2603,15 @@ (define with-boot0
   (package-with-explicit-inputs %boot0-inputs
                                 %bootstrap-guile))
 
+(define autoconf-boot0
+  (with-boot0
+   (package
+     (inherit autoconf)
+     (name "autoconf-boot0")
+     (native-inputs (list m4-boot0 perl-boot0))
+     (inputs '())
+     (arguments (list #:tests? #f)))))
+
 (define gnumach-headers-boot0
   (with-boot0
    (package
-- 
2.40.1

Janneke Nieuwenhuizen wrote on 18 Jul 2023 15:14

[PATCH 2/8] gnu: commencement: Add automake-boot0.

Recipients:(address . 64708@debbugs.gnu.org)

Message-ID:eddec7910064fe0bd1d2179d3faf71aef23ee98d.1689685374.git.janneke@gnu.org

* gnu/packages/commencement.scm (automake-boot0): New variable.

---

gnu/packages/commencement.scm | 13 +++++++++++++

1 file changed, 13 insertions(+)

Toggle diff (26 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 3ccd51ac52..d0603e62c8 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2612,6 +2612,19 @@ (define autoconf-boot0
      (inputs '())
      (arguments (list #:tests? #f)))))
 
+(define automake-boot0
+  (with-boot0
+   (package
+     (inherit automake)
+     (name "automake-boot0")
+     (source (origin
+               (inherit (package-source automake))
+               (patches '())))        ;test are skipped anyway
+     (native-inputs (list autoconf-boot0 m4-boot0 perl-boot0))
+     (inputs '())
+     (arguments
+      (list #:tests? #f)))))
+
 (define gnumach-headers-boot0
   (with-boot0
    (package
-- 
2.40.1

Janneke Nieuwenhuizen wrote on 18 Jul 2023 15:15

[PATCH 4/8] gnu: commencement: gnumach-headers-boot0: Update to 1.8+git20221224.

Recipients:(address . 64708@debbugs.gnu.org)

Message-ID:0d6b3e695340f343aa03eb279846eb3e4e6ecee3.1689685375.git.janneke@gnu.org

* gnu/packages/commencement.scm (gnumach-headers-boot0)[name]: New field; give
proper name.
[source]: Update to 1.8+git20221224, using git-fetch-from-tarball.
[native-inputs]: Add autoconf-boot0, automake-boot0, texinfo-boot0.
[arguments]: Add 'patch-compat' phase to allow building hurd-minimal etc, with
our out-of-date bootstsrap binaries.
---
 gnu/packages/commencement.scm | 46 ++++++++++++++++++++++++++---------
 1 file changed, 34 insertions(+), 12 deletions(-)

Toggle diff (59 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 1d69c2a80f..b15a71feb3 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2660,18 +2660,40 @@ (define gnumach-headers-boot0
   (with-boot0
    (package
      (inherit gnumach-headers)
-     (version "1.8-116-g28b53508")
-     (source (bootstrap-origin
-              (origin
-                (method url-fetch)
-                (uri (list (string-append "mirror://gnu/guix/mirror/gnumach-"
-                                          version ".tar.gz")
-                           (string-append "https://lilypond.org/janneke/hurd/"
-                                          "gnumach-" version ".tar.gz")))
-                (sha256
-                 (base32
-                  "006i0zgwy81vxarpfm12vip4q6i5mgmi5mmy5ldvxp5hx9h3l0zg")))))
-     (native-inputs '()))))
+     (name "gnumach-headers-boot0")
+     (version "1.8+git20221224")
+     (source
+      (origin
+        (method
+         (git-fetch-from-tarball
+          (origin
+            (method url-fetch)
+            (uri (string-append
+                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
+                  "gnumach-" version ".tar.gz"))
+            (sha256
+             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
+        (uri (git-reference
+              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
+              (commit version)))
+        (file-name (git-file-name name version))
+        (sha256
+         (base32
+          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
+     (native-inputs (list autoconf-boot0 automake-boot0 texinfo-boot0))
+     (arguments
+      (substitute-keyword-arguments (package-arguments gnumach-headers)
+        ((#:phases phases)
+         #~(modify-phases #$phases
+             (add-after 'unpack 'patch-compat
+               (lambda _
+                 (substitute* '("include/device/device_types.h"
+                                "include/mach_debug/slab_info.h"
+                                "include/mach_debug/vm_info.h")
+                   (("rpc_vm_size_t") "unsigned int")
+                   (("rpc_vm_offset_t") "unsigned int")
+                   (("rpc_long_natural_t") "unsigned long")
+                   (("long_natural_t") "unsigned long")))))))))))
 
 (define mig-boot0
   (let* ((mig (package
-- 
2.40.1

Janneke Nieuwenhuizen wrote on 18 Jul 2023 15:15

[PATCH 5/8] gnu: commencement: mig-boot0: Update to 1.8+git20230520.

Recipients:(address . 64708@debbugs.gnu.org)

Message-ID:96de3c312923a14bf2d1803aa13ace58ed9bd9c8.1689685375.git.janneke@gnu.org

* gnu/packages/commencement.scm (mig-boot0): Remove redundant
outer let.
[name]: New field, give proper name.
[version]: New field
[source]: Update to 1.8+git20230520, using git-fetch-from-tarball.
[native-inputs]: Add autoconf-boot0, automake-boot0.
[inputs]: Add gnumach-headers-boot0.
---
 gnu/packages/commencement.scm | 42 ++++++++++++++++++++++++++---------
 1 file changed, 31 insertions(+), 11 deletions(-)

Toggle diff (55 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index b15a71feb3..05a6badea0 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2696,17 +2696,37 @@ (define gnumach-headers-boot0
                    (("long_natural_t") "unsigned long")))))))))))
 
 (define mig-boot0
-  (let* ((mig (package
-                 (inherit (package-with-bootstrap-guile mig))
-                 (native-inputs `(("bison" ,bison-boot0)
-                                  ("flex" ,flex-boot0)))
-                 (inputs `(("flex" ,flex-boot0)))
-                 (arguments
-                  ;; TODO: On next rebuild cycle, reuse phases from 'mig'.
-                  `(#:configure-flags
-                    `(,(string-append "LDFLAGS=-Wl,-rpath="
-                                      (assoc-ref %build-inputs "flex") "/lib/")))))))
-    (with-boot0 mig)))
+  (with-boot0
+   (package
+     (inherit mig)
+     (name "mig-boot0")
+     (version "1.8+git20230520")
+     (source
+      (origin
+        (method
+         (git-fetch-from-tarball
+          (origin
+            (method url-fetch)
+            (uri (string-append
+                  "https://git.savannah.gnu.org/cgit/hurd/mig.git/snapshot/"
+                  "mig-" version ".tar.gz"))
+            (sha256
+             (base32 "1l1vfm4wap5yxylv91wssgpy7fnq22wp3akgd5nv995kychfa9jy")))))
+        (uri (git-reference
+              (url "https://git.savannah.gnu.org/git/hurd/mig.git")
+              (commit version)))
+        (file-name (git-file-name name version))
+        (sha256
+         (base32 "10r0fdjqjzqsy6ajb21rifvhw0wpjvrw6a1zdyliqlzqny5k0qlz"))))
+     (native-inputs (list autoconf-boot0 automake-boot0 bison-boot0 flex-boot0
+                          gnumach-headers-boot0))
+     (inputs (list flex-boot0 gnumach-headers-boot0))
+     (arguments
+      (list
+       #:configure-flags
+       #~(list (string-append "LDFLAGS=-Wl,-rpath="
+                              #$(this-package-native-input "flex")
+                              "/lib/")))))))
 
 (define hurd-version-boot0 "0.9-229-ga1efcee8")
 (define hurd-source-boot0
-- 
2.40.1

Janneke Nieuwenhuizen wrote on 18 Jul 2023 15:15

[PATCH 6/8] gnu: commencement: hurd-headers-boot0: Update to 0.9.git20230216.

Recipients:(address . 64708@debbugs.gnu.org)

Message-ID:3497e77b40e5573b2969bd171b486f7b7e7e0161.1689685375.git.janneke@gnu.org

* gnu/packages/commencement.scm (hurd-headers-boot0): Remove redundant outer
let and package-with-bootstrap-guile.
[name]: New field; give proper name.
[version]: New field.
[source]: Update to 0.9.git20230216, using git-fetch-from-tarball.
[inputs]: Add autoconf-boot0, automake-boot0, gnumach-headers-boot0.
---
 gnu/packages/commencement.scm | 31 +++++++++++++++++++++++++------
 1 file changed, 25 insertions(+), 6 deletions(-)

Toggle diff (44 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 05a6badea0..86246c3b08 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2742,12 +2742,31 @@ (define hurd-source-boot0
         "0bq2q2jisxcy0kgcm6rz0z2fddwxxm7azsama7li28a2m08kdpzy")))))
 
 (define hurd-headers-boot0
-  (let ((hurd-headers (package (inherit hurd-headers)
-                               (version hurd-version-boot0)
-                               (source hurd-source-boot0)
-                               (native-inputs `(("mig" ,mig-boot0)))
-                               (inputs '()))))
-    (with-boot0 (package-with-bootstrap-guile hurd-headers))))
+  (with-boot0
+   (package
+     (inherit hurd-headers)
+     (name "hurd-headers-boot0")
+     (version "0.9.git20230216")
+     (source
+      (origin
+        (method
+         (git-fetch-from-tarball
+          (origin
+            (method url-fetch)
+            (uri (string-append
+                  "https://git.savannah.gnu.org/cgit/hurd/hurd.git/snapshot/"
+                  "hurd-v" version ".tar.gz"))
+            (sha256
+             (base32 "1f75nlkcl00dqnnrbrj1frvzs2qibfpygj3gwywqi85aldjl48y7")))))
+        (uri (git-reference
+              (url "https://git.savannah.gnu.org/git/hurd/hurd.git")
+              (commit version)))
+        (file-name (git-file-name name version))
+        (sha256
+         (base32 "0jm1dnqkx4kdwmby0z5w0yqp9m5qp4hbxd4jxlyhiqm8nkw9mkvv"))))
+     (native-inputs
+      (list autoconf-boot0 automake-boot0 mig-boot0))
+     (inputs '()))))
 
 (define hurd-minimal-boot0
   (let ((hurd-minimal (package (inherit hurd-minimal)
-- 
2.40.1

Janneke Nieuwenhuizen wrote on 18 Jul 2023 15:14

[PATCH 3/8] gnu: commencement: Add git-fetch-from-tarball utility.

Recipients:(address . 64708@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:161bab138845fe307ec92b8fea181e00ece86e58.1689685375.git.janneke@gnu.org

* gnu/packages/commencement.scm (git-fetch-from-tarball): New procedure.

---

gnu/packages/commencement.scm | 31 +++++++++++++++++++++++++++++++

1 file changed, 31 insertions(+)

Toggle diff (52 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index d0603e62c8..1d69c2a80f 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -57,7 +57,10 @@ (define-module (gnu packages commencement)
   #:use-module (gnu packages xml)
   #:use-module (guix gexp)
   #:use-module (guix packages)
+  #:use-module ((guix store) #:select (%store-monad))
+  #:use-module (guix monads)
   #:use-module (guix download)
+  #:use-module ((guix git-download) #:select (git-reference git-file-name))
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system trivial)
   #:use-module ((guix licenses) #:prefix license:)
@@ -90,6 +93,34 @@ (define-module (gnu packages commencement)
 ;;;
 ;;; Code:
 
+(define* (git-fetch-from-tarball tarball)
+  (lambda* (url hash-algo hash
+                #:optional name
+                #:key (system (%current-system))
+                (guile %bootstrap-guile))
+    (mlet %store-monad ((guile (package->derivation guile system)))
+      (gexp->derivation
+       (or name "git-checkout")
+       (with-imported-modules '((guix build utils))
+         #~(begin
+             (use-modules (guix build utils)
+                          (ice-9 ftw)
+                          (ice-9 match))
+             (setenv "PATH"
+                     #+(file-append %bootstrap-coreutils&co "/bin"))
+             (invoke (string-append #+tar "/bin/tar")
+                     "xf" #$tarball)
+             (match (scandir ".")
+               (("." ".." directory)
+                (copy-recursively directory #$output)))))
+       #:recursive? #t
+       #:hash-algo hash-algo
+       #:hash hash
+       #:system system
+       #:guile-for-build guile
+       #:graft? #f
+       #:local-build? #t))))
+
 (define bootar
   (package
     (name "bootar")
-- 
2.40.1

Janneke Nieuwenhuizen wrote on 18 Jul 2023 15:15

[PATCH 7/8] gnu: commencement: hurd-minimal-boot0: Update to 0.9.git20230216.

Recipients:(address . 64708@debbugs.gnu.org)

Message-ID:6876c1d9bc1f349af9d9679127dfdbfebe100712.1689685375.git.janneke@gnu.org

* gnu/packages/commencement.scm (hurd-minimal-boot0): Remove redundant let
and package-with-bootstrap-guile.
[name]: New field; give proper name.
[version]: New field.
[source]: Update to 0.9.git20230216, using hurd-headers-boot0' source.
[inputs]: Add autoconf-boot0, automake-boot0, gnumach-headers-boot0.
[native-inputs]: Add autoconf-boot0, automake-boot0, and
gnumach-headers-boot0.
* gnu/packages/commencement.scm (hurd-version-boot0, hurd-source-boot0):
Remove.
---
 gnu/packages/commencement.scm | 27 ++++++++-------------------
 1 file changed, 8 insertions(+), 19 deletions(-)

Toggle diff (47 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 86246c3b08..7e988129a5 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2728,19 +2728,6 @@ (define mig-boot0
                               #$(this-package-native-input "flex")
                               "/lib/")))))))
 
-(define hurd-version-boot0 "0.9-229-ga1efcee8")
-(define hurd-source-boot0
-  (let ((version hurd-version-boot0))
-    (origin
-      (method url-fetch)
-      (uri (list (string-append "mirror://gnu/guix/mirror/hurd-v"
-                                version ".tar.gz")
-                 (string-append "https://lilypond.org/janneke/hurd/"
-                                "hurd-v" version ".tar.gz")))
-      (sha256
-       (base32
-        "0bq2q2jisxcy0kgcm6rz0z2fddwxxm7azsama7li28a2m08kdpzy")))))
-
 (define hurd-headers-boot0
   (with-boot0
    (package
@@ -2769,12 +2756,14 @@ (define hurd-headers-boot0
      (inputs '()))))
 
 (define hurd-minimal-boot0
-  (let ((hurd-minimal (package (inherit hurd-minimal)
-                               (version hurd-version-boot0)
-                               (source hurd-source-boot0)
-                               (native-inputs `(("mig" ,mig-boot0)))
-                               (inputs '()))))
-    (with-boot0 (package-with-bootstrap-guile hurd-minimal))))
+  (with-boot0
+   (package
+     (inherit hurd-minimal)
+     (name "hurd-minimal-boot0")
+     (source (package-source hurd-headers-boot0))
+     (native-inputs
+      (list autoconf-boot0 automake-boot0 gnumach-headers-boot0 mig-boot0))
+     (inputs (list gnumach-headers-boot0)))))
 
 (define/system-dependent hurd-core-headers-boot0
   ;; Return the Hurd and Mach headers as well as initial Hurd libraries for
-- 
2.40.1

Janneke Nieuwenhuizen wrote on 18 Jul 2023 15:15

[PATCH 8/8] gnu: mig: Update to 1.8+git20230520.

Recipients:(address . 64708@debbugs.gnu.org)(name . Josselin Poiret)(address . dev@jpoiret.xyz)

Message-ID:61835f33c6fee35457c7277e3216c10c15d0a391.1689685375.git.janneke@gnu.org

Now that mig-boot0 has been updated not to use mig's source, we can

safely update mig.

* gnu/packages/hurd.scm (mig)[source]: Update to 1.8+git20230520.

---

gnu/packages/hurd.scm | 20 ++++++++------------

1 file changed, 8 insertions(+), 12 deletions(-)

Toggle diff (39 lines)diff --git a/gnu/packages/hurd.scm b/gnu/packages/hurd.scm
index 3785c47750..4064a9e2e8 100644
--- a/gnu/packages/hurd.scm
+++ b/gnu/packages/hurd.scm
@@ -91,24 +91,20 @@ (define-public gnumach-headers
 (define-public mig
   (package
     (name "mig")
-    (version "1.8+git20220827")
+    (version "1.8+git20230520")
     (source (origin
-              (method url-fetch)
-              ;; XXX: Version 2.35 of glibc can only be built with an
-              ;; unreleased version of MiG:
-              ;; <https://lists.gnu.org/archive/html/bug-hurd/2023-03/msg00025.html>.
-              ;; It cannot be fetched from Git though, as the extra dependency
-              ;; on Autoconf/Automake would complicate bootstrapping.
-              (uri (string-append "mirror://gnu/guix/mirror/mig-"
-                                  version ".tar.gz"))
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://git.savannah.gnu.org/git/hurd/mig.git")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
               (sha256
                (base32
-                "163d37s9lscd6zxyfng421m9nl857464mgjj90xsrcl5ykbng5p2"))
-              (patches (search-patches "mig-cpu.h-generation.patch"))))
+                "10r0fdjqjzqsy6ajb21rifvhw0wpjvrw6a1zdyliqlzqny5k0qlz"))))
     (build-system gnu-build-system)
     ;; Flex is needed both at build and run time.
     (inputs (list gnumach-headers flex))
-    (native-inputs (list flex bison))
+    (native-inputs (list autoconf automake flex bison))
     (arguments
      (list #:tests? #f
            #:phases
-- 
2.40.1

Ludovic Courtès wrote on 18 Jul 2023 23:13

Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the Hurd.

Recipients:(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 64708@debbugs.gnu.org)

Message-ID:87lefcgbxl.fsf_-_@gnu.org

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (4 lines)

> From: Ludovic Courtès <ludo@gnu.org>

> * gnu/packages/commencement.scm (git-fetch-from-tarball): New procedure.

I guess this one is very weird so it deserves an explanation.

The problems we’re trying to solve are:

1. We cannot use ‘git-fetch’ at this point because that’d introduce a

circular dependency: https://issues.guix.gnu.org/63331.

2. Downloading a cgit-generated tarball is unsafe: the tarball might

be modified when cgit is updated or whatever, leading to a hash

mismatch that we wouldn’t be able to resolve (well,

disarchive.guix.gnu.org might be able to add an entry for that

tarball, but then there’s currently no fallback in case of hash

mismatch: https://issues.guix.gnu.org/28659).

So we need to obtain our Git checkouts by some other mean. There are

further complications:

3. We could use (web client), but without (gnutls) since it’s

unavailable at this stage.

4. We cannot use (guix swh) as is because Guile-JSON doesn’t build

with Guile 2.0, which is what we have here.

The trick below is the only way I could think of to reconcile these

issues: it essentially stacks two fixed-output derivation. The first

one downloads that auto-generated tarball (bad!), and the second one

extracts the contents of that tarball using ‘tar’ from

‘%bootstrap-coreutils&co’. The output of the second one is identical to

what ‘git-fetch’ would give us (or ‘swh-download’).

That makes it more robust. It also allows ‘guix lint -c archival’ to do

its work and similarly (well, almost) for ‘build-package-metadata.scm’,

the program that builds https://guix.gnu.org/sources.json, which SWH

periodically ingests.

Pfeww, long story.

Toggle quote (2 lines)

> +(define* (git-fetch-from-tarball tarball)

Maybe add this docstring:

"Return an <origin> method equivalent to 'git-fetch', except that it

fetches the checkout from TARBALL, a tarball containing said

checkout.

The purpose of this procedure is to work around bootstrapping issues:

'git-fetch' depends on Git, which is much higher in the dependency

graph."

Toggle quote (5 lines)

> + (setenv "PATH"

> + #+(file-append %bootstrap-coreutils&co "/bin"))

> + (invoke (string-append #+tar "/bin/tar")

> + "xf" #$tarball)

This should be: (invoke "tar" "xf" #$tarball).

Otherwise we’d get a cyclic dependency with ‘tar’.

(I didn’t notice because there’s a bug where ‘-s’ is ignored here:

./pre-inst-env guix build -e '(@@ (gnu packages commencement) mig-boot0)' \

-s i586-gnu -Sd

)

OK with these changes!

Ludo’.

Ludovic Courtès wrote on 18 Jul 2023 23:16

Recipients:(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 64708@debbugs.gnu.org)

Message-ID:87h6q0gbt6.fsf_-_@gnu.org

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (21 lines)> +     (name "gnumach-headers-boot0")
> +     (version "1.8+git20221224")
> +     (source
> +      (origin
> +        (method
> +         (git-fetch-from-tarball
> +          (origin
> +            (method url-fetch)
> +            (uri (string-append
> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
> +                  "gnumach-" version ".tar.gz"))
> +            (sha256
> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
> +        (uri (git-reference
> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
> +              (commit version)))
> +        (file-name (git-file-name name version))
> +        (sha256
> +         (base32
> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))

Since this is the same revision as in hurd.scm, we can reduce

duplication like so:

(origin

(inherit (package-source gnumach-headers))

(method (git-fetch-from-tarball …)))

Otherwise LGTM.

Ludovic Courtès wrote on 18 Jul 2023 23:17

Recipients:(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 64708@debbugs.gnu.org)

Message-ID:87cz0ogbrz.fsf_-_@gnu.org

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (18 lines)> +     (source
> +      (origin
> +        (method
> +         (git-fetch-from-tarball
> +          (origin
> +            (method url-fetch)
> +            (uri (string-append
> +                  "https://git.savannah.gnu.org/cgit/hurd/mig.git/snapshot/"
> +                  "mig-" version ".tar.gz"))
> +            (sha256
> +             (base32 "1l1vfm4wap5yxylv91wssgpy7fnq22wp3akgd5nv995kychfa9jy")))))
> +        (uri (git-reference
> +              (url "https://git.savannah.gnu.org/git/hurd/mig.git")
> +              (commit version)))
> +        (file-name (git-file-name name version))
> +        (sha256
> +         (base32 "10r0fdjqjzqsy6ajb21rifvhw0wpjvrw6a1zdyliqlzqny5k0qlz"))))

Likewise: (origin (inherit (package-source mig)) …)

Ludovic Courtès wrote on 18 Jul 2023 23:20

Recipients:(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)

Message-ID:874jm0gbml.fsf_-_@gnu.org

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (7 lines)

> From: Josselin Poiret <dev@jpoiret.xyz>

> Now that mig-boot0 has been updated not to use mig's source, we can

> safely update mig.

> * gnu/packages/hurd.scm (mig)[source]: Update to 1.8+git20230520.

This one could be avoided, at least for now, due to

4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting

on ftp.gnu.org. (We could also avoid the corresponding change in

‘mig-boot0’.)

But really, either way is fine, as long as both cross-compilation and

native compilation work after that. :-)

Ludo’.

Ludovic Courtès wrote on 18 Jul 2023 23:23

Recipients:(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 64708@debbugs.gnu.org)

Message-ID:87wmywewx9.fsf@gnu.org

Hi,

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (11 lines)> So...updating mig using git-fetch created a circular dependency because
> mig-boot0 re-used mig's source, and we cannot use git in commencement.
>
> This patch series now uses a hack by Ludo (thanks!) as suggested on IRC
> <https://logs.guix.gnu.org/guix/2023-07-17.log#233051> to not directly use
> generated cgit tarballs from savannah.
>
> The versions of gnumach-headers-boot0, mig-boot0, and hurd-headers-boot0 are
> updated so that native builds work again.  Because upstream does not produce
> release tarballs we now also need autoconf and automake in commencement.

I sent minor suggestions, but overall LGTM!

As of 4e11cac6be68a674c16965dc74e7da54cd3168df, we have

cross-compilation to i586-pc-gnu working again. After this series, we

should still have that + ‘-s i586-gnu’ (native compilation) working

(modulo test failures fixed by https://issues.guix.gnu.org/64711, of

course). I haven’t checked whether this is the case but that’s the

idea, right? :-)

Thanks again for this!

Ludo’.

Janneke Nieuwenhuizen wrote on 18 Jul 2023 23:31

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 64708@debbugs.gnu.org)

Message-ID:87ilagdhzs.fsf@gnu.org

Ludovic Courtès writes:

Hi!

Toggle quote (30 lines)> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> +     (name "gnumach-headers-boot0")
>> +     (version "1.8+git20221224")
>> +     (source
>> +      (origin
>> +        (method
>> +         (git-fetch-from-tarball
>> +          (origin
>> +            (method url-fetch)
>> +            (uri (string-append
>> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
>> +                  "gnumach-" version ".tar.gz"))
>> +            (sha256
>> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
>> +        (uri (git-reference
>> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
>> +              (commit version)))
>> +        (file-name (git-file-name name version))
>> +        (sha256
>> +         (base32
>> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
>
> Since this is the same revision as in hurd.scm, we can reduce
> duplication like so:
>
>   (origin
>     (inherit (package-source gnumach-headers))
>     (method (git-fetch-from-tarball …)))

Eh...no? Because gnumach-headers uses method git-fetch now!

Toggle quote (2 lines)

> Otherwise LGTM.

Thanks!

Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org

Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com

Janneke Nieuwenhuizen wrote on 18 Jul 2023 23:38

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87edl4dhne.fsf@gnu.org

Ludovic Courtès writes:

Hi!

Toggle quote (14 lines)> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> From: Josselin Poiret <dev@jpoiret.xyz>
>>
>> Now that mig-boot0 has been updated not to use mig's source, we can
>> safely update mig.
>>
>> * gnu/packages/hurd.scm (mig)[source]: Update to 1.8+git20230520.
>
> This one could be avoided, at least for now, due to
> 4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
> on ftp.gnu.org.  (We could also avoid the corresponding change in
> ‘mig-boot0’.)

I believe we'd rather re-instate this change to build mig fram git

so that we can do --with-commit to track all the neat things that

upstream is doing, right Josselin?

Toggle quote (3 lines)

> But really, either way is fine, as long as both cross-compilation and

> native compilation work after that. :-)

Sure!

Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org

Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com

Ludovic Courtès wrote on 19 Jul 2023 00:21

Recipients:(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 64708@debbugs.gnu.org)

Message-ID:87sf9keu9c.fsf_-_@gnu.org

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (32 lines)>> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>>
>>> +     (name "gnumach-headers-boot0")
>>> +     (version "1.8+git20221224")
>>> +     (source
>>> +      (origin
>>> +        (method
>>> +         (git-fetch-from-tarball
>>> +          (origin
>>> +            (method url-fetch)
>>> +            (uri (string-append
>>> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
>>> +                  "gnumach-" version ".tar.gz"))
>>> +            (sha256
>>> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
>>> +        (uri (git-reference
>>> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
>>> +              (commit version)))
>>> +        (file-name (git-file-name name version))
>>> +        (sha256
>>> +         (base32
>>> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
>>
>> Since this is the same revision as in hurd.scm, we can reduce
>> duplication like so:
>>
>>   (origin
>>     (inherit (package-source gnumach-headers))
>>     (method (git-fetch-from-tarball …)))
>
> Eh...no?  Because gnumach-headers uses method git-fetch now!

That’s the whole point: here we override the method with one that’s

close to the graph root, but the <git-reference> and hash are the same

as the one from ‘gnumach-headers’. Inheriting makes the intent

explicit.

Which is pretty cool no?

Ludo’.

Janneke Nieuwenhuizen wrote on 19 Jul 2023 07:00

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 64708@debbugs.gnu.org)

Message-ID:877cqwcx7t.fsf@gnu.org

Ludovic Courtès writes:

Toggle quote (32 lines)> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>>> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>>>
>>>> +     (name "gnumach-headers-boot0")
>>>> +     (version "1.8+git20221224")
>>>> +     (source
>>>> +      (origin
>>>> +        (method
>>>> +         (git-fetch-from-tarball
>>>> +          (origin
>>>> +            (method url-fetch)
>>>> +            (uri (string-append
>>>> +                  "https://git.savannah.gnu.org/cgit/hurd/gnumach.git/snapshot/"
>>>> +                  "gnumach-" version ".tar.gz"))
>>>> +            (sha256
>>>> +             (base32 "0vb19ynvrxz302snqxkd0wgizwa5fw2x06a4zjsllqb9ijbq9mc8")))))
>>>> +        (uri (git-reference
>>>> +              (url "https://git.savannah.gnu.org/git/hurd/gnumach.git")
>>>> +              (commit version)))
>>>> +        (file-name (git-file-name name version))
>>>> +        (sha256
>>>> +         (base32
>>>> +          "0f49zqxf64ds75rmskizpybl2mw7sxs05k59gjp3pgspvr87w7gs"))))
>>>
>>> Since this is the same revision as in hurd.scm, we can reduce
>>> duplication like so:
>>>
>>>   (origin
>>>     (inherit (package-source gnumach-headers))
>>>     (method (git-fetch-from-tarball …)))

Ah, I didn't really read this properly; pretty cool!

Toggle quote (7 lines)

>> Eh...no? Because gnumach-headers uses method git-fetch now!

> That’s the whole point: here we override the method with one that’s

> close to the graph root, but the <git-reference> and hash are the same

> as the one from ‘gnumach-headers’. Inheriting makes the intent

> explicit.

I still wonder a bit if it's a good idea to have commencement packages /

this commencement package inherit from hurd.scm; it means a world

rebuild whenever hurd.scm is updated (unless you put something back like

the verbose origin declaration above).

I'll take with your suggestion and go for the less code duplication

option right now, we can change it again when the need arises.

Toggle quote (2 lines)

> Which is pretty cool no?

Yeah! Sorry for the confusion :)

Greetings,

Janneke

Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org

Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com

Janneke Nieuwenhuizen wrote on 19 Jul 2023 07:53

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 64708-done@debbugs.gnu.org)

Message-ID:871qh4cuqz.fsf@gnu.org

Ludovic Courtès writes:

Hello,

Toggle quote (11 lines)> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> The versions of gnumach-headers-boot0, mig-boot0, and hurd-headers-boot0 are
>> updated so that native builds work again.  Because upstream does not produce
>> release tarballs we now also need autoconf and automake in commencement.
>
> I sent minor suggestions, but overall LGTM!
>
> As of 4e11cac6be68a674c16965dc74e7da54cd3168df, we have
> cross-compilation to i586-pc-gnu working again.

That's right, someone beat me to it ;-) (thanks!!)

Toggle quote (5 lines)

> After this series, we should still have that + ‘-s i586-gnu’ (native

> compilation) working (modulo test failures fixed by

> <https://issues.guix.gnu.org/64711>, of course). I haven’t checked

> whether this is the case but that’s the idea, right? :-)

Certainly!

Toggle quote (2 lines)

> Thanks again for this!

That's always good to hear.

Pushed to master as 8831d3ba9f799b7342ab466d625097f1982513a9

Greetings,

Janneke

Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org

Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com

Closed

Josselin Poiret wrote on 19 Jul 2023 11:08

Recipients:(address . 64708@debbugs.gnu.org)

Message-ID:87sf9ke0b0.fsf@jpoiret.xyz

Hi Janneke and Ludo,

Janneke Nieuwenhuizen <janneke@gnu.org> writes:

Toggle quote (10 lines)> Ludovic Courtès writes:
>> This one could be avoided, at least for now, due to
>> 4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
>> on ftp.gnu.org.  (We could also avoid the corresponding change in
>> ‘mig-boot0’.)
>
> I believe we'd rather re-instate this change to build mig fram git
> so that we can do --with-commit to track all the neat things that
> upstream is doing, right Josselin?

Yes, that was the point, we don't want to upload a new tarball to the
ftp all the time (esp. because I can't do it myself :) ).

Best,
-- 
Josselin Poiret

Josselin Poiret wrote on 19 Jul 2023 11:11

Re: [bug#64708] [PATCH 0/8] Fix cross build and native build for the Hurd.

Recipients:(address . 64708@debbugs.gnu.org)

Message-ID:87pm4oe05x.fsf@jpoiret.xyz

Hi Ludo and Janneke.

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (12 lines)> The trick below is the only way I could think of to reconcile these
> issues: it essentially stacks two fixed-output derivation.  The first
> one downloads that auto-generated tarball (bad!), and the second one
> extracts the contents of that tarball using ‘tar’ from
> ‘%bootstrap-coreutils&co’.  The output of the second one is identical to
> what ‘git-fetch’ would give us (or ‘swh-download’).
>
> That makes it more robust.  It also allows ‘guix lint -c archival’ to do
> its work and similarly (well, almost) for ‘build-package-metadata.scm’,
> the program that builds <https://guix.gnu.org/sources.json>, which SWH
> periodically ingests.

I don't 100% understand what this resolves though, since this still
depends on the unstable tarball in the end!  In both cases, we'd rely on
CI to have the stable copy, or SWH providing it, since we wouldn't be
able to rebuild it.

Best,
-- 
Josselin Poiret

Janneke Nieuwenhuizen wrote on 19 Jul 2023 11:32

Re: bug#64708: [PATCH 0/8] Fix cross build and native build for the Hurd.

Recipients:(name . Josselin Poiret)(address . dev@jpoiret.xyz)

Message-ID:87cz0ob61v.fsf@gnu.org

Josselin Poiret writes:

Hi Josselin and Ludo,

Toggle quote (15 lines)> Janneke Nieuwenhuizen <janneke@gnu.org> writes:
>
>> Ludovic Courtès writes:
>>> This one could be avoided, at least for now, due to
>>> 4e11cac6be68a674c16965dc74e7da54cd3168df and the tarball we’re hosting
>>> on ftp.gnu.org.  (We could also avoid the corresponding change in
>>> ‘mig-boot0’.)
>>
>> I believe we'd rather re-instate this change to build mig fram git
>> so that we can do --with-commit to track all the neat things that
>> upstream is doing, right Josselin?
>
> Yes, that was the point, we don't want to upload a new tarball to the
> ftp all the time (esp. because I can't do it myself :) ).

So, that's what I've done, I think we're OK!

Thanks for the confirmation,

Janneke

Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org

Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com

Your comment

This issue is archived.

To comment on this conversation send an email to 64708@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 64708

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date