[PATCH core-updates 0/6] Run builds in C.UTF-8 locale

  • Done
  • quality assurance status badge
Details
2 participants
  • Janneke Nieuwenhuizen
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 14 Dec 2023 11:51
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
cover.1702550735.git.ludo@gnu.org
Hello,

This is the logical followup to https://issues.guix.gnu.org/67686:
running builds in the C.UTF-8 locale (instead of en_US.UTF-8) and
removing ‘glibc-utf8-locales’ from the set of implicit inputs.

Tested on x86_64-linux with:

guix build texinfo nss-certs
make check TESTS=tests/profiles.scm

I’d like to merge it soon so we can move forward.

Let me know what you think!

Ludo’.

Ludovic Courtès (6):
packages: Remove reference to ‘glibc-utf8-locales’ in
‘patch-and-repack’.
build-system/gnu: Change default locale to C.UTF-8.
gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%final-inputs’.
profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
packages: Output and error ports are line-buffered in
‘patch-and-repack’.
gnu: glibc-utf8-locales: Add the C.UTF-8 locale.

gnu/packages/base.scm | 23 ++++++++++++++++++-----
gnu/packages/commencement.scm | 31 +++++++++++++++----------------
gnu/packages/gawk.scm | 6 +++++-
gnu/packages/texinfo.scm | 7 +++++--
guix/build-system/gnu.scm | 4 ++--
guix/packages.scm | 34 +++++++++++++++++-----------------
guix/profiles.scm | 33 +++++----------------------------
tests/profiles.scm | 7 +------
8 files changed, 68 insertions(+), 77 deletions(-)


base-commit: e4013b31266b40dc9cb250c44b2088d3931cb3b3
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 1/6] packages: Remove reference to ‘glibc-utf8-locales’ in ‘pa tch-and-repack’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
eee58768e5b7514c836ea4b12abe61850205a359.1702550735.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/packages.scm (%standard-patch-inputs): Remove “locales” entry.
(patch-and-repack): Remove ‘locales’ variable.
[build]: Remove reference to ‘locales’. Use “C.UTF-8” unconditionally.

Change-Id: I54219b232880e05534599fbba8738cc00fb5d568
---
guix/packages.scm | 29 ++++++++++++-----------------
1 file changed, 12 insertions(+), 17 deletions(-)

Toggle diff (56 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 930b1a3b0e..4b0c478ff4 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -866,13 +866,7 @@ (define (%standard-patch-inputs system)
("gzip" ,(ref '(gnu packages compression) 'gzip))
("lzip" ,(ref '(gnu packages compression) 'lzip))
("unzip" ,(ref '(gnu packages compression) 'unzip))
- ("patch" ,(ref '(gnu packages base) 'patch))
- ("locales"
- ,(parameterize ((%current-target-system #f)
- (%current-system system))
- (canonical
- ((module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))))))))
+ ("patch" ,(ref '(gnu packages base) 'patch)))))
(define (default-guile)
"Return the default Guile package used to run the build code of
@@ -955,7 +949,6 @@ (define* (patch-and-repack source patches
(lzip (lookup-input "lzip"))
(xz (lookup-input "xz"))
(patch (lookup-input "patch"))
- (locales (lookup-input "locales"))
(comp (and=> (compressor source-file-name) lookup-input))
(patches (map instantiate-patch patches)))
(define build
@@ -1017,15 +1010,17 @@ (define* (patch-and-repack source patches
;; Encoding/decoding errors shouldn't be silent.
(fluid-set! %default-port-conversion-strategy 'error)
- (when #+locales
- ;; First of all, install a UTF-8 locale so that UTF-8 file names
- ;; are correctly interpreted. During bootstrap, LOCALES is #f.
- (setenv "LOCPATH"
- (string-append #+locales "/lib/locale/"
- #+(and locales
- (version-major+minor
- (package-version locales)))))
- (setlocale LC_ALL "en_US.utf8"))
+ ;; First of all, install a UTF-8 locale so that UTF-8 file names
+ ;; are correctly interpreted. During bootstrap, locales are
+ ;; missing.
+ (let ((locale "C.UTF-8"))
+ (catch 'system-error
+ (lambda ()
+ (setlocale LC_ALL locale))
+ (lambda args
+ (format (current-error-port)
+ "failed to install '~a' locale: ~a~%"
+ locale (system-error-errno args)))))
(setenv "PATH"
(string-append #+xz "/bin"
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 2/6] build-system/gnu: Change default locale to C.UTF-8.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
1df36313ccd91f5249261ae14d222658d5564183.1702550735.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): #:locale now
defaults to "C.UTF-8".

Change-Id: I9a4bfe564bcd8bbd0d57ba04568eb6b8020bd051
---
guix/build-system/gnu.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/guix/build-system/gnu.scm b/guix/build-system/gnu.scm
index ed1dc33c76..0f886fe21d 100644
--- a/guix/build-system/gnu.scm
+++ b/guix/build-system/gnu.scm
@@ -361,7 +361,7 @@ (define* (gnu-build name inputs
(make-dynamic-linker-cache? #t)
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
@@ -501,7 +501,7 @@ (define* (gnu-cross-build name
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 3/6] gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%fina l-inputs’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
a65975f7dcf62d4d629fae5ca1cfb43e9ace4488.1702550735.git.ludo@gnu.org
* gnu/packages/commencement.scm (%boot5-inputs): Make an alias for
‘%boot4-inputs’.
(with-boot5): Turn into an alias for ‘with-boot4’.
(%final-inputs): Override ‘native-inputs’ of GAWK. Remove "locales"
entry.
(canonical-package): Add “glibc-utf8-locales” entry.
* gnu/packages/base.scm (libc-utf8-locales-for-target): Use
‘canonical-package’.
* gnu/packages/gawk.scm (gawk)[native-inputs]: New field.
* gnu/packages/texinfo.scm (texinfo)[native-inputs]: Add
‘libc-utf8-locales-for-target’.

Change-Id: I2283ffc6c6000c98ad2db89c04d581e050667c94
---
gnu/packages/base.scm | 11 ++++++++---
gnu/packages/commencement.scm | 31 +++++++++++++++----------------
gnu/packages/gawk.scm | 6 +++++-
gnu/packages/texinfo.scm | 7 +++++--
4 files changed, 33 insertions(+), 22 deletions(-)

Toggle diff (136 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 235ad7fecc..d31c0b0772 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1593,9 +1593,14 @@ (define* (libc-locales-for-target #:optional
(define* (libc-utf8-locales-for-target #:optional
(target (or (%current-target-system)
(%current-system))))
- (if (target-hurd? target)
- glibc-utf8-locales/hurd
- glibc-utf8-locales))
+ "Return the glibc UTF-8 locale package for TARGET."
+ ;; Note: To avoid circular dependencies (such as: texinfo ->
+ ;; glibc-utf8-locales -> glibc -> texinfo), refer to
+ ;; 'glibc-utf8-locales-final' via 'canonical-package'.
+ (canonical-package
+ (if (target-hurd? target)
+ glibc-utf8-locales/hurd
+ glibc-utf8-locales)))
(define-public tzdata
(package
diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 988e497ee7..f4882f7cdd 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -3394,17 +3394,11 @@ (define-public ld-gold-wrapper
#:guile guile-final
#:bash bash-final))
-(define (%boot5-inputs)
- ;; Now with UTF-8 locales. Remember that the bootstrap binaries were built
- ;; with an older libc, which cannot load the new locale format. See
- ;; <https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html>.
- `(("locales" ,(if (target-hurd?)
- glibc-utf8-locales-final/hurd
- glibc-utf8-locales-final))
- ,@(%boot4-inputs)))
-
-(define with-boot5
- (package-with-explicit-inputs %boot5-inputs))
+;; There used to be a "stage 5" including a variant of the
+;; 'glibc-utf8-locales' package. This is no longer necessary since 'glibc'
+;; embeds the "C.UTF-8" locale, but these aliases are kept for convenience.
+(define %boot5-inputs %boot4-inputs)
+(define with-boot5 with-boot4)
(define gnu-make-final
;; The final GNU Make, which uses the final Guile.
@@ -3493,7 +3487,11 @@ (define-public %final-inputs
("diffutils" ,diffutils)
("patch" ,patch)
("findutils" ,findutils)
- ("gawk" ,gawk)))
+ ("gawk" ,(package/inherit gawk
+ (native-inputs
+ (list (if (target-hurd?)
+ glibc-utf8-locales-final/hurd
+ glibc-utf8-locales-final)))))))
("sed" ,sed-final)
("grep" ,grep-final)
("xz" ,xz-final)
@@ -3504,10 +3502,7 @@ (define-public %final-inputs
("binutils" ,binutils-final)
("gcc" ,gcc-final)
("libc" ,glibc-final)
- ("libc:static" ,glibc-final "static")
- ("locales" ,(if (target-hurd? (%current-system))
- glibc-utf8-locales-final/hurd
- glibc-utf8-locales-final)))))))
+ ("libc:static" ,glibc-final "static"))))))
(define-public canonical-package
(let ((name->package (mlambda (system)
@@ -3518,6 +3513,10 @@ (define-public canonical-package
package result))))
vlist-null
`(("guile" ,guile-final)
+ ("glibc-utf8-locales"
+ ,(if (target-hurd? system)
+ glibc-utf8-locales-final/hurd
+ glibc-utf8-locales-final))
,@(%final-inputs system))))))
(lambda (package)
"Return the 'canonical' variant of PACKAGE---i.e., if PACKAGE is one of
diff --git a/gnu/packages/gawk.scm b/gnu/packages/gawk.scm
index 2d8bc581ef..476865ada3 100644
--- a/gnu/packages/gawk.scm
+++ b/gnu/packages/gawk.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2016, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2018, 2022, 2023 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2021, 2022 Marius Bakke <marius@gnu.org>
@@ -79,6 +79,10 @@ (define-public gawk
(("#! /usr/bin/gawk")
(string-append "#!" (which "gawk")))))))))
+ ;; Tests explicitly require en_US.UTF-8, fr_FR.UTF-8, ru_RU.UTF-8 and
+ ;; ja_JP.UTF-8.
+ (native-inputs (list (libc-utf8-locales-for-target)))
+
(inputs (list libsigsegv
;; Use the full-fledged Bash package, otherwise the test suite
;; sometimes fail non-deterministically.
diff --git a/gnu/packages/texinfo.scm b/gnu/packages/texinfo.scm
index ab4583071e..92c9d800a9 100644
--- a/gnu/packages/texinfo.scm
+++ b/gnu/packages/texinfo.scm
@@ -38,6 +38,7 @@ (define-module (gnu packages texinfo)
#:use-module (guix build-system gnu)
#:use-module (gnu packages)
#:use-module (gnu packages autotools)
+ #:use-module (gnu packages base)
#:use-module (gnu packages compression)
#:use-module (gnu packages gettext)
#:use-module (gnu packages ncurses)
@@ -79,10 +80,12 @@ (define-public texinfo
#:tests? ,(and (not (target-hurd?))
(not (%current-target-system)))))
(inputs (list ncurses perl))
+
;; When cross-compiling, texinfo will build some of its own binaries with
;; the native compiler. This means ncurses is needed both in both inputs
- ;; and native-inputs.
- (native-inputs (list perl ncurses))
+ ;; and native-inputs. Some of its tests require extra locales such as
+ ;; fr_FR.UTF-8.
+ (native-inputs (list perl ncurses (libc-utf8-locales-for-target)))
(native-search-paths
;; This is the variable used by the standalone Info reader.
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 4/6] profiles: Use C.UTF-8 in stead of ‘glibc-utf8-locales’ where pos sible.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
f40d9b9bbe08c279eab122d2bd29db2426a45731.1702550735.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/profiles.scm (ca-certificate-bundle)[libc-utf8-locales-for-target]:
Remove.
[build]: Use “C.UTF-8” instead of “en_US.utf8”.
(profile-derivation)[libc-utf8-locales-for-target]: Remove.
[set-utf8-locale]: Rewrite to a gexp that calls ‘setlocale’ for “C.UTF-8”.
* tests/profiles.scm ("profile-derivation, cross-compilation"): Remove
‘locales’ variable and related check.

Change-Id: I7cb148b9149fe5fbe5b9b1b25fdce1002ad1f37e
---
guix/profiles.scm | 33 +++++----------------------------
tests/profiles.scm | 7 +------
2 files changed, 6 insertions(+), 34 deletions(-)

Toggle diff (97 lines)
diff --git a/guix/profiles.scm b/guix/profiles.scm
index ce2f8337bf..da7790d819 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -1125,11 +1125,6 @@ (define* (ca-certificate-bundle manifest #:optional system)
MANIFEST. Single-file bundles are required by programs such as Git and Lynx."
;; See <http://lists.gnu.org/archive/html/guix-devel/2015-02/msg00429.html>
;; for a discussion.
-
- (define libc-utf8-locales-for-target ;lazy reference
- (module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))
-
(define build
(with-imported-modules '((guix build utils))
#~(begin
@@ -1161,13 +1156,7 @@ (define* (ca-certificate-bundle manifest #:optional system)
;; Some file names in the NSS certificates are UTF-8 encoded so
;; install a UTF-8 locale.
- (setenv "LOCPATH"
- (string-append #+(libc-utf8-locales-for-target system)
- "/lib/locale/"
- #+(version-major+minor
- (package-version
- (libc-utf8-locales-for-target system)))))
- (setlocale LC_ALL "en_US.utf8")
+ (setlocale LC_ALL "C.UTF-8")
(match (append-map ca-files '#$(manifest-inputs manifest))
(()
@@ -1960,8 +1949,7 @@ (define* (profile-derivation manifest
or TARGET is set, raise an error if MANIFEST contains a package that does not
support SYSTEM.
-When LOCALES? is true, the build is performed under a UTF-8 locale; this adds
-a dependency on the 'glibc-utf8-locales' package.
+When LOCALES? is true, the build is performed under a UTF-8 locale.
When RELATIVE-SYMLINKS? is true, use relative file names for symlink targets.
This is one of the things to do for the result to be relocatable.
@@ -2004,21 +1992,10 @@ (define* (profile-derivation manifest
(and (derivation? drv) (gexp-input drv)))
extras))
- (define libc-utf8-locales-for-target ;lazy reference
- (module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))
-
(define set-utf8-locale
- ;; Some file names (e.g., in 'nss-certs') are UTF-8 encoded so
- ;; install a UTF-8 locale.
- (let ((locales (libc-utf8-locales-for-target
- (or system (%current-system)))))
- #~(begin
- (setenv "LOCPATH"
- #$(file-append locales "/lib/locale/"
- (version-major+minor
- (package-version locales))))
- (setlocale LC_ALL "en_US.utf8"))))
+ ;; Some file names (e.g., in 'nss-certs') are UTF-8 encoded so install a
+ ;; UTF-8 locale. Assume libc comes with a copy of C.UTF-8.
+ #~(setlocale LC_ALL "C.UTF-8"))
(define builder
(with-imported-modules '((guix build profiles)
diff --git a/tests/profiles.scm b/tests/profiles.scm
index ddd6d74f3b..e448137cff 100644
--- a/tests/profiles.scm
+++ b/tests/profiles.scm
@@ -463,7 +463,6 @@ (define glibc
(target -> "arm-linux-gnueabihf")
(grep (package->cross-derivation packages:grep target))
(sed (package->cross-derivation packages:sed target))
- (locales (package->derivation (packages:libc-utf8-locales-for-target)))
(drv (profile-derivation manifest
#:hooks '()
#:locales? #t
@@ -475,15 +474,11 @@ (define glibc
(and (string-suffix? name input) input)))
(derivation-inputs drv))))
- ;; The inputs for grep and sed should be cross-build derivations, but that
- ;; for the glibc-utf8-locales should be a native build.
(return (and (string=? (derivation-system drv) (%current-system))
(string=? (find-input packages:grep)
(derivation-file-name grep))
(string=? (find-input packages:sed)
- (derivation-file-name sed))
- (string=? (find-input (packages:libc-utf8-locales-for-target))
- (derivation-file-name locales))))))
+ (derivation-file-name sed))))))
(test-assert "package->manifest-entry defaults to \"out\""
(let ((outputs (package-outputs packages:glibc)))
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 5/6] packages: Output and err or ports are line-buffered in ‘patch-and-repa ck’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
2533db50b4cf1f21baf85b63d2266055d53618a2.1702550735.git.ludo@gnu.org
* guix/packages.scm (patch-and-repack): Add ‘setvbuf’ calls.

Change-Id: I039bb6407263d5172bf0bc716bda6860dc2615fb
---
guix/packages.scm | 5 +++++
1 file changed, 5 insertions(+)

Toggle diff (18 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 4b0c478ff4..cb8db925f8 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -1007,6 +1007,11 @@ (define* (patch-and-repack source patches
'("--no-recursion"
"--files-from=.file_list"))))
+ (let ((line (cond-expand (guile-2.0 _IOLBF)
+ (else 'line))))
+ (setvbuf (current-output-port) line)
+ (setvbuf (current-error-port) line))
+
;; Encoding/decoding errors shouldn't be silent.
(fluid-set! %default-port-conversion-strategy 'error)
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 6/6] gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
48fba13f955b3552c7f011672763052752cc7c35.1702550735.git.ludo@gnu.org
* gnu/packages/base.scm (%default-utf8-locales): Add "C".
(make-glibc-utf8-locales): Delete "C" from LOCALES for glibc versions
lower than or equal to 2.35.

Change-Id: I6a6faa68dc18bf0f87c372109a17f476d3561278
---
gnu/packages/base.scm | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index d31c0b0772..eab9ff411d 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1396,7 +1396,8 @@ (define-public (make-glibc-locales glibc)
(define %default-utf8-locales
;; These are the locales commonly used for tests---e.g., in Guile's i18n
;; tests.
- '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+ '("C" "de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+
(define*-public (make-glibc-utf8-locales glibc #:key
(locales %default-utf8-locales)
(name "glibc-utf8-locales"))
@@ -1444,7 +1445,14 @@ (define*-public (make-glibc-utf8-locales glibc #:key
(symlink (string-append locale ".utf8")
(string-append localedir "/"
locale ".UTF-8")))
- '#$locales)))))
+
+ ;; The C.UTF-8 locale was introduced in 2.35 but it
+ ;; fails to build there:
+ ;; <https://sourceware.org/bugzilla/show_bug.cgi?id=28861>.
+ '#$(if (version>? (package-version this-package)
+ "2.35")
+ locales
+ (delete "C" locales)))))))
(native-inputs (list glibc gzip))
(synopsis (if default-locales?
(P_ "Small sample of UTF-8 locales")
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 16 Dec 2023 09:49
Re: [bug#67824] [PATCH core-updates 0/6] Run builds in C.UTF-8 locale
(name . Ludovic Courtès)(address . ludo@gnu.org)
87h6kir0pp.fsf@gnu.org
Ludovic Courtès writes:

Hi,

Toggle quote (13 lines)
> This is the logical followup to https://issues.guix.gnu.org/67686:
> running builds in the C.UTF-8 locale (instead of en_US.UTF-8) and
> removing ‘glibc-utf8-locales’ from the set of implicit inputs.
>
> Tested on x86_64-linux with:
>
> guix build texinfo nss-certs
> make check TESTS=tests/profiles.scm
>
> I’d like to merge it soon so we can move forward.
>
> Let me know what you think!

I like the idea but I cannot build it, i.e., building gcc-mesboot1 hangs
for me in the gmp configure:

Toggle snippet (8 lines)
./pre-inst-env guix build '(@@ (gnu packages commencement) gcc-mesboot1)'
[..]
(Configuring in host-i686-unknown-linux-gnu/gmp)
checking whether sscanf needs writable input... no
checking for struct pst_processor.psp_iticksperclktick... no
<hang>

I've tried at least 6 times. Gash running configure just hangs.

On a related note, I found that running

Toggle snippet (3 lines)
./pre-inst-env guix shell --bootstrap --no-grafts -e '(@@ (gnu packages commencement) gcc-mesboot1)'

wants to build gcc-mesboot1 when attempting to easily create a shell in
which to run and debug the hanging configure command.

I guess someone needs to patch gcc-mesboot1 and dive into it.

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 16 Dec 2023 12:27
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
87h6kis7zo.fsf@gnu.org
Hi!

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (10 lines)
> I like the idea but I cannot build it, i.e., building gcc-mesboot1 hangs
> for me in the gmp configure:
>
> ./pre-inst-env guix build '(@@ (gnu packages commencement) gcc-mesboot1)'
> [..]
> (Configuring in host-i686-unknown-linux-gnu/gmp)
> checking whether sscanf needs writable input... no
> checking for struct pst_processor.psp_iticksperclktick... no
> <hang>

Hmm it works for me, weird.

Now I realize I forgot to change the default value of #:locale in
‘gnu-build-system.scm’ plus a couple of other things, so I’m starting
another round of rebuilds to test it.

It’s all about patience. :-)

Ludo’.
L
L
Ludovic Courtès wrote on 16 Dec 2023 17:47
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
874jgirt62.fsf@gnu.org
Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (13 lines)
> I like the idea but I cannot build it, i.e., building gcc-mesboot1 hangs
> for me in the gmp configure:
>
> ./pre-inst-env guix build '(@@ (gnu packages commencement) gcc-mesboot1)'
> [..]
> (Configuring in host-i686-unknown-linux-gnu/gmp)
> checking whether sscanf needs writable input... no
> checking for struct pst_processor.psp_iticksperclktick... no
> <hang>
>
>
> I've tried at least 6 times. Gash running configure just hangs.

I experienced a similar thing just now (I had removed #:parallel-build?
#f from ‘binutils-mesboot0’):

Toggle snippet (19 lines)
$ pstree -p 29600
.guile-real(29600)???conftest(45270)
??make(34404)???.guile-real(34406)???make(34421)???.guile-real(6216)???.guile-real(6247)???make(629+
??{.guile-real}(29610)
ludo@guix-hpc5 ~$ sudo cat /proc/29600/cmdline |xargs -0
/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile --no-auto-compile -L /gnu/store/4zibvq0ffm75xrmmkm0997rnwl3i835p-module-import -C /gnu/store/idi8frs4vz0ympplpm1kddp8932kgkr1-module-import-compiled /gnu/store/kj2wyb3w6c6zqdjfmizv826rgfb383rf-binutils-mesboot0-2.20.1a-builder
$ sudo guix processes |guix shell recutils -- recsel -e 'LockHeld ~ "binutils"'
SessionPID: 17495
ClientPID: 17477
ClientCommand: /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/libexec/guix/guile \ /run/current-system/profile/bin/guix build -c32 -M3 /gnu/store/icfjxv22fysqpip9zxfqv7xjdahziv94-hello-2.12.1.drv
LockHeld: /gnu/store/pkaxhawivjhhff55ncglr30c6d9dzlaz-binutils-mesboot0-2.20.1a.lock
ChildPID: 17496
ChildCommand: : /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/libexec/guix/guile \ /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/bin/guix substitute --query
ChildPID: 17522
ChildCommand: : /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/libexec/guix/guile \ /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/bin/guix offload x86_64-linux 0 1 0
ChildPID: 29600
ChildCommand: : /gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile --no-auto-compile -L /gnu/store/4zibvq0ffm75xrmmkm0997rnwl3i835p-module-import -C /gnu/store/idi8frs4vz0ympplpm1kddp8932kgkr1-module-import-compiled /gnu/store/kj2wyb3w6c6zqdjfmizv826rgfb383rf-binutils-mesboot0-2.20.1a-builder

Here ‘conftest’ is a zombie, which means that the shell running
./configure (gash-boot-0.3.0) did not call ‘waitpid’ (bug #1). There’s
another bunch of .guile-real zombie processes (child processes of ‘make
DESTDIR= RPATH_ENVVAR=LD_LIBRARY_PATH
TARGET_SUBDIR=i686-unknown-linux-gnu
bindir=/gnu/store/pkaxhawivjhhff55ncglr30c6d9dzlaz-binutils-mesboot0-2.20.1a/bin
[…]’).

Thus the ‘conftest’ process got reparented to the Guile process that
runs the build script (PID 1), and then we run into (bug #2):


Ludo’.
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 1/9] packages: Remove refere nce to ‘glibc-utf8-locales’ in ‘patch-and-repack’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
eee58768e5b7514c836ea4b12abe61850205a359.1702916397.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/packages.scm (%standard-patch-inputs): Remove “locales” entry.
(patch-and-repack): Remove ‘locales’ variable.
[build]: Remove reference to ‘locales’. Use “C.UTF-8” unconditionally.

Change-Id: I54219b232880e05534599fbba8738cc00fb5d568
---
guix/packages.scm | 29 ++++++++++++-----------------
1 file changed, 12 insertions(+), 17 deletions(-)

Toggle diff (56 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 930b1a3b0e..4b0c478ff4 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -866,13 +866,7 @@ (define (%standard-patch-inputs system)
("gzip" ,(ref '(gnu packages compression) 'gzip))
("lzip" ,(ref '(gnu packages compression) 'lzip))
("unzip" ,(ref '(gnu packages compression) 'unzip))
- ("patch" ,(ref '(gnu packages base) 'patch))
- ("locales"
- ,(parameterize ((%current-target-system #f)
- (%current-system system))
- (canonical
- ((module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))))))))
+ ("patch" ,(ref '(gnu packages base) 'patch)))))
(define (default-guile)
"Return the default Guile package used to run the build code of
@@ -955,7 +949,6 @@ (define* (patch-and-repack source patches
(lzip (lookup-input "lzip"))
(xz (lookup-input "xz"))
(patch (lookup-input "patch"))
- (locales (lookup-input "locales"))
(comp (and=> (compressor source-file-name) lookup-input))
(patches (map instantiate-patch patches)))
(define build
@@ -1017,15 +1010,17 @@ (define* (patch-and-repack source patches
;; Encoding/decoding errors shouldn't be silent.
(fluid-set! %default-port-conversion-strategy 'error)
- (when #+locales
- ;; First of all, install a UTF-8 locale so that UTF-8 file names
- ;; are correctly interpreted. During bootstrap, LOCALES is #f.
- (setenv "LOCPATH"
- (string-append #+locales "/lib/locale/"
- #+(and locales
- (version-major+minor
- (package-version locales)))))
- (setlocale LC_ALL "en_US.utf8"))
+ ;; First of all, install a UTF-8 locale so that UTF-8 file names
+ ;; are correctly interpreted. During bootstrap, locales are
+ ;; missing.
+ (let ((locale "C.UTF-8"))
+ (catch 'system-error
+ (lambda ()
+ (setlocale LC_ALL locale))
+ (lambda args
+ (format (current-error-port)
+ "failed to install '~a' locale: ~a~%"
+ locale (system-error-errno args)))))
(setenv "PATH"
(string-append #+xz "/bin"
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 0/9] Run builds in C.UTF-8 locale
(address . 67824@debbugs.gnu.org)
cover.1702916397.git.ludo@gnu.org
Hello Guix!

This is an updated version of the patch set. Changes since v1:

• Change default value of #:locale in ‘gnu-build-system.scm’,
since all the build systems except ‘gnu’ rely on it.

• Remove customization of util-linux tests that was due to
the lack of C.UTF-8 on our side before.

• Skip failing GLib test, which should allow us to go further
towards evaluating ‘core-updates’ (!).

• Add useful x86_64-gnu patch in glibc (piggy-backed, not related
to the issues at hand).

• Include fix for https://issues.guix.gnu.org/30948. Probably
not strictly necessary but a good opportunity, especially since
Gash/Guile & co. seem to leave zombies when building the early
‘mesboot’ packages (something Janneke and I discussed on IRC
a couple of days ago).

Feedback welcome!

Ludo’.

Ludovic Courtès (9):
packages: Remove reference to ‘glibc-utf8-locales’ in
‘patch-and-repack’.
build-system/gnu: Change default locale to C.UTF-8.
gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%final-inputs’.
profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
packages: Output and error ports are line-buffered in
‘patch-and-repack’.
gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
gnu: glibc: Add patch for ‘ucontext’ on x86_64-gnu (GNU/Hurd).
build-system/gnu: Turn PID 1 into an “init”-style process by default.
gnu: glib: Skip test that fails with glibc 2.38.

gnu/local.mk | 1 +
gnu/packages/base.scm | 24 +++--
gnu/packages/commencement.scm | 31 ++++---
gnu/packages/gawk.scm | 6 +-
gnu/packages/glib.scm | 6 +-
gnu/packages/linux.scm | 6 +-
.../patches/glibc-2.38-hurd-ucontext.patch | 87 +++++++++++++++++++
gnu/packages/texinfo.scm | 7 +-
guix/build-system/gnu.scm | 8 +-
guix/build/gnu-build-system.scm | 41 ++++++++-
guix/packages.scm | 34 ++++----
guix/profiles.scm | 33 ++-----
tests/profiles.scm | 7 +-
13 files changed, 206 insertions(+), 85 deletions(-)
create mode 100644 gnu/packages/patches/glibc-2.38-hurd-ucontext.patch


base-commit: e4013b31266b40dc9cb250c44b2088d3931cb3b3
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 2/9] build-system/gnu: Change default locale to C.UTF-8.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
6fbee1e6544d1abcaf781fc246004de494396868.1702916397.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): #:locale now
defaults to "C.UTF-8".
* guix/build/gnu-build-system.scm (install-locale): Likewise.
* gnu/packages/linux.scm (util-linux)[arguments]: In ‘pre-check’ phase,
remove ‘invalid-multibyte’ modification so the test runs under C.UTF-8.

Change-Id: I9a4bfe564bcd8bbd0d57ba04568eb6b8020bd051
---
gnu/packages/linux.scm | 6 +-----
guix/build-system/gnu.scm | 4 ++--
guix/build/gnu-build-system.scm | 2 +-
3 files changed, 4 insertions(+), 8 deletions(-)

Toggle diff (54 lines)
diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index 430645e4ad..0572ad3a73 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -2257,11 +2257,7 @@ (define-public util-linux
;; Change the test to refer to the right file.
(substitute* "tests/ts/misc/mcookie"
(("/etc/services")
- services))
-
- ;; The C.UTF-8 locale does not exist in our libc.
- (substitute* "tests/ts/column/invalid-multibyte"
- (("C\\.UTF-8") "en_US.utf8")))))
+ services)))))
(add-before 'check 'disable-setarch-test
(lambda _
;; The setarch tests are unreliable in QEMU's user-mode
diff --git a/guix/build-system/gnu.scm b/guix/build-system/gnu.scm
index ed1dc33c76..0f886fe21d 100644
--- a/guix/build-system/gnu.scm
+++ b/guix/build-system/gnu.scm
@@ -361,7 +361,7 @@ (define* (gnu-build name inputs
(make-dynamic-linker-cache? #t)
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
@@ -501,7 +501,7 @@ (define* (gnu-cross-build name
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
index ef5873d793..39707e7ace 100644
--- a/guix/build/gnu-build-system.scm
+++ b/guix/build/gnu-build-system.scm
@@ -123,7 +123,7 @@ (define* (set-paths #:key target inputs native-inputs
native-search-paths)))
(define* (install-locale #:key
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(locale-category LC_ALL)
#:allow-other-keys)
"Try to install LOCALE; emit a warning if that fails. The main goal is to
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 3/9] gnu: commencement: Remo ve ‘glibc-utf8-locales’ from ‘%fi nal-inputs’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
8f609fdf9e20d9140f149c8255b92334fae84f62.1702916397.git.ludo@gnu.org
* gnu/packages/commencement.scm (%boot5-inputs): Make an alias for
‘%boot4-inputs’.
(with-boot5): Turn into an alias for ‘with-boot4’.
(%final-inputs): Override ‘native-inputs’ of GAWK. Remove "locales"
entry.
(canonical-package): Add “glibc-utf8-locales” entry.
* gnu/packages/base.scm (libc-utf8-locales-for-target): Use
‘canonical-package’.
* gnu/packages/gawk.scm (gawk)[native-inputs]: New field.
* gnu/packages/texinfo.scm (texinfo)[native-inputs]: Add
‘libc-utf8-locales-for-target’.

Change-Id: I2283ffc6c6000c98ad2db89c04d581e050667c94
---
gnu/packages/base.scm | 11 ++++++++---
gnu/packages/commencement.scm | 31 +++++++++++++++----------------
gnu/packages/gawk.scm | 6 +++++-
gnu/packages/texinfo.scm | 7 +++++--
4 files changed, 33 insertions(+), 22 deletions(-)

Toggle diff (136 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 235ad7fecc..d31c0b0772 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1593,9 +1593,14 @@ (define* (libc-locales-for-target #:optional
(define* (libc-utf8-locales-for-target #:optional
(target (or (%current-target-system)
(%current-system))))
- (if (target-hurd? target)
- glibc-utf8-locales/hurd
- glibc-utf8-locales))
+ "Return the glibc UTF-8 locale package for TARGET."
+ ;; Note: To avoid circular dependencies (such as: texinfo ->
+ ;; glibc-utf8-locales -> glibc -> texinfo), refer to
+ ;; 'glibc-utf8-locales-final' via 'canonical-package'.
+ (canonical-package
+ (if (target-hurd? target)
+ glibc-utf8-locales/hurd
+ glibc-utf8-locales)))
(define-public tzdata
(package
diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 988e497ee7..f4882f7cdd 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -3394,17 +3394,11 @@ (define-public ld-gold-wrapper
#:guile guile-final
#:bash bash-final))
-(define (%boot5-inputs)
- ;; Now with UTF-8 locales. Remember that the bootstrap binaries were built
- ;; with an older libc, which cannot load the new locale format. See
- ;; <https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html>.
- `(("locales" ,(if (target-hurd?)
- glibc-utf8-locales-final/hurd
- glibc-utf8-locales-final))
- ,@(%boot4-inputs)))
-
-(define with-boot5
- (package-with-explicit-inputs %boot5-inputs))
+;; There used to be a "stage 5" including a variant of the
+;; 'glibc-utf8-locales' package. This is no longer necessary since 'glibc'
+;; embeds the "C.UTF-8" locale, but these aliases are kept for convenience.
+(define %boot5-inputs %boot4-inputs)
+(define with-boot5 with-boot4)
(define gnu-make-final
;; The final GNU Make, which uses the final Guile.
@@ -3493,7 +3487,11 @@ (define-public %final-inputs
("diffutils" ,diffutils)
("patch" ,patch)
("findutils" ,findutils)
- ("gawk" ,gawk)))
+ ("gawk" ,(package/inherit gawk
+ (native-inputs
+ (list (if (target-hurd?)
+ glibc-utf8-locales-final/hurd
+ glibc-utf8-locales-final)))))))
("sed" ,sed-final)
("grep" ,grep-final)
("xz" ,xz-final)
@@ -3504,10 +3502,7 @@ (define-public %final-inputs
("binutils" ,binutils-final)
("gcc" ,gcc-final)
("libc" ,glibc-final)
- ("libc:static" ,glibc-final "static")
- ("locales" ,(if (target-hurd? (%current-system))
- glibc-utf8-locales-final/hurd
- glibc-utf8-locales-final)))))))
+ ("libc:static" ,glibc-final "static"))))))
(define-public canonical-package
(let ((name->package (mlambda (system)
@@ -3518,6 +3513,10 @@ (define-public canonical-package
package result))))
vlist-null
`(("guile" ,guile-final)
+ ("glibc-utf8-locales"
+ ,(if (target-hurd? system)
+ glibc-utf8-locales-final/hurd
+ glibc-utf8-locales-final))
,@(%final-inputs system))))))
(lambda (package)
"Return the 'canonical' variant of PACKAGE---i.e., if PACKAGE is one of
diff --git a/gnu/packages/gawk.scm b/gnu/packages/gawk.scm
index 2d8bc581ef..476865ada3 100644
--- a/gnu/packages/gawk.scm
+++ b/gnu/packages/gawk.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2016, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2018, 2022, 2023 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2021, 2022 Marius Bakke <marius@gnu.org>
@@ -79,6 +79,10 @@ (define-public gawk
(("#! /usr/bin/gawk")
(string-append "#!" (which "gawk")))))))))
+ ;; Tests explicitly require en_US.UTF-8, fr_FR.UTF-8, ru_RU.UTF-8 and
+ ;; ja_JP.UTF-8.
+ (native-inputs (list (libc-utf8-locales-for-target)))
+
(inputs (list libsigsegv
;; Use the full-fledged Bash package, otherwise the test suite
;; sometimes fail non-deterministically.
diff --git a/gnu/packages/texinfo.scm b/gnu/packages/texinfo.scm
index ab4583071e..92c9d800a9 100644
--- a/gnu/packages/texinfo.scm
+++ b/gnu/packages/texinfo.scm
@@ -38,6 +38,7 @@ (define-module (gnu packages texinfo)
#:use-module (guix build-system gnu)
#:use-module (gnu packages)
#:use-module (gnu packages autotools)
+ #:use-module (gnu packages base)
#:use-module (gnu packages compression)
#:use-module (gnu packages gettext)
#:use-module (gnu packages ncurses)
@@ -79,10 +80,12 @@ (define-public texinfo
#:tests? ,(and (not (target-hurd?))
(not (%current-target-system)))))
(inputs (list ncurses perl))
+
;; When cross-compiling, texinfo will build some of its own binaries with
;; the native compiler. This means ncurses is needed both in both inputs
- ;; and native-inputs.
- (native-inputs (list perl ncurses))
+ ;; and native-inputs. Some of its tests require extra locales such as
+ ;; fr_FR.UTF-8.
+ (native-inputs (list perl ncurses (libc-utf8-locales-for-target)))
(native-search-paths
;; This is the variable used by the standalone Info reader.
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 4/9] profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
bd105b2312e12569bb657b24ef8f9f6e4fb4521b.1702916397.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/profiles.scm (ca-certificate-bundle)[libc-utf8-locales-for-target]:
Remove.
[build]: Use “C.UTF-8” instead of “en_US.utf8”.
(profile-derivation)[libc-utf8-locales-for-target]: Remove.
[set-utf8-locale]: Rewrite to a gexp that calls ‘setlocale’ for “C.UTF-8”.
* tests/profiles.scm ("profile-derivation, cross-compilation"): Remove
‘locales’ variable and related check.

Change-Id: I7cb148b9149fe5fbe5b9b1b25fdce1002ad1f37e
---
guix/profiles.scm | 33 +++++----------------------------
tests/profiles.scm | 7 +------
2 files changed, 6 insertions(+), 34 deletions(-)

Toggle diff (97 lines)
diff --git a/guix/profiles.scm b/guix/profiles.scm
index ce2f8337bf..da7790d819 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -1125,11 +1125,6 @@ (define* (ca-certificate-bundle manifest #:optional system)
MANIFEST. Single-file bundles are required by programs such as Git and Lynx."
;; See <http://lists.gnu.org/archive/html/guix-devel/2015-02/msg00429.html>
;; for a discussion.
-
- (define libc-utf8-locales-for-target ;lazy reference
- (module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))
-
(define build
(with-imported-modules '((guix build utils))
#~(begin
@@ -1161,13 +1156,7 @@ (define* (ca-certificate-bundle manifest #:optional system)
;; Some file names in the NSS certificates are UTF-8 encoded so
;; install a UTF-8 locale.
- (setenv "LOCPATH"
- (string-append #+(libc-utf8-locales-for-target system)
- "/lib/locale/"
- #+(version-major+minor
- (package-version
- (libc-utf8-locales-for-target system)))))
- (setlocale LC_ALL "en_US.utf8")
+ (setlocale LC_ALL "C.UTF-8")
(match (append-map ca-files '#$(manifest-inputs manifest))
(()
@@ -1960,8 +1949,7 @@ (define* (profile-derivation manifest
or TARGET is set, raise an error if MANIFEST contains a package that does not
support SYSTEM.
-When LOCALES? is true, the build is performed under a UTF-8 locale; this adds
-a dependency on the 'glibc-utf8-locales' package.
+When LOCALES? is true, the build is performed under a UTF-8 locale.
When RELATIVE-SYMLINKS? is true, use relative file names for symlink targets.
This is one of the things to do for the result to be relocatable.
@@ -2004,21 +1992,10 @@ (define* (profile-derivation manifest
(and (derivation? drv) (gexp-input drv)))
extras))
- (define libc-utf8-locales-for-target ;lazy reference
- (module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))
-
(define set-utf8-locale
- ;; Some file names (e.g., in 'nss-certs') are UTF-8 encoded so
- ;; install a UTF-8 locale.
- (let ((locales (libc-utf8-locales-for-target
- (or system (%current-system)))))
- #~(begin
- (setenv "LOCPATH"
- #$(file-append locales "/lib/locale/"
- (version-major+minor
- (package-version locales))))
- (setlocale LC_ALL "en_US.utf8"))))
+ ;; Some file names (e.g., in 'nss-certs') are UTF-8 encoded so install a
+ ;; UTF-8 locale. Assume libc comes with a copy of C.UTF-8.
+ #~(setlocale LC_ALL "C.UTF-8"))
(define builder
(with-imported-modules '((guix build profiles)
diff --git a/tests/profiles.scm b/tests/profiles.scm
index ddd6d74f3b..e448137cff 100644
--- a/tests/profiles.scm
+++ b/tests/profiles.scm
@@ -463,7 +463,6 @@ (define glibc
(target -> "arm-linux-gnueabihf")
(grep (package->cross-derivation packages:grep target))
(sed (package->cross-derivation packages:sed target))
- (locales (package->derivation (packages:libc-utf8-locales-for-target)))
(drv (profile-derivation manifest
#:hooks '()
#:locales? #t
@@ -475,15 +474,11 @@ (define glibc
(and (string-suffix? name input) input)))
(derivation-inputs drv))))
- ;; The inputs for grep and sed should be cross-build derivations, but that
- ;; for the glibc-utf8-locales should be a native build.
(return (and (string=? (derivation-system drv) (%current-system))
(string=? (find-input packages:grep)
(derivation-file-name grep))
(string=? (find-input packages:sed)
- (derivation-file-name sed))
- (string=? (find-input (packages:libc-utf8-locales-for-target))
- (derivation-file-name locales))))))
+ (derivation-file-name sed))))))
(test-assert "package->manifest-entry defaults to \"out\""
(let ((outputs (package-outputs packages:glibc)))
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 5/9] packages: Output and error ports are line-buffered in ‘patch-and-r epack’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
568767c79a40cf44a01fe9b57372defce13ec620.1702916397.git.ludo@gnu.org
* guix/packages.scm (patch-and-repack): Add ‘setvbuf’ calls.

Change-Id: I039bb6407263d5172bf0bc716bda6860dc2615fb
---
guix/packages.scm | 5 +++++
1 file changed, 5 insertions(+)

Toggle diff (18 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 4b0c478ff4..cb8db925f8 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -1007,6 +1007,11 @@ (define* (patch-and-repack source patches
'("--no-recursion"
"--files-from=.file_list"))))
+ (let ((line (cond-expand (guile-2.0 _IOLBF)
+ (else 'line))))
+ (setvbuf (current-output-port) line)
+ (setvbuf (current-error-port) line))
+
;; Encoding/decoding errors shouldn't be silent.
(fluid-set! %default-port-conversion-strategy 'error)
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 6/9] gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
7f7ca40f6236f1456cdc5604549a60f8070d6514.1702916397.git.ludo@gnu.org
* gnu/packages/base.scm (%default-utf8-locales): Add "C".
(make-glibc-utf8-locales): Delete "C" from LOCALES for glibc versions
lower than or equal to 2.35.

Change-Id: I6a6faa68dc18bf0f87c372109a17f476d3561278
---
gnu/packages/base.scm | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index d31c0b0772..eab9ff411d 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1396,7 +1396,8 @@ (define-public (make-glibc-locales glibc)
(define %default-utf8-locales
;; These are the locales commonly used for tests---e.g., in Guile's i18n
;; tests.
- '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+ '("C" "de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+
(define*-public (make-glibc-utf8-locales glibc #:key
(locales %default-utf8-locales)
(name "glibc-utf8-locales"))
@@ -1444,7 +1445,14 @@ (define*-public (make-glibc-utf8-locales glibc #:key
(symlink (string-append locale ".utf8")
(string-append localedir "/"
locale ".UTF-8")))
- '#$locales)))))
+
+ ;; The C.UTF-8 locale was introduced in 2.35 but it
+ ;; fails to build there:
+ ;; <https://sourceware.org/bugzilla/show_bug.cgi?id=28861>.
+ '#$(if (version>? (package-version this-package)
+ "2.35")
+ locales
+ (delete "C" locales)))))))
(native-inputs (list glibc gzip))
(synopsis (if default-locales?
(P_ "Small sample of UTF-8 locales")
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 7/9] gnu: glibc: Add patch for ‘ucontext’ on x86_64-gnu (GNU/ Hurd).
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
2f3b64b9d967b4eea5cbdb32c859f4e3ac3b1a83.1702916397.git.ludo@gnu.org
Suggested by Sergey Bugaev.

* gnu/packages/patches/glibc-2.38-hurd-ucontext.patch: New file.
* gnu/packages/base.scm (glibc)[source]: Use it.
* gnu/local.mk (dist_patch_DATA): Add it.

Change-Id: Ib74e93f2414eda2ec4b3e01fc2bb67b02838fabb
---
gnu/local.mk | 1 +
gnu/packages/base.scm | 1 +
.../patches/glibc-2.38-hurd-ucontext.patch | 87 +++++++++++++++++++
3 files changed, 89 insertions(+)
create mode 100644 gnu/packages/patches/glibc-2.38-hurd-ucontext.patch

Toggle diff (119 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index ff2f82a23a..e92cf871ad 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1320,6 +1320,7 @@ dist_patch_DATA = \
%D%/packages/patches/glibc-2.37-hurd-clock_t_centiseconds.patch \
%D%/packages/patches/glibc-2.37-hurd-local-clock_gettime_MONOTONIC.patch \
%D%/packages/patches/glibc-2.37-versioned-locpath.patch \
+ %D%/packages/patches/glibc-2.38-hurd-ucontext.patch \
%D%/packages/patches/glibc-2.38-ldd-x86_64.patch \
%D%/packages/patches/glibc-hurd-clock_t_centiseconds.patch \
%D%/packages/patches/glibc-hurd-getauxval.patch \
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index eab9ff411d..e2a952cc42 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -813,6 +813,7 @@ (define-public glibc
"glibc-supported-locales.patch"
"glibc-2.37-hurd-clock_t_centiseconds.patch"
"glibc-2.37-hurd-local-clock_gettime_MONOTONIC.patch"
+ "glibc-2.38-hurd-ucontext.patch"
"glibc-hurd-mach-print.patch"
"glibc-hurd-gettyent.patch"
"glibc-hurd-getauxval.patch"))))
diff --git a/gnu/packages/patches/glibc-2.38-hurd-ucontext.patch b/gnu/packages/patches/glibc-2.38-hurd-ucontext.patch
new file mode 100644
index 0000000000..03a6132666
--- /dev/null
+++ b/gnu/packages/patches/glibc-2.38-hurd-ucontext.patch
@@ -0,0 +1,87 @@
+commit 29d4591b07a4da53320e949557c6946c62c26bde
+Author: Samuel Thibault <samuel.thibault@ens-lyon.org>
+Date: Thu Sep 28 00:07:18 2023 +0200
+
+ hurd: Drop REG_GSFS and REG_ESDS from x86_64's ucontext
+
+ These are useless on x86_64, and __NGREG was actually wrong with them.
+
+diff --git a/sysdeps/mach/hurd/x86/trampoline.c b/sysdeps/mach/hurd/x86/trampoline.c
+index 6318c9528a..bc3f15e1e0 100644
+--- a/sysdeps/mach/hurd/x86/trampoline.c
++++ b/sysdeps/mach/hurd/x86/trampoline.c
+@@ -79,8 +79,8 @@ static void fill_ucontext (ucontext_t *uc, const struct sigcontext *sc)
+
+ /* Registers. */
+ #ifdef __x86_64__
+- memcpy (&uc->uc_mcontext.gregs[REG_GSFS], &sc->sc_gs,
+- (REG_ERR - REG_GSFS) * sizeof (long));
++ memcpy (&uc->uc_mcontext.gregs[REG_R8], &sc->sc_r8,
++ (REG_ERR - REG_R8) * sizeof (long));
+ #else
+ memcpy (&uc->uc_mcontext.gregs[REG_GS], &sc->sc_gs,
+ (REG_TRAPNO - REG_GS) * sizeof (int));
+diff --git a/sysdeps/mach/hurd/x86_64/bits/sigcontext.h b/sysdeps/mach/hurd/x86_64/bits/sigcontext.h
+index 6396054463..7facc587b8 100644
+--- a/sysdeps/mach/hurd/x86_64/bits/sigcontext.h
++++ b/sysdeps/mach/hurd/x86_64/bits/sigcontext.h
+@@ -59,13 +59,7 @@ struct sigcontext
+ }
+ trampoline.c knows this, so it must be changed if this changes. */
+
+-#define sc_i386_thread_state sc_gs /* Beginning of correspondence. */
+- /* Segment registers. */
+- int sc_gs;
+- int sc_fs;
+- int sc_es;
+- int sc_ds;
+-
++#define sc_i386_thread_state sc_r8 /* Beginning of correspondence. */
+ long sc_r8;
+ long sc_r9;
+ long sc_r10;
+diff --git a/sysdeps/mach/x86/thread_state.h b/sysdeps/mach/x86/thread_state.h
+index 8c419515f9..e237e46cb2 100644
+--- a/sysdeps/mach/x86/thread_state.h
++++ b/sysdeps/mach/x86/thread_state.h
+@@ -34,12 +34,13 @@
+ #define PC rip
+ #define SP ursp
+ #define SYSRETURN rax
++#define MACHINE_THREAD_STATE_FIX_NEW(ts) do { \
++ asm ("mov %%cs, %w0" : "=q" ((ts)->cs)); \
++} while(0)
+ #else
+ #define PC eip
+ #define SP uesp
+ #define SYSRETURN eax
+-#endif
+-
+ #define MACHINE_THREAD_STATE_FIX_NEW(ts) do { \
+ asm ("mov %%cs, %w0" : "=q" ((ts)->cs)); \
+ asm ("mov %%ds, %w0" : "=q" ((ts)->ds)); \
+@@ -47,6 +48,7 @@
+ asm ("mov %%fs, %w0" : "=q" ((ts)->fs)); \
+ asm ("mov %%gs, %w0" : "=q" ((ts)->gs)); \
+ } while(0)
++#endif
+
+ struct machine_thread_all_state
+ {
+diff --git a/sysdeps/x86_64/sys/ucontext.h b/sysdeps/x86_64/sys/ucontext.h
+index d73a893795..f1b6be77a7 100644
+--- a/sysdeps/x86_64/sys/ucontext.h
++++ b/sysdeps/x86_64/sys/ucontext.h
+@@ -47,11 +47,7 @@ typedef greg_t gregset_t[__NGREG];
+ /* Number of each register in the `gregset_t' array. */
+ enum
+ {
+- REG_GSFS = 0, /* Actually int gs, fs. */
+-# define REG_GSFS REG_GSFS
+- REG_ESDS, /* Actually int es, ds. */
+-# define REG_ESDS REG_ESDS
+- REG_R8,
++ REG_R8 = 0,
+ # define REG_R8 REG_R8
+ REG_R9,
+ # define REG_R9 REG_R9
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 9/9] gnu: glib: Skip test that fails with glibc 2.38.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
08c866533e2a76bcf91149208c1323ecbfcff4cd.1702916397.git.ludo@gnu.org
* gnu/packages/glib.scm (glib)[arguments]: In ‘disable-failing-tests’
phase, substitute “glib/tests/error.c”.

Change-Id: I9e3f5f1f5f55d08c83fbde3b79931987df27ebdf
---
gnu/packages/glib.scm | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/glib.scm b/gnu/packages/glib.scm
index dc8cf5800b..13f957f810 100644
--- a/gnu/packages/glib.scm
+++ b/gnu/packages/glib.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2013, 2014, 2015, 2016, 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2013, 2014, 2015, 2016, 2019, 2020, 2021, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2013, 2015 Andreas Enge <andreas@enge.fr>
;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
;;; Copyright © 2014, 2015, 2016, 2017, 2018, 2021 Mark H Weaver <mhw@netris.org>
@@ -275,6 +275,10 @@ (define glib
(substitute* '("contenttype.c" "gdbus-address-get-session.c"
"gdbus-peer.c" "appinfo.c" "desktop-app-info.c")
(("[ \t]*g_test_add_func.*;") "")))
+ (substitute* "glib/tests/error.c"
+ ;; This test segfaults with glibc 2.38.
+ (("g_test_add_func.*new-valist/invalid.*" all)
+ (string-append "//" all "\n")))
#$@(if (target-x86-32?)
;; Comment out parts of timer.c that fail on i686 due to
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 8/9] build-system/gnu: Turn PID 1 into an “init”-style process by default.
(address . 67824@debbugs.gnu.org)
d5497ca9b8d069d31dc905ac2aedddcff2614792.1702916397.git.ludo@gnu.org

* guix/build/gnu-build-system.scm (separate-from-pid1): New procedure.
(%standard-phases): Add it.
* guix/build-system/gnu.scm (gnu-build): Add #:separate-from-pid1? and
honor it.
(gnu-cross-build): Likewise.

Reported-by: Carlo Zancanaro <carlo@zancanaro.id.au>
Change-Id: I6f3bc8d8186d1a571f983a38d5e3fd178ffa2678
---
guix/build-system/gnu.scm | 4 ++++
guix/build/gnu-build-system.scm | 39 ++++++++++++++++++++++++++++++++-
2 files changed, 42 insertions(+), 1 deletion(-)

Toggle diff (95 lines)
diff --git a/guix/build-system/gnu.scm b/guix/build-system/gnu.scm
index 0f886fe21d..6a89bcc0d8 100644
--- a/guix/build-system/gnu.scm
+++ b/guix/build-system/gnu.scm
@@ -362,6 +362,7 @@ (define* (gnu-build name inputs
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
(locale "C.UTF-8")
+ (separate-from-pid1? #t)
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
@@ -404,6 +405,7 @@ (define* (gnu-build name inputs
(sexp->gexp phases)
phases)
#:locale #$locale
+ #:separate-from-pid1? #$separate-from-pid1?
#:bootstrap-scripts #$bootstrap-scripts
#:configure-flags #$(if (pair? configure-flags)
(sexp->gexp configure-flags)
@@ -502,6 +504,7 @@ (define* (gnu-cross-build name
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
(locale "C.UTF-8")
+ (separate-from-pid1? #t)
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
@@ -547,6 +550,7 @@ (define* (gnu-cross-build name
(sexp->gexp phases)
phases)
#:locale #$locale
+ #:separate-from-pid1? #$separate-from-pid1?
#:bootstrap-scripts #$bootstrap-scripts
#:configure-flags #$configure-flags
#:make-flags #$make-flags
diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
index 39707e7ace..51b8f9acbf 100644
--- a/guix/build/gnu-build-system.scm
+++ b/guix/build/gnu-build-system.scm
@@ -72,6 +72,42 @@ (define (first-subdirectory directory)
((first . _) first)
(_ #f)))
+(define* (separate-from-pid1 #:key (separate-from-pid1? #t)
+ #:allow-other-keys)
+ "When running as PID 1 and SEPARATE-FROM-PID1? is true, run build phases as
+a child process; PID 1 then becomes responsible for reaping child processes."
+ (if separate-from-pid1?
+ (if (= 1 (getpid))
+ (dynamic-wind
+ (const #t)
+ (lambda ()
+ (match (primitive-fork)
+ (0 #t)
+ (builder-pid
+ (format (current-error-port)
+ "build process now running as PID ~a~%"
+ builder-pid)
+ (let loop ()
+ ;; Running as PID 1 so take responsibility for reaping
+ ;; child processes.
+ (match (waitpid WAIT_ANY)
+ ((pid . status)
+ (if (= pid builder-pid)
+ (if (zero? status)
+ (primitive-exit 0)
+ (begin
+ (format (current-error-port)
+ "build process ~a exited with status ~a~%"
+ pid status)
+ (primitive-exit 1)))
+ (loop))))))))
+ (const #t))
+ (format (current-error-port) "not running as PID 1 (PID: ~a)~%"
+ (getpid)))
+ (format (current-error-port)
+ "build process running as PID ~a; not forking~%"
+ (getpid))))
+
(define* (set-paths #:key target inputs native-inputs
(search-paths '()) (native-search-paths '())
#:allow-other-keys)
@@ -872,7 +908,8 @@ (define %standard-phases
;; Standard build phases, as a list of symbol/procedure pairs.
(let-syntax ((phases (syntax-rules ()
((_ p ...) `((p . ,p) ...)))))
- (phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
+ (phases separate-from-pid1
+ set-SOURCE-DATE-EPOCH set-paths install-locale unpack
bootstrap
patch-usr-bin-file
patch-source-shebangs configure patch-generated-file-shebangs
--
2.41.0
L
L
Ludovic Courtès wrote on 19 Dec 2023 23:54
Re: [bug#67824] [PATCH core-updates v2 0/9] Run builds in C.UTF-8 locale
(address . 67824-done@debbugs.gnu.org)
875y0tiz0t.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (12 lines)
> packages: Remove reference to ‘glibc-utf8-locales’ in
> ‘patch-and-repack’.
> build-system/gnu: Change default locale to C.UTF-8.
> gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%final-inputs’.
> profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
> packages: Output and error ports are line-buffered in
> ‘patch-and-repack’.
> gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
> gnu: glibc: Add patch for ‘ucontext’ on x86_64-gnu (GNU/Hurd).
> build-system/gnu: Turn PID 1 into an “init”-style process by default.
> gnu: glib: Skip test that fails with glibc 2.38.

Pushed v2 as 62e67aa7994f40c438ef5a528675e85699d7af76.

Now to see what ci.guix thinks about it…

Ludo’.
Closed
J
J
Janneke Nieuwenhuizen wrote on 20 Dec 2023 08:20
(name . Ludovic Courtès)(address . ludo@gnu.org)
874jgdpcfb.fsf@gnu.org
Ludovic Courtès writes:

Toggle quote (18 lines)
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> packages: Remove reference to ‘glibc-utf8-locales’ in
>> ‘patch-and-repack’.
>> build-system/gnu: Change default locale to C.UTF-8.
>> gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%final-inputs’.
>> profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
>> packages: Output and error ports are line-buffered in
>> ‘patch-and-repack’.
>> gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
>> gnu: glibc: Add patch for ‘ucontext’ on x86_64-gnu (GNU/Hurd).
>> build-system/gnu: Turn PID 1 into an “init”-style process by default.
>> gnu: glib: Skip test that fails with glibc 2.38.
>
> Pushed v2 as 62e67aa7994f40c438ef5a528675e85699d7af76.
>
> Now to see what ci.guix thinks about it…

+1

Haven't built everyting just yet, but it's looking good.
Thank you!

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
Closed
?