[PATCH core-updates 0/6] Run builds in C.UTF-8 locale

  • Done
  • quality assurance status badge
Details
2 participants
  • Janneke Nieuwenhuizen
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 14 Dec 2023 11:51
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
cover.1702550735.git.ludo@gnu.org
Hello,

This is the logical followup to https://issues.guix.gnu.org/67686:
running builds in the C.UTF-8 locale (instead of en_US.UTF-8) and
removing ‘glibc-utf8-locales’ from the set of implicit inputs.

Tested on x86_64-linux with:

guix build texinfo nss-certs
make check TESTS=tests/profiles.scm

I’d like to merge it soon so we can move forward.

Let me know what you think!

Ludo’.

Ludovic Courtès (6):
packages: Remove reference to ‘glibc-utf8-locales’ in
‘patch-and-repack’.
build-system/gnu: Change default locale to C.UTF-8.
gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%final-inputs’.
profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
packages: Output and error ports are line-buffered in
‘patch-and-repack’.
gnu: glibc-utf8-locales: Add the C.UTF-8 locale.

gnu/packages/base.scm | 23 ++++++++++++++++++-----
gnu/packages/commencement.scm | 31 +++++++++++++++----------------
gnu/packages/gawk.scm | 6 +++++-
gnu/packages/texinfo.scm | 7 +++++--
guix/build-system/gnu.scm | 4 ++--
guix/packages.scm | 34 +++++++++++++++++-----------------
guix/profiles.scm | 33 +++++----------------------------
tests/profiles.scm | 7 +------
8 files changed, 68 insertions(+), 77 deletions(-)


base-commit: e4013b31266b40dc9cb250c44b2088d3931cb3b3
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 1/6] packages: Remove reference to ‘glibc-utf8-locales’ in ‘pa tch-and-repack’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
eee58768e5b7514c836ea4b12abe61850205a359.1702550735.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/packages.scm (%standard-patch-inputs): Remove “locales” entry.
(patch-and-repack): Remove ‘locales’ variable.
[build]: Remove reference to ‘locales’. Use “C.UTF-8” unconditionally.

Change-Id: I54219b232880e05534599fbba8738cc00fb5d568
---
guix/packages.scm | 29 ++++++++++++-----------------
1 file changed, 12 insertions(+), 17 deletions(-)

Toggle diff (56 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 930b1a3b0e..4b0c478ff4 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -866,13 +866,7 @@ (define (%standard-patch-inputs system)
("gzip" ,(ref '(gnu packages compression) 'gzip))
("lzip" ,(ref '(gnu packages compression) 'lzip))
("unzip" ,(ref '(gnu packages compression) 'unzip))
- ("patch" ,(ref '(gnu packages base) 'patch))
- ("locales"
- ,(parameterize ((%current-target-system #f)
- (%current-system system))
- (canonical
- ((module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))))))))
+ ("patch" ,(ref '(gnu packages base) 'patch)))))
(define (default-guile)
"Return the default Guile package used to run the build code of
@@ -955,7 +949,6 @@ (define* (patch-and-repack source patches
(lzip (lookup-input "lzip"))
(xz (lookup-input "xz"))
(patch (lookup-input "patch"))
- (locales (lookup-input "locales"))
(comp (and=> (compressor source-file-name) lookup-input))
(patches (map instantiate-patch patches)))
(define build
@@ -1017,15 +1010,17 @@ (define* (patch-and-repack source patches
;; Encoding/decoding errors shouldn't be silent.
(fluid-set! %default-port-conversion-strategy 'error)
- (when #+locales
- ;; First of all, install a UTF-8 locale so that UTF-8 file names
- ;; are correctly interpreted. During bootstrap, LOCALES is #f.
- (setenv "LOCPATH"
- (string-append #+locales "/lib/locale/"
- #+(and locales
- (version-major+minor
- (package-version locales)))))
- (setlocale LC_ALL "en_US.utf8"))
+ ;; First of all, install a UTF-8 locale so that UTF-8 file names
+ ;; are correctly interpreted. During bootstrap, locales are
+ ;; missing.
+ (let ((locale "C.UTF-8"))
+ (catch 'system-error
+ (lambda ()
+ (setlocale LC_ALL locale))
+ (lambda args
+ (format (current-error-port)
+ "failed to install '~a' locale: ~a~%"
+ locale (system-error-errno args)))))
(setenv "PATH"
(string-append #+xz "/bin"
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 2/6] build-system/gnu: Change default locale to C.UTF-8.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
1df36313ccd91f5249261ae14d222658d5564183.1702550735.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): #:locale now
defaults to "C.UTF-8".

Change-Id: I9a4bfe564bcd8bbd0d57ba04568eb6b8020bd051
---
guix/build-system/gnu.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/guix/build-system/gnu.scm b/guix/build-system/gnu.scm
index ed1dc33c76..0f886fe21d 100644
--- a/guix/build-system/gnu.scm
+++ b/guix/build-system/gnu.scm
@@ -361,7 +361,7 @@ (define* (gnu-build name inputs
(make-dynamic-linker-cache? #t)
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
@@ -501,7 +501,7 @@ (define* (gnu-cross-build name
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 3/6] gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%fina l-inputs’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
a65975f7dcf62d4d629fae5ca1cfb43e9ace4488.1702550735.git.ludo@gnu.org
* gnu/packages/commencement.scm (%boot5-inputs): Make an alias for
‘%boot4-inputs’.
(with-boot5): Turn into an alias for ‘with-boot4’.
(%final-inputs): Override ‘native-inputs’ of GAWK. Remove "locales"
entry.
(canonical-package): Add “glibc-utf8-locales” entry.
* gnu/packages/base.scm (libc-utf8-locales-for-target): Use
‘canonical-package’.
* gnu/packages/gawk.scm (gawk)[native-inputs]: New field.
* gnu/packages/texinfo.scm (texinfo)[native-inputs]: Add
‘libc-utf8-locales-for-target’.

Change-Id: I2283ffc6c6000c98ad2db89c04d581e050667c94
---
gnu/packages/base.scm | 11 ++++++++---
gnu/packages/commencement.scm | 31 +++++++++++++++----------------
gnu/packages/gawk.scm | 6 +++++-
gnu/packages/texinfo.scm | 7 +++++--
4 files changed, 33 insertions(+), 22 deletions(-)

Toggle diff (136 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 235ad7fecc..d31c0b0772 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1593,9 +1593,14 @@ (define* (libc-locales-for-target #:optional
(define* (libc-utf8-locales-for-target #:optional
(target (or (%current-target-system)
(%current-system))))
- (if (target-hurd? target)
- glibc-utf8-locales/hurd
- glibc-utf8-locales))
+ "Return the glibc UTF-8 locale package for TARGET."
+ ;; Note: To avoid circular dependencies (such as: texinfo ->
+ ;; glibc-utf8-locales -> glibc -> texinfo), refer to
+ ;; 'glibc-utf8-locales-final' via 'canonical-package'.
+ (canonical-package
+ (if (target-hurd? target)
+ glibc-utf8-locales/hurd
+ glibc-utf8-locales)))
(define-public tzdata
(package
diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 988e497ee7..f4882f7cdd 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -3394,17 +3394,11 @@ (define-public ld-gold-wrapper
#:guile guile-final
#:bash bash-final))
-(define (%boot5-inputs)
- ;; Now with UTF-8 locales. Remember that the bootstrap binaries were built
- ;; with an older libc, which cannot load the new locale format. See
- ;; <https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html>.
- `(("locales" ,(if (target-hurd?)
- glibc-utf8-locales-final/hurd
- glibc-utf8-locales-final))
- ,@(%boot4-inputs)))
-
-(define with-boot5
- (package-with-explicit-inputs %boot5-inputs))
+;; There used to be a "stage 5" including a variant of the
+;; 'glibc-utf8-locales' package. This is no longer necessary since 'glibc'
+;; embeds the "C.UTF-8" locale, but these aliases are kept for convenience.
+(define %boot5-inputs %boot4-inputs)
+(define with-boot5 with-boot4)
(define gnu-make-final
;; The final GNU Make, which uses the final Guile.
@@ -3493,7 +3487,11 @@ (define-public %final-inputs
("diffutils" ,diffutils)
("patch" ,patch)
("findutils" ,findutils)
- ("gawk" ,gawk)))
+ ("gawk" ,(package/inherit gawk
+ (native-inputs
+ (list (if (target-hurd?)
+ glibc-utf8-locales-final/hurd
+ glibc-utf8-locales-final)))))))
("sed" ,sed-final)
("grep" ,grep-final)
("xz" ,xz-final)
@@ -3504,10 +3502,7 @@ (define-public %final-inputs
("binutils" ,binutils-final)
("gcc" ,gcc-final)
("libc" ,glibc-final)
- ("libc:static" ,glibc-final "static")
- ("locales" ,(if (target-hurd? (%current-system))
- glibc-utf8-locales-final/hurd
- glibc-utf8-locales-final)))))))
+ ("libc:static" ,glibc-final "static"))))))
(define-public canonical-package
(let ((name->package (mlambda (system)
@@ -3518,6 +3513,10 @@ (define-public canonical-package
package result))))
vlist-null
`(("guile" ,guile-final)
+ ("glibc-utf8-locales"
+ ,(if (target-hurd? system)
+ glibc-utf8-locales-final/hurd
+ glibc-utf8-locales-final))
,@(%final-inputs system))))))
(lambda (package)
"Return the 'canonical' variant of PACKAGE---i.e., if PACKAGE is one of
diff --git a/gnu/packages/gawk.scm b/gnu/packages/gawk.scm
index 2d8bc581ef..476865ada3 100644
--- a/gnu/packages/gawk.scm
+++ b/gnu/packages/gawk.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2016, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2018, 2022, 2023 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2021, 2022 Marius Bakke <marius@gnu.org>
@@ -79,6 +79,10 @@ (define-public gawk
(("#! /usr/bin/gawk")
(string-append "#!" (which "gawk")))))))))
+ ;; Tests explicitly require en_US.UTF-8, fr_FR.UTF-8, ru_RU.UTF-8 and
+ ;; ja_JP.UTF-8.
+ (native-inputs (list (libc-utf8-locales-for-target)))
+
(inputs (list libsigsegv
;; Use the full-fledged Bash package, otherwise the test suite
;; sometimes fail non-deterministically.
diff --git a/gnu/packages/texinfo.scm b/gnu/packages/texinfo.scm
index ab4583071e..92c9d800a9 100644
--- a/gnu/packages/texinfo.scm
+++ b/gnu/packages/texinfo.scm
@@ -38,6 +38,7 @@ (define-module (gnu packages texinfo)
#:use-module (guix build-system gnu)
#:use-module (gnu packages)
#:use-module (gnu packages autotools)
+ #:use-module (gnu packages base)
#:use-module (gnu packages compression)
#:use-module (gnu packages gettext)
#:use-module (gnu packages ncurses)
@@ -79,10 +80,12 @@ (define-public texinfo
#:tests? ,(and (not (target-hurd?))
(not (%current-target-system)))))
(inputs (list ncurses perl))
+
;; When cross-compiling, texinfo will build some of its own binaries with
;; the native compiler. This means ncurses is needed both in both inputs
- ;; and native-inputs.
- (native-inputs (list perl ncurses))
+ ;; and native-inputs. Some of its tests require extra locales such as
+ ;; fr_FR.UTF-8.
+ (native-inputs (list perl ncurses (libc-utf8-locales-for-target)))
(native-search-paths
;; This is the variable used by the standalone Info reader.
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 4/6] profiles: Use C.UTF-8 in stead of ‘glibc-utf8-locales’ where pos sible.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
f40d9b9bbe08c279eab122d2bd29db2426a45731.1702550735.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/profiles.scm (ca-certificate-bundle)[libc-utf8-locales-for-target]:
Remove.
[build]: Use “C.UTF-8” instead of “en_US.utf8”.
(profile-derivation)[libc-utf8-locales-for-target]: Remove.
[set-utf8-locale]: Rewrite to a gexp that calls ‘setlocale’ for “C.UTF-8”.
* tests/profiles.scm ("profile-derivation, cross-compilation"): Remove
‘locales’ variable and related check.

Change-Id: I7cb148b9149fe5fbe5b9b1b25fdce1002ad1f37e
---
guix/profiles.scm | 33 +++++----------------------------
tests/profiles.scm | 7 +------
2 files changed, 6 insertions(+), 34 deletions(-)

Toggle diff (97 lines)
diff --git a/guix/profiles.scm b/guix/profiles.scm
index ce2f8337bf..da7790d819 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -1125,11 +1125,6 @@ (define* (ca-certificate-bundle manifest #:optional system)
MANIFEST. Single-file bundles are required by programs such as Git and Lynx."
;; See <http://lists.gnu.org/archive/html/guix-devel/2015-02/msg00429.html>
;; for a discussion.
-
- (define libc-utf8-locales-for-target ;lazy reference
- (module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))
-
(define build
(with-imported-modules '((guix build utils))
#~(begin
@@ -1161,13 +1156,7 @@ (define* (ca-certificate-bundle manifest #:optional system)
;; Some file names in the NSS certificates are UTF-8 encoded so
;; install a UTF-8 locale.
- (setenv "LOCPATH"
- (string-append #+(libc-utf8-locales-for-target system)
- "/lib/locale/"
- #+(version-major+minor
- (package-version
- (libc-utf8-locales-for-target system)))))
- (setlocale LC_ALL "en_US.utf8")
+ (setlocale LC_ALL "C.UTF-8")
(match (append-map ca-files '#$(manifest-inputs manifest))
(()
@@ -1960,8 +1949,7 @@ (define* (profile-derivation manifest
or TARGET is set, raise an error if MANIFEST contains a package that does not
support SYSTEM.
-When LOCALES? is true, the build is performed under a UTF-8 locale; this adds
-a dependency on the 'glibc-utf8-locales' package.
+When LOCALES? is true, the build is performed under a UTF-8 locale.
When RELATIVE-SYMLINKS? is true, use relative file names for symlink targets.
This is one of the things to do for the result to be relocatable.
@@ -2004,21 +1992,10 @@ (define* (profile-derivation manifest
(and (derivation? drv) (gexp-input drv)))
extras))
- (define libc-utf8-locales-for-target ;lazy reference
- (module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))
-
(define set-utf8-locale
- ;; Some file names (e.g., in 'nss-certs') are UTF-8 encoded so
- ;; install a UTF-8 locale.
- (let ((locales (libc-utf8-locales-for-target
- (or system (%current-system)))))
- #~(begin
- (setenv "LOCPATH"
- #$(file-append locales "/lib/locale/"
- (version-major+minor
- (package-version locales))))
- (setlocale LC_ALL "en_US.utf8"))))
+ ;; Some file names (e.g., in 'nss-certs') are UTF-8 encoded so install a
+ ;; UTF-8 locale. Assume libc comes with a copy of C.UTF-8.
+ #~(setlocale LC_ALL "C.UTF-8"))
(define builder
(with-imported-modules '((guix build profiles)
diff --git a/tests/profiles.scm b/tests/profiles.scm
index ddd6d74f3b..e448137cff 100644
--- a/tests/profiles.scm
+++ b/tests/profiles.scm
@@ -463,7 +463,6 @@ (define glibc
(target -> "arm-linux-gnueabihf")
(grep (package->cross-derivation packages:grep target))
(sed (package->cross-derivation packages:sed target))
- (locales (package->derivation (packages:libc-utf8-locales-for-target)))
(drv (profile-derivation manifest
#:hooks '()
#:locales? #t
@@ -475,15 +474,11 @@ (define glibc
(and (string-suffix? name input) input)))
(derivation-inputs drv))))
- ;; The inputs for grep and sed should be cross-build derivations, but that
- ;; for the glibc-utf8-locales should be a native build.
(return (and (string=? (derivation-system drv) (%current-system))
(string=? (find-input packages:grep)
(derivation-file-name grep))
(string=? (find-input packages:sed)
- (derivation-file-name sed))
- (string=? (find-input (packages:libc-utf8-locales-for-target))
- (derivation-file-name locales))))))
+ (derivation-file-name sed))))))
(test-assert "package->manifest-entry defaults to \"out\""
(let ((outputs (package-outputs packages:glibc)))
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 5/6] packages: Output and err or ports are line-buffered in ‘patch-and-repa ck’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
2533db50b4cf1f21baf85b63d2266055d53618a2.1702550735.git.ludo@gnu.org
* guix/packages.scm (patch-and-repack): Add ‘setvbuf’ calls.

Change-Id: I039bb6407263d5172bf0bc716bda6860dc2615fb
---
guix/packages.scm | 5 +++++
1 file changed, 5 insertions(+)

Toggle diff (18 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 4b0c478ff4..cb8db925f8 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -1007,6 +1007,11 @@ (define* (patch-and-repack source patches
'("--no-recursion"
"--files-from=.file_list"))))
+ (let ((line (cond-expand (guile-2.0 _IOLBF)
+ (else 'line))))
+ (setvbuf (current-output-port) line)
+ (setvbuf (current-error-port) line))
+
;; Encoding/decoding errors shouldn't be silent.
(fluid-set! %default-port-conversion-strategy 'error)
--
2.41.0
L
L
Ludovic Courtès wrote on 14 Dec 2023 14:37
[PATCH core-updates 6/6] gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
48fba13f955b3552c7f011672763052752cc7c35.1702550735.git.ludo@gnu.org
* gnu/packages/base.scm (%default-utf8-locales): Add "C".
(make-glibc-utf8-locales): Delete "C" from LOCALES for glibc versions
lower than or equal to 2.35.

Change-Id: I6a6faa68dc18bf0f87c372109a17f476d3561278
---
gnu/packages/base.scm | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index d31c0b0772..eab9ff411d 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1396,7 +1396,8 @@ (define-public (make-glibc-locales glibc)
(define %default-utf8-locales
;; These are the locales commonly used for tests---e.g., in Guile's i18n
;; tests.
- '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+ '("C" "de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+
(define*-public (make-glibc-utf8-locales glibc #:key
(locales %default-utf8-locales)
(name "glibc-utf8-locales"))
@@ -1444,7 +1445,14 @@ (define*-public (make-glibc-utf8-locales glibc #:key
(symlink (string-append locale ".utf8")
(string-append localedir "/"
locale ".UTF-8")))
- '#$locales)))))
+
+ ;; The C.UTF-8 locale was introduced in 2.35 but it
+ ;; fails to build there:
+ ;; <https://sourceware.org/bugzilla/show_bug.cgi?id=28861>.
+ '#$(if (version>? (package-version this-package)
+ "2.35")
+ locales
+ (delete "C" locales)))))))
(native-inputs (list glibc gzip))
(synopsis (if default-locales?
(P_ "Small sample of UTF-8 locales")
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 16 Dec 2023 09:49
Re: [bug#67824] [PATCH core-updates 0/6] Run builds in C.UTF-8 locale
(name . Ludovic Courtès)(address . ludo@gnu.org)
87h6kir0pp.fsf@gnu.org
Ludovic Courtès writes:

Hi,

Toggle quote (13 lines)
> This is the logical followup to https://issues.guix.gnu.org/67686:
> running builds in the C.UTF-8 locale (instead of en_US.UTF-8) and
> removing ‘glibc-utf8-locales’ from the set of implicit inputs.
>
> Tested on x86_64-linux with:
>
> guix build texinfo nss-certs
> make check TESTS=tests/profiles.scm
>
> I’d like to merge it soon so we can move forward.
>
> Let me know what you think!

I like the idea but I cannot build it, i.e., building gcc-mesboot1 hangs
for me in the gmp configure:

Toggle snippet (8 lines)
./pre-inst-env guix build '(@@ (gnu packages commencement) gcc-mesboot1)'
[..]
(Configuring in host-i686-unknown-linux-gnu/gmp)
checking whether sscanf needs writable input... no
checking for struct pst_processor.psp_iticksperclktick... no
<hang>

I've tried at least 6 times. Gash running configure just hangs.

On a related note, I found that running

Toggle snippet (3 lines)
./pre-inst-env guix shell --bootstrap --no-grafts -e '(@@ (gnu packages commencement) gcc-mesboot1)'

wants to build gcc-mesboot1 when attempting to easily create a shell in
which to run and debug the hanging configure command.

I guess someone needs to patch gcc-mesboot1 and dive into it.

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 16 Dec 2023 12:27
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
87h6kis7zo.fsf@gnu.org
Hi!

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (10 lines)
> I like the idea but I cannot build it, i.e., building gcc-mesboot1 hangs
> for me in the gmp configure:
>
> ./pre-inst-env guix build '(@@ (gnu packages commencement) gcc-mesboot1)'
> [..]
> (Configuring in host-i686-unknown-linux-gnu/gmp)
> checking whether sscanf needs writable input... no
> checking for struct pst_processor.psp_iticksperclktick... no
> <hang>

Hmm it works for me, weird.

Now I realize I forgot to change the default value of #:locale in
‘gnu-build-system.scm’ plus a couple of other things, so I’m starting
another round of rebuilds to test it.

It’s all about patience. :-)

Ludo’.
L
L
Ludovic Courtès wrote on 16 Dec 2023 17:47
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
874jgirt62.fsf@gnu.org
Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (13 lines)
> I like the idea but I cannot build it, i.e., building gcc-mesboot1 hangs
> for me in the gmp configure:
>
> ./pre-inst-env guix build '(@@ (gnu packages commencement) gcc-mesboot1)'
> [..]
> (Configuring in host-i686-unknown-linux-gnu/gmp)
> checking whether sscanf needs writable input... no
> checking for struct pst_processor.psp_iticksperclktick... no
> <hang>
>
>
> I've tried at least 6 times. Gash running configure just hangs.

I experienced a similar thing just now (I had removed #:parallel-build?
#f from ‘binutils-mesboot0’):

Toggle snippet (19 lines)
$ pstree -p 29600
.guile-real(29600)???conftest(45270)
??make(34404)???.guile-real(34406)???make(34421)???.guile-real(6216)???.guile-real(6247)???make(629+
??{.guile-real}(29610)
ludo@guix-hpc5 ~$ sudo cat /proc/29600/cmdline |xargs -0
/gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile --no-auto-compile -L /gnu/store/4zibvq0ffm75xrmmkm0997rnwl3i835p-module-import -C /gnu/store/idi8frs4vz0ympplpm1kddp8932kgkr1-module-import-compiled /gnu/store/kj2wyb3w6c6zqdjfmizv826rgfb383rf-binutils-mesboot0-2.20.1a-builder
$ sudo guix processes |guix shell recutils -- recsel -e 'LockHeld ~ "binutils"'
SessionPID: 17495
ClientPID: 17477
ClientCommand: /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/libexec/guix/guile \ /run/current-system/profile/bin/guix build -c32 -M3 /gnu/store/icfjxv22fysqpip9zxfqv7xjdahziv94-hello-2.12.1.drv
LockHeld: /gnu/store/pkaxhawivjhhff55ncglr30c6d9dzlaz-binutils-mesboot0-2.20.1a.lock
ChildPID: 17496
ChildCommand: : /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/libexec/guix/guile \ /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/bin/guix substitute --query
ChildPID: 17522
ChildCommand: : /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/libexec/guix/guile \ /gnu/store/n2amn1hpl41r403n0rq7cjbawvzfx51m-guix-1.4.0-16.aeb4943/bin/guix offload x86_64-linux 0 1 0
ChildPID: 29600
ChildCommand: : /gnu/store/lgi9x15a0w35mcpd7g1kb9274r6wy4pv-guile-bootstrap-2.0/bin/guile --no-auto-compile -L /gnu/store/4zibvq0ffm75xrmmkm0997rnwl3i835p-module-import -C /gnu/store/idi8frs4vz0ympplpm1kddp8932kgkr1-module-import-compiled /gnu/store/kj2wyb3w6c6zqdjfmizv826rgfb383rf-binutils-mesboot0-2.20.1a-builder

Here ‘conftest’ is a zombie, which means that the shell running
./configure (gash-boot-0.3.0) did not call ‘waitpid’ (bug #1). There’s
another bunch of .guile-real zombie processes (child processes of ‘make
DESTDIR= RPATH_ENVVAR=LD_LIBRARY_PATH
TARGET_SUBDIR=i686-unknown-linux-gnu
bindir=/gnu/store/pkaxhawivjhhff55ncglr30c6d9dzlaz-binutils-mesboot0-2.20.1a/bin
[…]’).

Thus the ‘conftest’ process got reparented to the Guile process that
runs the build script (PID 1), and then we run into (bug #2):


Ludo’.
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 1/9] packages: Remove refere nce to ‘glibc-utf8-locales’ in ‘patch-and-repack’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
eee58768e5b7514c836ea4b12abe61850205a359.1702916397.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/packages.scm (%standard-patch-inputs): Remove “locales” entry.
(patch-and-repack): Remove ‘locales’ variable.
[build]: Remove reference to ‘locales’. Use “C.UTF-8” unconditionally.

Change-Id: I54219b232880e05534599fbba8738cc00fb5d568
---
guix/packages.scm | 29 ++++++++++++-----------------
1 file changed, 12 insertions(+), 17 deletions(-)

Toggle diff (56 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 930b1a3b0e..4b0c478ff4 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -866,13 +866,7 @@ (define (%standard-patch-inputs system)
("gzip" ,(ref '(gnu packages compression) 'gzip))
("lzip" ,(ref '(gnu packages compression) 'lzip))
("unzip" ,(ref '(gnu packages compression) 'unzip))
- ("patch" ,(ref '(gnu packages base) 'patch))
- ("locales"
- ,(parameterize ((%current-target-system #f)
- (%current-system system))
- (canonical
- ((module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))))))))
+ ("patch" ,(ref '(gnu packages base) 'patch)))))
(define (default-guile)
"Return the default Guile package used to run the build code of
@@ -955,7 +949,6 @@ (define* (patch-and-repack source patches
(lzip (lookup-input "lzip"))
(xz (lookup-input "xz"))
(patch (lookup-input "patch"))
- (locales (lookup-input "locales"))
(comp (and=> (compressor source-file-name) lookup-input))
(patches (map instantiate-patch patches)))
(define build
@@ -1017,15 +1010,17 @@ (define* (patch-and-repack source patches
;; Encoding/decoding errors shouldn't be silent.
(fluid-set! %default-port-conversion-strategy 'error)
- (when #+locales
- ;; First of all, install a UTF-8 locale so that UTF-8 file names
- ;; are correctly interpreted. During bootstrap, LOCALES is #f.
- (setenv "LOCPATH"
- (string-append #+locales "/lib/locale/"
- #+(and locales
- (version-major+minor
- (package-version locales)))))
- (setlocale LC_ALL "en_US.utf8"))
+ ;; First of all, install a UTF-8 locale so that UTF-8 file names
+ ;; are correctly interpreted. During bootstrap, locales are
+ ;; missing.
+ (let ((locale "C.UTF-8"))
+ (catch 'system-error
+ (lambda ()
+ (setlocale LC_ALL locale))
+ (lambda args
+ (format (current-error-port)
+ "failed to install '~a' locale: ~a~%"
+ locale (system-error-errno args)))))
(setenv "PATH"
(string-append #+xz "/bin"
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 0/9] Run builds in C.UTF-8 locale
(address . 67824@debbugs.gnu.org)
cover.1702916397.git.ludo@gnu.org
Hello Guix!

This is an updated version of the patch set. Changes since v1:

• Change default value of #:locale in ‘gnu-build-system.scm’,
since all the build systems except ‘gnu’ rely on it.

• Remove customization of util-linux tests that was due to
the lack of C.UTF-8 on our side before.

• Skip failing GLib test, which should allow us to go further
towards evaluating ‘core-updates’ (!).

• Add useful x86_64-gnu patch in glibc (piggy-backed, not related
to the issues at hand).

• Include fix for https://issues.guix.gnu.org/30948. Probably
not strictly necessary but a good opportunity, especially since
Gash/Guile & co. seem to leave zombies when building the early
‘mesboot’ packages (something Janneke and I discussed on IRC
a couple of days ago).

Feedback welcome!

Ludo’.

Ludovic Courtès (9):
packages: Remove reference to ‘glibc-utf8-locales’ in
‘patch-and-repack’.
build-system/gnu: Change default locale to C.UTF-8.
gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%final-inputs’.
profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
packages: Output and error ports are line-buffered in
‘patch-and-repack’.
gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
gnu: glibc: Add patch for ‘ucontext’ on x86_64-gnu (GNU/Hurd).
build-system/gnu: Turn PID 1 into an “init”-style process by default.
gnu: glib: Skip test that fails with glibc 2.38.

gnu/local.mk | 1 +
gnu/packages/base.scm | 24 +++--
gnu/packages/commencement.scm | 31 ++++---
gnu/packages/gawk.scm | 6 +-
gnu/packages/glib.scm | 6 +-
gnu/packages/linux.scm | 6 +-
.../patches/glibc-2.38-hurd-ucontext.patch | 87 +++++++++++++++++++
gnu/packages/texinfo.scm | 7 +-
guix/build-system/gnu.scm | 8 +-
guix/build/gnu-build-system.scm | 41 ++++++++-
guix/packages.scm | 34 ++++----
guix/profiles.scm | 33 ++-----
tests/profiles.scm | 7 +-
13 files changed, 206 insertions(+), 85 deletions(-)
create mode 100644 gnu/packages/patches/glibc-2.38-hurd-ucontext.patch


base-commit: e4013b31266b40dc9cb250c44b2088d3931cb3b3
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 2/9] build-system/gnu: Change default locale to C.UTF-8.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
6fbee1e6544d1abcaf781fc246004de494396868.1702916397.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): #:locale now
defaults to "C.UTF-8".
* guix/build/gnu-build-system.scm (install-locale): Likewise.
* gnu/packages/linux.scm (util-linux)[arguments]: In ‘pre-check’ phase,
remove ‘invalid-multibyte’ modification so the test runs under C.UTF-8.

Change-Id: I9a4bfe564bcd8bbd0d57ba04568eb6b8020bd051
---
gnu/packages/linux.scm | 6 +-----
guix/build-system/gnu.scm | 4 ++--
guix/build/gnu-build-system.scm | 2 +-
3 files changed, 4 insertions(+), 8 deletions(-)

Toggle diff (54 lines)
diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index 430645e4ad..0572ad3a73 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -2257,11 +2257,7 @@ (define-public util-linux
;; Change the test to refer to the right file.
(substitute* "tests/ts/misc/mcookie"
(("/etc/services")
- services))
-
- ;; The C.UTF-8 locale does not exist in our libc.
- (substitute* "tests/ts/column/invalid-multibyte"
- (("C\\.UTF-8") "en_US.utf8")))))
+ services)))))
(add-before 'check 'disable-setarch-test
(lambda _
;; The setarch tests are unreliable in QEMU's user-mode
diff --git a/guix/build-system/gnu.scm b/guix/build-system/gnu.scm
index ed1dc33c76..0f886fe21d 100644
--- a/guix/build-system/gnu.scm
+++ b/guix/build-system/gnu.scm
@@ -361,7 +361,7 @@ (define* (gnu-build name inputs
(make-dynamic-linker-cache? #t)
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
@@ -501,7 +501,7 @@ (define* (gnu-cross-build name
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
index ef5873d793..39707e7ace 100644
--- a/guix/build/gnu-build-system.scm
+++ b/guix/build/gnu-build-system.scm
@@ -123,7 +123,7 @@ (define* (set-paths #:key target inputs native-inputs
native-search-paths)))
(define* (install-locale #:key
- (locale "en_US.utf8")
+ (locale "C.UTF-8")
(locale-category LC_ALL)
#:allow-other-keys)
"Try to install LOCALE; emit a warning if that fails. The main goal is to
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 3/9] gnu: commencement: Remo ve ‘glibc-utf8-locales’ from ‘%fi nal-inputs’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
8f609fdf9e20d9140f149c8255b92334fae84f62.1702916397.git.ludo@gnu.org
* gnu/packages/commencement.scm (%boot5-inputs): Make an alias for
‘%boot4-inputs’.
(with-boot5): Turn into an alias for ‘with-boot4’.
(%final-inputs): Override ‘native-inputs’ of GAWK. Remove "locales"
entry.
(canonical-package): Add “glibc-utf8-locales” entry.
* gnu/packages/base.scm (libc-utf8-locales-for-target): Use
‘canonical-package’.
* gnu/packages/gawk.scm (gawk)[native-inputs]: New field.
* gnu/packages/texinfo.scm (texinfo)[native-inputs]: Add
‘libc-utf8-locales-for-target’.

Change-Id: I2283ffc6c6000c98ad2db89c04d581e050667c94
---
gnu/packages/base.scm | 11 ++++++++---
gnu/packages/commencement.scm | 31 +++++++++++++++----------------
gnu/packages/gawk.scm | 6 +++++-
gnu/packages/texinfo.scm | 7 +++++--
4 files changed, 33 insertions(+), 22 deletions(-)

Toggle diff (136 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 235ad7fecc..d31c0b0772 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1593,9 +1593,14 @@ (define* (libc-locales-for-target #:optional
(define* (libc-utf8-locales-for-target #:optional
(target (or (%current-target-system)
(%current-system))))
- (if (target-hurd? target)
- glibc-utf8-locales/hurd
- glibc-utf8-locales))
+ "Return the glibc UTF-8 locale package for TARGET."
+ ;; Note: To avoid circular dependencies (such as: texinfo ->
+ ;; glibc-utf8-locales -> glibc -> texinfo), refer to
+ ;; 'glibc-utf8-locales-final' via 'canonical-package'.
+ (canonical-package
+ (if (target-hurd? target)
+ glibc-utf8-locales/hurd
+ glibc-utf8-locales)))
(define-public tzdata
(package
diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 988e497ee7..f4882f7cdd 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -3394,17 +3394,11 @@ (define-public ld-gold-wrapper
#:guile guile-final
#:bash bash-final))
-(define (%boot5-inputs)
- ;; Now with UTF-8 locales. Remember that the bootstrap binaries were built
- ;; with an older libc, which cannot load the new locale format. See
- ;; <https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html>.
- `(("locales" ,(if (target-hurd?)
- glibc-utf8-locales-final/hurd
- glibc-utf8-locales-final))
- ,@(%boot4-inputs)))
-
-(define with-boot5
- (package-with-explicit-inputs %boot5-inputs))
+;; There used to be a "stage 5" including a variant of the
+;; 'glibc-utf8-locales' package. This is no longer necessary since 'glibc'
+;; embeds the "C.UTF-8" locale, but these aliases are kept for convenience.
+(define %boot5-inputs %boot4-inputs)
+(define with-boot5 with-boot4)
(define gnu-make-final
;; The final GNU Make, which uses the final Guile.
@@ -3493,7 +3487,11 @@ (define-public %final-inputs
("diffutils" ,diffutils)
("patch" ,patch)
("findutils" ,findutils)
- ("gawk" ,gawk)))
+ ("gawk" ,(package/inherit gawk
+ (native-inputs
+ (list (if (target-hurd?)
+ glibc-utf8-locales-final/hurd
+ glibc-utf8-locales-final)))))))
("sed" ,sed-final)
("grep" ,grep-final)
("xz" ,xz-final)
@@ -3504,10 +3502,7 @@ (define-public %final-inputs
("binutils" ,binutils-final)
("gcc" ,gcc-final)
("libc" ,glibc-final)
- ("libc:static" ,glibc-final "static")
- ("locales" ,(if (target-hurd? (%current-system))
- glibc-utf8-locales-final/hurd
- glibc-utf8-locales-final)))))))
+ ("libc:static" ,glibc-final "static"))))))
(define-public canonical-package
(let ((name->package (mlambda (system)
@@ -3518,6 +3513,10 @@ (define-public canonical-package
package result))))
vlist-null
`(("guile" ,guile-final)
+ ("glibc-utf8-locales"
+ ,(if (target-hurd? system)
+ glibc-utf8-locales-final/hurd
+ glibc-utf8-locales-final))
,@(%final-inputs system))))))
(lambda (package)
"Return the 'canonical' variant of PACKAGE---i.e., if PACKAGE is one of
diff --git a/gnu/packages/gawk.scm b/gnu/packages/gawk.scm
index 2d8bc581ef..476865ada3 100644
--- a/gnu/packages/gawk.scm
+++ b/gnu/packages/gawk.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2016, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2018, 2022, 2023 Efraim Flashner <efraim@flashner.co.il>
;;; Copyright © 2021, 2022 Marius Bakke <marius@gnu.org>
@@ -79,6 +79,10 @@ (define-public gawk
(("#! /usr/bin/gawk")
(string-append "#!" (which "gawk")))))))))
+ ;; Tests explicitly require en_US.UTF-8, fr_FR.UTF-8, ru_RU.UTF-8 and
+ ;; ja_JP.UTF-8.
+ (native-inputs (list (libc-utf8-locales-for-target)))
+
(inputs (list libsigsegv
;; Use the full-fledged Bash package, otherwise the test suite
;; sometimes fail non-deterministically.
diff --git a/gnu/packages/texinfo.scm b/gnu/packages/texinfo.scm
index ab4583071e..92c9d800a9 100644
--- a/gnu/packages/texinfo.scm
+++ b/gnu/packages/texinfo.scm
@@ -38,6 +38,7 @@ (define-module (gnu packages texinfo)
#:use-module (guix build-system gnu)
#:use-module (gnu packages)
#:use-module (gnu packages autotools)
+ #:use-module (gnu packages base)
#:use-module (gnu packages compression)
#:use-module (gnu packages gettext)
#:use-module (gnu packages ncurses)
@@ -79,10 +80,12 @@ (define-public texinfo
#:tests? ,(and (not (target-hurd?))
(not (%current-target-system)))))
(inputs (list ncurses perl))
+
;; When cross-compiling, texinfo will build some of its own binaries with
;; the native compiler. This means ncurses is needed both in both inputs
- ;; and native-inputs.
- (native-inputs (list perl ncurses))
+ ;; and native-inputs. Some of its tests require extra locales such as
+ ;; fr_FR.UTF-8.
+ (native-inputs (list perl ncurses (libc-utf8-locales-for-target)))
(native-search-paths
;; This is the variable used by the standalone Info reader.
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 4/9] profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
bd105b2312e12569bb657b24ef8f9f6e4fb4521b.1702916397.git.ludo@gnu.org
This is a followup to 1cebc334a77030c0c94955981652f4df7608c9e3.

* guix/profiles.scm (ca-certificate-bundle)[libc-utf8-locales-for-target]:
Remove.
[build]: Use “C.UTF-8” instead of “en_US.utf8”.
(profile-derivation)[libc-utf8-locales-for-target]: Remove.
[set-utf8-locale]: Rewrite to a gexp that calls ‘setlocale’ for “C.UTF-8”.
* tests/profiles.scm ("profile-derivation, cross-compilation"): Remove
‘locales’ variable and related check.

Change-Id: I7cb148b9149fe5fbe5b9b1b25fdce1002ad1f37e
---
guix/profiles.scm | 33 +++++----------------------------
tests/profiles.scm | 7 +------
2 files changed, 6 insertions(+), 34 deletions(-)

Toggle diff (97 lines)
diff --git a/guix/profiles.scm b/guix/profiles.scm
index ce2f8337bf..da7790d819 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -1125,11 +1125,6 @@ (define* (ca-certificate-bundle manifest #:optional system)
MANIFEST. Single-file bundles are required by programs such as Git and Lynx."
;; See <http://lists.gnu.org/archive/html/guix-devel/2015-02/msg00429.html>
;; for a discussion.
-
- (define libc-utf8-locales-for-target ;lazy reference
- (module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))
-
(define build
(with-imported-modules '((guix build utils))
#~(begin
@@ -1161,13 +1156,7 @@ (define* (ca-certificate-bundle manifest #:optional system)
;; Some file names in the NSS certificates are UTF-8 encoded so
;; install a UTF-8 locale.
- (setenv "LOCPATH"
- (string-append #+(libc-utf8-locales-for-target system)
- "/lib/locale/"
- #+(version-major+minor
- (package-version
- (libc-utf8-locales-for-target system)))))
- (setlocale LC_ALL "en_US.utf8")
+ (setlocale LC_ALL "C.UTF-8")
(match (append-map ca-files '#$(manifest-inputs manifest))
(()
@@ -1960,8 +1949,7 @@ (define* (profile-derivation manifest
or TARGET is set, raise an error if MANIFEST contains a package that does not
support SYSTEM.
-When LOCALES? is true, the build is performed under a UTF-8 locale; this adds
-a dependency on the 'glibc-utf8-locales' package.
+When LOCALES? is true, the build is performed under a UTF-8 locale.
When RELATIVE-SYMLINKS? is true, use relative file names for symlink targets.
This is one of the things to do for the result to be relocatable.
@@ -2004,21 +1992,10 @@ (define* (profile-derivation manifest
(and (derivation? drv) (gexp-input drv)))
extras))
- (define libc-utf8-locales-for-target ;lazy reference
- (module-ref (resolve-interface '(gnu packages base))
- 'libc-utf8-locales-for-target))
-
(define set-utf8-locale
- ;; Some file names (e.g., in 'nss-certs') are UTF-8 encoded so
- ;; install a UTF-8 locale.
- (let ((locales (libc-utf8-locales-for-target
- (or system (%current-system)))))
- #~(begin
- (setenv "LOCPATH"
- #$(file-append locales "/lib/locale/"
- (version-major+minor
- (package-version locales))))
- (setlocale LC_ALL "en_US.utf8"))))
+ ;; Some file names (e.g., in 'nss-certs') are UTF-8 encoded so install a
+ ;; UTF-8 locale. Assume libc comes with a copy of C.UTF-8.
+ #~(setlocale LC_ALL "C.UTF-8"))
(define builder
(with-imported-modules '((guix build profiles)
diff --git a/tests/profiles.scm b/tests/profiles.scm
index ddd6d74f3b..e448137cff 100644
--- a/tests/profiles.scm
+++ b/tests/profiles.scm
@@ -463,7 +463,6 @@ (define glibc
(target -> "arm-linux-gnueabihf")
(grep (package->cross-derivation packages:grep target))
(sed (package->cross-derivation packages:sed target))
- (locales (package->derivation (packages:libc-utf8-locales-for-target)))
(drv (profile-derivation manifest
#:hooks '()
#:locales? #t
@@ -475,15 +474,11 @@ (define glibc
(and (string-suffix? name input) input)))
(derivation-inputs drv))))
- ;; The inputs for grep and sed should be cross-build derivations, but that
- ;; for the glibc-utf8-locales should be a native build.
(return (and (string=? (derivation-system drv) (%current-system))
(string=? (find-input packages:grep)
(derivation-file-name grep))
(string=? (find-input packages:sed)
- (derivation-file-name sed))
- (string=? (find-input (packages:libc-utf8-locales-for-target))
- (derivation-file-name locales))))))
+ (derivation-file-name sed))))))
(test-assert "package->manifest-entry defaults to \"out\""
(let ((outputs (package-outputs packages:glibc)))
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 5/9] packages: Output and error ports are line-buffered in ‘patch-and-r epack’.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
568767c79a40cf44a01fe9b57372defce13ec620.1702916397.git.ludo@gnu.org
* guix/packages.scm (patch-and-repack): Add ‘setvbuf’ calls.

Change-Id: I039bb6407263d5172bf0bc716bda6860dc2615fb
---
guix/packages.scm | 5 +++++
1 file changed, 5 insertions(+)

Toggle diff (18 lines)
diff --git a/guix/packages.scm b/guix/packages.scm
index 4b0c478ff4..cb8db925f8 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -1007,6 +1007,11 @@ (define* (patch-and-repack source patches
'("--no-recursion"
"--files-from=.file_list"))))
+ (let ((line (cond-expand (guile-2.0 _IOLBF)
+ (else 'line))))
+ (setvbuf (current-output-port) line)
+ (setvbuf (current-error-port) line))
+
;; Encoding/decoding errors shouldn't be silent.
(fluid-set! %default-port-conversion-strategy 'error)
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 6/9] gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
7f7ca40f6236f1456cdc5604549a60f8070d6514.1702916397.git.ludo@gnu.org
* gnu/packages/base.scm (%default-utf8-locales): Add "C".
(make-glibc-utf8-locales): Delete "C" from LOCALES for glibc versions
lower than or equal to 2.35.

Change-Id: I6a6faa68dc18bf0f87c372109a17f476d3561278
---
gnu/packages/base.scm | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index d31c0b0772..eab9ff411d 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1396,7 +1396,8 @@ (define-public (make-glibc-locales glibc)
(define %default-utf8-locales
;; These are the locales commonly used for tests---e.g., in Guile's i18n
;; tests.
- '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+ '("C" "de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+
(define*-public (make-glibc-utf8-locales glibc #:key
(locales %default-utf8-locales)
(name "glibc-utf8-locales"))
@@ -1444,7 +1445,14 @@ (define*-public (make-glibc-utf8-locales glibc #:key
(symlink (string-append locale ".utf8")
(string-append localedir "/"
locale ".UTF-8")))
- '#$locales)))))
+
+ ;; The C.UTF-8 locale was introduced in 2.35 but it
+ ;; fails to build there:
+ ;; <https://sourceware.org/bugzilla/show_bug.cgi?id=28861>.
+ '#$(if (version>? (package-version this-package)
+ "2.35")
+ locales
+ (delete "C" locales)))))))
(native-inputs (list glibc gzip))
(synopsis (if default-locales?
(P_ "Small sample of UTF-8 locales")
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 7/9] gnu: glibc: Add patch for ‘ucontext’ on x86_64-gnu (GNU/ Hurd).
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
2f3b64b9d967b4eea5cbdb32c859f4e3ac3b1a83.1702916397.git.ludo@gnu.org
Suggested by Sergey Bugaev.

* gnu/packages/patches/glibc-2.38-hurd-ucontext.patch: New file.
* gnu/packages/base.scm (glibc)[source]: Use it.
* gnu/local.mk (dist_patch_DATA): Add it.

Change-Id: Ib74e93f2414eda2ec4b3e01fc2bb67b02838fabb
---
gnu/local.mk | 1 +
gnu/packages/base.scm | 1 +
.../patches/glibc-2.38-hurd-ucontext.patch | 87 +++++++++++++++++++
3 files changed, 89 insertions(+)
create mode 100644 gnu/packages/patches/glibc-2.38-hurd-ucontext.patch

Toggle diff (119 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index ff2f82a23a..e92cf871ad 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1320,6 +1320,7 @@ dist_patch_DATA = \
%D%/packages/patches/glibc-2.37-hurd-clock_t_centiseconds.patch \
%D%/packages/patches/glibc-2.37-hurd-local-clock_gettime_MONOTONIC.patch \
%D%/packages/patches/glibc-2.37-versioned-locpath.patch \
+ %D%/packages/patches/glibc-2.38-hurd-ucontext.patch \
%D%/packages/patches/glibc-2.38-ldd-x86_64.patch \
%D%/packages/patches/glibc-hurd-clock_t_centiseconds.patch \
%D%/packages/patches/glibc-hurd-getauxval.patch \
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index eab9ff411d..e2a952cc42 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -813,6 +813,7 @@ (define-public glibc
"glibc-supported-locales.patch"
"glibc-2.37-hurd-clock_t_centiseconds.patch"
"glibc-2.37-hurd-local-clock_gettime_MONOTONIC.patch"
+ "glibc-2.38-hurd-ucontext.patch"
"glibc-hurd-mach-print.patch"
"glibc-hurd-gettyent.patch"
"glibc-hurd-getauxval.patch"))))
diff --git a/gnu/packages/patches/glibc-2.38-hurd-ucontext.patch b/gnu/packages/patches/glibc-2.38-hurd-ucontext.patch
new file mode 100644
index 0000000000..03a6132666
--- /dev/null
+++ b/gnu/packages/patches/glibc-2.38-hurd-ucontext.patch
@@ -0,0 +1,87 @@
+commit 29d4591b07a4da53320e949557c6946c62c26bde
+Author: Samuel Thibault <samuel.thibault@ens-lyon.org>
+Date: Thu Sep 28 00:07:18 2023 +0200
+
+ hurd: Drop REG_GSFS and REG_ESDS from x86_64's ucontext
+
+ These are useless on x86_64, and __NGREG was actually wrong with them.
+
+diff --git a/sysdeps/mach/hurd/x86/trampoline.c b/sysdeps/mach/hurd/x86/trampoline.c
+index 6318c9528a..bc3f15e1e0 100644
+--- a/sysdeps/mach/hurd/x86/trampoline.c
++++ b/sysdeps/mach/hurd/x86/trampoline.c
+@@ -79,8 +79,8 @@ static void fill_ucontext (ucontext_t *uc, const struct sigcontext *sc)
+
+ /* Registers. */
+ #ifdef __x86_64__
+- memcpy (&uc->uc_mcontext.gregs[REG_GSFS], &sc->sc_gs,
+- (REG_ERR - REG_GSFS) * sizeof (long));
++ memcpy (&uc->uc_mcontext.gregs[REG_R8], &sc->sc_r8,
++ (REG_ERR - REG_R8) * sizeof (long));
+ #else
+ memcpy (&uc->uc_mcontext.gregs[REG_GS], &sc->sc_gs,
+ (REG_TRAPNO - REG_GS) * sizeof (int));
+diff --git a/sysdeps/mach/hurd/x86_64/bits/sigcontext.h b/sysdeps/mach/hurd/x86_64/bits/sigcontext.h
+index 6396054463..7facc587b8 100644
+--- a/sysdeps/mach/hurd/x86_64/bits/sigcontext.h
++++ b/sysdeps/mach/hurd/x86_64/bits/sigcontext.h
+@@ -59,13 +59,7 @@ struct sigcontext
+ }
+ trampoline.c knows this, so it must be changed if this changes. */
+
+-#define sc_i386_thread_state sc_gs /* Beginning of correspondence. */
+- /* Segment registers. */
+- int sc_gs;
+- int sc_fs;
+- int sc_es;
+- int sc_ds;
+-
++#define sc_i386_thread_state sc_r8 /* Beginning of correspondence. */
+ long sc_r8;
+ long sc_r9;
+ long sc_r10;
+diff --git a/sysdeps/mach/x86/thread_state.h b/sysdeps/mach/x86/thread_state.h
+index 8c419515f9..e237e46cb2 100644
+--- a/sysdeps/mach/x86/thread_state.h
++++ b/sysdeps/mach/x86/thread_state.h
+@@ -34,12 +34,13 @@
+ #define PC rip
+ #define SP ursp
+ #define SYSRETURN rax
++#define MACHINE_THREAD_STATE_FIX_NEW(ts) do { \
++ asm ("mov %%cs, %w0" : "=q" ((ts)->cs)); \
++} while(0)
+ #else
+ #define PC eip
+ #define SP uesp
+ #define SYSRETURN eax
+-#endif
+-
+ #define MACHINE_THREAD_STATE_FIX_NEW(ts) do { \
+ asm ("mov %%cs, %w0" : "=q" ((ts)->cs)); \
+ asm ("mov %%ds, %w0" : "=q" ((ts)->ds)); \
+@@ -47,6 +48,7 @@
+ asm ("mov %%fs, %w0" : "=q" ((ts)->fs)); \
+ asm ("mov %%gs, %w0" : "=q" ((ts)->gs)); \
+ } while(0)
++#endif
+
+ struct machine_thread_all_state
+ {
+diff --git a/sysdeps/x86_64/sys/ucontext.h b/sysdeps/x86_64/sys/ucontext.h
+index d73a893795..f1b6be77a7 100644
+--- a/sysdeps/x86_64/sys/ucontext.h
++++ b/sysdeps/x86_64/sys/ucontext.h
+@@ -47,11 +47,7 @@ typedef greg_t gregset_t[__NGREG];
+ /* Number of each register in the `gregset_t' array. */
+ enum
+ {
+- REG_GSFS = 0, /* Actually int gs, fs. */
+-# define REG_GSFS REG_GSFS
+- REG_ESDS, /* Actually int es, ds. */
+-# define REG_ESDS REG_ESDS
+- REG_R8,
++ REG_R8 = 0,
+ # define REG_R8 REG_R8
+ REG_R9,
+ # define REG_R9 REG_R9
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 9/9] gnu: glib: Skip test that fails with glibc 2.38.
(address . 67824@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
08c866533e2a76bcf91149208c1323ecbfcff4cd.1702916397.git.ludo@gnu.org
* gnu/packages/glib.scm (glib)[arguments]: In ‘disable-failing-tests’
phase, substitute “glib/tests/error.c”.

Change-Id: I9e3f5f1f5f55d08c83fbde3b79931987df27ebdf
---
gnu/packages/glib.scm | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/glib.scm b/gnu/packages/glib.scm
index dc8cf5800b..13f957f810 100644
--- a/gnu/packages/glib.scm
+++ b/gnu/packages/glib.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2013, 2014, 2015, 2016, 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2013, 2014, 2015, 2016, 2019, 2020, 2021, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2013, 2015 Andreas Enge <andreas@enge.fr>
;;; Copyright © 2013 Nikita Karetnikov <nikita@karetnikov.org>
;;; Copyright © 2014, 2015, 2016, 2017, 2018, 2021 Mark H Weaver <mhw@netris.org>
@@ -275,6 +275,10 @@ (define glib
(substitute* '("contenttype.c" "gdbus-address-get-session.c"
"gdbus-peer.c" "appinfo.c" "desktop-app-info.c")
(("[ \t]*g_test_add_func.*;") "")))
+ (substitute* "glib/tests/error.c"
+ ;; This test segfaults with glibc 2.38.
+ (("g_test_add_func.*new-valist/invalid.*" all)
+ (string-append "//" all "\n")))
#$@(if (target-x86-32?)
;; Comment out parts of timer.c that fail on i686 due to
--
2.41.0
L
L
Ludovic Courtès wrote on 18 Dec 2023 17:30
[PATCH core-updates v2 8/9] build-system/gnu: Turn PID 1 into an “init”-style process by default.
(address . 67824@debbugs.gnu.org)
d5497ca9b8d069d31dc905ac2aedddcff2614792.1702916397.git.ludo@gnu.org

* guix/build/gnu-build-system.scm (separate-from-pid1): New procedure.
(%standard-phases): Add it.
* guix/build-system/gnu.scm (gnu-build): Add #:separate-from-pid1? and
honor it.
(gnu-cross-build): Likewise.

Reported-by: Carlo Zancanaro <carlo@zancanaro.id.au>
Change-Id: I6f3bc8d8186d1a571f983a38d5e3fd178ffa2678
---
guix/build-system/gnu.scm | 4 ++++
guix/build/gnu-build-system.scm | 39 ++++++++++++++++++++++++++++++++-
2 files changed, 42 insertions(+), 1 deletion(-)

Toggle diff (95 lines)
diff --git a/guix/build-system/gnu.scm b/guix/build-system/gnu.scm
index 0f886fe21d..6a89bcc0d8 100644
--- a/guix/build-system/gnu.scm
+++ b/guix/build-system/gnu.scm
@@ -362,6 +362,7 @@ (define* (gnu-build name inputs
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
(locale "C.UTF-8")
+ (separate-from-pid1? #t)
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
@@ -404,6 +405,7 @@ (define* (gnu-build name inputs
(sexp->gexp phases)
phases)
#:locale #$locale
+ #:separate-from-pid1? #$separate-from-pid1?
#:bootstrap-scripts #$bootstrap-scripts
#:configure-flags #$(if (pair? configure-flags)
(sexp->gexp configure-flags)
@@ -502,6 +504,7 @@ (define* (gnu-cross-build name
(license-file-regexp %license-file-regexp)
(phases '%standard-phases)
(locale "C.UTF-8")
+ (separate-from-pid1? #t)
(system (%current-system))
(build (nix-system->gnu-triplet system))
(imported-modules %default-gnu-imported-modules)
@@ -547,6 +550,7 @@ (define* (gnu-cross-build name
(sexp->gexp phases)
phases)
#:locale #$locale
+ #:separate-from-pid1? #$separate-from-pid1?
#:bootstrap-scripts #$bootstrap-scripts
#:configure-flags #$configure-flags
#:make-flags #$make-flags
diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
index 39707e7ace..51b8f9acbf 100644
--- a/guix/build/gnu-build-system.scm
+++ b/guix/build/gnu-build-system.scm
@@ -72,6 +72,42 @@ (define (first-subdirectory directory)
((first . _) first)
(_ #f)))
+(define* (separate-from-pid1 #:key (separate-from-pid1? #t)
+ #:allow-other-keys)
+ "When running as PID 1 and SEPARATE-FROM-PID1? is true, run build phases as
+a child process; PID 1 then becomes responsible for reaping child processes."
+ (if separate-from-pid1?
+ (if (= 1 (getpid))
+ (dynamic-wind
+ (const #t)
+ (lambda ()
+ (match (primitive-fork)
+ (0 #t)
+ (builder-pid
+ (format (current-error-port)
+ "build process now running as PID ~a~%"
+ builder-pid)
+ (let loop ()
+ ;; Running as PID 1 so take responsibility for reaping
+ ;; child processes.
+ (match (waitpid WAIT_ANY)
+ ((pid . status)
+ (if (= pid builder-pid)
+ (if (zero? status)
+ (primitive-exit 0)
+ (begin
+ (format (current-error-port)
+ "build process ~a exited with status ~a~%"
+ pid status)
+ (primitive-exit 1)))
+ (loop))))))))
+ (const #t))
+ (format (current-error-port) "not running as PID 1 (PID: ~a)~%"
+ (getpid)))
+ (format (current-error-port)
+ "build process running as PID ~a; not forking~%"
+ (getpid))))
+
(define* (set-paths #:key target inputs native-inputs
(search-paths '()) (native-search-paths '())
#:allow-other-keys)
@@ -872,7 +908,8 @@ (define %standard-phases
;; Standard build phases, as a list of symbol/procedure pairs.
(let-syntax ((phases (syntax-rules ()
((_ p ...) `((p . ,p) ...)))))
- (phases set-SOURCE-DATE-EPOCH set-paths install-locale unpack
+ (phases separate-from-pid1
+ set-SOURCE-DATE-EPOCH set-paths install-locale unpack
bootstrap
patch-usr-bin-file
patch-source-shebangs configure patch-generated-file-shebangs
--
2.41.0
L
L
Ludovic Courtès wrote on 19 Dec 2023 23:54
Re: [bug#67824] [PATCH core-updates v2 0/9] Run builds in C.UTF-8 locale
(address . 67824-done@debbugs.gnu.org)
875y0tiz0t.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (12 lines)
> packages: Remove reference to ‘glibc-utf8-locales’ in
> ‘patch-and-repack’.
> build-system/gnu: Change default locale to C.UTF-8.
> gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%final-inputs’.
> profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
> packages: Output and error ports are line-buffered in
> ‘patch-and-repack’.
> gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
> gnu: glibc: Add patch for ‘ucontext’ on x86_64-gnu (GNU/Hurd).
> build-system/gnu: Turn PID 1 into an “init”-style process by default.
> gnu: glib: Skip test that fails with glibc 2.38.

Pushed v2 as 62e67aa7994f40c438ef5a528675e85699d7af76.

Now to see what ci.guix thinks about it…

Ludo’.
Closed
J
J
Janneke Nieuwenhuizen wrote on 20 Dec 2023 08:20
(name . Ludovic Courtès)(address . ludo@gnu.org)
874jgdpcfb.fsf@gnu.org
Ludovic Courtès writes:

Toggle quote (18 lines)
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> packages: Remove reference to ‘glibc-utf8-locales’ in
>> ‘patch-and-repack’.
>> build-system/gnu: Change default locale to C.UTF-8.
>> gnu: commencement: Remove ‘glibc-utf8-locales’ from ‘%final-inputs’.
>> profiles: Use C.UTF-8 instead of ‘glibc-utf8-locales’ where possible.
>> packages: Output and error ports are line-buffered in
>> ‘patch-and-repack’.
>> gnu: glibc-utf8-locales: Add the C.UTF-8 locale.
>> gnu: glibc: Add patch for ‘ucontext’ on x86_64-gnu (GNU/Hurd).
>> build-system/gnu: Turn PID 1 into an “init”-style process by default.
>> gnu: glib: Skip test that fails with glibc 2.38.
>
> Pushed v2 as 62e67aa7994f40c438ef5a528675e85699d7af76.
>
> Now to see what ci.guix thinks about it…

+1

Haven't built everyting just yet, but it's looking good.
Thank you!

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 67824@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 67824
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch