Wrong ‘glibc-utf8-locales’ package used on GNU/Hurd

  • Done
  • quality assurance status badge
Details
2 participants
  • Janneke Nieuwenhuizen
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 11 Oct 2023 23:42
(address . bug-guix@gnu.org)
87wmvsq1ia.fsf@inria.fr
Hi!

We discussed it briefly on IRC the other day: our packages get built on
i586-gnu with the wrong ‘glibc-utf8-locales’ package (2.35 instead of
2.37), which causes Coreutils among others to fail to build:

Toggle snippet (14 lines)
environment variable `GUIX_LOCPATH' set to `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
phase `set-paths' succeeded after 0.0 seconds
starting phase `install-locale'
warning: failed to install 'en_US.utf8' locale: Invalid argument
phase `install-locale' succeeded after 0.0 seconds
[…]
starting phase `remove-tests'
error: in phase 'remove-tests': uncaught exception:
decoding-error "decode-char" "input decoding error" 1073741930 #<input: tests/misc/ls-misc.pl 15>
phase `remove-tests' failed after 0.1 seconds
[…]
builder for `/gnu/store/vvp0yxvyxsrwmmzli7dsxinr6p9ba3mj-coreutils-9.1.drv' failed with exit code 1

commit cdbd81ce144f17644ceebd3d08723aa244696a05.)

So we need a better fix than the local workaround in
21deb89e287b5821975544118bf137562a91d4e1.

Thoughts? Perhaps you’ve looked into it already?

Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 12 Oct 2023 16:12
Re: bug#66472: Wrong ‘glibc-utf8-locales ’ package used on GNU/Hurd
(name . Ludovic Courtès)(address . ludo@gnu.org)
87il7ckjz5.fsf@gnu.org
Ludovic Courtès writes:

Hey!

Toggle quote (25 lines)
> We discussed it briefly on IRC the other day: our packages get built on
> i586-gnu with the wrong ‘glibc-utf8-locales’ package (2.35 instead of
> 2.37), which causes Coreutils among others to fail to build:
>
> environment variable `GUIX_LOCPATH' set to `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
> phase `set-paths' succeeded after 0.0 seconds
> starting phase `install-locale'
> warning: failed to install 'en_US.utf8' locale: Invalid argument
> phase `install-locale' succeeded after 0.0 seconds
> […]
> starting phase `remove-tests'
> error: in phase 'remove-tests': uncaught exception:
> decoding-error "decode-char" "input decoding error" 1073741930 #<input: tests/misc/ls-misc.pl 15>
> phase `remove-tests' failed after 0.1 seconds
> […]
> builder for `/gnu/store/vvp0yxvyxsrwmmzli7dsxinr6p9ba3mj-coreutils-9.1.drv' failed with exit code 1
>
> (This is from <https://ci.guix.gnu.org/build/2062597/details>, made with
> commit cdbd81ce144f17644ceebd3d08723aa244696a05.)
>
> So we need a better fix than the local workaround in
> 21deb89e287b5821975544118bf137562a91d4e1.
>
> Thoughts? Perhaps you’ve looked into it already?

Hmm. I've briefly looked at this but failed to reproduce it. I've
tried building coreutils, and coreutils-final in a childhurd created
from "a recent" hurd-team branch.

Toggle snippet (13 lines)
root@guixydevel ~/src/guix/hurd-team [env]# ./pre-inst-env guix build --keep-failed -e '(@@ (gnu packages commencement) coreutils-final)' --without-tests=coreutils
[..]
environment variable `GUIX_LOCPATH' set to `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
[..]
phase `unpack' succeeded after 10.4 seconds
starting phase `remove-tests'
phase `remove-tests' succeeded after 0.5 seconds
starting phase `bootstrap'
[..]
successfully built /gnu/store/zryfw42ayqpmk3s15a7s2cn231xsyjf0-coreutils-9.1.drv
/gnu/store/zbdppljxvvw3vc6lz64h5ic3fvihdr7q-coreutils-9.1

and similar for coreutils.

I've seen a similar error before trying to build guile-avahi a while ago
(before 21deb89e287b5821975544118bf137562a91d4e1) and it really puzzled
me. The idea that a mismatch between GUIX_LOCPATH's glibc version for
locales (2.35) and the glibc actually used (2.37) would cause this
mysterious bug, is kind of a relief...

...although I've got no idea what causes this mismatch or how to fix it
;)

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 14 Oct 2023 19:41
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
87o7h1cda6.fsf@gnu.org
Hi,

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (4 lines)
>> starting phase `remove-tests'
>> error: in phase 'remove-tests': uncaught exception:
>> decoding-error "decode-char" "input decoding error" 1073741930 #<input: tests/misc/ls-misc.pl 15>

[...]

Toggle quote (12 lines)
> Hmm. I've briefly looked at this but failed to reproduce it. I've
> tried building coreutils, and coreutils-final in a childhurd created
> from "a recent" hurd-team branch.
>
> root@guixydevel ~/src/guix/hurd-team [env]# ./pre-inst-env guix build --keep-failed -e '(@@ (gnu packages commencement) coreutils-final)' --without-tests=coreutils
> [..]
> environment variable `GUIX_LOCPATH' set to `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
> [..]
> phase `unpack' succeeded after 10.4 seconds
> starting phase `remove-tests'
> phase `remove-tests' succeeded after 0.5 seconds

Maybe something differs on ‘hurd-team’? For me it’s 100% reproducible
on ‘master’, even though my childhurd has
/run/current-system/locale/2.37 (I thought this could interfere but
luckily it doesn’t.)

Anyway, in both cases the core issue remains: we’re building packages
with the wrong locale data.

The mismatch comes from the fact that ‘glibc-utf8-locales’ is a
system-independent package: you get 2.35 regardless of the system you’re
targeting.

Thanks,
Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 14 Oct 2023 22:22
(name . Ludovic Courtès)(address . ludo@gnu.org)
871qdxj6o8.fsf@gnu.org
Ludovic Courtès writes:

Toggle quote (28 lines)
> Hi,
>
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>>> starting phase `remove-tests'
>>> error: in phase 'remove-tests': uncaught exception:
>>> decoding-error "decode-char" "input decoding error" 1073741930
>>> #<input: tests/misc/ls-misc.pl 15>
>
> [...]
>
>> Hmm. I've briefly looked at this but failed to reproduce it. I've
>> tried building coreutils, and coreutils-final in a childhurd created
>> from "a recent" hurd-team branch.
>>
>> root@guixydevel ~/src/guix/hurd-team [env]# ./pre-inst-env guix
>> build --keep-failed -e '(@@ (gnu packages commencement)
>> coreutils-final)' --without-tests=coreutils
>> [..]
>> environment variable `GUIX_LOCPATH' set to
>> `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
>> [..]
>> phase `unpack' succeeded after 10.4 seconds
>> starting phase `remove-tests'
>> phase `remove-tests' succeeded after 0.5 seconds
>
> Maybe something differs on ‘hurd-team’?

Well, yeah. I've been building in a childhurd created from
gnu/system/examples/devel-hurd.tmpl, which currently has

(locale-libcs (if (system-hurd?)
(list glibc/hurd)
%default-locale-libcs))

Toggle quote (12 lines)
> For me it’s 100% reproducible
> on ‘master’, even though my childhurd has
> /run/current-system/locale/2.37 (I thought this could interfere but
> luckily it doesn’t.)
>
> Anyway, in both cases the core issue remains: we’re building packages
> with the wrong locale data.
>
> The mismatch comes from the fact that ‘glibc-utf8-locales’ is a
> system-independent package: you get 2.35 regardless of the system you’re
> targeting.

Right. Is that easy, difficult, or impossible to change?

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 21 Oct 2023 16:34
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
87r0looxhs.fsf@gnu.org
Hi,

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (2 lines)
> Ludovic Courtès writes:

[...]

Toggle quote (9 lines)
>> Anyway, in both cases the core issue remains: we’re building packages
>> with the wrong locale data.
>>
>> The mismatch comes from the fact that ‘glibc-utf8-locales’ is a
>> system-independent package: you get 2.35 regardless of the system you’re
>> targeting.
>
> Right. Is that easy, difficult, or impossible to change?

We could define ‘glibc-utf8-locales’ with ‘define/system-dependent’, as
we’ve done in commencement.scm. However, I don’t think that’s feasible
because then every place that does:

(module-ref (resolve-interface '(gnu packages base))
'glibc-utf8-locales)

will suddenly be broken, and that’s not acceptable.

So I’m not sure what to do. Again I feel that maintaining two libc
variants is too costly. Time to upgrade in ‘core-updates’?

Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 22 Oct 2023 14:26
(name . Ludovic Courtès)(address . ludo@gnu.org)
87fs22g7w4.fsf@gnu.org
Ludovic Courtès writes:

Hello,

Toggle quote (22 lines)
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Ludovic Courtès writes:
>>
>>> Anyway, in both cases the core issue remains: we’re building packages
>>> with the wrong locale data.
>>>
>>> The mismatch comes from the fact that ‘glibc-utf8-locales’ is a
>>> system-independent package: you get 2.35 regardless of the system you’re
>>> targeting.
>>
>> Right. Is that easy, difficult, or impossible to change?
>
> We could define ‘glibc-utf8-locales’ with ‘define/system-dependent’, as
> we’ve done in commencement.scm. However, I don’t think that’s feasible
> because then every place that does:
>
> (module-ref (resolve-interface '(gnu packages base))
> 'glibc-utf8-locales)
>
> will suddenly be broken, and that’s not acceptable.

Well, unless maybe in the same patch it could also be un-broken?

Toggle quote (3 lines)
> So I’m not sure what to do. Again I feel that maintaining two libc
> variants is too costly. Time to upgrade in ‘core-updates’?

Yeah, that would work...until we really need another glibc update for
the Hurd. I really don't know what's wise here.

So...I've tried the attached to patches "how hard could it be?" (that's
not using define/system-dependent just yet) but get

Toggle snippet (4 lines)
error: failed to load 'guix/self.scm':
ice-9/eval.scm:293:34: Wrong type to apply: #<promise #<procedure 7fae0716b620 at ice-9/eval.scm:330:13 ()>>

...and now I feel stuck.

Greetings,
Janneke
From da6027537f2146bb0d62228de2ea15fb271027ea Mon Sep 17 00:00:00 2001
Message-ID: <da6027537f2146bb0d62228de2ea15fb271027ea.1697977363.git.janneke@gnu.org>
From: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Wed, 7 Jun 2023 19:19:01 +0200
Subject: [PATCH 1/2] gnu: Add libc-locales-for-target and glibc-locales/hurd.

* gnu/packages/base.scm (glibc-locales/hurd): New variable
(libc-locales-for-target): Use it in new procedure.
(glibc-utf8-locales/hurd): New variable.
(libc-utf8-locales-for-target): Use it in new procedure.
---
gnu/packages/base.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (49 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 2d8e9143cd..5c0e056261 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -76,6 +76,8 @@ (define-module (gnu packages base)
#:use-module (srfi srfi-26)
#:export (glibc
libc-for-target
+ libc-locales-for-target
+ libc-utf8-locales-for-target
make-ld-wrapper
libiconv-if-needed
%final-inputs))
@@ -1521,6 +1523,31 @@ (define* (libc-for-target #:optional
(_
glibc)))
+(define-public glibc-locales/hurd
+ (make-glibc-locales glibc/hurd))
+
+(define* (libc-locales-for-target #:optional
+ (target (or (%current-target-system)
+ (%current-system))))
+ (match target
+ ((? target-hurd?)
+ glibc-locales/hurd)
+ (_
+ glibc-locales)))
+
+(define-public glibc-utf8-locales/hurd
+ (hidden-package
+ (make-glibc-utf8-locales glibc/hurd)))
+
+(define* (libc-utf8-locales-for-target #:optional
+ (target (or (%current-target-system)
+ (%current-system))))
+ (match target
+ ((? target-hurd?)
+ glibc-utf8-locales/hurd)
+ (_
+ glibc-utf8-locales)))
+
(define-public tzdata
(package
(name "tzdata")

base-commit: e6af40d7b46b5c9e397a38c62c885fb42ccd9d26
--
2.41.0
Attachment: 0002-DRAFT-Use-libc-utf8-locales-for-target.patch
--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 25 Nov 2023 17:51
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
877cm54vz5.fsf@gnu.org
Hello!

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (2 lines)
> Ludovic Courtès writes:

[...]

Toggle quote (9 lines)
>> We could define ‘glibc-utf8-locales’ with ‘define/system-dependent’, as
>> we’ve done in commencement.scm. However, I don’t think that’s feasible
>> because then every place that does:
>>
>> (module-ref (resolve-interface '(gnu packages base))
>> 'glibc-utf8-locales)
>>
>> will suddenly be broken, and that’s not acceptable.

[...]

Toggle quote (11 lines)
>>From da6027537f2146bb0d62228de2ea15fb271027ea Mon Sep 17 00:00:00 2001
> Message-ID: <da6027537f2146bb0d62228de2ea15fb271027ea.1697977363.git.janneke@gnu.org>
> From: Janneke Nieuwenhuizen <janneke@gnu.org>
> Date: Wed, 7 Jun 2023 19:19:01 +0200
> Subject: [PATCH 1/2] gnu: Add libc-locales-for-target and glibc-locales/hurd.
>
> * gnu/packages/base.scm (glibc-locales/hurd): New variable
> (libc-locales-for-target): Use it in new procedure.
> (glibc-utf8-locales/hurd): New variable.
> (libc-utf8-locales-for-target): Use it in new procedure.

[...]

Toggle quote (8 lines)
>>From 345683fea1be7e6f186fe45b59198caa9ba36890 Mon Sep 17 00:00:00 2001
> Message-ID: <345683fea1be7e6f186fe45b59198caa9ba36890.1697977363.git.janneke@gnu.org>
> In-Reply-To: <da6027537f2146bb0d62228de2ea15fb271027ea.1697977363.git.janneke@gnu.org>
> References: <da6027537f2146bb0d62228de2ea15fb271027ea.1697977363.git.janneke@gnu.org>
> From: Janneke Nieuwenhuizen <janneke@gnu.org>
> Date: Sun, 22 Oct 2023 10:23:19 +0200
> Subject: [PATCH 2/2] DRAFT Use libc-utf8-locales-for-target.

I think we’ll need these two patches eventually; for now, commit
95ea1277ae2ebd278bdb51a7887f5ba1116fbc64 fixes the default
‘glibc-utf8-locales’ package, the one that’s added implicitly by all
build systems, which unlocks basic builds.

it up!

Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 27 Nov 2023 18:12
(name . Ludovic Courtès)(address . ludo@gnu.org)
87jzq3ce8d.fsf@gnu.org
Ludovic Courtès writes:

Hi!

Toggle quote (22 lines)
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Ludovic Courtès writes:
>
> [...]
>
>>> We could define ‘glibc-utf8-locales’ with ‘define/system-dependent’, as
>>> we’ve done in commencement.scm. However, I don’t think that’s feasible
>>> because then every place that does:
>>>
>>> (module-ref (resolve-interface '(gnu packages base))
>>> 'glibc-utf8-locales)
>>>
>>> will suddenly be broken, and that’s not acceptable.
>
> [...]
>
>> Subject: [PATCH 1/2] gnu: Add libc-locales-for-target and glibc-locales/hurd.
>> Subject: [PATCH 2/2] DRAFT Use libc-utf8-locales-for-target.
>
> I think we’ll need these two patches eventually; for now,

Ok, we'll see; git is patient :)

Toggle quote (8 lines)
> commit
> 95ea1277ae2ebd278bdb51a7887f5ba1116fbc64 fixes the default
> ‘glibc-utf8-locales’ package, the one that’s added implicitly by all
> build systems, which unlocks basic builds.
>
> Now waiting for <https://ci.guix.gnu.org/jobset/hurd-packages> to pick
> it up!

Awesome...yeah, it's "evaluating"...

Seems like we're (almost?) there!

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 7 Dec 2023 10:29
control message for bug #66472
(address . control@debbugs.gnu.org)
878r66fjiz.fsf@gnu.org
close 66472
quit
?