Wrong ‘glibc-utf8-locales’ package used on GNU/Hurd

  • Done
  • quality assurance status badge
Details
2 participants
  • Janneke Nieuwenhuizen
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 11 Oct 2023 23:42
(address . bug-guix@gnu.org)
87wmvsq1ia.fsf@inria.fr
Hi!

We discussed it briefly on IRC the other day: our packages get built on
i586-gnu with the wrong ‘glibc-utf8-locales’ package (2.35 instead of
2.37), which causes Coreutils among others to fail to build:

Toggle snippet (14 lines)
environment variable `GUIX_LOCPATH' set to `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
phase `set-paths' succeeded after 0.0 seconds
starting phase `install-locale'
warning: failed to install 'en_US.utf8' locale: Invalid argument
phase `install-locale' succeeded after 0.0 seconds
[…]
starting phase `remove-tests'
error: in phase 'remove-tests': uncaught exception:
decoding-error "decode-char" "input decoding error" 1073741930 #<input: tests/misc/ls-misc.pl 15>
phase `remove-tests' failed after 0.1 seconds
[…]
builder for `/gnu/store/vvp0yxvyxsrwmmzli7dsxinr6p9ba3mj-coreutils-9.1.drv' failed with exit code 1

commit cdbd81ce144f17644ceebd3d08723aa244696a05.)

So we need a better fix than the local workaround in
21deb89e287b5821975544118bf137562a91d4e1.

Thoughts? Perhaps you’ve looked into it already?

Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 12 Oct 2023 16:12
Re: bug#66472: Wrong ‘glibc-utf8-locales ’ package used on GNU/Hurd
(name . Ludovic Courtès)(address . ludo@gnu.org)
87il7ckjz5.fsf@gnu.org
Ludovic Courtès writes:

Hey!

Toggle quote (25 lines)
> We discussed it briefly on IRC the other day: our packages get built on
> i586-gnu with the wrong ‘glibc-utf8-locales’ package (2.35 instead of
> 2.37), which causes Coreutils among others to fail to build:
>
> environment variable `GUIX_LOCPATH' set to `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
> phase `set-paths' succeeded after 0.0 seconds
> starting phase `install-locale'
> warning: failed to install 'en_US.utf8' locale: Invalid argument
> phase `install-locale' succeeded after 0.0 seconds
> […]
> starting phase `remove-tests'
> error: in phase 'remove-tests': uncaught exception:
> decoding-error "decode-char" "input decoding error" 1073741930 #<input: tests/misc/ls-misc.pl 15>
> phase `remove-tests' failed after 0.1 seconds
> […]
> builder for `/gnu/store/vvp0yxvyxsrwmmzli7dsxinr6p9ba3mj-coreutils-9.1.drv' failed with exit code 1
>
> (This is from <https://ci.guix.gnu.org/build/2062597/details>, made with
> commit cdbd81ce144f17644ceebd3d08723aa244696a05.)
>
> So we need a better fix than the local workaround in
> 21deb89e287b5821975544118bf137562a91d4e1.
>
> Thoughts? Perhaps you’ve looked into it already?

Hmm. I've briefly looked at this but failed to reproduce it. I've
tried building coreutils, and coreutils-final in a childhurd created
from "a recent" hurd-team branch.

Toggle snippet (13 lines)
root@guixydevel ~/src/guix/hurd-team [env]# ./pre-inst-env guix build --keep-failed -e '(@@ (gnu packages commencement) coreutils-final)' --without-tests=coreutils
[..]
environment variable `GUIX_LOCPATH' set to `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
[..]
phase `unpack' succeeded after 10.4 seconds
starting phase `remove-tests'
phase `remove-tests' succeeded after 0.5 seconds
starting phase `bootstrap'
[..]
successfully built /gnu/store/zryfw42ayqpmk3s15a7s2cn231xsyjf0-coreutils-9.1.drv
/gnu/store/zbdppljxvvw3vc6lz64h5ic3fvihdr7q-coreutils-9.1

and similar for coreutils.

I've seen a similar error before trying to build guile-avahi a while ago
(before 21deb89e287b5821975544118bf137562a91d4e1) and it really puzzled
me. The idea that a mismatch between GUIX_LOCPATH's glibc version for
locales (2.35) and the glibc actually used (2.37) would cause this
mysterious bug, is kind of a relief...

...although I've got no idea what causes this mismatch or how to fix it
;)

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 14 Oct 2023 19:41
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
87o7h1cda6.fsf@gnu.org
Hi,

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (4 lines)
>> starting phase `remove-tests'
>> error: in phase 'remove-tests': uncaught exception:
>> decoding-error "decode-char" "input decoding error" 1073741930 #<input: tests/misc/ls-misc.pl 15>

[...]

Toggle quote (12 lines)
> Hmm. I've briefly looked at this but failed to reproduce it. I've
> tried building coreutils, and coreutils-final in a childhurd created
> from "a recent" hurd-team branch.
>
> root@guixydevel ~/src/guix/hurd-team [env]# ./pre-inst-env guix build --keep-failed -e '(@@ (gnu packages commencement) coreutils-final)' --without-tests=coreutils
> [..]
> environment variable `GUIX_LOCPATH' set to `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
> [..]
> phase `unpack' succeeded after 10.4 seconds
> starting phase `remove-tests'
> phase `remove-tests' succeeded after 0.5 seconds

Maybe something differs on ‘hurd-team’? For me it’s 100% reproducible
on ‘master’, even though my childhurd has
/run/current-system/locale/2.37 (I thought this could interfere but
luckily it doesn’t.)

Anyway, in both cases the core issue remains: we’re building packages
with the wrong locale data.

The mismatch comes from the fact that ‘glibc-utf8-locales’ is a
system-independent package: you get 2.35 regardless of the system you’re
targeting.

Thanks,
Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 14 Oct 2023 22:22
(name . Ludovic Courtès)(address . ludo@gnu.org)
871qdxj6o8.fsf@gnu.org
Ludovic Courtès writes:

Toggle quote (28 lines)
> Hi,
>
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>>> starting phase `remove-tests'
>>> error: in phase 'remove-tests': uncaught exception:
>>> decoding-error "decode-char" "input decoding error" 1073741930
>>> #<input: tests/misc/ls-misc.pl 15>
>
> [...]
>
>> Hmm. I've briefly looked at this but failed to reproduce it. I've
>> tried building coreutils, and coreutils-final in a childhurd created
>> from "a recent" hurd-team branch.
>>
>> root@guixydevel ~/src/guix/hurd-team [env]# ./pre-inst-env guix
>> build --keep-failed -e '(@@ (gnu packages commencement)
>> coreutils-final)' --without-tests=coreutils
>> [..]
>> environment variable `GUIX_LOCPATH' set to
>> `/gnu/store/sq6w1nfi59askjfq6b1nqq6z8ld5zh1l-glibc-utf8-locales-2.35/lib/locale'
>> [..]
>> phase `unpack' succeeded after 10.4 seconds
>> starting phase `remove-tests'
>> phase `remove-tests' succeeded after 0.5 seconds
>
> Maybe something differs on ‘hurd-team’?

Well, yeah. I've been building in a childhurd created from
gnu/system/examples/devel-hurd.tmpl, which currently has

(locale-libcs (if (system-hurd?)
(list glibc/hurd)
%default-locale-libcs))

Toggle quote (12 lines)
> For me it’s 100% reproducible
> on ‘master’, even though my childhurd has
> /run/current-system/locale/2.37 (I thought this could interfere but
> luckily it doesn’t.)
>
> Anyway, in both cases the core issue remains: we’re building packages
> with the wrong locale data.
>
> The mismatch comes from the fact that ‘glibc-utf8-locales’ is a
> system-independent package: you get 2.35 regardless of the system you’re
> targeting.

Right. Is that easy, difficult, or impossible to change?

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 21 Oct 2023 16:34
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
87r0looxhs.fsf@gnu.org
Hi,

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (2 lines)
> Ludovic Courtès writes:

[...]

Toggle quote (9 lines)
>> Anyway, in both cases the core issue remains: we’re building packages
>> with the wrong locale data.
>>
>> The mismatch comes from the fact that ‘glibc-utf8-locales’ is a
>> system-independent package: you get 2.35 regardless of the system you’re
>> targeting.
>
> Right. Is that easy, difficult, or impossible to change?

We could define ‘glibc-utf8-locales’ with ‘define/system-dependent’, as
we’ve done in commencement.scm. However, I don’t think that’s feasible
because then every place that does:

(module-ref (resolve-interface '(gnu packages base))
'glibc-utf8-locales)

will suddenly be broken, and that’s not acceptable.

So I’m not sure what to do. Again I feel that maintaining two libc
variants is too costly. Time to upgrade in ‘core-updates’?

Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 22 Oct 2023 14:26
(name . Ludovic Courtès)(address . ludo@gnu.org)
87fs22g7w4.fsf@gnu.org
Ludovic Courtès writes:

Hello,

Toggle quote (22 lines)
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Ludovic Courtès writes:
>>
>>> Anyway, in both cases the core issue remains: we’re building packages
>>> with the wrong locale data.
>>>
>>> The mismatch comes from the fact that ‘glibc-utf8-locales’ is a
>>> system-independent package: you get 2.35 regardless of the system you’re
>>> targeting.
>>
>> Right. Is that easy, difficult, or impossible to change?
>
> We could define ‘glibc-utf8-locales’ with ‘define/system-dependent’, as
> we’ve done in commencement.scm. However, I don’t think that’s feasible
> because then every place that does:
>
> (module-ref (resolve-interface '(gnu packages base))
> 'glibc-utf8-locales)
>
> will suddenly be broken, and that’s not acceptable.

Well, unless maybe in the same patch it could also be un-broken?

Toggle quote (3 lines)
> So I’m not sure what to do. Again I feel that maintaining two libc
> variants is too costly. Time to upgrade in ‘core-updates’?

Yeah, that would work...until we really need another glibc update for
the Hurd. I really don't know what's wise here.

So...I've tried the attached to patches "how hard could it be?" (that's
not using define/system-dependent just yet) but get

Toggle snippet (4 lines)
error: failed to load 'guix/self.scm':
ice-9/eval.scm:293:34: Wrong type to apply: #<promise #<procedure 7fae0716b620 at ice-9/eval.scm:330:13 ()>>

...and now I feel stuck.

Greetings,
Janneke
From da6027537f2146bb0d62228de2ea15fb271027ea Mon Sep 17 00:00:00 2001
Message-ID: <da6027537f2146bb0d62228de2ea15fb271027ea.1697977363.git.janneke@gnu.org>
From: Janneke Nieuwenhuizen <janneke@gnu.org>
Date: Wed, 7 Jun 2023 19:19:01 +0200
Subject: [PATCH 1/2] gnu: Add libc-locales-for-target and glibc-locales/hurd.

* gnu/packages/base.scm (glibc-locales/hurd): New variable
(libc-locales-for-target): Use it in new procedure.
(glibc-utf8-locales/hurd): New variable.
(libc-utf8-locales-for-target): Use it in new procedure.
---
gnu/packages/base.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (49 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 2d8e9143cd..5c0e056261 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -76,6 +76,8 @@ (define-module (gnu packages base)
#:use-module (srfi srfi-26)
#:export (glibc
libc-for-target
+ libc-locales-for-target
+ libc-utf8-locales-for-target
make-ld-wrapper
libiconv-if-needed
%final-inputs))
@@ -1521,6 +1523,31 @@ (define* (libc-for-target #:optional
(_
glibc)))
+(define-public glibc-locales/hurd
+ (make-glibc-locales glibc/hurd))
+
+(define* (libc-locales-for-target #:optional
+ (target (or (%current-target-system)
+ (%current-system))))
+ (match target
+ ((? target-hurd?)
+ glibc-locales/hurd)
+ (_
+ glibc-locales)))
+
+(define-public glibc-utf8-locales/hurd
+ (hidden-package
+ (make-glibc-utf8-locales glibc/hurd)))
+
+(define* (libc-utf8-locales-for-target #:optional
+ (target (or (%current-target-system)
+ (%current-system))))
+ (match target
+ ((? target-hurd?)
+ glibc-utf8-locales/hurd)
+ (_
+ glibc-utf8-locales)))
+
(define-public tzdata
(package
(name "tzdata")

base-commit: e6af40d7b46b5c9e397a38c62c885fb42ccd9d26
--
2.41.0
Attachment: 0002-DRAFT-Use-libc-utf8-locales-for-target.patch
--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 25 Nov 2023 17:51
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
877cm54vz5.fsf@gnu.org
Hello!

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (2 lines)
> Ludovic Courtès writes:

[...]

Toggle quote (9 lines)
>> We could define ‘glibc-utf8-locales’ with ‘define/system-dependent’, as
>> we’ve done in commencement.scm. However, I don’t think that’s feasible
>> because then every place that does:
>>
>> (module-ref (resolve-interface '(gnu packages base))
>> 'glibc-utf8-locales)
>>
>> will suddenly be broken, and that’s not acceptable.

[...]

Toggle quote (11 lines)
>>From da6027537f2146bb0d62228de2ea15fb271027ea Mon Sep 17 00:00:00 2001
> Message-ID: <da6027537f2146bb0d62228de2ea15fb271027ea.1697977363.git.janneke@gnu.org>
> From: Janneke Nieuwenhuizen <janneke@gnu.org>
> Date: Wed, 7 Jun 2023 19:19:01 +0200
> Subject: [PATCH 1/2] gnu: Add libc-locales-for-target and glibc-locales/hurd.
>
> * gnu/packages/base.scm (glibc-locales/hurd): New variable
> (libc-locales-for-target): Use it in new procedure.
> (glibc-utf8-locales/hurd): New variable.
> (libc-utf8-locales-for-target): Use it in new procedure.

[...]

Toggle quote (8 lines)
>>From 345683fea1be7e6f186fe45b59198caa9ba36890 Mon Sep 17 00:00:00 2001
> Message-ID: <345683fea1be7e6f186fe45b59198caa9ba36890.1697977363.git.janneke@gnu.org>
> In-Reply-To: <da6027537f2146bb0d62228de2ea15fb271027ea.1697977363.git.janneke@gnu.org>
> References: <da6027537f2146bb0d62228de2ea15fb271027ea.1697977363.git.janneke@gnu.org>
> From: Janneke Nieuwenhuizen <janneke@gnu.org>
> Date: Sun, 22 Oct 2023 10:23:19 +0200
> Subject: [PATCH 2/2] DRAFT Use libc-utf8-locales-for-target.

I think we’ll need these two patches eventually; for now, commit
95ea1277ae2ebd278bdb51a7887f5ba1116fbc64 fixes the default
‘glibc-utf8-locales’ package, the one that’s added implicitly by all
build systems, which unlocks basic builds.

it up!

Ludo’.
J
J
Janneke Nieuwenhuizen wrote on 27 Nov 2023 18:12
(name . Ludovic Courtès)(address . ludo@gnu.org)
87jzq3ce8d.fsf@gnu.org
Ludovic Courtès writes:

Hi!

Toggle quote (22 lines)
> Janneke Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Ludovic Courtès writes:
>
> [...]
>
>>> We could define ‘glibc-utf8-locales’ with ‘define/system-dependent’, as
>>> we’ve done in commencement.scm. However, I don’t think that’s feasible
>>> because then every place that does:
>>>
>>> (module-ref (resolve-interface '(gnu packages base))
>>> 'glibc-utf8-locales)
>>>
>>> will suddenly be broken, and that’s not acceptable.
>
> [...]
>
>> Subject: [PATCH 1/2] gnu: Add libc-locales-for-target and glibc-locales/hurd.
>> Subject: [PATCH 2/2] DRAFT Use libc-utf8-locales-for-target.
>
> I think we’ll need these two patches eventually; for now,

Ok, we'll see; git is patient :)

Toggle quote (8 lines)
> commit
> 95ea1277ae2ebd278bdb51a7887f5ba1116fbc64 fixes the default
> ‘glibc-utf8-locales’ package, the one that’s added implicitly by all
> build systems, which unlocks basic builds.
>
> Now waiting for <https://ci.guix.gnu.org/jobset/hurd-packages> to pick
> it up!

Awesome...yeah, it's "evaluating"...

Seems like we're (almost?) there!

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 7 Dec 2023 10:29
control message for bug #66472
(address . control@debbugs.gnu.org)
878r66fjiz.fsf@gnu.org
close 66472
quit
?
Your comment

This issue is archived.

To comment on this conversation send an email to 66472@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 66472
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch