'glibc-locales' lacks symlinks for locale names with normalized codeset

  • Done
  • quality assurance status badge
Details
3 participants
  • Giovanni Biscuolo
  • Jack Hill
  • Ludovic Courtès
Owner
unassigned
Submitted by
Jack Hill
Severity
important
J
J
Jack Hill wrote on 3 Jun 2019 20:39
Manual should clarify that glibc-utf8-locales is needed by default on foreign distros
(address . bug-guix@gnu.org)
alpine.DEB.2.20.1906031430060.5164@marsh.hcoop.net
Hi Guix,

While setting up Guix on a foreign distribution (CentOS 7), I elected to
use the full glibc-locales while following section 2.6.1 of the manual for
application setup. I installed the glibc-locales package in both my user's
profile and root's so that the locales would be available to guix-daemon.

However, I was surprised that even though I had the full set of locales
available guix-daemon couldn't find the locale it was looking for. This is
because the default systemd unit for guix-daemon configures it to use the
en_US.utf8 locale which is provided by glibc-utf8-locales (en_US.UTF-8 is
in glibc-locales).

I think it would be good to clarify in the manual which locale package is
needed when using the default daemon configuration on systemd foreign
distros.

I also talked about my experience on help-guix: https://lists.gnu.org/archive/html/help-guix/2019-06/msg00024.html

Best,
Jack
L
L
Ludovic Courtès wrote on 5 Jun 2019 15:17
(name . Jack Hill)(address . jackhill@jackhill.us)(address . 36076@debbugs.gnu.org)
87v9xk2mt6.fsf@gnu.org
Hi,

Jack Hill <jackhill@jackhill.us> skribis:

Toggle quote (10 lines)
> However, I was surprised that even though I had the full set of
> locales available guix-daemon couldn't find the locale it was looking
> for. This is because the default systemd unit for guix-daemon
> configures it to use the en_US.utf8 locale which is provided by
> glibc-utf8-locales (en_US.UTF-8 is in glibc-locales).
>
> I think it would be good to clarify in the manual which locale package
> is needed when using the default daemon configuration on systemd
> foreign distros.

I honestly don’t see how it could be clearer:


Could you propose a patch to this section?

Thanks,
Ludo’.
G
G
Giovanni Biscuolo wrote on 5 Jun 2019 19:28
87o93ct006.fsf@roquette.mug.biscuolo.net
Hello Jack,

I'm also on foreign distro so maybe I could help to clarify what's
missing here

Did you use the last shell installer script [1] you or did it
"manually" using the last binary tarball?


Jack Hill <jackhill@jackhill.us> writes:

[...]

Toggle quote (4 lines)
> application setup. I installed the glibc-locales package in both my user's
> profile and root's so that the locales would be available to
> guix-daemon.

I installed glibc-utf8-locales in the root profile:

Toggle snippet (3 lines)
glibc-utf8-locales 2.28 out /gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28

and both in my user profile:

Toggle snippet (4 lines)
glibc-utf8-locales 2.28 out /gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28
glibc-locales 2.28 out /gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28

Toggle quote (3 lines)
> However, I was surprised that even though I had the full set of locales
> available guix-daemon couldn't find the locale it was looking for.

Please are you able to reproduce the error you got from guix-daemon?
What's the error message?

Toggle quote (5 lines)
> This is
> because the default systemd unit for guix-daemon configures it to use the
> en_US.utf8 locale which is provided by glibc-utf8-locales (en_US.UTF-8 is
> in glibc-locales).

Well, actually both en_US.utf8 and en_US.UTF-8 are both in
glibc-utf8-locales:

Toggle snippet (5 lines)
~$ ls -lah /var/guix/profiles/per-user/root/guix-profile/lib/locale/2.28/ | grep en_US
dr-xr-xr-x 3 root root 4.0K Jan 1 1970 en_US.utf8
lrwxrwxrwx 3 root root 10 Jan 1 1970 en_US.UTF-8 -> en_US.utf8

and glibc-locales

Toggle snippet (6 lines)
$ ls -lah /var/guix/profiles/per-user/giovanni/guix-profile/lib/locale/2.28/ | grep en_US
lrwxrwxrwx 33 root root 84 Jan 1 1970 en_US -> /gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28/lib/locale/2.28/en_US
lrwxrwxrwx 33 root root 94 Jan 1 1970 en_US.utf8 -> /gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28/lib/locale/2.28/en_US.utf8
dr-xr-xr-x 3 root root 4.0K Jan 1 1970 en_US.UTF-8

So AFAIU glibc-locales provides both en_US.UTF-8 and en_US.utf8 that is
used by guix-daemon.

HTH! Gio'.

--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEERcxjuFJYydVfNLI5030Op87MORIFAlz3+7oACgkQ030Op87M
ORIJ1A/7B/5/MNfqiUsGouZ6NL+1+Cc8WCwbmWra67CnBOE63y5NG2OZpwLaxNcp
dIQ60ArRNq/hq06QV27UvRLDIuMO2j3Y4wPHJViCigjl8mAyLgTlaf2mQc3MCPoy
O1nywcEA+u7IdOCuvy4/U2peKZxVoFNWIa93atyivfCLnLhYW4vf7ygewfBeupNH
sE21tsfljFlCJ+ECg5xjQ4YUCJCDVhTsOWJam31o7l4tC2f86nKtC1VEzG+Uxv43
HVvXU2GOPp61rlebgc9jk/Xp6SgDIPND9CyrmEFfEXlZxFdnBlMw580qdI0zTiHO
u0VDO/4yE/GEcjAVBq/Lmiu6jILylVCgRVjUf84dnYjMgd2/g5pYZ3xTNdQfAM+B
rEWxWuqCwIA6I0j/+HrEzuBvdgYJvHt8leUdcf7X4a9Xyg3rbdMtaPUjFI40GyiK
zwbXTWPbJ8c9Uph24FtjWck7RJPUmsRFWilVYpfmCqVgOKwURMrf3r+TC6/LlvQ2
rbPNS94OznwONIUHvEI3ez/OG3cXoS+Yuz+XAUQLbgdH+3Bv7iIVISQaqGD+Xx2W
d4QvBRTtPH8AIzXkWLGMqKFNbZlwR/sR9jAvpvuACuSBqaS/Qp64Z6kQ1gY4naNQ
CCsLg7qo8jmhnNwDdmxfLCx/YyAJlBsnVHees5BTCf3t9WzPTic=
=WzSr
-----END PGP SIGNATURE-----

J
J
Jack Hill wrote on 6 Jun 2019 06:32
(name . Giovanni Biscuolo)(address . g@xelera.eu)(address . 36076@debbugs.gnu.org)
alpine.DEB.2.20.1906051702120.5164@marsh.hcoop.net
On Wed, 5 Jun 2019, Giovanni Biscuolo wrote:

Toggle quote (5 lines)
> Hello Jack,
>
> I'm also on foreign distro so maybe I could help to clarify what's
> missing here

Thanks for helping

Toggle quote (5 lines)
> Did you use the last shell installer script [1] you or did it
> "manually" using the last binary tarball?
>
> [1] https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh

I used the shell install script.

Toggle quote (27 lines)
> Jack Hill <jackhill@jackhill.us> writes:
>
> [...]
>
>> application setup. I installed the glibc-locales package in both my user's
>> profile and root's so that the locales would be available to
>> guix-daemon.
>
> I installed glibc-utf8-locales in the root profile:
>
> --8<---------------cut here---------------start------------->8---
> glibc-utf8-locales 2.28 out /gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28
> --8<---------------cut here---------------end--------------->8---
>
> and both in my user profile:
>
> --8<---------------cut here---------------start------------->8---
> glibc-utf8-locales 2.28 out /gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28
> glibc-locales 2.28 out /gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28
> --8<---------------cut here---------------end--------------->8---
>
>> However, I was surprised that even though I had the full set of locales
>> available guix-daemon couldn't find the locale it was looking for.
>
> Please are you able to reproduce the error you got from guix-daemon?
> What's the error message?

I have been able to reproduce this with on a fresh CentOS 7 host with a
freshly installed Guix from the installation script.

I see two error messages. The first occurs anytime a substitute is requested:

```
substitute: /gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
```

The second is in the guix-daemon logs:

```
guix-daemon[37753]: guile: warning: failed to install locale
```

I have included a full terminal session that I believe demonstrates the
problem at the end of this message.

Toggle quote (14 lines)
>> This is
>> because the default systemd unit for guix-daemon configures it to use the
>> en_US.utf8 locale which is provided by glibc-utf8-locales (en_US.UTF-8 is
>> in glibc-locales).
>
> Well, actually both en_US.utf8 and en_US.UTF-8 are both in
> glibc-utf8-locales:
>
> --8<---------------cut here---------------start------------->8---
> ~$ ls -lah /var/guix/profiles/per-user/root/guix-profile/lib/locale/2.28/ | grep en_US
> dr-xr-xr-x 3 root root 4.0K Jan 1 1970 en_US.utf8
> lrwxrwxrwx 3 root root 10 Jan 1 1970 en_US.UTF-8 -> en_US.utf8
> --8<---------------cut here---------------end--------------->8---

ah :)

Toggle quote (12 lines)
> and glibc-locales
>
> --8<---------------cut here---------------start------------->8---
> $ ls -lah /var/guix/profiles/per-user/giovanni/guix-profile/lib/locale/2.28/ | grep en_US
> lrwxrwxrwx 33 root root 84 Jan 1 1970 en_US -> /gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28/lib/locale/2.28/en_US
> lrwxrwxrwx 33 root root 94 Jan 1 1970 en_US.utf8 -> /gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28/lib/locale/2.28/en_US.utf8
> dr-xr-xr-x 3 root root 4.0K Jan 1 1970 en_US.UTF-8
> --8<---------------cut here---------------end--------------->8---
>
> So AFAIU glibc-locales provides both en_US.UTF-8 and en_US.utf8 that is
> used by guix-daemon.

That's not how I read the above lines. It looks like en_US.utf8 is a link
to the 94k5…glib-utf8-locales-2.28 store item.

On my system acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28 does not
contain en_US.utf8

```
$ pwd
/gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28/lib/locale/2.28
$ find -iname '*en_US*'
./en_US
./en_US.UTF-8

Best,
Jack

[root@guix-jackhill-dev-01 ~]# type guix
guix is hashed (/root/.config/guix/current/bin/guix)
[root@guix-jackhill-dev-01 ~]# guix describe
Generation 2 Jun 05 2019 16:52:04 (current)
guix c0f6eeb
branch: master
commit: c0f6eebb6d9f6ca9b62344f32ce5f82dab601d53
[root@guix-jackhill-dev-01 ~]# guix package -l
Generation 4 Jun 05 2019 16:53:10 (current)
hello 2.10 out /gnu/store/md2plii4g5sk66wg9cgwc964l3xwhrm9-hello-2.10
glibc-locales 2.28 out /gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28

[root@guix-jackhill-dev-01 ~]# systemctl stop guix-daemon
[root@guix-jackhill-dev-01 ~]# systemctl start guix-daemon
[root@guix-jackhill-dev-01 ~]# systemctl status -l guix-daemon
? guix-daemon.service - Build daemon for GNU Guix
Loaded: loaded (/etc/systemd/system/guix-daemon.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2019-06-05 16:54:24 EDT; 4s ago
Main PID: 37753 (guix-daemon)
Tasks: 1 (limit: 8192)
CGroup: /system.slice/guix-daemon.service
??37753 guix-daemon --build-users-group=guixbuild

Jun 05 16:54:24 guix-jackhill-dev-01.lib.duke.edu systemd[1]: Started Build daemon for GNU Guix.
Jun 05 16:54:24 guix-jackhill-dev-01.lib.duke.edu guix-daemon[37753]: guile: warning: failed to install locale
[root@guix-jackhill-dev-01 ~]# guix package -i recutils
substitute: /gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
substitute: guile: warning: failed to install locale
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following package will be installed:
recutils 1.8 /gnu/store/163pczynvjfq4l6p7nam5sx9wbz07bgz-recutils-1.8

[unrelated lines elided]

/gnu/store/xa0bg9kq6nn82fzssz0wjsw5gnrl8z00-ca-certificate-bundle.drv
/gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
guile: warning: failed to install locale
module-import-compiled 121KiB 4.2MiB/s 00:00 [##################] 100.0%

/gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
guile: warning: failed to install locale
attr-2.4.47 92KiB 3.0MiB/s 00:00 [##################] 100.0%

[more similar lines elided]

/gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
guile: warning: failed to install locale
recutils-1.8 1.1MiB 1.9MiB/s 00:01 [##################] 100.0%

[unrelated lines elided]

building /gnu/store/63hrmvk6rp7bw0iz530ya4a02s14ld6q-profile.drv...
3 packages in profile
[root@guix-jackhill-dev-01 ~]# guix package -i glibc-utf8-locales
The following package will be installed:
glibc-utf8-locales 2.28 /gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28

substitute: /gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
substitute: guile: warning: failed to install locale
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivation will be built:
/gnu/store/xb5inry74xm915f5pbynp3i1q3qjj6h5-profile.drv
The following profile hooks will be built:
/gnu/store/2dw212srwmqc23qgp44dvsipr9z729rq-manual-database.drv
/gnu/store/d92npn7xlj4dkkl7mk1jirsn1bnf6rlb-ca-certificate-bundle.drv
/gnu/store/fmcbxm998xr40hnpdzs2wizq17zhwgml-info-dir.drv
/gnu/store/rdwrrznp57467vza8d2x079zrf4clxmk-fonts-dir.drv
building CA certificate bundle...
building fonts directory...
building directory of Info manuals...
building database for manual pages...
building /gnu/store/xb5inry74xm915f5pbynp3i1q3qjj6h5-profile.drv...
4 packages in profile
[root@guix-jackhill-dev-01 ~]# systemctl stop guix-daemon
(reverse-i-search)`sta': systemctl ^Catus -l guix-daemon
[root@guix-jackhill-dev-01 ~]# systemctl start guix-daemon
[root@guix-jackhill-dev-01 ~]# systemctl status -l guix-daemon
? guix-daemon.service - Build daemon for GNU Guix
Loaded: loaded (/etc/systemd/system/guix-daemon.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2019-06-05 16:56:06 EDT; 1s ago
Main PID: 38352 (guix-daemon)
Tasks: 1 (limit: 8192)
CGroup: /system.slice/guix-daemon.service
??38352 guix-daemon --build-users-group=guixbuild

Jun 05 16:56:06 guix-jackhill-dev-01.lib.duke.edu systemd[1]: Started Build daemon for GNU Guix.
[root@guix-jackhill-dev-01 ~]# guix package -i units
[n.b. no local errors this time]
updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following package will be installed:
units 2.18 /gnu/store/iqkf11sld63f8f3rs57qlivlgisvrvjj-units-2.18

substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivation will be built:
/gnu/store/ws08mnazgqsbqs0ib2m96vvj2h7cpb2d-profile.drv
52.2 MB will be downloaded:
/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0

[unrelated lines elided]

/gnu/store/zzhq9wv0ikfh7s01a036xivn2b0z42aa-info-dir.drv
python-asn1crypto-0.24.0 204KiB 3.1MiB/s 00:00 [##################] 100.0%

[more similar lines elided]

applying 1 graft for /gnu/store/vs5m35vyskvdg94xk4g42gd9jrc7hczz-python-cffi-1.11.5.drv...
units-2.18 250KiB 740KiB/s 00:00 [##################] 100.0%

[unrelated lines elided]

building /gnu/store/ws08mnazgqsbqs0ib2m96vvj2h7cpb2d-profile.drv...
5 packages in profile
J
J
Jack Hill wrote on 6 Jun 2019 06:46
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 36076@debbugs.gnu.org)
alpine.DEB.2.20.1906060036470.5164@marsh.hcoop.net
Hi Ludo’

On Wed, 5 Jun 2019, Ludovic Courtès wrote:

Toggle quote (6 lines)
> I honestly don’t see how it could be clearer:
>
> https://www.gnu.org/software/guix/manual/en/html_node/Application-Setup.html#Locales
>
> Could you propose a patch to this section?

I think what is missing is a statement that the systemd unit for
guix-daemon on foreign distos runs guix-daemon with a locale that is only
provided in glibc-utf8-locales.

I clairification could be something like, "The provided unit for running
guix-daemon on a foreign distribution configures the daemon to run with a
local that is only provided in glibc-utf8-locales. Therefore,
glibc-utf8-locacles should be installed in root's profile, or the unit
should be configured to use a locale of your choosing."

Of course, that only makes sense if I'm correct about what is going on. I
can propose a patch when Gio' and I figure it out.

Best,
Jack
L
L
Ludovic Courtès wrote on 6 Jun 2019 09:57
(name . Jack Hill)(address . jackhill@jackhill.us)(address . 36076@debbugs.gnu.org)
871s07rvqw.fsf@gnu.org
Hi Jack,

Jack Hill <jackhill@jackhill.us> skribis:

Toggle quote (19 lines)
> On Wed, 5 Jun 2019, Ludovic Courtès wrote:
>
>> I honestly don’t see how it could be clearer:
>>
>> https://www.gnu.org/software/guix/manual/en/html_node/Application-Setup.html#Locales
>>
>> Could you propose a patch to this section?
>
> I think what is missing is a statement that the systemd unit for
> guix-daemon on foreign distos runs guix-daemon with a locale that is
> only provided in glibc-utf8-locales.
>
> I clairification could be something like, "The provided unit for
> running guix-daemon on a foreign distribution configures the daemon to
> run with a local that is only provided in
> glibc-utf8-locales. Therefore, glibc-utf8-locacles should be installed
> in root's profile, or the unit should be configured to use a locale of
> your choosing."

‘glibc-utf8-locales’ is embedded in the ‘guix’ package (the binary
tarball), such that even if you don’t install it, “en_US.utf8” is
available to ‘guix substitute’ & co.

However, IIUC, the problem is with the Bash used by
libexec/guix/substitute, is that right?

Thanks,
Ludo’.
G
G
Giovanni Biscuolo wrote on 6 Jun 2019 11:03
(name . Jack Hill)(address . jackhill@jackhill.us)
87imtjt79n.fsf@roquette.mug.biscuolo.net
Hello Jack,

AFAIU you have found a bug, but not in the manual :-)

The manual states that glibc-utf8-locales is an alternative to
glibc-locales, limited to a few UTF-8 locales; this is how it should be.

The issue is that glibc-locales package does not create symlinks to the
"normalized codeset" like glibc-utf8-locales does (see below for
details): **all* normalized codes like en_US.utf8 are missing.

If I'm right we should change the issue subject to something like
"glibc-locales does not generate normalized codesets", but I'm
not used to debbugs so I'll let this part to others **if applicable**

As a temporary (or permanent :-) ) workaround you could add this in the
env of guix-daemon adding this line to
"/etc/systemd/system/guix-daemon.service":

Toggle snippet (3 lines)
Environment=LC_ALL=en_US.UTF-8

You should also use en_US.UTF-8 instead of en_US.utf8 in your user env.

Jack Hill <jackhill@jackhill.us> writes:

[...]

Toggle quote (13 lines)
>> --8<---------------cut here---------------start------------->8---
>> $ ls -lah /var/guix/profiles/per-user/giovanni/guix-profile/lib/locale/2.28/ | grep en_US
>> lrwxrwxrwx 33 root root 84 Jan 1 1970 en_US -> /gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28/lib/locale/2.28/en_US
>> lrwxrwxrwx 33 root root 94 Jan 1 1970 en_US.utf8 -> /gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28/lib/locale/2.28/en_US.utf8
>> dr-xr-xr-x 3 root root 4.0K Jan 1 1970 en_US.UTF-8
>> --8<---------------cut here---------------end--------------->8---
>>
>> So AFAIU glibc-locales provides both en_US.UTF-8 and en_US.utf8 that is
>> used by guix-daemon.
>
> That's not how I read the above lines. It looks like en_US.utf8 is a link
> to the 94k5…glib-utf8-locales-2.28 store item.

You are right, I overlooked that: I removed glibc-utf8-locales from my
user profile, I just have glibc-locales now:

Toggle snippet (5 lines)
~$ ls -lah /var/guix/profiles/per-user/giovanni/guix-profile/lib/locale/2.28/ | grep en_US
dr-xr-xr-x 3 root root 4.0K Jan 1 1970 en_US
dr-xr-xr-x 3 root root 4.0K Jan 1 1970 en_US.UTF-8

"en_US.utf8" is missing; that "incidentally" is also the default locale
of my user, so:

Toggle snippet (4 lines)
~$ guix package -I
guile: warning: failed to install locale

Setting "LANG=en_US.UTF-8" fixes the problem.

In fact make-glibc-locales in base.scm "just" installs all locales in
"not normalized codeset" with:

Toggle snippet (8 lines)
[...]
(replace 'build
(lambda _
(invoke "make" "localedata/install-locales"
"-j" (number->string (parallel-job-count)))))
[...]

while make-glibc-utf8-locales generates a limited list locales via localedef in
"normalized codeset" and creates symlinks for "not normalized codeset"

Toggle snippet (22 lines)
(for-each (lambda (locale)
(define file
;; Use the "normalized codeset" by
;; default--e.g., "en_US.utf8".
(string-append localedir "/" locale ".utf8"))

(invoke "localedef" "--no-archive"
"--prefix" localedir
"-i" locale
"-f" "UTF-8" file)

;; For backward compatibility with Guix
;; <= 0.8.3, add "xx_YY.UTF-8".
(symlink (string-append locale ".utf8")
(string-append localedir "/"
locale ".UTF-8")))

;; These are the locales commonly used for
;; tests---e.g., in Guile's i18n tests.
'("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))

I don't understand if "normalized codeset" are mandatory or facultative
(see info guix "Locales"), but AFAIU are missing from the making of
glibc localedata/install-locales; for sure they are used by default by a
lot of users in their locale env, and in the default locale of
guix-daemon.

AFAIU one possible workaround is to write a patch that normalizes [1]
all the codesets

Toggle snippet (7 lines)
by applying the following rules:

1. Remove all characters besides numbers and letters.
2. Fold letters to lowercase.
3. If the same only contains digits prepend the string "iso".

and creates a symlink to not normalized ones

An alternative could be to create share/locale/locale.alias with
"normalized codes" aliases

Sorry I'm still not able to propose such patch for make-glibc-locales.

To deprecate "normalized codeset" is not an alternative IMHO

HTH! Gio'.

[...]


--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEERcxjuFJYydVfNLI5030Op87MORIFAlz41vQACgkQ030Op87M
ORKBJg//ba4IV+TqE5JG/SoFXlpT8AXyXIHx+yrpBeY+UWx4mZPNKJo4FzduC8Eg
He7qA4nTuDWwSxuNNmMT6bdWhRtjsFaLgriqMXD+0WNYTVZ3bT3DbfDY+JONW3Ld
x5ik9O92CuFThUqKJhGZ3m1EuJt87IvGFzLFKor1scS9hUbC7xiKamDjrWHhSRkI
pGM8SZQaCK6I91RV1t19weSLb9u/EVEY/MeuS/SrQpTEz2OzrQ+fHuBXz1W9O0yY
5Wz/ldXuQubd18mLrWE2BkfKju1osu6eoKmwBgOpG+UPaHxz6vtOMRJrKS/LjqN/
Cr+ZEt4+gkZGEoHYCeMjH6pce6dfxNsYsoG0dIfsSQYLpTx12k2qg86SkLY6711x
LFW4ZymaYmdE545mHXIaSEZutcaMPRDuxWyALgEoWrpznuR7zwJaCzVdKscm3DNQ
zCokGCZipld6cIdSoCfHaM8TuLpYCL5A5+pqgRQUlUeqV8yzZyTgjQZGQP3rpaB+
QID/OM0FmkN2O+npkybz5WjTak6RDfa1/l3Hhs1SZzfC0BTczqlFFGpC1KFZiam6
o8AeV/jJXDP+2AZ7/yxjl3DP0ihc0YksYwptky+85KkirRrTNkPDFY6v/h5bxaRn
EASIPNjbEMe9rrgF0N0x2cl3qhJd9CWwSUMi4Mnum3+Nawho5Es=
=ZM7Q
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 6 Jun 2019 17:38
(name . Giovanni Biscuolo)(address . g@xelera.eu)
87y32elo5z.fsf@gnu.org
Hi Giovanni,

Giovanni Biscuolo <g@xelera.eu> skribis:

Toggle quote (7 lines)
> You are right, I overlooked that: I removed glibc-utf8-locales from my
> user profile, I just have glibc-locales now:
>
> ~$ ls -lah /var/guix/profiles/per-user/giovanni/guix-profile/lib/locale/2.28/ | grep en_US
> dr-xr-xr-x 3 root root 4.0K Jan 1 1970 en_US
> dr-xr-xr-x 3 root root 4.0K Jan 1 1970 en_US.UTF-8

D’oh!

Toggle quote (9 lines)
> In fact make-glibc-locales in base.scm "just" installs all locales in
> "not normalized codeset" with:
>
> [...]
> (replace 'build
> (lambda _
> (invoke "make" "localedata/install-locales"
> "-j" (number->string (parallel-job-count)))))

We should fix it, notably because libc automatically falls back to the
normalized codeset (that is, if you choose “en_US.uTF--8”, it eventually
tries to read data for “en_US.utf8”.) I wonder why the
‘install-locales’ rule doesn’t to it.

The “normalized codeset” is described in the libc manual (info "(libc)
Using gettextized software") and (info "(libc) Locale Names").

Anyway, I’ve posted patches to refactor things a bit and to create those
symlinks in ‘glibc-locales’:


Let me know what you think.

Thanks!

Ludo’.
J
J
Jack Hill wrote on 6 Jun 2019 18:56
(name . Ludovic Courtès)(address . ludo@gnu.org)
alpine.DEB.2.20.1906061253240.5164@marsh.hcoop.net
On Thu, 6 Jun 2019, Ludovic Courtès wrote:

Toggle quote (2 lines)
> Let me know what you think.

I follow all of the explanation, so it makes sense to me. Previously, I
didn't know about normalized names, but now I do, so thanks!

Once normalized locales are present in glibc-locales, I don't think any
change will be needed in the manual.

Best,
Jack
L
L
Ludovic Courtès wrote on 7 Jun 2019 23:11
control message for bug #36076
(address . control@debbugs.gnu.org)
87o9399k3r.fsf@gnu.org
retitle 36076 'glibc-locales' lacks symlinks for locale names with normalized codeset
quit
L
L
Ludovic Courtès wrote on 7 Jun 2019 23:11
(address . control@debbugs.gnu.org)
87muit9k3n.fsf@gnu.org
severity 36076 important
quit
L
L
Ludovic Courtès wrote on 7 Jun 2019 23:12
Re: bug#36076: Manual should clarify that glibc-utf8-locales is needed by default on foreign distros
(name . Jack Hill)(address . jackhill@jackhill.us)
87imth9k2m.fsf@gnu.org
Hi,

Jack Hill <jackhill@jackhill.us> skribis:

Toggle quote (10 lines)
> On Thu, 6 Jun 2019, Ludovic Courtès wrote:
>
>> Let me know what you think.
>
> I follow all of the explanation, so it makes sense to me. Previously,
> I didn't know about normalized names, but now I do, so thanks!
>
> Once normalized locales are present in glibc-locales, I don't think
> any change will be needed in the manual.

Fixed in 0e6cee21a48294b81a5e57e00602728fe7f7075f, thanks!

Ludo’.
Closed
J
J
Jack Hill wrote on 8 Jun 2019 00:59
(name . Ludovic Courtès)(address . ludo@gnu.org)
alpine.DEB.2.20.1906071859070.5164@marsh.hcoop.net
On Fri, 7 Jun 2019, Ludovic Courtès wrote:

Toggle quote (2 lines)
> Fixed in 0e6cee21a48294b81a5e57e00602728fe7f7075f, thanks!

Awesome, thanks for the developing the fix!

Jack
Closed
G
G
Giovanni Biscuolo wrote on 11 Jun 2019 16:49
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 36076@debbugs.gnu.org)
87imtcp47p.fsf@roquette.mug.biscuolo.net
Hi Ludo'.

Sorry I could not reply earlier.

Ludovic Courtès <ludo@gnu.org> writes:

[...]

Toggle quote (5 lines)
> We should fix it, notably because libc automatically falls back to the
> normalized codeset (that is, if you choose “en_US.uTF--8”, it eventually
> tries to read data for “en_US.utf8”.) I wonder why the
> ‘install-locales’ rule doesn’t to it.

Me too: do you think it's appropriate to report this upstream?
I can manage this if needed, of course.

Toggle quote (3 lines)
> The “normalized codeset” is described in the libc manual (info "(libc)
> Using gettextized software") and (info "(libc) Locale Names").

Herm... I'm still too web dependent, I have to correctly install all
infos :-O

Toggle quote (7 lines)
> Anyway, I’ve posted patches to refactor things a bit and to create those
> symlinks in ‘glibc-locales’:
>
> https://issues.guix.gnu.org/issue/36116
>
> Let me know what you think.

I'm still not skilled enough to think anything useful about that patches
:-)... exept they looks great AFAIU! Thank you for fixing that so
quickly!

Reading your refactored build-locale in (gnu build locale), as used
e.g. in (gnu system locale), makes me wonder if Guix could allow users to
define their preferred locales in a (operating-system ...) declaration,
having "guix system" building only the needed ones... but that's another
story

Thanks! Gio'.

--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEERcxjuFJYydVfNLI5030Op87MORIFAlz/v2oACgkQ030Op87M
ORJzCQ/9FrX3JYdtnpLXkwbSESldFjouKqDFc6OB8lMLaU+b8EpUYAbye0uhnEBy
uQWeOX+GOA4u4M2fTywFJ8hinc6VfHMsExL1320kF0eka3Eqm85khVia9LhlcEg9
fWTzVgGge0POHkpGQm4CnsguRwrUvZTBTkni4XvV3/V6wpUmj12feYv5fAQk2y8q
MT5ZFolJWneV0yAi6fbWfa85I3co5os0QNF1sqzRs4Cl2xNoIZ4UoFYL7ABPVuj+
z5IDyP8EHy4y9jqz1LVHQb/lLBb9BeYBFnrsETColgDamR805uMZjhWomv3KUrW+
1PUheCvRdkBYvqcMAv+RPwObSs5BjSa6pnihtrhTejCpo0wIi67Vs7UcpMaiChig
cctpcGfz96KD1ZUuuRaCYOV2yun5NIEz86UfWnpfbw08/ejrjV0BZ+JZgn00bH3X
iS/hYCoZQZq+NEEbAKBGzHS3z2TVqPOqyC1OAyp5my41E+r2/+KZ630R68I9xuHu
d6HhBk3P3ro4aMrxTuH4VaiCkOW9ouT0k+gLu77+1XmMGg/XXL1Jah4tE6pvj6ZA
9xIgmGL2SJsOZ+bnp56yX5NweiYOksNtnpmo0izvtc6SZG4emd+a04N/Fd6ZjcRk
cmpPnLuVFlsNoDOW5nzy9i9ZZPBQUp5DbJYV1wBZijfAgJWBDfg=
=x7yS
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 12 Jun 2019 16:40
(name . Giovanni Biscuolo)(address . g@xelera.eu)(address . 36076@debbugs.gnu.org)
875zpavpdm.fsf@gnu.org
Hello Giovanni,

Giovanni Biscuolo <g@xelera.eu> skribis:

Toggle quote (12 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>> We should fix it, notably because libc automatically falls back to the
>> normalized codeset (that is, if you choose “en_US.uTF--8”, it eventually
>> tries to read data for “en_US.utf8”.) I wonder why the
>> ‘install-locales’ rule doesn’t to it.
>
> Me too: do you think it's appropriate to report this upstream?
> I can manage this if needed, of course.

It would be great, yes!

Toggle quote (6 lines)
>> The “normalized codeset” is described in the libc manual (info "(libc)
>> Using gettextized software") and (info "(libc) Locale Names").
>
> Herm... I'm still too web dependent, I have to correctly install all
> infos :-O

Just type these ‘info’ commands and it should Just Work on Guix System.
:-)

Toggle quote (6 lines)
> Reading your refactored build-locale in (gnu build locale), as used
> e.g. in (gnu system locale), makes me wonder if Guix could allow users to
> define their preferred locales in a (operating-system ...) declaration,
> having "guix system" building only the needed ones... but that's another
> story

That’s already the case.

It would be nice to have it for Guix on foreign distros, as discussed in
the “Parameterized packages” thread.

Thanks,
Ludo’.
?