guix-daemon cannot find UTF-8 locale out-of-the-box on foreign distros

  • Done
  • quality assurance status badge
Details
4 participants
  • Ludovic Courtès
  • Ludovic Courtès
  • Maxim Cournoyer
  • Meiyo Peng
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 10 May 2019 18:31
'glibc-utf8-locales' should include C.UTF-8 locale
(name . Bug Guix)(address . bug-guix@gnu.org)
877eayz2x0.fsf@inria.fr
Hello,

I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
the C.UTF-8 locale by default, even if the machine is otherwise
configured with another locale. Consequently, ‘guix substitute’ keeps
complaining about locales and all that, which is terrible.

Thus, I think ‘glibc-utf8-locales’ should provide that locale.

Ludo’.
M
M
Meiyo Peng wrote on 11 May 2019 05:48
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 35671@debbugs.gnu.org)
871s15vefu.fsf@riseup.net
Hi Ludovic,

Ludovic Courtès writes:

Toggle quote (7 lines)
> I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
> the C.UTF-8 locale by default, even if the machine is otherwise
> configured with another locale. Consequently, ‘guix substitute’ keeps
> complaining about locales and all that, which is terrible.
>
> Thus, I think ‘glibc-utf8-locales’ should provide that locale.

And the zh_CN.UTF-8, zh_TW.UTF-8 please.


--
Meiyo Peng
L
L
Ludovic Courtès wrote on 13 May 2019 22:44
(address . 35671@debbugs.gnu.org)
87o946ozj3.fsf@gnu.org
Hello,

Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

Toggle quote (5 lines)
> I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
> the C.UTF-8 locale by default, even if the machine is otherwise
> configured with another locale. Consequently, ‘guix substitute’ keeps
> complaining about locales and all that, which is terrible.

It turns out that the “C.UTF-8” locale doesn’t exist upstream in glibc:


But anyway, I think we should just set “LC_ALL=en_US.utf8” in the
.service file: that’s what we do on Guix System, and the choice of a
locale doesn’t matter since ‘guix substitute’ honors the client’s
locale.

Meiyo Peng <meiyo@riseup.net> skribis:

Toggle quote (2 lines)
> Ludovic Courtès writes:

[...]

Toggle quote (4 lines)
>> Thus, I think ‘glibc-utf8-locales’ should provide that locale.
>
> And the zh_CN.UTF-8, zh_TW.UTF-8 please.

‘glibc-utf8-locales’ has always been described as an *arbitrary* sample
of UTF-8 locales (info "(guix) Application Setup"). Initially its main
purpose was to use it in build processes, but we’ve come to more or less
recommend it for users, which is not great because we know it only works
for some users.

I built a ‘glibc-utf8-locales’ package that provides all the supported
UTF-8 locales, and only UTF-8 locales:

Toggle snippet (12 lines)
$ guix size /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28
store item total self
/gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28 855.7 855.7 100.0%
total: 855.7 MiB
$ ls -1d /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28/lib/locale/2.28/*.utf8 |wc -l
312
$ guix size glibc-locales
store item total self
/gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28 916.7 916.7 100.0%
total: 916.7 MiB

As you can see, it’s almost the same size as ‘glibc-locales’, so there’s
no point in having such a ‘glibc-utf8-locales’ package.

In Guix System, the (gnu system locale) module generates just the needed
locales. I’m thinking we should do the same at the ‘guix package’
level. For example, ‘guix package --install-locales’ would
automatically install whatever $LANG or $LC_* refers to.

Thoughts?

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 13 May 2019 23:44
control message for bug #35671
(address . control@debbugs.gnu.org)
87ef52owr4.fsf@gnu.org
retitle 35671 guix-daemon cannot find UTF-8 locale out-of-the-box on foreign distros
M
M
Meiyo Peng wrote on 14 May 2019 04:10
Re: bug#35671: 'glibc-utf8-locales' should include C.UTF-8 locale
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 35671@debbugs.gnu.org)
87a7fp23bo.fsf@riseup.net
Hi Ludovic,

Ludovic Courtès writes:

Toggle quote (5 lines)
> But anyway, I think we should just set “LC_ALL=en_US.utf8” in the
> .service file: that’s what we do on Guix System, and the choice of a
> locale doesn’t matter since ‘guix substitute’ honors the client’s
> locale.

That's what I did to my guix-daemon.service and guix-publish.service
files on a foreign distro after getting locale warnings.

Toggle quote (16 lines)
> Meiyo Peng <meiyo@riseup.net> skribis:
>
>> Ludovic Courtès writes:
>
> [...]
>
>>> Thus, I think ‘glibc-utf8-locales’ should provide that locale.
>>
>> And the zh_CN.UTF-8, zh_TW.UTF-8 please.
>
> ‘glibc-utf8-locales’ has always been described as an *arbitrary* sample
> of UTF-8 locales (info "(guix) Application Setup"). Initially its main
> purpose was to use it in build processes, but we’ve come to more or less
> recommend it for users, which is not great because we know it only works
> for some users.

I was surprised last week (maybe the week before that) when I found out
that glibc-utf8-locales only provides locales for de_DE, el_GR, en_US,
fr_FR, tr_TR. And in our manual, we tell people to install
glibc-utf8-locales after installing Guix on a foreign distro in order to
solve locale problems. The name "glibc-utf8-locales" indicates it
provides ALL UTF-8 locales while in fact it does not. This caused me a
trouble because I have to set the locale of Emacs to zh_CN.UTF-8 in
order to enable the ibus input method in Emacs. (This is a known
problem of Emacs. Locale has to be set to one of CJK locale in order to
enable external input methods.) Since the zh_CN.UTF-8 locale is not in
$GUIX_LOCPATH, ibus does not work in Emacs. I ended up installing the
"glibc-locales" package and solved the problem.

Toggle quote (26 lines)
> I built a ‘glibc-utf8-locales’ package that provides all the supported
> UTF-8 locales, and only UTF-8 locales:
>
> --8<---------------cut here---------------start------------->8---
> $ guix size /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28
> store item total self
> /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28 855.7 855.7 100.0%
> total: 855.7 MiB
> $ ls -1d /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28/lib/locale/2.28/*.utf8 |wc -l
> 312
> $ guix size glibc-locales
> store item total self
> /gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28 916.7 916.7 100.0%
> total: 916.7 MiB
> --8<---------------cut here---------------end--------------->8---
>
> As you can see, it’s almost the same size as ‘glibc-locales’, so there’s
> no point in having such a ‘glibc-utf8-locales’ package.
>
> In Guix System, the (gnu system locale) module generates just the needed
> locales. I’m thinking we should do the same at the ‘guix package’
> level. For example, ‘guix package --install-locales’ would
> automatically install whatever $LANG or $LC_* refers to.
>
> Thoughts?

I think it would be better to simply tell people to install the
"glibc-locales" package. There may be a case where different
applications are set to different locales. For example, my system is
set to the en_US.UTF-8 locale but my Emacs is set to the zh_CN.UTF-8
locale. So $LC_* only refers to en_US.UTF-8 if guix tries to detect the
locale. If ‘guix package --install-locales’ only install what $LANG or
$LC_* refers to, zh_CN.UTF-8 won't be installed for me.

We are unable to detect what locales a user will need later. Installing
a full package like "glibc-locales" or letting the user explicitly
specify what locales they need can avoid a lot of special problems.


--
Meiyo Peng
L
L
Ludovic Courtès wrote on 13 May 2019 23:50
(address . 35671@debbugs.gnu.org)
87a7fqowhe.fsf@gnu.org
Hi!

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (16 lines)
> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
>> the C.UTF-8 locale by default, even if the machine is otherwise
>> configured with another locale. Consequently, ‘guix substitute’ keeps
>> complaining about locales and all that, which is terrible.
>
> It turns out that the “C.UTF-8” locale doesn’t exist upstream in glibc:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=902094
>
> But anyway, I think we should just set “LC_ALL=en_US.utf8” in the
> .service file: that’s what we do on Guix System, and the choice of a
> locale doesn’t matter since ‘guix substitute’ honors the client’s
> locale.

Done in 489d6c0dc03f1d95a3fefa6301f36cac9b50b2c3.

I got the syntax from
but I wouldn’t mind if someone would double-check!

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 14 May 2019 22:57
(name . Meiyo Peng)(address . meiyo@riseup.net)(address . 35671@debbugs.gnu.org)
87y338wy7q.fsf@gnu.org
Hi,

Meiyo Peng <meiyo@riseup.net> skribis:

Toggle quote (13 lines)
> I was surprised last week (maybe the week before that) when I found out
> that glibc-utf8-locales only provides locales for de_DE, el_GR, en_US,
> fr_FR, tr_TR. And in our manual, we tell people to install
> glibc-utf8-locales after installing Guix on a foreign distro in order to
> solve locale problems. The name "glibc-utf8-locales" indicates it
> provides ALL UTF-8 locales while in fact it does not. This caused me a
> trouble because I have to set the locale of Emacs to zh_CN.UTF-8 in
> order to enable the ibus input method in Emacs. (This is a known
> problem of Emacs. Locale has to be set to one of CJK locale in order to
> enable external input methods.) Since the zh_CN.UTF-8 locale is not in
> $GUIX_LOCPATH, ibus does not work in Emacs. I ended up installing the
> "glibc-locales" package and solved the problem.

To be fair, the manual has always recommended ‘glibc-locales’, not
‘glibc-utf8-locales’, and the latter is explicitly described as “limited
to a few UTF-8 locales”:


The hint in (guix ui) is less clear, though.

Toggle quote (8 lines)
> I think it would be better to simply tell people to install the
> "glibc-locales" package. There may be a case where different
> applications are set to different locales. For example, my system is
> set to the en_US.UTF-8 locale but my Emacs is set to the zh_CN.UTF-8
> locale. So $LC_* only refers to en_US.UTF-8 if guix tries to detect the
> locale. If ‘guix package --install-locales’ only install what $LANG or
> $LC_* refers to, zh_CN.UTF-8 won't be installed for me.

That command could take a parameter, of course.

See also:

Thanks for your feedback!

Ludo’.
M
M
Meiyo Peng wrote on 15 May 2019 03:35
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 35671@debbugs.gnu.org)
87ftpg5wl2.fsf@riseup.net
Hi Ludovic,

Ludovic Courtès writes:

Toggle quote (21 lines)
> Meiyo Peng <meiyo@riseup.net> skribis:
>
>> I was surprised last week (maybe the week before that) when I found out
>> that glibc-utf8-locales only provides locales for de_DE, el_GR, en_US,
>> fr_FR, tr_TR. And in our manual, we tell people to install
>> glibc-utf8-locales after installing Guix on a foreign distro in order to
>> solve locale problems. The name "glibc-utf8-locales" indicates it
>> provides ALL UTF-8 locales while in fact it does not. This caused me a
>> trouble because I have to set the locale of Emacs to zh_CN.UTF-8 in
>> order to enable the ibus input method in Emacs. (This is a known
>> problem of Emacs. Locale has to be set to one of CJK locale in order to
>> enable external input methods.) Since the zh_CN.UTF-8 locale is not in
>> $GUIX_LOCPATH, ibus does not work in Emacs. I ended up installing the
>> "glibc-locales" package and solved the problem.
>
> To be fair, the manual has always recommended ‘glibc-locales’, not
> ‘glibc-utf8-locales’, and the latter is explicitly described as “limited
> to a few UTF-8 locales”:
>
> https://www.gnu.org/software/guix/manual/en/html_node/Application-Setup.html

You are right. My mistake. But when I saw the text:

#+begin_quote
Note that the glibc-locales package contains data for all the locales
supported by the GNU libc and weighs in at around 110 MiB.
Alternatively, the glibc-utf8-locales is smaller but limited to a
few UTF-8 locales.
#+end_quote

I thought the glibc-utf8-locales should satisfy my requirements since
all my locales are in UTF-8, and it's preferred if disk space is
limited. The name of glibc-utf8-locales is misleading. In fact, the
"few" UTF-8 locales it provides are too few. We'd better modify the
text a little bit if we are going to reach a broader user base.

Toggle quote (12 lines)
> The hint in (guix ui) is less clear, though.
>
>> I think it would be better to simply tell people to install the
>> "glibc-locales" package. There may be a case where different
>> applications are set to different locales. For example, my system is
>> set to the en_US.UTF-8 locale but my Emacs is set to the zh_CN.UTF-8
>> locale. So $LC_* only refers to en_US.UTF-8 if guix tries to detect the
>> locale. If ‘guix package --install-locales’ only install what $LANG or
>> $LC_* refers to, zh_CN.UTF-8 won't be installed for me.
>
> That command could take a parameter, of course.

That makes sense.


--
Meiyo Peng
M
M
Maxim Cournoyer wrote on 14 Dec 2021 18:55
Re: bug#35671: guix-daemon cannot find UTF-8 locale out-of-the-box on foreign distros
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 35671-done@debbugs.gnu.org)
87wnk79im8.fsf_-_@gmail.com
Hello,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (26 lines)
> Hi!
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>>
>>> I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
>>> the C.UTF-8 locale by default, even if the machine is otherwise
>>> configured with another locale. Consequently, ‘guix substitute’ keeps
>>> complaining about locales and all that, which is terrible.
>>
>> It turns out that the “C.UTF-8” locale doesn’t exist upstream in glibc:
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=902094
>>
>> But anyway, I think we should just set “LC_ALL=en_US.utf8” in the
>> .service file: that’s what we do on Guix System, and the choice of a
>> locale doesn’t matter since ‘guix substitute’ honors the client’s
>> locale.
>
> Done in 489d6c0dc03f1d95a3fefa6301f36cac9b50b2c3.
>
> I got the syntax from
> <https://www.freedesktop.org/software/systemd/man/systemd.service.html>
> but I wouldn’t mind if someone would double-check!

Closing, then :-).

Maxim
Closed
?