[PATCH] gnu: Add make-glibc-locales-collection.

  • Open
  • quality assurance status badge
Details
5 participants
  • bdju
  • Efraim Flashner
  • Leo Famulari
  • Ludovic Courtès
  • Miguel Ángel Arruga Vivas
Owner
unassigned
Submitted by
Efraim Flashner
Severity
normal
E
E
Efraim Flashner wrote on 19 Oct 2020 08:47
(address . guix-patches@gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)
20201019064739.4736-1-efraim@flashner.co.il
* gnu/packages/base.scm (make-glibc-locales-collection): New macro.
(en_us-glibc-locales): New variable.
---
gnu/packages/base.scm | 73 ++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 72 insertions(+), 1 deletion(-)

Toggle diff (100 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..41d3aaf865 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -62,7 +62,8 @@
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (glibc
- libiconv-if-needed))
+ libiconv-if-needed
+ make-custom-glibc-locales))
;;; Commentary:
;;;
@@ -1106,6 +1107,69 @@ to the @code{share/locale} sub-directory of this package.")
,(version-major+minor
(package-version glibc)))))))))))
+(define* (make-glibc-locales-collection
+ glibc
+ #:optional (locales
+ '(list "en_US.utf8" "en_US.ISO-8859-1")))
+ ;; This list for testing
+ ;'(list "el_GR.UTF-8" "en_US.utf8" "he_IL.ISO-8859-8" "ja_JP.EUC-JP" "zh_CN.GB18030" "zh_CN.GBK" "hy_AM.ARMSCII-8")))
+ (package
+ (name "glibc-locales-collection")
+ (version (package-version glibc))
+ (source #f)
+ (build-system trivial-build-system)
+ (arguments
+ `(#:modules ((guix build utils))
+ #:builder
+ (begin
+ (use-modules (guix build utils))
+
+ (let* ((libc (assoc-ref %build-inputs "glibc"))
+ (gzip (assoc-ref %build-inputs "gzip"))
+ (out (assoc-ref %outputs "out"))
+ (localedir (string-append out "/lib/locale/"
+ ,(version-major+minor version))))
+ ;; 'localedef' needs 'gzip'.
+ (setenv "PATH" (string-append libc "/bin:" gzip "/bin"))
+
+ (mkdir-p localedir)
+ (for-each
+ (lambda (locale)
+ (let* ((contains-dot? (string-index locale #\.))
+ (encoding-type (substring locale (1+ contains-dot?)))
+ (raw-locale (substring locale 0 contains-dot?))
+ (utf8? (or (number? (string-contains locale ".utf8"))
+ (number? (string-contains locale ".UTF-8"))))
+ (file (if utf8?
+ (string-append localedir "/" raw-locale ".utf8")
+ (if (string-contains locale ".ISO")
+ (string-append localedir "/" raw-locale)
+ (string-append localedir "/" locale)))))
+
+ (invoke "localedef" "--no-archive"
+ "--prefix" localedir
+ "-i" raw-locale
+ "-f" (if (equal? "utf8" encoding-type)
+ "UTF-8"
+ encoding-type)
+ file)
+
+ ;; Is it utf8 or UTF-8? NO ONE KNOWS!
+ (when utf8?
+ (symlink (string-append raw-locale ".utf8")
+ (string-append localedir "/"
+ raw-locale ".UTF-8")))))
+ ,locales)
+ #t))))
+ (native-inputs `(("glibc" ,glibc)
+ ("gzip" ,gzip)))
+ (synopsis "Customizable collection of locales")
+ (description
+ "This package provides a custom collection of locales useful for
+providing exactly the locales requested when size matters.")
+ (home-page (package-home-page glibc))
+ (license (package-license glibc))))
+
(define-public (make-glibc-utf8-locales glibc)
(package
(name "glibc-utf8-locales")
@@ -1161,6 +1225,13 @@ test environments.")
(define-public glibc-utf8-locales
(make-glibc-utf8-locales glibc))
+(define-public en_us-glibc-locales
+ (package
+ (inherit (make-glibc-locales-collection
+ glibc
+ '(list "en_US.utf8" "en_US.ISO-8859-1")))
+ (name "en-us-glibc-locales")))
+
;; Packages provided to ease use of binaries linked against the previous libc.
(define-public glibc-locales-2.29
(package (inherit (make-glibc-locales glibc-2.29))
--
2.28.0
M
M
Miguel Ángel Arruga Vivas wrote on 19 Oct 2020 15:17
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 44075@debbugs.gnu.org)
87tuuqnym5.fsf@gmail.com
Hi Efraim,

I've been taking a look into your patch. One issue are the comments
about utf8 and UTF-8, as the issue is already explained in
make-glibc[-utf8]-locales.

Other point is:

Efraim Flashner <efraim@flashner.co.il> writes:
Toggle quote (7 lines)
> +(define* (make-glibc-locales-collection
> + glibc
> + #:optional (locales
> + '(list "en_US.utf8" "en_US.ISO-8859-1")))
> (... Removed for clarity ...)
> + ,locales)

I would have used list there like (list ,@locales) or '(,@locales), this
looks a bit odd to my eyes at least. I'd expect this kind of calling code:

(let ((locales '("de_CH.utf8" ... "de_DE.utf8"))
(my-glibc ...))
(make-glibc-locales-collection myglibc locales))

Enforcing an extra quotation for no real reason on the calling site, as
strings are self-evaluating objects, and the use of the symbol list,
whose meaning depends on other context of execution, doesn't seem
necessary. Even worse, my example would raise an error as "de_CH.utf8"
is not a procedure.

What do you think about replacing make-glibc-utf8-locales with a call of
the new function (using that code) ensuring that the generated
derivation stays the same for that case (i.e. it's optimized for the
UTF-8 case)?

Happy hacking!
Miguel
-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEEiIeExBRZrMuD5+hMY0xuiXn6vsIFAl+NkgIACgkQY0xuiXn6
vsIPmgv7BmwzC3IaxvbOLJbgKF5MYvOFJoZHoUQuKsJAT/6UwJZ3J9H6dZ1kNJuO
Sgm/pKuCmrTMDqOIIp5SrYulGjBnjP4vTsO8bwBEFyJ7y5cWREXDgoOp4J9vQXyH
S4Yjy72GwNO+WBK8S1ZyquNnLr0vuA1kuNGoaaTw8ACtirlOAQoLKTmSsLZNZIk1
fJQUOfz08gUwg0Y/AMzTgDGhflQA+rHMDC/Qs1uwnmB64xhxfHC2jemC8ZttklqB
edhc2Hw4AJAKoBLwnANM5d/q73vief2FL7Y3tZiP5+V/qVRVaZOhGnSQyCQd9cnF
kB9dlqLmJwC47b78oOQxK/fDicKbhoi7XJHkkRrRB/6Adf8LfJVJQ8T1iLgCu8zw
rT+1eTd2FkOyNYRV4j8FVATjUB/wabDLB23BgeOpiAaLMFj7A8QEgLV0WRlTJoBK
aqCD/zXVGmd/SDVX1jxxKmLJ7+0veD5dXgYoCoecBBWibjBhf5j07maphuttXIsq
9dNtpxXo
=o6XK
-----END PGP SIGNATURE-----

E
E
Efraim Flashner wrote on 19 Oct 2020 16:02
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)(address . 44075@debbugs.gnu.org)
20201019140236.GF9117@E5400
On Mon, Oct 19, 2020 at 03:17:54PM +0200, Miguel Ángel Arruga Vivas wrote:
Toggle quote (6 lines)
> Hi Efraim,
>
> I've been taking a look into your patch. One issue are the comments
> about utf8 and UTF-8, as the issue is already explained in
> make-glibc[-utf8]-locales.

Thanks for taking a look. For the utf8 vs UTF-8 there are a couple of
comments in the code:
The above phase does not install locales with names using
the "normalized codeset." Thus, create symlinks like:
en_US.utf8 -> en_US.UTF-8

and also:
For backward compatibility with Guix
<= 0.8.3, add "xx_YY.UTF-8".

When I check on one of my Debian boxes:
$localectl list-locales
C.UTF-8
en_US.utf8

I've learned that 'C.UTF-8' isn't from upstream, Debian has a patch for
it. Having written this patch a week or so ago and returning to it I
can't tell you which is the correct one. I think it's best to offer both
so it's not confusing which is correct. That works for the logic for
accepting either in the list of locales, but I'm concerned that skipping
the symlink to the other one could cause problems.

Toggle quote (24 lines)
>
> Other point is:
>
> Efraim Flashner <efraim@flashner.co.il> writes:
> > +(define* (make-glibc-locales-collection
> > + glibc
> > + #:optional (locales
> > + '(list "en_US.utf8" "en_US.ISO-8859-1")))
> > (... Removed for clarity ...)
> > + ,locales)
>
> I would have used list there like (list ,@locales) or '(,@locales), this
> looks a bit odd to my eyes at least. I'd expect this kind of calling code:
>
> (let ((locales '("de_CH.utf8" ... "de_DE.utf8"))
> (my-glibc ...))
> (make-glibc-locales-collection myglibc locales))
>
> Enforcing an extra quotation for no real reason on the calling site, as
> strings are self-evaluating objects, and the use of the symbol list,
> whose meaning depends on other context of execution, doesn't seem
> necessary. Even worse, my example would raise an error as "de_CH.utf8"
> is not a procedure.

My scheme-foo isn't terribly strong, sometimes I just hack at it until
the code does what I want, and this is one of those times. I've changed
it so that locales takes a list and not an item that is a list.

Toggle quote (5 lines)
> What do you think about replacing make-glibc-utf8-locales with a call of
> the new function (using that code) ensuring that the generated
> derivation stays the same for that case (i.e. it's optimized for the
> UTF-8 case)?

This is what I originally wanted to do, but there's a glibc-locales
buried in the bootstrap path so it's not so easy to just swap it out. I
can make the change in core-updates. I'll play around with it and see if
I can come out with the same derivation using a different function, but
I'm not expecting it to turn out identical.

Toggle quote (7 lines)
>
> Happy hacking!
> Miguel
>



--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
From 3c2dfe702ad1c62b2f73a1f1036da3bcfa007c15 Mon Sep 17 00:00:00 2001
From: Efraim Flashner <efraim@flashner.co.il>
Date: Thu, 1 Oct 2020 22:50:32 +0300
Subject: [PATCH v2] gnu: Add make-glibc-locales-collection.

* gnu/packages/base.scm (make-glibc-locales-collection): New macro.
(en_us-glibc-locales): New variable.
---
gnu/packages/base.scm | 73 ++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 72 insertions(+), 1 deletion(-)

Toggle diff (100 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..4ea31c2ab6 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -62,7 +62,8 @@
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (glibc
- libiconv-if-needed))
+ libiconv-if-needed
+ make-custom-glibc-locales))
;;; Commentary:
;;;
@@ -1106,6 +1107,69 @@ to the @code{share/locale} sub-directory of this package.")
,(version-major+minor
(package-version glibc)))))))))))
+(define* (make-glibc-locales-collection
+ glibc
+ #:optional (locales
+ (list "en_US.utf8" "en_US.ISO-8859-1")))
+ ;; This list for testing
+ ;(list "el_GR.UTF-8" "en_US.utf8" "he_IL.ISO-8859-8" "ja_JP.EUC-JP" "zh_CN.GB18030" "zh_CN.GBK" "hy_AM.ARMSCII-8")))
+ (package
+ (name "glibc-locales-collection")
+ (version (package-version glibc))
+ (source #f)
+ (build-system trivial-build-system)
+ (arguments
+ `(#:modules ((guix build utils))
+ #:builder
+ (begin
+ (use-modules (guix build utils))
+
+ (let* ((libc (assoc-ref %build-inputs "glibc"))
+ (gzip (assoc-ref %build-inputs "gzip"))
+ (out (assoc-ref %outputs "out"))
+ (localedir (string-append out "/lib/locale/"
+ ,(version-major+minor version))))
+ ;; 'localedef' needs 'gzip'.
+ (setenv "PATH" (string-append libc "/bin:" gzip "/bin"))
+
+ (mkdir-p localedir)
+ (for-each
+ (lambda (locale)
+ (let* ((contains-dot? (string-index locale #\.))
+ (encoding-type (substring locale (1+ contains-dot?)))
+ (raw-locale (substring locale 0 contains-dot?))
+ (utf8? (or (number? (string-contains locale ".utf8"))
+ (number? (string-contains locale ".UTF-8"))))
+ (file (if utf8?
+ (string-append localedir "/" raw-locale ".utf8")
+ (if (string-contains locale ".ISO")
+ (string-append localedir "/" raw-locale)
+ (string-append localedir "/" locale)))))
+
+ (invoke "localedef" "--no-archive"
+ "--prefix" localedir
+ "-i" raw-locale
+ "-f" (if (equal? "utf8" encoding-type)
+ "UTF-8"
+ encoding-type)
+ file)
+
+ ;; Is it utf8 or UTF-8? NO ONE KNOWS!
+ (when utf8?
+ (symlink (string-append raw-locale ".utf8")
+ (string-append localedir "/"
+ raw-locale ".UTF-8")))))
+ (list ,@locales))
+ #t))))
+ (native-inputs `(("glibc" ,glibc)
+ ("gzip" ,gzip)))
+ (synopsis "Customizable collection of locales")
+ (description
+ "This package provides a custom collection of locales useful for
+providing exactly the locales requested when size matters.")
+ (home-page (package-home-page glibc))
+ (license (package-license glibc))))
+
(define-public (make-glibc-utf8-locales glibc)
(package
(name "glibc-utf8-locales")
@@ -1161,6 +1225,13 @@ test environments.")
(define-public glibc-utf8-locales
(make-glibc-utf8-locales glibc))
+(define-public en_us-glibc-locales
+ (package
+ (inherit (make-glibc-locales-collection
+ glibc
+ (list "en_US.utf8" "en_US.ISO-8859-1")))
+ (name "en-us-glibc-locales")))
+
;; Packages provided to ease use of binaries linked against the previous libc.
(define-public glibc-locales-2.29
(package (inherit (make-glibc-locales glibc-2.29))
--
2.28.0
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl+NnHwACgkQQarn3Mo9
g1Hr9w//QqDSEeBZTuHDJc7Bq75J/9U7oI5+HACp8TwyRzY2W2Rc44SQwgGeZmFb
Cu4mc2pxTaZ7/C2V1eJf794+/ACv52sVzSQAyxZCXz8xvYf7ALsHT640y99vdThf
DtMh8uPjJXkg2Lc01npUmB8oGEMriofYNMPeEcWcPnV4c3LmGTW1owvcgzVT9kBo
y1LGqbY939/kYriI4HvaEbh1+lMwuG+kQONSg9Qp1Pn+C4NlVa9BtGsNiIJ7+GXS
7LP0TCCZBtc4hKRB411P8Y5WGeB7LbC11ZW7lYzfFeTKSFthHuwAuHvLSITRJh+7
E+sjSJ0wqLzqOuGJEcc5HGtkywnm2wiYLriux7HMDTyj/NkCVpgIpuMRa8Il2jDJ
Kw0EPFuf7ov0i3+4y4Ix7VJ+T0BgVZQ285O8b+6UGNgJSISdKe90OSVSvbvwiQX4
4xWJZDnfeM5caWrxTjT36jCvXKDa7NFc/1LjWw9oFbntVrUh/N+aQmzKjsNmnZSB
A5kJvyVwC0jIv4WZGgcxbBvhIge7nl100zew+UsjKuSYUYFPASp66D3+IKKi0JYv
liUM44iHsNovns2JTgFg8li7P8z0Yp5nccO2b08At9KK9JsF8dvcEhTtia6PFfkr
BkoCG0WK7LiNW/ulPx57oLU2Ql05Gvl7M5YAywo30ZhiTaNrybI=
=Tdd4
-----END PGP SIGNATURE-----


M
M
Miguel Ángel Arruga Vivas wrote on 19 Oct 2020 19:43
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 44075@debbugs.gnu.org)
87mu0im7r4.fsf@gmail.com
Hello,

Efraim Flashner <efraim@flashner.co.il> writes:
Toggle quote (2 lines)
> Thanks for taking a look.

No problem at all, I was thinking about something along these lines too,
but then I saw your patch. :-)

Toggle quote (9 lines)
> For the utf8 vs UTF-8 there are a couple of comments in the code:
> The above phase does not install locales with names using
> the "normalized codeset." Thus, create symlinks like:
> en_US.utf8 -> en_US.UTF-8
>
> and also:
> For backward compatibility with Guix
> <= 0.8.3, add "xx_YY.UTF-8".

Yes, what I mean is that the comments along the code may need to be
clarified, but adding a "nobody knows" doesn't add much information.
The actual source[1] says that the correct value is utf8, following
the rules for the 'normalized codeset' naming, that I copy here:

1. Remove all characters besides numbers and letters.
2. Fold letters to lowercase.
3. If the same only contains digits prepend the string ‘"iso"’.

We should stick to that naming regarding libc locales (note to self),
even though we keep the links for compatibility. That includes the
other locale at en_us-glibc-locales.

Toggle quote (8 lines)
>> What do you think about replacing make-glibc-utf8-locales with a call of
>> the new function (using that code) ensuring that the generated
>> derivation stays the same for that case (i.e. it's optimized for the
>> UTF-8 case)?
>
> This is what I originally wanted to do, but there's a glibc-locales
> buried in the bootstrap path so it's not so easy to just swap it out.

I guess you mean glibc-utf8-locales, if not I'm missing where that
reference could come from. That's why I insist on leaving exactly the
same derivation.

Toggle quote (4 lines)
> I can make the change in core-updates. I'll play around with it and
> see if I can come out with the same derivation using a different
> function, but I'm not expecting it to turn out identical.

The colour of my lisp-rate belt isn't even close to some you can see
around here but I could bring you some help if you want, because I think
it's easier to do it without any change in current packages than it
might sound. Not the definitions in scheme, of course, I mean the
derivations the daemon actually sees.

Said this, I've seen that [single-]locale-directory does mostly the
same, and there is a build-locale function in gnu/build/locale.scm... so
I'm starting to see as a better idea to clean up glibc-utf8-locales up
in core-updates using the latter, as it would lead to cleaner code for
the builder.

Could you check that and tell if you consider feasible what I propose?

Happy hacking!
Miguel

[1] info "(libc)Using gettextized software"
E
E
Efraim Flashner wrote on 21 Oct 2020 09:01
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)(address . 44075@debbugs.gnu.org)
20201021070155.GQ9117@E5400
On Mon, Oct 19, 2020 at 07:43:27PM +0200, Miguel Ángel Arruga Vivas wrote:
Toggle quote (8 lines)
> Hello,
>
> Efraim Flashner <efraim@flashner.co.il> writes:
> > Thanks for taking a look.
>
> No problem at all, I was thinking about something along these lines too,
> but then I saw your patch. :-)

Your patch was what got me to actually send mine to guix-patches :)

Toggle quote (12 lines)
> > For the utf8 vs UTF-8 there are a couple of comments in the code:
> > The above phase does not install locales with names using
> > the "normalized codeset." Thus, create symlinks like:
> > en_US.utf8 -> en_US.UTF-8
> >
> > and also:
> > For backward compatibility with Guix
> > <= 0.8.3, add "xx_YY.UTF-8".
>
> Yes, what I mean is that the comments along the code may need to be
> clarified, but adding a "nobody knows" doesn't add much information.

That is a good point. I'll change it to something actually useful,
mentioning that there's a convention but that people often get it wrong.

Toggle quote (11 lines)
> The actual source[1] says that the correct value is utf8, following
> the rules for the 'normalized codeset' naming, that I copy here:
>
> 1. Remove all characters besides numbers and letters.
> 2. Fold letters to lowercase.
> 3. If the same only contains digits prepend the string ‘"iso"’.
>
> We should stick to that naming regarding libc locales (note to self),
> even though we keep the links for compatibility. That includes the
> other locale at en_us-glibc-locales.

I agree it should be renamed to en-us. I have some bikeshedding to think
about for en-us-glibc-locales vs glibc-locales-en-us. And I'll also test
out one for glibc-locales-guile-tests.

Toggle quote (12 lines)
> >> What do you think about replacing make-glibc-utf8-locales with a call of
> >> the new function (using that code) ensuring that the generated
> >> derivation stays the same for that case (i.e. it's optimized for the
> >> UTF-8 case)?
> >
> > This is what I originally wanted to do, but there's a glibc-locales
> > buried in the bootstrap path so it's not so easy to just swap it out.
>
> I guess you mean glibc-utf8-locales, if not I'm missing where that
> reference could come from. That's why I insist on leaving exactly the
> same derivation.

I meant in commencement.scm. There's a glibc-utf8-locales-final near the
definition for %boot5-inputs.

Toggle quote (18 lines)
> > I can make the change in core-updates. I'll play around with it and
> > see if I can come out with the same derivation using a different
> > function, but I'm not expecting it to turn out identical.
>
> The colour of my lisp-rate belt isn't even close to some you can see
> around here but I could bring you some help if you want, because I think
> it's easier to do it without any change in current packages than it
> might sound. Not the definitions in scheme, of course, I mean the
> derivations the daemon actually sees.
>
> Said this, I've seen that [single-]locale-directory does mostly the
> same, and there is a build-locale function in gnu/build/locale.scm... so
> I'm starting to see as a better idea to clean up glibc-utf8-locales up
> in core-updates using the latter, as it would lead to cleaner code for
> the builder.
>
> Could you check that and tell if you consider feasible what I propose?

I hadn't seen gnu/build/locales.scm. I'll have to see what we can do
about using it. We normally don't allow cross-over from the build side
to the packages side.

Toggle quote (5 lines)
> Happy hacking!
> Miguel
>
> [1] info "(libc)Using gettextized software"

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl+P3OAACgkQQarn3Mo9
g1Es1Q/8CNLcoZtmwR/84oN5iZF+f+U5jqgaPAi/r9/VjnREPvhVPxiAMbwDBLlO
2vxB6kxn+iP9sDKpcafNNdNHSbOe98blIb/+xXBbG48dzw9Jncq2abN77970MJXg
5S70r1t01iX4qHl46YqttFyWhYVrMoadRj7hJrMzAnTlO8/yAimNDWhf+Z6VHv5n
qHiUpdYZXlNdnpcI851OueqIEqNCn7+FF5O9oF9g3qlfEL8F+NyuvWd0zmXAjcfg
UnmMKHjMXvopx8v/YSyo+gP+XPpJzEtvW6rD3EeAMFQE67UNU3sKJNytpubtZ4a+
5xielVXNS/ndvbOLrP/8+lESv1YJHrPWQLtiufEwqBkEfCwtJIHnL37ceODgXSWN
v9rs0MFC8F2BGqBqyHn7SSVfTc+nW/RmSv8vymbsKPwZqqTXmNWNFZxe6CmLpkCc
1NwnrMAx0VyODPCd20e7deJxScjDNz2nXGCkR2usy2/AY88wQABYVduecUO8uu8g
5Zyya3BuJ77IKzFRBovgYfK0DOSBaTt1Ly9pp/SqeLKwx5H2h9faFPKigl3CDE9P
ZJfaG0DyJBZQ3+E2JXW+gOiExwfa32OMEf+GtYYI/yVj6+Hi0n9v3MF8IMQwiA/3
7JiQiwdYSZCu6AyfqxdtCq8SLEONhxqxl1PpUEU03dLyBy9jV2s=
=eAqn
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 21 Oct 2020 19:09
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 44075@debbugs.gnu.org)
878sbz1p71.fsf@gnu.org
Hi!

Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (3 lines)
> * gnu/packages/base.scm (make-glibc-locales-collection): New macro.
> (en_us-glibc-locales): New variable.

Cool!

A while back I posted the attached patch, which reuses existing code
from (gnu system locale) and (gnu packages base). I think I used it
just to see how much space all the UTF-8 locales take, compared to all
the locales.

Would it make sense to borrow from that?

At any rate, it’s the kind of package that’ll be more useful if/when we
have parameterized packages.

Ludo’.
Toggle diff (97 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..3fc43b04da 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -53,6 +53,8 @@
#:use-module (gnu packages python)
#:use-module (gnu packages gettext)
#:use-module (guix utils)
+ #:use-module (guix gexp)
+ #:use-module (guix modules)
#:use-module (guix packages)
#:use-module (guix download)
#:use-module (guix git-download)
@@ -62,6 +64,8 @@
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (glibc
+ %default-utf8-locales
+ make-glibc-utf8-locales
libiconv-if-needed))
;;; Commentary:
@@ -1106,7 +1110,12 @@ to the @code{share/locale} sub-directory of this package.")
,(version-major+minor
(package-version glibc)))))))))))
-(define-public (make-glibc-utf8-locales glibc)
+(define %default-utf8-locales
+ '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+
+(define* (make-glibc-utf8-locales glibc #:optional
+ (locales %default-utf8-locales)
+ (locale-file #f))
(package
(name "glibc-utf8-locales")
(version (package-version glibc))
@@ -1145,10 +1154,17 @@ to the @code{share/locale} sub-directory of this package.")
;; These are the locales commonly used for
;; tests---e.g., in Guile's i18n tests.
- '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+ ,(if locale-file
+ `(call-with-input-file
+ (assoc-ref %build-inputs "locale-file")
+ read)
+ `',locales))
#t))))
(native-inputs `(("glibc" ,glibc)
- ("gzip" ,gzip)))
+ ("gzip" ,gzip)
+ ,@(if locale-file
+ `(("locale-file" ,locale-file))
+ '())))
(synopsis "Small sample of UTF-8 locales")
(description
"This package provides a small sample of UTF-8 locales mostly useful in
@@ -1169,6 +1185,40 @@ test environments.")
(package (inherit (make-glibc-utf8-locales glibc-2.29))
(name "glibc-utf8-locales-2.29")))
+(define (glibc-supported-locales libc)
+ ((module-ref (resolve-interface '(gnu system locale)) ;FIXME: hack
+ 'glibc-supported-locales)
+ libc))
+
+(define* (make-glibc-utf8-locales/full #:optional (glibc glibc))
+ (define utf8-locales
+ (computed-file "glibc-supported-utf8-locales.scm"
+ #~(begin
+ (use-modules (srfi srfi-1)
+ (ice-9 match)
+ (ice-9 pretty-print))
+
+ (define locales
+ (call-with-input-file
+ #+(glibc-supported-locales glibc)
+ read))
+
+ (define utf8-locales
+ (filter-map (match-lambda
+ ((name . "UTF-8")
+ (if (string-suffix? ".UTF-8" name)
+ (string-drop-right name 6)
+ name))
+ (_ #f))
+ locales))
+
+ (call-with-output-file #$output
+ (lambda (port)
+ (pretty-print utf8-locales port))))))
+
+ (make-glibc-utf8-locales glibc #:locale-file utf8-locales))
+
+
(define-public which
(package
(name "which")
M
M
Miguel Ángel Arruga Vivas wrote on 22 Oct 2020 01:18
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 44075@debbugs.gnu.org)
87blgvjhhj.fsf@gmail.com
Hello,

Efraim Flashner <efraim@flashner.co.il> writes:
Toggle quote (2 lines)
> Your patch was what got me to actually send mine to guix-patches :)

Thanks. :-)

Toggle quote (3 lines)
> That is a good point. I'll change it to something actually useful,
> mentioning that there's a convention but that people often get it wrong.

Great, the more knowledge, the easier to overcome that common
misunderstanding. :-)

Toggle quote (4 lines)
> I agree it should be renamed to en-us. I have some bikeshedding to think
> about for en-us-glibc-locales vs glibc-locales-en-us. And I'll also test
> out one for glibc-locales-guile-tests.

I meant the ISO-8859-1, the locale creation should be en_EN.iso88591,
the locale name can be en_EN.88591 too...

Although in that case shouldn't be glibc-en-us-locales? I'd like green
lights for the bike shed, please. ;-)

Toggle quote (4 lines)
> I hadn't seen gnu/build/locales.scm. I'll have to see what we can do
> about using it. We normally don't allow cross-over from the build side
> to the packages side.

It is already being used in the build of glibc-locales and I guess it
would be harmless to move it to guix build if that is a problem, but I
think it can be used as it is right now, even I didn't check yet. In
this for-each(gnu/packages/base.scm:1084), for example:
------------------------->8-------------begin----------------------
(for-each (lambda (directory)
(let*-values (((base)
(basename directory))
((name codeset)
(locale->name+codeset base))
((normalized)
(normalize-codeset codeset)))
(unless (string=? codeset normalized)
(symlink base
(string-append (dirname directory)
"/" name "."
normalized)))))
locales)
-------------------------8<--------------end-----------------------

I would write something like this in the builder code:
------------------------->8-------------begin----------------------
(define (locale-builder directory)
(lambda (locale)
(let-values (((name codeset)
(locale->name+codeset)))
(if codeset
(build-locale name
#:directory directory
#:codeset (normalize-codeset codeset))
(build-locale name
#:directory directory
#:codeset "utf8")))))
(let ...
(for-each (locale-builder localedir) '(,@locales))
...)
-------------------------8<--------------end-----------------------

I leave you here the function I was hacking apart (please, do not
confuse with working code ;-)) if you want to take any idea other from
it:
------------------------->8-------------begin----------------------
(define*-public (make-glibc-selected-locales
glibc
#:key
(suffix "-utf8-locales")
(locales %default-locales)
(alias %default-alias))
(package
(name (string-append (package-name glibc) suffix))
(version (package-version glibc))
(source #f)
(build-system trivial-build-system)
(arguments
`(#:modules ((guix build utils) (gnu build locale))
#:builder
(begin
(use-modules (guix build utils) (gnu build locale))
;; Print the common alias for locale
(define (print-locale-alias locale)
(let-values (((name codeset)
(locale->name+codeset)))
(if codeset
(or (string-equal codeset (normalize-codeset codeset))
(format #t "~a.~a ~a.~a~%" name codeset
name (normalize-codeset codeset)))
(format #t "~a.UTF-8 ~a.utf8" name name))))

;; Generate a function that builds the locale into directory.
(define (locale-builder directory)
(lambda (locale)
(let-values (((name codeset)
(locale->name+codeset)))
(if codeset
(build-locale name
#:directory directory
#:codeset (normalize-codeset codeset))
(build-locale name
#:directory directory
#:codeset "utf8")))))

;; Do the job.
(let* ((libc (assoc-ref %build-inputs "glibc"))
(gzip (assoc-ref %build-inputs "gzip"))
(out (assoc-ref %outputs "out"))
(localedir (string-append out "/lib/locale/"
,(version-major+minor version)))
(alias (string-append localedir "/locale.alias"))
(locales '(,@locales)))

;; 'localedef' needs 'gzip'.
(setenv "PATH" (string-append libc "/bin:" gzip "/bin"))

(mkdir-p localedir)
;; Generate each locale provided.
(for-each (locale-builder localedir) locales)
;; Generate alias file.
(with-output-to-file alias
(lambda ()
(format #t "~a~%" "# Aliases for common codeset names.")
(for-each print-locale-alias locales)
(format #t "~a~%" "# Other aliases.")
(for-each (lambda (line)
(format #t "~a~%" line))
'(,@alias)))))))
(native-inputs `(("glibc" ,glibc)
("gzip" ,gzip)))
(synopsis "Configured sample of locales")
(description
"TODO.")
(home-page (package-home-page glibc))
(license (package-license glibc))))
-------------------------8<--------------end-----------------------

Happy hacking,
Miguel
-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEEiIeExBRZrMuD5+hMY0xuiXn6vsIFAl+QwbgACgkQY0xuiXn6
vsIAwwv/V+iIDRtlXgmgrhnCHBkNBV8gIadbxLQxZnc1so/psQbcGDThb1whjL4J
EsAEpGvzSD5XFmahmv2vaRcoTAaeCKgYpWH/NXyXuzop00dteXctvq302WhEUB35
qVobkwJrff/Qfw+xvoHYyz+cGWLTQdKnD+v8D2RYyRxDjj528CSpOI8cDtTU2oSV
8KbzSnEtCPfLHRjyj96ZNzDPtLpbq0ROIFV2G9qqNahXE5pEDi10Bu8iiAnwn95e
GTXCSARMrtrSQtg81wnOS+A8UgpTpGiN6el3fjOVuolx64n6aZJYJQze2s64Wmtc
3z1jMFXhdvKzhdU5vjnBG9FWXc7FMZv36ry/QLh/CQpGVDp75yng9PcTzZw8IwEB
k6rJNJGz7hG4uuzQdS0OjrMNX7XhNMUGlRHkeUSm1bLGS/HymiRQG3EeLU24o85O
3xrOyI4SZspr9e3opINLUMNu0no5N0tOrhDnQhBe9OK9xGclpM2oiQ4z901nAMNh
9JZfbd7K
=X5L6
-----END PGP SIGNATURE-----

M
M
Miguel Ángel Arruga Vivas wrote on 4 Nov 2020 17:12
(name . Ludovic Courtès)(address . ludo@gnu.org)
87a6vx85j4.fsf@gmail.com
Hi Efraim and Ludo!

This patch could be an starting point to even master, it allows to
extend easily glibc-utf8-locales to the desired ones.

I've used the following steps to test it[1]:
------------------------------>8----------------------------------
$ echo '(define-module (t) #:use-module (gnu packages base))
(define-public glibc-utf8-test
(make-glibc-utf8-locales glibc #:locales (list "es_ES")
#:name "glibc-utf8-test"))
glibc-utf8-test' > tmp/t.scm
$ ./pre-inst-env guix build -f tmp/t.scm
$ ./pre-inst-env guix package -L tmp --show=glibc-utf8-test
------------------------------8<----------------------------------

Something like glibc-utf8-<language>-locales could be easily implemented
on top of this, and I think it could be helpful for the end user too.
WDYT?

Happy hacking!
Miguel

[1] After checking that ./pre-inst-env guix build glibc-utf8-locales
didn't rebuild anything at all, of course.
From b0d2bfbe20a0a48a23a8dd4f14c0acce4ef5842f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
<rosen644835@gmail.com>
Date: Wed, 4 Nov 2020 15:48:45 +0100
Subject: [PATCH] gnu: base: Add optional locales keyword to
make-glibc-utf8-locales.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* gnu/packages/base.scm (make-glibc-utf8-locales): Add keyword parameter
locales with the old value as default.
[arguments]: Use locales value.

Co-authored-by: Efraim Flashner <efraim@flashner.co.il>
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
---
gnu/packages/base.scm | 31 +++++++++++++++++++++++--------
1 file changed, 23 insertions(+), 8 deletions(-)

Toggle diff (71 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..5170a77270 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -52,13 +52,16 @@
#:use-module (gnu packages pkg-config)
#:use-module (gnu packages python)
#:use-module (gnu packages gettext)
+ #:use-module (guix i18n)
#:use-module (guix utils)
#:use-module (guix packages)
#:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix build-system gnu)
#:use-module (guix build-system trivial)
+ #:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 optargs)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (glibc
@@ -1106,9 +1109,16 @@ to the @code{share/locale} sub-directory of this package.")
,(version-major+minor
(package-version glibc)))))))))))
-(define-public (make-glibc-utf8-locales glibc)
+(define %default-utf8-locales
+ ;; These are the locales commonly used for tests---e.g., in Guile's i18n
+ ;; tests.
+ '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+(define*-public (make-glibc-utf8-locales glibc #:key
+ (locales %default-utf8-locales)
+ (name "glibc-utf8-locales"))
+ (define default-locales? (equal? locales %default-utf8-locales))
(package
- (name "glibc-utf8-locales")
+ (name name)
(version (package-version glibc))
(source #f)
(build-system trivial-build-system)
@@ -1142,17 +1152,22 @@ to the @code{share/locale} sub-directory of this package.")
(symlink (string-append locale ".utf8")
(string-append localedir "/"
locale ".UTF-8")))
-
- ;; These are the locales commonly used for
- ;; tests---e.g., in Guile's i18n tests.
- '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+ ',locales)
#t))))
(native-inputs `(("glibc" ,glibc)
("gzip" ,gzip)))
- (synopsis "Small sample of UTF-8 locales")
+ (synopsis (if default-locales?
+ (P_ "Small sample of UTF-8 locales")
+ (P_ "Customized sample of UTF-8 locales")))
(description
- "This package provides a small sample of UTF-8 locales mostly useful in
+ (if default-locales?
+ (P_ "This package provides a small sample of UTF-8 locales mostly useful in
test environments.")
+ (format #f (P_ "This package provides the following UTF-8 locales:
+@itemize
+~{@item ~a~%~}
+@end itemize~%")
+ locales)))
(home-page (package-home-page glibc))
(license (package-license glibc))))
--
2.28.0
-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEEiIeExBRZrMuD5+hMY0xuiXn6vsIFAl+i0v8ACgkQY0xuiXn6
vsLvVgwAlFel35w8Arwy7EwqBEGzfory3AfNvrAPaZxXsODZISybpTpyhe0ts8U5
sBXqCJddyGdNoemqXD5YStAOyCT2SgaK4XnXTX61g0Gmm/n5izcw16TusjG1dSpg
MfjIqRZfsVjbe4WEjepfb1n0T6GAfU0tleqaXeiBQbcTjG8mOy1mNKS373LTR8LW
Vuwhhe9vyk4n5R5RRqQqpZ0QkciY9aKVSe4kkVlOMa3utFxZhb9CMcJ2W3sZ6PWY
QDD5sxgNdGle23wsp0RD0Vc2t9Akg/ERfKK5vbbqF5cFK9TzpSmzx/MUfz4vNw5L
ZDYtizs//JkKupgQxLA/8gH//0CCX0TndktDNH+kM71SR3tNfVY4uVv5pKVO4+oX
evu7CqDPx9hRtqzMo0D15Y8qUfuDjVqlm7CuJJjDVEwiDJW4hqoqFCtj8KwsRvIp
mG/fC5fInwJn3W2wZh7jkkbwTs27LK9enAHQ6y3NEFLFvIeDNvlSP7p/Fl/ayljB
TBn3+Nqa
=Oa2h
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 6 Nov 2020 16:58
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)
87d00qxyrx.fsf@gnu.org
Hi,

Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:

Toggle quote (17 lines)
> From b0d2bfbe20a0a48a23a8dd4f14c0acce4ef5842f Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
> <rosen644835@gmail.com>
> Date: Wed, 4 Nov 2020 15:48:45 +0100
> Subject: [PATCH] gnu: base: Add optional locales keyword to
> make-glibc-utf8-locales.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> * gnu/packages/base.scm (make-glibc-utf8-locales): Add keyword parameter
> locales with the old value as default.
> [arguments]: Use locales value.
>
> Co-authored-by: Efraim Flashner <efraim@flashner.co.il>
> Co-authored-by: Ludovic Courtès <ludo@gnu.org>

Looks reasonable to me.

The next question is how we’ll use it. I think it’s an excellent case
for parameterized packages, for which I’ve proposed a rough design
before, but we need someone to champion on that one.

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 18 Nov 2020 23:10
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)
873616joyn.fsf@gnu.org
Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (21 lines)
> Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:
>
>> From b0d2bfbe20a0a48a23a8dd4f14c0acce4ef5842f Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
>> <rosen644835@gmail.com>
>> Date: Wed, 4 Nov 2020 15:48:45 +0100
>> Subject: [PATCH] gnu: base: Add optional locales keyword to
>> make-glibc-utf8-locales.
>> MIME-Version: 1.0
>> Content-Type: text/plain; charset=UTF-8
>> Content-Transfer-Encoding: 8bit
>>
>> * gnu/packages/base.scm (make-glibc-utf8-locales): Add keyword parameter
>> locales with the old value as default.
>> [arguments]: Use locales value.
>>
>> Co-authored-by: Efraim Flashner <efraim@flashner.co.il>
>> Co-authored-by: Ludovic Courtès <ludo@gnu.org>
>
> Looks reasonable to me.

I think it’s OK for ‘master’.

Toggle quote (4 lines)
> The next question is how we’ll use it. I think it’s an excellent case
> for parameterized packages, for which I’ve proposed a rough design
> before, but we need someone to champion on that one.

It’s not there yet but there’s hope:


Ludo’.
B
B
bdju wrote on 24 Dec 2020 10:38
[PATCH] gnu: Add make-glibc-locales-collection.
(address . 44075@debbugs.gnu.org)
2eb2cb4a63cfbe5e0f1fe1e6634ceca9@tilde.team
Could the en_SE.utf8 locale be added? For those who want their system in English, but with YMD date and 24 hour time.
L
L
Leo Famulari wrote on 24 Dec 2020 23:04
(name . bdju--- via Guix-patches via)(address . guix-patches@gnu.org)(address . 44075@debbugs.gnu.org)
X+UQedmoDguQHb1m@jasmine.lan
On Thu, Dec 24, 2020 at 09:38:51AM +0000, bdju--- via Guix-patches via wrote:
Toggle quote (2 lines)
> Could the en_SE.utf8 locale be added? For those who want their system in English, but with YMD date and 24 hour time.

Is this contained in the glibc-locales package?

The glibc-utf8-locales package is basically just a small package variant
for testing and we are planning to remove it completely.
L
L
Leo Famulari wrote on 24 Dec 2020 23:05
(name . bdju--- via Guix-patches via)(address . guix-patches@gnu.org)(address . 44075@debbugs.gnu.org)
X+UQtQI+3JyT6/N+@jasmine.lan
On Thu, Dec 24, 2020 at 05:04:41PM -0500, Leo Famulari wrote:
Toggle quote (8 lines)
> On Thu, Dec 24, 2020 at 09:38:51AM +0000, bdju--- via Guix-patches via wrote:
> > Could the en_SE.utf8 locale be added? For those who want their system in English, but with YMD date and 24 hour time.
>
> Is this contained in the glibc-locales package?
>
> The glibc-utf8-locales package is basically just a small package variant
> for testing and we are planning to remove it completely.

I read more of this thread and realized I misunderstood its point.
Please disregard my previous message.
M
M
Miguel Ángel Arruga Vivas wrote on 7 Jan 2021 13:00
Re: bug#44075: [PATCH] gnu: Add make-glibc-locales-collection.
(name . Ludovic Courtès)(address . ludo@gnu.org)
875z490xf2.fsf_-_@gmail.com
Hi,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (2 lines)
> I think it’s OK for ‘master’.

Oops, sorry, I forgot completely about this. :-(

I've just pushed as 0d3f2716a8 to master.

Thank you!
Miguel
?