[PATCH] gnu: Add make-glibc-locales-collection.

Open

Details

5 participants

bdju
Efraim Flashner
Leo Famulari
Ludovic Courtès
Miguel Ángel Arruga Vivas

Owner: unassigned

Submitted by: Efraim Flashner

Severity: normal

Efraim Flashner wrote on 19 Oct 2020 08:47

Recipients:(address . guix-patches@gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:20201019064739.4736-1-efraim@flashner.co.il

* gnu/packages/base.scm (make-glibc-locales-collection): New macro.
(en_us-glibc-locales): New variable.
---
 gnu/packages/base.scm | 73 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 72 insertions(+), 1 deletion(-)

Toggle diff (100 lines)diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..41d3aaf865 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -62,7 +62,8 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (glibc
-            libiconv-if-needed))
+            libiconv-if-needed
+            make-custom-glibc-locales))
 
 ;;; Commentary:
 ;;;
@@ -1106,6 +1107,69 @@ to the @code{share/locale} sub-directory of this package.")
                                         ,(version-major+minor
                                           (package-version glibc)))))))))))
 
+(define* (make-glibc-locales-collection
+           glibc
+           #:optional (locales
+                        '(list "en_US.utf8" "en_US.ISO-8859-1")))
+                        ;; This list for testing
+                        ;'(list "el_GR.UTF-8" "en_US.utf8" "he_IL.ISO-8859-8" "ja_JP.EUC-JP" "zh_CN.GB18030" "zh_CN.GBK" "hy_AM.ARMSCII-8")))
+  (package
+    (name "glibc-locales-collection")
+    (version (package-version glibc))
+    (source #f)
+    (build-system trivial-build-system)
+    (arguments
+     `(#:modules ((guix build utils))
+       #:builder
+       (begin
+         (use-modules (guix build utils))
+
+         (let* ((libc      (assoc-ref %build-inputs "glibc"))
+                (gzip      (assoc-ref %build-inputs "gzip"))
+                (out       (assoc-ref %outputs "out"))
+                (localedir (string-append out "/lib/locale/"
+                                          ,(version-major+minor version))))
+           ;; 'localedef' needs 'gzip'.
+           (setenv "PATH" (string-append libc "/bin:" gzip "/bin"))
+
+           (mkdir-p localedir)
+           (for-each
+             (lambda (locale)
+               (let* ((contains-dot? (string-index locale #\.))
+                      (encoding-type (substring locale (1+ contains-dot?)))
+                      (raw-locale    (substring locale 0 contains-dot?))
+                      (utf8? (or (number? (string-contains locale ".utf8"))
+                                 (number? (string-contains locale ".UTF-8"))))
+                      (file (if utf8?
+                              (string-append localedir "/" raw-locale ".utf8")
+                              (if (string-contains locale ".ISO")
+                                (string-append localedir "/" raw-locale)
+                                (string-append localedir "/" locale)))))
+
+                 (invoke "localedef" "--no-archive"
+                         "--prefix" localedir
+                         "-i" raw-locale
+                         "-f" (if (equal? "utf8" encoding-type)
+                                "UTF-8"
+                                encoding-type)
+                         file)
+
+                 ;; Is it utf8 or UTF-8? NO ONE KNOWS!
+                 (when utf8?
+                   (symlink (string-append raw-locale ".utf8")
+                            (string-append localedir "/"
+                                           raw-locale ".UTF-8")))))
+             ,locales)
+           #t))))
+    (native-inputs `(("glibc" ,glibc)
+                     ("gzip" ,gzip)))
+    (synopsis "Customizable collection of locales")
+    (description
+     "This package provides a custom collection of locales useful for
+providing exactly the locales requested when size matters.")
+    (home-page (package-home-page glibc))
+    (license (package-license glibc))))
+
 (define-public (make-glibc-utf8-locales glibc)
   (package
     (name "glibc-utf8-locales")
@@ -1161,6 +1225,13 @@ test environments.")
 (define-public glibc-utf8-locales
   (make-glibc-utf8-locales glibc))
 
+(define-public en_us-glibc-locales
+  (package
+    (inherit (make-glibc-locales-collection
+               glibc
+               '(list "en_US.utf8" "en_US.ISO-8859-1")))
+    (name "en-us-glibc-locales")))
+
 ;; Packages provided to ease use of binaries linked against the previous libc.
 (define-public glibc-locales-2.29
   (package (inherit (make-glibc-locales glibc-2.29))
-- 
2.28.0

Miguel Ángel Arruga Vivas wrote on 19 Oct 2020 15:17

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 44075@debbugs.gnu.org)

Message-ID:87tuuqnym5.fsf@gmail.com

Hi Efraim,

I've been taking a look into your patch. One issue are the comments

about utf8 and UTF-8, as the issue is already explained in

make-glibc[-utf8]-locales.

Other point is:

Efraim Flashner <efraim@flashner.co.il> writes:

Toggle quote (7 lines)

> +(define* (make-glibc-locales-collection

> + glibc

> + #:optional (locales

> + '(list "en_US.utf8" "en_US.ISO-8859-1")))

> (... Removed for clarity ...)

> + ,locales)

I would have used list there like (list ,@locales) or '(,@locales), this

looks a bit odd to my eyes at least. I'd expect this kind of calling code:

(let ((locales '("de_CH.utf8" ... "de_DE.utf8"))

(my-glibc ...))

(make-glibc-locales-collection myglibc locales))

Enforcing an extra quotation for no real reason on the calling site, as

strings are self-evaluating objects, and the use of the symbol list,

whose meaning depends on other context of execution, doesn't seem

necessary. Even worse, my example would raise an error as "de_CH.utf8"

is not a procedure.

What do you think about replacing make-glibc-utf8-locales with a call of

the new function (using that code) ensuring that the generated

derivation stays the same for that case (i.e. it's optimized for the

UTF-8 case)?

Happy hacking!

Miguel

Efraim Flashner wrote on 19 Oct 2020 16:02

Recipients:(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)(address . 44075@debbugs.gnu.org)

Message-ID:20201019140236.GF9117@E5400

On Mon, Oct 19, 2020 at 03:17:54PM +0200, Miguel Ángel Arruga Vivas wrote:

Toggle quote (6 lines)

> Hi Efraim,

> I've been taking a look into your patch. One issue are the comments

> about utf8 and UTF-8, as the issue is already explained in

> make-glibc[-utf8]-locales.

Thanks for taking a look. For the utf8 vs UTF-8 there are a couple of
comments in the code:
The above phase does not install locales with names using
the "normalized codeset."  Thus, create symlinks like:
en_US.utf8 -> en_US.UTF-8

and also:
For backward compatibility with Guix
<= 0.8.3, add "xx_YY.UTF-8".

When I check on one of my Debian boxes:
$localectl list-locales
C.UTF-8
en_US.utf8

I've learned that 'C.UTF-8' isn't from upstream, Debian has a patch for
it. Having written this patch a week or so ago and returning to it I
can't tell you which is the correct one. I think it's best to offer both
so it's not confusing which is correct. That works for the logic for
accepting either in the list of locales, but I'm concerned that skipping
the symlink to the other one could cause problems.

Toggle quote (24 lines)> 
> Other point is:
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> > +(define* (make-glibc-locales-collection
> > +           glibc
> > +           #:optional (locales
> > +                        '(list "en_US.utf8" "en_US.ISO-8859-1")))
> > (... Removed for clarity ...)
> > +             ,locales)
> 
> I would have used list there like (list ,@locales) or '(,@locales), this
> looks a bit odd to my eyes at least.  I'd expect this kind of calling code:
> 
> (let ((locales '("de_CH.utf8" ... "de_DE.utf8"))
>       (my-glibc ...))
>   (make-glibc-locales-collection myglibc locales))
> 
> Enforcing an extra quotation for no real reason on the calling site, as
> strings are self-evaluating objects, and the use of the symbol list,
> whose meaning depends on other context of execution, doesn't seem
> necessary.  Even worse, my example would raise an error as "de_CH.utf8"
> is not a procedure.

My scheme-foo isn't terribly strong, sometimes I just hack at it until

the code does what I want, and this is one of those times. I've changed

it so that locales takes a list and not an item that is a list.

Toggle quote (5 lines)

> What do you think about replacing make-glibc-utf8-locales with a call of

> the new function (using that code) ensuring that the generated

> derivation stays the same for that case (i.e. it's optimized for the

> UTF-8 case)?

This is what I originally wanted to do, but there's a glibc-locales
buried in the bootstrap path so it's not so easy to just swap it out. I
can make the change in core-updates. I'll play around with it and see if
I can come out with the same derivation using a different function, but
I'm not expecting it to turn out identical.

Toggle quote (7 lines)

> Happy hacking!

> Miguel

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

From 3c2dfe702ad1c62b2f73a1f1036da3bcfa007c15 Mon Sep 17 00:00:00 2001
From: Efraim Flashner <efraim@flashner.co.il>
Date: Thu, 1 Oct 2020 22:50:32 +0300
Subject: [PATCH v2] gnu: Add make-glibc-locales-collection.

* gnu/packages/base.scm (make-glibc-locales-collection): New macro.
(en_us-glibc-locales): New variable.
---
 gnu/packages/base.scm | 73 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 72 insertions(+), 1 deletion(-)

Toggle diff (100 lines)diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..4ea31c2ab6 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -62,7 +62,8 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (glibc
-            libiconv-if-needed))
+            libiconv-if-needed
+            make-custom-glibc-locales))
 
 ;;; Commentary:
 ;;;
@@ -1106,6 +1107,69 @@ to the @code{share/locale} sub-directory of this package.")
                                         ,(version-major+minor
                                           (package-version glibc)))))))))))
 
+(define* (make-glibc-locales-collection
+           glibc
+           #:optional (locales
+                        (list "en_US.utf8" "en_US.ISO-8859-1")))
+                        ;; This list for testing
+                        ;(list "el_GR.UTF-8" "en_US.utf8" "he_IL.ISO-8859-8" "ja_JP.EUC-JP" "zh_CN.GB18030" "zh_CN.GBK" "hy_AM.ARMSCII-8")))
+  (package
+    (name "glibc-locales-collection")
+    (version (package-version glibc))
+    (source #f)
+    (build-system trivial-build-system)
+    (arguments
+     `(#:modules ((guix build utils))
+       #:builder
+       (begin
+         (use-modules (guix build utils))
+
+         (let* ((libc      (assoc-ref %build-inputs "glibc"))
+                (gzip      (assoc-ref %build-inputs "gzip"))
+                (out       (assoc-ref %outputs "out"))
+                (localedir (string-append out "/lib/locale/"
+                                          ,(version-major+minor version))))
+           ;; 'localedef' needs 'gzip'.
+           (setenv "PATH" (string-append libc "/bin:" gzip "/bin"))
+
+           (mkdir-p localedir)
+           (for-each
+             (lambda (locale)
+               (let* ((contains-dot? (string-index locale #\.))
+                      (encoding-type (substring locale (1+ contains-dot?)))
+                      (raw-locale    (substring locale 0 contains-dot?))
+                      (utf8? (or (number? (string-contains locale ".utf8"))
+                                 (number? (string-contains locale ".UTF-8"))))
+                      (file (if utf8?
+                              (string-append localedir "/" raw-locale ".utf8")
+                              (if (string-contains locale ".ISO")
+                                (string-append localedir "/" raw-locale)
+                                (string-append localedir "/" locale)))))
+
+                 (invoke "localedef" "--no-archive"
+                         "--prefix" localedir
+                         "-i" raw-locale
+                         "-f" (if (equal? "utf8" encoding-type)
+                                "UTF-8"
+                                encoding-type)
+                         file)
+
+                 ;; Is it utf8 or UTF-8? NO ONE KNOWS!
+                 (when utf8?
+                   (symlink (string-append raw-locale ".utf8")
+                            (string-append localedir "/"
+                                           raw-locale ".UTF-8")))))
+             (list ,@locales))
+           #t))))
+    (native-inputs `(("glibc" ,glibc)
+                     ("gzip" ,gzip)))
+    (synopsis "Customizable collection of locales")
+    (description
+     "This package provides a custom collection of locales useful for
+providing exactly the locales requested when size matters.")
+    (home-page (package-home-page glibc))
+    (license (package-license glibc))))
+
 (define-public (make-glibc-utf8-locales glibc)
   (package
     (name "glibc-utf8-locales")
@@ -1161,6 +1225,13 @@ test environments.")
 (define-public glibc-utf8-locales
   (make-glibc-utf8-locales glibc))
 
+(define-public en_us-glibc-locales
+  (package
+    (inherit (make-glibc-locales-collection
+               glibc
+               (list "en_US.utf8" "en_US.ISO-8859-1")))
+    (name "en-us-glibc-locales")))
+
 ;; Packages provided to ease use of binaries linked against the previous libc.
 (define-public glibc-locales-2.29
   (package (inherit (make-glibc-locales glibc-2.29))
-- 
2.28.0

Miguel Ángel Arruga Vivas wrote on 19 Oct 2020 19:43

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 44075@debbugs.gnu.org)

Message-ID:87mu0im7r4.fsf@gmail.com

Hello,

Efraim Flashner <efraim@flashner.co.il> writes:

Toggle quote (2 lines)

> Thanks for taking a look.

No problem at all, I was thinking about something along these lines too,

but then I saw your patch. :-)

Toggle quote (9 lines)

> For the utf8 vs UTF-8 there are a couple of comments in the code:

> The above phase does not install locales with names using

> the "normalized codeset." Thus, create symlinks like:

> en_US.utf8 -> en_US.UTF-8

> and also:

> For backward compatibility with Guix

> <= 0.8.3, add "xx_YY.UTF-8".

Yes, what I mean is that the comments along the code may need to be

clarified, but adding a "nobody knows" doesn't add much information.

The actual source[1] says that the correct value is utf8, following

the rules for the 'normalized codeset' naming, that I copy here:

1. Remove all characters besides numbers and letters.

2. Fold letters to lowercase.

3. If the same only contains digits prepend the string ‘"iso"’.

We should stick to that naming regarding libc locales (note to self),

even though we keep the links for compatibility. That includes the

other locale at en_us-glibc-locales.

Toggle quote (8 lines)

>> What do you think about replacing make-glibc-utf8-locales with a call of

>> the new function (using that code) ensuring that the generated

>> derivation stays the same for that case (i.e. it's optimized for the

>> UTF-8 case)?

> This is what I originally wanted to do, but there's a glibc-locales

> buried in the bootstrap path so it's not so easy to just swap it out.

I guess you mean glibc-utf8-locales, if not I'm missing where that

reference could come from. That's why I insist on leaving exactly the

same derivation.

Toggle quote (4 lines)

> I can make the change in core-updates. I'll play around with it and

> see if I can come out with the same derivation using a different

> function, but I'm not expecting it to turn out identical.

The colour of my lisp-rate belt isn't even close to some you can see

around here but I could bring you some help if you want, because I think

it's easier to do it without any change in current packages than it

might sound. Not the definitions in scheme, of course, I mean the

derivations the daemon actually sees.

Said this, I've seen that [single-]locale-directory does mostly the

same, and there is a build-locale function in gnu/build/locale.scm... so

I'm starting to see as a better idea to clean up glibc-utf8-locales up

in core-updates using the latter, as it would lead to cleaner code for

the builder.

Could you check that and tell if you consider feasible what I propose?

Happy hacking!

Miguel

[1] info "(libc)Using gettextized software"

Efraim Flashner wrote on 21 Oct 2020 09:01

Recipients:(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)(address . 44075@debbugs.gnu.org)

Message-ID:20201021070155.GQ9117@E5400

On Mon, Oct 19, 2020 at 07:43:27PM +0200, Miguel Ángel Arruga Vivas wrote:

Toggle quote (8 lines)

> Hello,

> Efraim Flashner <efraim@flashner.co.il> writes:

> > Thanks for taking a look.

> No problem at all, I was thinking about something along these lines too,

> but then I saw your patch. :-)

Your patch was what got me to actually send mine to guix-patches :)

Toggle quote (12 lines)> > For the utf8 vs UTF-8 there are a couple of comments in the code:
> > The above phase does not install locales with names using
> > the "normalized codeset."  Thus, create symlinks like:
> > en_US.utf8 -> en_US.UTF-8
> >
> > and also:
> > For backward compatibility with Guix
> > <= 0.8.3, add "xx_YY.UTF-8".
> 
> Yes, what I mean is that the comments along the code may need to be
> clarified, but adding a "nobody knows" doesn't add much information.

That is a good point. I'll change it to something actually useful,

mentioning that there's a convention but that people often get it wrong.

Toggle quote (11 lines)> The actual source[1] says that the correct value is utf8, following
> the rules for the 'normalized codeset' naming, that I copy here:
> 
>   1. Remove all characters besides numbers and letters.
>   2. Fold letters to lowercase.
>   3. If the same only contains digits prepend the string ‘"iso"’.
> 
> We should stick to that naming regarding libc locales (note to self),
> even though we keep the links for compatibility.  That includes the
> other locale at en_us-glibc-locales.

I agree it should be renamed to en-us. I have some bikeshedding to think

about for en-us-glibc-locales vs glibc-locales-en-us. And I'll also test

out one for glibc-locales-guile-tests.

Toggle quote (12 lines)> >> What do you think about replacing make-glibc-utf8-locales with a call of
> >> the new function (using that code) ensuring that the generated
> >> derivation stays the same for that case (i.e. it's optimized for the
> >> UTF-8 case)?
> >
> > This is what I originally wanted to do, but there's a glibc-locales
> > buried in the bootstrap path so it's not so easy to just swap it out.
> 
> I guess you mean glibc-utf8-locales, if not I'm missing where that
> reference could come from.  That's why I insist on leaving exactly the
> same derivation.

I meant in commencement.scm. There's a glibc-utf8-locales-final near the

definition for %boot5-inputs.

Toggle quote (18 lines)> > I can make the change in core-updates. I'll play around with it and
> > see if I can come out with the same derivation using a different
> > function, but I'm not expecting it to turn out identical.
> 
> The colour of my lisp-rate belt isn't even close to some you can see
> around here but I could bring you some help if you want, because I think
> it's easier to do it without any change in current packages than it
> might sound.  Not the definitions in scheme, of course, I mean the
> derivations the daemon actually sees.
> 
> Said this, I've seen that [single-]locale-directory does mostly the
> same, and there is a build-locale function in gnu/build/locale.scm... so
> I'm starting to see as a better idea to clean up glibc-utf8-locales up
> in core-updates using the latter, as it would lead to cleaner code for
> the builder.
> 
> Could you check that and tell if you consider feasible what I propose?

I hadn't seen gnu/build/locales.scm. I'll have to see what we can do

about using it. We normally don't allow cross-over from the build side

to the packages side.

Toggle quote (5 lines)

> Happy hacking!

> Miguel

> [1] info "(libc)Using gettextized software"

-- 
Efraim Flashner   <efraim@flashner.co.il>   ????? ?????
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Ludovic Courtès wrote on 21 Oct 2020 19:09

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 44075@debbugs.gnu.org)

Message-ID:878sbz1p71.fsf@gnu.org

Hi!

Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (3 lines)

> * gnu/packages/base.scm (make-glibc-locales-collection): New macro.

> (en_us-glibc-locales): New variable.

Cool!

A while back I posted the attached patch, which reuses existing code

from (gnu system locale) and (gnu packages base). I think I used it

just to see how much space all the UTF-8 locales take, compared to all

the locales.

Would it make sense to borrow from that?

At any rate, it’s the kind of package that’ll be more useful if/when we

have parameterized packages.

Ludo’.

Toggle diff (97 lines)diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..3fc43b04da 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -53,6 +53,8 @@
   #:use-module (gnu packages python)
   #:use-module (gnu packages gettext)
   #:use-module (guix utils)
+  #:use-module (guix gexp)
+  #:use-module (guix modules)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
@@ -62,6 +64,8 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (glibc
+            %default-utf8-locales
+            make-glibc-utf8-locales
             libiconv-if-needed))
 
 ;;; Commentary:
@@ -1106,7 +1110,12 @@ to the @code{share/locale} sub-directory of this package.")
                                         ,(version-major+minor
                                           (package-version glibc)))))))))))
 
-(define-public (make-glibc-utf8-locales glibc)
+(define %default-utf8-locales
+  '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+
+(define* (make-glibc-utf8-locales glibc #:optional
+                                  (locales %default-utf8-locales)
+                                  (locale-file #f))
   (package
     (name "glibc-utf8-locales")
     (version (package-version glibc))
@@ -1145,10 +1154,17 @@ to the @code{share/locale} sub-directory of this package.")
 
                                ;; These are the locales commonly used for
                                ;; tests---e.g., in Guile's i18n tests.
-                               '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+                               ,(if locale-file
+                                    `(call-with-input-file
+                                         (assoc-ref %build-inputs "locale-file")
+                                       read)
+                                    `',locales))
                      #t))))
     (native-inputs `(("glibc" ,glibc)
-                     ("gzip" ,gzip)))
+                     ("gzip" ,gzip)
+                     ,@(if locale-file
+                           `(("locale-file" ,locale-file))
+                           '())))
     (synopsis "Small sample of UTF-8 locales")
     (description
      "This package provides a small sample of UTF-8 locales mostly useful in
@@ -1169,6 +1185,40 @@ test environments.")
   (package (inherit (make-glibc-utf8-locales glibc-2.29))
            (name "glibc-utf8-locales-2.29")))
 
+(define (glibc-supported-locales libc)
+  ((module-ref (resolve-interface '(gnu system locale)) ;FIXME: hack
+               'glibc-supported-locales)
+   libc))
+
+(define* (make-glibc-utf8-locales/full #:optional (glibc glibc))
+  (define utf8-locales
+    (computed-file "glibc-supported-utf8-locales.scm"
+                   #~(begin
+                       (use-modules (srfi srfi-1)
+                                    (ice-9 match)
+                                    (ice-9 pretty-print))
+
+                       (define locales
+                         (call-with-input-file
+                             #+(glibc-supported-locales glibc)
+                           read))
+
+                       (define utf8-locales
+                         (filter-map (match-lambda
+                                       ((name . "UTF-8")
+                                        (if (string-suffix? ".UTF-8" name)
+                                            (string-drop-right name 6)
+                                            name))
+                                       (_ #f))
+                                     locales))
+
+                       (call-with-output-file #$output
+                         (lambda (port)
+                           (pretty-print utf8-locales port))))))
+
+  (make-glibc-utf8-locales glibc #:locale-file utf8-locales))
+
+
 (define-public which
   (package
     (name "which")

Miguel Ángel Arruga Vivas wrote on 22 Oct 2020 01:18

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 44075@debbugs.gnu.org)

Message-ID:87blgvjhhj.fsf@gmail.com

Hello,

Efraim Flashner <efraim@flashner.co.il> writes:

Toggle quote (2 lines)

> Your patch was what got me to actually send mine to guix-patches :)

Thanks. :-)

Toggle quote (3 lines)

> That is a good point. I'll change it to something actually useful,

> mentioning that there's a convention but that people often get it wrong.

Great, the more knowledge, the easier to overcome that common

misunderstanding. :-)

Toggle quote (4 lines)

> I agree it should be renamed to en-us. I have some bikeshedding to think

> about for en-us-glibc-locales vs glibc-locales-en-us. And I'll also test

> out one for glibc-locales-guile-tests.

I meant the ISO-8859-1, the locale creation should be en_EN.iso88591,

the locale name can be en_EN.88591 too...

Although in that case shouldn't be glibc-en-us-locales? I'd like green

lights for the bike shed, please. ;-)

Toggle quote (4 lines)

> I hadn't seen gnu/build/locales.scm. I'll have to see what we can do

> about using it. We normally don't allow cross-over from the build side

> to the packages side.

It is already being used in the build of glibc-locales and I guess it
would be harmless to move it to guix build if that is a problem, but I
think it can be used as it is right now, even I didn't check yet.  In
this for-each(gnu/packages/base.scm:1084), for example:
------------------------->8-------------begin----------------------
(for-each (lambda (directory)
            (let*-values (((base)
                           (basename directory))
                          ((name codeset)
                           (locale->name+codeset base))
                          ((normalized)
                           (normalize-codeset codeset)))
              (unless (string=? codeset normalized)
                (symlink base
                         (string-append (dirname directory)
                                        "/" name "."
                                        normalized)))))
          locales)
-------------------------8<--------------end-----------------------

I would write something like this in the builder code:
------------------------->8-------------begin----------------------
(define (locale-builder directory)
  (lambda (locale)
    (let-values (((name codeset)
                  (locale->name+codeset)))
      (if codeset
          (build-locale name
                        #:directory directory
                        #:codeset (normalize-codeset codeset))
          (build-locale name
                        #:directory directory
                        #:codeset "utf8")))))
(let ...
  (for-each (locale-builder localedir) '(,@locales))
  ...)
-------------------------8<--------------end-----------------------

I leave you here the function I was hacking apart (please, do not
confuse with working code ;-)) if you want to take any idea other from
it:
------------------------->8-------------begin----------------------
(define*-public (make-glibc-selected-locales
                 glibc
                 #:key
                 (suffix "-utf8-locales")
                 (locales %default-locales)
                 (alias %default-alias))
  (package
    (name (string-append (package-name glibc) suffix))
    (version (package-version glibc))
    (source #f)
    (build-system trivial-build-system)
    (arguments
     `(#:modules ((guix build utils) (gnu build locale))
       #:builder
       (begin
         (use-modules (guix build utils) (gnu build locale))
         ;; Print the common alias for locale
         (define (print-locale-alias locale)
           (let-values (((name codeset)
                         (locale->name+codeset)))
             (if codeset
                 (or (string-equal codeset (normalize-codeset codeset))
                     (format #t "~a.~a ~a.~a~%" name codeset
                             name (normalize-codeset codeset)))
                 (format #t "~a.UTF-8 ~a.utf8" name name))))

         ;; Generate a function that builds the locale into directory.
         (define (locale-builder directory)
           (lambda (locale)
             (let-values (((name codeset)
                           (locale->name+codeset)))
               (if codeset
                   (build-locale name
                                 #:directory directory
                                 #:codeset (normalize-codeset codeset))
                   (build-locale name
                                 #:directory directory
                                 #:codeset "utf8")))))

         ;; Do the job.
         (let* ((libc      (assoc-ref %build-inputs "glibc"))
                (gzip      (assoc-ref %build-inputs "gzip"))
                (out       (assoc-ref %outputs "out"))
                (localedir (string-append out "/lib/locale/"
                                          ,(version-major+minor version)))
                (alias     (string-append localedir "/locale.alias"))
                (locales   '(,@locales)))

           ;; 'localedef' needs 'gzip'.
           (setenv "PATH" (string-append libc "/bin:" gzip "/bin"))

           (mkdir-p localedir)
           ;; Generate each locale provided.
           (for-each (locale-builder localedir) locales)
           ;; Generate alias file.
           (with-output-to-file alias
             (lambda ()
               (format #t "~a~%" "# Aliases for common codeset names.")
               (for-each print-locale-alias locales)
               (format #t "~a~%" "# Other aliases.")
               (for-each (lambda (line)
                           (format #t "~a~%" line))
                         '(,@alias)))))))
    (native-inputs `(("glibc" ,glibc)
                     ("gzip" ,gzip)))
    (synopsis "Configured sample of locales")
    (description
     "TODO.")
    (home-page (package-home-page glibc))
    (license (package-license glibc))))
-------------------------8<--------------end-----------------------

Happy hacking,
Miguel

Miguel Ángel Arruga Vivas wrote on 4 Nov 2020 17:12

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87a6vx85j4.fsf@gmail.com

Hi Efraim and Ludo!

This patch could be an starting point to even master, it allows to

extend easily glibc-utf8-locales to the desired ones.

I've used the following steps to test it[1]:

------------------------------>8----------------------------------

$ echo '(define-module (t) #:use-module (gnu packages base))

(define-public glibc-utf8-test

(make-glibc-utf8-locales glibc #:locales (list "es_ES")

#:name "glibc-utf8-test"))

glibc-utf8-test' > tmp/t.scm

$ ./pre-inst-env guix build -f tmp/t.scm

$ ./pre-inst-env guix package -L tmp --show=glibc-utf8-test

------------------------------8<----------------------------------

Something like glibc-utf8-<language>-locales could be easily implemented

on top of this, and I think it could be helpful for the end user too.

WDYT?

Happy hacking!

Miguel

[1] After checking that ./pre-inst-env guix build glibc-utf8-locales

didn't rebuild anything at all, of course.

From b0d2bfbe20a0a48a23a8dd4f14c0acce4ef5842f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
 <rosen644835@gmail.com>
Date: Wed, 4 Nov 2020 15:48:45 +0100
Subject: [PATCH] gnu: base: Add optional locales keyword to
 make-glibc-utf8-locales.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* gnu/packages/base.scm (make-glibc-utf8-locales): Add keyword parameter
locales with the old value as default.
[arguments]: Use locales value.

Co-authored-by: Efraim Flashner <efraim@flashner.co.il>
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
---
 gnu/packages/base.scm | 31 +++++++++++++++++++++++--------
 1 file changed, 23 insertions(+), 8 deletions(-)

Toggle diff (71 lines)diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..5170a77270 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -52,13 +52,16 @@
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages gettext)
+  #:use-module (guix i18n)
   #:use-module (guix utils)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system trivial)
+  #:use-module (ice-9 format)
   #:use-module (ice-9 match)
+  #:use-module (ice-9 optargs)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (glibc
@@ -1106,9 +1109,16 @@ to the @code{share/locale} sub-directory of this package.")
                                         ,(version-major+minor
                                           (package-version glibc)))))))))))
 
-(define-public (make-glibc-utf8-locales glibc)
+(define %default-utf8-locales
+  ;; These are the locales commonly used for tests---e.g., in Guile's i18n
+  ;; tests.
+  '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+(define*-public (make-glibc-utf8-locales glibc #:key
+                                         (locales %default-utf8-locales)
+                                         (name "glibc-utf8-locales"))
+  (define default-locales? (equal? locales %default-utf8-locales))
   (package
-    (name "glibc-utf8-locales")
+    (name name)
     (version (package-version glibc))
     (source #f)
     (build-system trivial-build-system)
@@ -1142,17 +1152,22 @@ to the @code{share/locale} sub-directory of this package.")
                                  (symlink (string-append locale ".utf8")
                                           (string-append localedir "/"
                                                          locale ".UTF-8")))
-
-                               ;; These are the locales commonly used for
-                               ;; tests---e.g., in Guile's i18n tests.
-                               '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+                               ',locales)
                      #t))))
     (native-inputs `(("glibc" ,glibc)
                      ("gzip" ,gzip)))
-    (synopsis "Small sample of UTF-8 locales")
+    (synopsis (if default-locales?
+                  (P_ "Small sample of UTF-8 locales")
+                  (P_ "Customized sample of UTF-8 locales")))
     (description
-     "This package provides a small sample of UTF-8 locales mostly useful in
+     (if default-locales?
+         (P_ "This package provides a small sample of UTF-8 locales mostly useful in
 test environments.")
+         (format #f (P_ "This package provides the following UTF-8 locales:
+@itemize
+~{@item ~a~%~}
+@end itemize~%")
+                 locales)))
     (home-page (package-home-page glibc))
     (license (package-license glibc))))
 
-- 
2.28.0

Ludovic Courtès wrote on 6 Nov 2020 16:58

Recipients:(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)

Message-ID:87d00qxyrx.fsf@gnu.org

Hi,

Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:

Toggle quote (17 lines)> From b0d2bfbe20a0a48a23a8dd4f14c0acce4ef5842f Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
>  <rosen644835@gmail.com>
> Date: Wed, 4 Nov 2020 15:48:45 +0100
> Subject: [PATCH] gnu: base: Add optional locales keyword to
>  make-glibc-utf8-locales.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> * gnu/packages/base.scm (make-glibc-utf8-locales): Add keyword parameter
> locales with the old value as default.
> [arguments]: Use locales value.
>
> Co-authored-by: Efraim Flashner <efraim@flashner.co.il>
> Co-authored-by: Ludovic Courtès <ludo@gnu.org>

Looks reasonable to me.

The next question is how we’ll use it. I think it’s an excellent case

for parameterized packages, for which I’ve proposed a rough design

before, but we need someone to champion on that one.

Thanks,

Ludo’.

Ludovic Courtès wrote on 18 Nov 2020 23:10

Recipients:(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)

Message-ID:873616joyn.fsf@gnu.org

Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (21 lines)> Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:
>
>> From b0d2bfbe20a0a48a23a8dd4f14c0acce4ef5842f Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
>>  <rosen644835@gmail.com>
>> Date: Wed, 4 Nov 2020 15:48:45 +0100
>> Subject: [PATCH] gnu: base: Add optional locales keyword to
>>  make-glibc-utf8-locales.
>> MIME-Version: 1.0
>> Content-Type: text/plain; charset=UTF-8
>> Content-Transfer-Encoding: 8bit
>>
>> * gnu/packages/base.scm (make-glibc-utf8-locales): Add keyword parameter
>> locales with the old value as default.
>> [arguments]: Use locales value.
>>
>> Co-authored-by: Efraim Flashner <efraim@flashner.co.il>
>> Co-authored-by: Ludovic Courtès <ludo@gnu.org>
>
> Looks reasonable to me.

I think it’s OK for ‘master’.

Toggle quote (4 lines)

> The next question is how we’ll use it. I think it’s an excellent case

> for parameterized packages, for which I’ve proposed a rough design

> before, but we need someone to champion on that one.

It’s not there yet but there’s hope:

https://lists.gnu.org/archive/html/guix-devel/2020-11/msg00312.html

Ludo’.

bdju wrote on 24 Dec 2020 10:38

[PATCH] gnu: Add make-glibc-locales-collection.

Recipients:(address . 44075@debbugs.gnu.org)

Message-ID:2eb2cb4a63cfbe5e0f1fe1e6634ceca9@tilde.team

Could the en_SE.utf8 locale be added? For those who want their system in English, but with YMD date and 24 hour time.

Leo Famulari wrote on 24 Dec 2020 23:04

Recipients:(name . bdju--- via Guix-patches via)(address . guix-patches@gnu.org)(address . 44075@debbugs.gnu.org)

Message-ID:X+UQedmoDguQHb1m@jasmine.lan

On Thu, Dec 24, 2020 at 09:38:51AM +0000, bdju--- via Guix-patches via wrote:

Toggle quote (2 lines)

> Could the en_SE.utf8 locale be added? For those who want their system in English, but with YMD date and 24 hour time.

Is this contained in the glibc-locales package?

The glibc-utf8-locales package is basically just a small package variant

for testing and we are planning to remove it completely.

Leo Famulari wrote on 24 Dec 2020 23:05

Recipients:(name . bdju--- via Guix-patches via)(address . guix-patches@gnu.org)(address . 44075@debbugs.gnu.org)

Message-ID:X+UQtQI+3JyT6/N+@jasmine.lan

On Thu, Dec 24, 2020 at 05:04:41PM -0500, Leo Famulari wrote:

Toggle quote (8 lines)

> On Thu, Dec 24, 2020 at 09:38:51AM +0000, bdju--- via Guix-patches via wrote:

> > Could the en_SE.utf8 locale be added? For those who want their system in English, but with YMD date and 24 hour time.

> Is this contained in the glibc-locales package?

> The glibc-utf8-locales package is basically just a small package variant

> for testing and we are planning to remove it completely.

I read more of this thread and realized I misunderstood its point.

Please disregard my previous message.

Miguel Ángel Arruga Vivas wrote on 7 Jan 2021 13:00

Re: bug#44075: [PATCH] gnu: Add make-glibc-locales-collection.

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:875z490xf2.fsf_-_@gmail.com

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (2 lines)

> I think it’s OK for ‘master’.

Oops, sorry, I forgot completely about this. :-(

I've just pushed as 0d3f2716a8 to master.

Thank you!

Miguel

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 44075@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 44075

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date