Time zone database improvements

  • Open
  • quality assurance status badge
Details
2 participants
  • Leo Famulari
  • Ludovic Courtès
Owner
unassigned
Submitted by
Leo Famulari
Severity
normal
L
L
Leo Famulari wrote on 21 Apr 2023 22:59
(address . guix-patches@gnu.org)
ZEL5PdiZbj243ZRD@jasmine.lan
Currently, almost the entire distro is rebuilt when we need to update
the time zone database, tzdata.

We need to do this a few times a year, and it doesn't make any sense for
packages to keep a static reference to this database which will be out
of date in a few months.

Instead, they should look up the time zones at run-time, which most
programs that actually use time zones do.

These patches start improving the situation.

Libical has committed a fix for this almost 5 years ago, but still
hasn't released it, so we cherry-pick Nix's patch:


Go wants to embed a copy of the database in case one cannot be found. I
don't see why having an incorrect time zone is better than no time
zones, so here's a patch to remove the embedded copy.

And after those two changes, we are able to update tzdata to the current
version.

We have more work to do to remove or replace python-pytz, which
unfortunately contains its own embedded copy of the database and a few
thousand dependents. Luckily, it seems that Python 3.9 has made pytz
obsolete:


And there are still some small reverse dependency graphs of packages
that mistakenly depend on tzdata.
L
L
Leo Famulari wrote on 21 Apr 2023 23:01
[PATCH 1/3] gnu: libical: Find the time zone database in the environment.
(address . 63006@debbugs.gnu.org)
724078bec9b692f2d0e3fae4a03a23ef96d94ee5.1682110862.git.leo@famulari.name
* gnu/packages/patches/libical-respect-tzdir.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/calendar.scm (libical)[source]: Use it.
[inputs]: Remove tzdata.
---
gnu/local.mk | 1 +
gnu/packages/calendar.scm | 3 +-
.../patches/libical-respect-tzdir.patch | 38 +++++++++++++++++++
3 files changed, 41 insertions(+), 1 deletion(-)
create mode 100644 gnu/packages/patches/libical-respect-tzdir.patch

Toggle diff (79 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index b631fa6d5a..f301c33cfa 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1393,6 +1393,7 @@ dist_patch_DATA = \
%D%/packages/patches/libffi-3.3-powerpc-fixes.patch \
%D%/packages/patches/libffi-float128-powerpc64le.patch \
%D%/packages/patches/libgeotiff-fix-tests-with-proj-9.1.1.patch \
+ %D%/packages/patches/libical-respect-tzdir.patch \
%D%/packages/patches/libobjc2-unbundle-robin-map.patch \
%D%/packages/patches/librime-fix-build-with-gcc10.patch \
%D%/packages/patches/libvirt-add-install-prefix.patch \
diff --git a/gnu/packages/calendar.scm b/gnu/packages/calendar.scm
index cf4286100d..030f9c52e7 100644
--- a/gnu/packages/calendar.scm
+++ b/gnu/packages/calendar.scm
@@ -127,6 +127,7 @@ (define-public libical
(uri (string-append
"https://github.com/libical/libical/releases/download/v"
version "/libical-" version ".tar.gz"))
+ (patches (search-patches "libical-respect-tzdir.patch"))
(sha256
(base32
"0cqc1wpalxmxjx8dmcaga9w8kd5l7944hqmidz43hifaf7fhaixl"))))
@@ -166,7 +167,7 @@ (define-public libical
pkg-config
vala))
(inputs
- (list glib libxml2 tzdata))
+ (list glib libxml2))
(propagated-inputs
;; In Requires.private of libical.pc.
(list icu4c))
diff --git a/gnu/packages/patches/libical-respect-tzdir.patch b/gnu/packages/patches/libical-respect-tzdir.patch
new file mode 100644
index 0000000000..f749addf44
--- /dev/null
+++ b/gnu/packages/patches/libical-respect-tzdir.patch
@@ -0,0 +1,38 @@
+Copied from Nix:
+
+https://github.com/NixOS/nixpkgs/blob/0336bd874374a9e33947ace8a289e5c785e44a19/pkgs/development/libraries/libical/respect-env-tzdir.patch
+
+This functionality will be available in libical 3.1.0, whenever that is
+released:
+
+https://github.com/libical/libical/issues/350
+
+--- a/src/libical/icaltz-util.c
++++ b/src/libical/icaltz-util.c
+@@ -94,9 +94,9 @@
+ static const char *zdir = NULL;
+
+ static const char *search_paths[] = {
++ "/etc/zoneinfo",
+ "/usr/share/zoneinfo",
+ "/usr/lib/zoneinfo",
+- "/etc/zoneinfo",
+ "/usr/share/lib/zoneinfo"
+ };
+
+@@ -178,6 +178,15 @@
+ const char *fname = ZONES_TAB_SYSTEM_FILENAME;
+ size_t i, num_search_paths;
+
++ const char *env_tzdir = getenv ("TZDIR");
++ if (env_tzdir) {
++ sprintf (file_path, "%s/%s", env_tzdir, fname);
++ if (!access (file_path, F_OK|R_OK)) {
++ zdir = env_tzdir;
++ return;
++ }
++ }
++
+ num_search_paths = sizeof(search_paths) / sizeof(search_paths[0]);
+ for (i = 0; i < num_search_paths; i++) {
+ snprintf(file_path, MAXPATHLEN, "%s/%s", search_paths[i], fname);
--
2.39.2
L
L
Leo Famulari wrote on 21 Apr 2023 23:01
[PATCH 2/3] gnu: Go: Do not depend on tzdata.
(address . 63006@debbugs.gnu.org)
4e15e1076c425b3e8e924edfc8adf61cba0df036.1682110862.git.leo@famulari.name
* gnu/packages/golang.scm (go-1.4)[inputs]: Remove tzdata.
[arguments]: Adjust accordingly.
(go-1.14, go-1.16, go-1.17)[arguments]: Adjust accordingly.
---
gnu/packages/golang.scm | 27 +++++----------------------
1 file changed, 5 insertions(+), 22 deletions(-)

Toggle diff (96 lines)
diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index 24dae34e20..73c70f55fb 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -139,8 +139,6 @@ (define-public go-1.4
(ld (string-append (assoc-ref inputs "libc") "/lib"))
(loader (car (find-files ld "^ld-linux.+")))
(net-base (assoc-ref inputs "net-base"))
- (tzdata-path
- (string-append (assoc-ref inputs "tzdata") "/share/zoneinfo"))
(output (assoc-ref outputs "out")))
;; Removing net/ tests, which fail when attempting to access
@@ -195,8 +193,6 @@ (define-public go-1.4
(substitute* "net/lookup_unix.go"
(("/etc/protocols") (string-append net-base "/etc/protocols")))
- (substitute* "time/zoneinfo_unix.go"
- (("/usr/share/zoneinfo/") tzdata-path))
(substitute* (find-files "cmd" "asm.c")
(("/lib/ld-linux.*\\.so\\.[0-9]") loader))
#t)))
@@ -238,8 +234,7 @@ (define-public go-1.4
(copy-recursively "../" output)
#t))))))
(inputs
- `(("tzdata" ,tzdata)
- ("pcre" ,pcre)
+ `(("pcre" ,pcre)
("gcc:lib" ,(canonical-package gcc) "lib")))
(native-inputs
(list pkg-config which net-base perl))
@@ -285,8 +280,6 @@ (define-public go-1.14
(loader (car (append (find-files ld "^ld-linux.+")
(find-files ld "^ld(64)?\\.so.+"))))
(net-base (assoc-ref inputs "net-base"))
- (tzdata-path
- (string-append (assoc-ref inputs "tzdata") "/share/zoneinfo"))
(output (assoc-ref outputs "out")))
;; Having the patch in the 'patches' field of <origin> breaks
@@ -400,8 +393,6 @@ (define-public go-1.14
(("/etc/protocols") (string-append net-base "/etc/protocols")))
(substitute* "net/port_unix.go"
(("/etc/services") (string-append net-base "/etc/services")))
- (substitute* "time/zoneinfo_unix.go"
- (("/usr/share/zoneinfo/") tzdata-path))
(substitute* (find-files "cmd" "\\.go")
(("/lib(64)?/ld-linux.*\\.so\\.[0-9]") loader))
#t)))
@@ -497,9 +488,7 @@ (define-public go-1.16
(replace 'prebuild
(lambda* (#:key inputs outputs #:allow-other-keys)
(let* ((gcclib (string-append (assoc-ref inputs "gcc:lib") "/lib"))
- (net-base (assoc-ref inputs "net-base"))
- (tzdata-path
- (string-append (assoc-ref inputs "tzdata") "/share/zoneinfo")))
+ (net-base (assoc-ref inputs "net-base")))
;; Having the patch in the 'patches' field of <origin> breaks
;; the 'TestServeContent' test due to the fact that
@@ -591,9 +580,7 @@ (define-public go-1.16
(substitute* "net/lookup_unix.go"
(("/etc/protocols") (string-append net-base "/etc/protocols")))
(substitute* "net/port_unix.go"
- (("/etc/services") (string-append net-base "/etc/services")))
- (substitute* "time/zoneinfo_unix.go"
- (("/usr/share/zoneinfo/") tzdata-path)))))
+ (("/etc/services") (string-append net-base "/etc/services"))))))
(replace 'build
(lambda* (#:key inputs outputs (parallel-build? #t)
#:allow-other-keys)
@@ -665,9 +652,7 @@ (define-public go-1.17
(setenv "GOCACHE" "/tmp/go-cache"))))
(add-after 'unpack 'patch-source
(lambda* (#:key inputs outputs #:allow-other-keys)
- (let* ((net-base (assoc-ref inputs "net-base"))
- (tzdata-path (string-append (assoc-ref inputs "tzdata")
- "/share/zoneinfo")))
+ (let* ((net-base (assoc-ref inputs "net-base")))
;; XXX: Remove when #49729 is merged?
(for-each make-file-writable (find-files "src"))
@@ -696,9 +681,7 @@ (define-public go-1.17
(string-append net-base "/etc/protocols")))
(substitute* "src/net/port_unix.go"
(("/etc/services")
- (string-append net-base "/etc/services")))
- (substitute* "src/time/zoneinfo_unix.go"
- (("/usr/share/zoneinfo/") tzdata-path)))))
+ (string-append net-base "/etc/services"))))))
;; Keep this synchronized with the package inputs.
;; Also keep syncthonized with later versions of go.
,@(if (or (target-arm?) (target-ppc64le?))
--
2.39.2
L
L
Leo Famulari wrote on 21 Apr 2023 23:01
[PATCH 3/3] gnu: tzdata: Update to 2023c.
(address . 63006@debbugs.gnu.org)
b374e42bf86a6b6bf7f9524073817105d32f049b.1682110862.git.leo@famulari.name
* gnu/packages/base.scm (tzdata): Update to 2023c.
(tzdata-for-tests): Pin at version 2022a and hide.
* gnu/packages/time.scm (python-pytz): Update to 2023.3.
---
gnu/packages/base.scm | 19 +++++++++++++++----
gnu/packages/time.scm | 4 ++--
2 files changed, 17 insertions(+), 6 deletions(-)

Toggle diff (74 lines)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index fa5f1ef32b..2aef52ddfe 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -1358,7 +1358,7 @@ (define-public tzdata
(name "tzdata")
;; This package should be kept in sync with python-pytz in (gnu packages
;; time).
- (version "2022a")
+ (version "2023c")
(source (origin
(method url-fetch)
(uri (string-append
@@ -1366,7 +1366,7 @@ (define-public tzdata
version ".tar.gz"))
(sha256
(base32
- "0r0nhwpk9nyxj5kkvjy58nr5d85568m04dcb69c4y3zmykczyzzg"))))
+ "0p4nvp5bdxxdqh269nvvcfrpycbbfwm31al5whwbpsaa3dfhnl9z"))))
(build-system gnu-build-system)
(arguments
(list #:tests? #f
@@ -1434,7 +1434,7 @@ (define-public tzdata
version ".tar.gz"))
(sha256
(base32
- "1iysv8fdkm79k8wh8jizmjmq075q4qjhk090vxjy57my6dz5wmzq")))))
+ "1rqln88ki0jagi372nqyn7bs03rf2l33081sy2835mwsn4mpzla6")))))
(home-page "https://www.iana.org/time-zones")
(synopsis "Database of current and historical time zones")
(description "The Time Zone Database (often called tz or zoneinfo)
@@ -1452,7 +1452,18 @@ (define-public tzdata
;;;
;;; Please make this a hidden-package if it is different from the primary tzdata
;;; package.
-(define-public tzdata-for-tests tzdata)
+(define-public tzdata-for-tests ; TODO rename to tzdata/pinned
+ (hidden-package
+ (package (inherit tzdata)
+ (version "2022a")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append
+ "https://data.iana.org/time-zones/releases/tzdata"
+ version ".tar.gz"))
+ (sha256
+ (base32
+ "0r0nhwpk9nyxj5kkvjy58nr5d85568m04dcb69c4y3zmykczyzzg")))))))
(define-public libiconv
(package
diff --git a/gnu/packages/time.scm b/gnu/packages/time.scm
index c0bcb848b3..9416199015 100644
--- a/gnu/packages/time.scm
+++ b/gnu/packages/time.scm
@@ -124,14 +124,14 @@ (define-public python-pytz
(package
(name "python-pytz")
;; This package should be kept in sync with tzdata in (gnu packages base).
- (version "2022.1")
+ (version "2023.3")
(source
(origin
(method url-fetch)
(uri (pypi-uri "pytz" version))
(sha256
(base32
- "19ya5sh7if819flgmszz585glailhi7rr8frng03n5m8wqphwxhy"))))
+ "1265s3my0r1d841w3sh07ngvaanqhw1nvviqafsiy6c9n6fy530x"))))
(build-system python-build-system)
(home-page "http://pythonhosted.org/pytz")
(synopsis "Python timezone library")
--
2.39.2
L
L
Ludovic Courtès wrote on 30 Apr 2023 23:00
Re: bug#63006: Time zone database improvements
(name . Leo Famulari)(address . leo@famulari.name)(address . 63006@debbugs.gnu.org)
87ttwxozjs.fsf@gnu.org
Hello!

Leo Famulari <leo@famulari.name> skribis:

Toggle quote (10 lines)
> Currently, almost the entire distro is rebuilt when we need to update
> the time zone database, tzdata.
>
> We need to do this a few times a year, and it doesn't make any sense for
> packages to keep a static reference to this database which will be out
> of date in a few months.
>
> Instead, they should look up the time zones at run-time, which most
> programs that actually use time zones do.

Most programs behave fine though, right? In particular, glibc honors
$TZDIR and we set TZDIR to the latest tzdata in (gnu system).

Toggle quote (7 lines)
> These patches start improving the situation.
>
> Libical has committed a fix for this almost 5 years ago, but still
> hasn't released it, so we cherry-pick Nix's patch:
>
> https://github.com/libical/libical/issues/350

LGTM!

Toggle quote (4 lines)
> Go wants to embed a copy of the database in case one cannot be found. I
> don't see why having an incorrect time zone is better than no time
> zones, so here's a patch to remove the embedded copy.

Sounds good, but do we need to build it in a branch first?

Toggle quote (7 lines)
> We have more work to do to remove or replace python-pytz, which
> unfortunately contains its own embedded copy of the database and a few
> thousand dependents. Luckily, it seems that Python 3.9 has made pytz
> obsolete:
>
> https://fedoraproject.org/wiki/Changes/DeprecatePytz#Deprecate_pytz

OK.

Thanks!

Ludo’.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 63006@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 63006
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch