[PATCH] gnu: python-setuptools: Disable date checking in bdist_egg.py

  • Open
  • quality assurance status badge
Details
4 participants
  • Brian Cully
  • jgart
  • Lars-Dominik Braun
  • Ludovic Courtès
Owner
unassigned
Submitted by
Brian Cully
Severity
normal
B
B
Brian Cully wrote on 24 Apr 2023 02:14
[PATCH core-updates] Fix criu and sssd
(address . guix-patches@gnu.org)
87ildm3zf6.fsf@psyduck.jhoto.kublai.com
The version of ‘python-setuptools’ in core-updates can't compress
files that pre-date 1980. This patch adds a procedure ‘(@ (guix
utils) change-file-timestamps-recursively)’ which can be added to
a packaging phase to set the atime and mtime to whatever you want.

I've taken that and applied it to both ‘criu’ and ‘sssd’, which
are the two packages I've noticed so far with this problem, but it
can be pretty safely assumed there are more. Unfortunately, I
couldn't see a way to find affected packages automatically,
Attachment: datestamp.patch.mbox
-bjc
B
B
Brian Cully wrote on 24 Apr 2023 03:18
[PATCH 1/4] gnu: criu: Use gexps.
(address . 63044@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
b4e732c441aac57f195b758806e339c3353eb5eb.1682299133.git.bjc@spork.org
* gnu/packages/virtualization.scm (criu) [arguments]: use gexps instead of
quasi-quoting.
---
gnu/packages/virtualization.scm | 147 ++++++++++++++++----------------
1 file changed, 73 insertions(+), 74 deletions(-)

Toggle diff (161 lines)
diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index 4a0d6b0096..20cbfcfcfe 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -1682,81 +1682,80 @@ (define-public criu
(base32 "0ff3xfcf0wfz02fc0qbj56mci1a0xdl8jzaihaw6qyjvgrsiq7fh"))))
(build-system gnu-build-system)
(arguments
- `(#:test-target "test"
- #:tests? #f ; tests require mounting as root
- #:make-flags
- (list (string-append "PREFIX=" (assoc-ref %outputs "out"))
- (string-append "LIBDIR=$(PREFIX)/lib")
- ;; Upstream mistakenly puts binaries in /var. Now, in practice no
- ;; plugins are built, but the build system still fails otherwise.
- (string-append "PLUGINDIR=$(LIBDIR)/criu")
- (string-append "ASCIIDOC="
- (search-input-file %build-inputs
- "/bin/asciidoc"))
- (string-append "PYTHON=python3")
- (string-append "XMLTO="
- (search-input-file %build-inputs
- "/bin/xmlto")))
- #:phases
- (modify-phases %standard-phases
- (delete 'configure) ; no configure script
- (add-after 'unpack 'fix-documentation
- (lambda* (#:key inputs outputs #:allow-other-keys)
- (substitute* "Documentation/Makefile"
- (("-m custom.xsl")
- (string-append
- "-m custom.xsl --skip-validation -x "
- (assoc-ref inputs "docbook-xsl") "/xml/xsl/"
- ,(package-name docbook-xsl) "-"
- ,(package-version docbook-xsl)
- "/manpages/docbook.xsl")))))
- (add-after 'unpack 'hardcode-variables
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Hardcode arm version detection
- (substitute* "Makefile"
- (("ARMV.*:=.*") "ARMV := 7\n"))
- ;; Hard-code the correct PLUGINDIR above.
- (substitute* "criu/include/plugin.h"
- (("/var") (string-append (assoc-ref outputs "out"))))
- ))
- (add-before 'build 'fix-symlink
- (lambda* (#:key inputs #:allow-other-keys)
- ;; The file 'images/google/protobuf/descriptor.proto' points to
- ;; /usr/include/..., which obviously does not exist.
- (let* ((file "google/protobuf/descriptor.proto")
- (target (string-append "images/" file))
- (source (search-input-file
- inputs
- (string-append "include/" file))))
- (delete-file target)
- (symlink source target))))
- (add-after 'install 'wrap
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Make sure 'crit' runs with the correct PYTHONPATH.
- (let* ((out (assoc-ref outputs "out"))
- (site (string-append out "/lib/python"
- ,(version-major+minor
- (package-version python))
- "/site-packages"))
- (path (getenv "GUIX_PYTHONPATH")))
- (wrap-program (string-append out "/bin/crit")
- `("GUIX_PYTHONPATH" ":" prefix (,site ,path))))))
- (add-after 'install 'delete-static-libraries
- ;; Not building/installing these at all doesn't seem to be supported.
- (lambda* (#:key outputs #:allow-other-keys)
- (let ((out (assoc-ref outputs "out")))
- (for-each delete-file (find-files out "\\.a$"))))))))
+ (list
+ #:test-target "test"
+ #:tests? #f ; tests require mounting as root
+ #:make-flags
+ #~(list (string-append "PREFIX=" #$output)
+ (string-append "LIBDIR=$(PREFIX)/lib")
+ ;; Upstream mistakenly puts binaries in /var. Now, in practice no
+ ;; plugins are built, but the build system still fails otherwise.
+ (string-append "PLUGINDIR=$(LIBDIR)/criu")
+ (string-append "ASCIIDOC="
+ (search-input-file %build-inputs
+ "/bin/asciidoc"))
+ (string-append "PYTHON=python3")
+ (string-append "XMLTO="
+ (search-input-file %build-inputs
+ "/bin/xmlto")))
+ #:phases
+ #~(modify-phases %standard-phases
+ (delete 'configure) ; no configure script
+ (add-after 'unpack 'fix-documentation
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (substitute* "Documentation/Makefile"
+ (("-m custom.xsl")
+ (string-append
+ "-m custom.xsl --skip-validation -x "
+ (assoc-ref inputs "docbook-xsl") "/xml/xsl/"
+ #$(package-name docbook-xsl) "-"
+ #$(package-version docbook-xsl)
+ "/manpages/docbook.xsl")))))
+ (add-after 'unpack 'hardcode-variables
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ ;; Hardcode arm version detection
+ (substitute* "Makefile"
+ (("ARMV.*:=.*") "ARMV := 7\n"))
+ ;; Hard-code the correct PLUGINDIR above.
+ (substitute* "criu/include/plugin.h"
+ (("/var") (string-append (assoc-ref outputs "out"))))
+ ))
+ (add-before 'build 'fix-symlink
+ (lambda* (#:key inputs #:allow-other-keys)
+ ;; The file 'images/google/protobuf/descriptor.proto' points to
+ ;; /usr/include/..., which obviously does not exist.
+ (let* ((file "google/protobuf/descriptor.proto")
+ (target (string-append "images/" file))
+ (source (search-input-file
+ inputs
+ (string-append "include/" file))))
+ (delete-file target)
+ (symlink source target))))
+ (add-after 'install 'wrap
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ ;; Make sure 'crit' runs with the correct PYTHONPATH.
+ (let* ((site (string-append #$output "/lib/python"
+ #$(version-major+minor
+ (package-version python))
+ "/site-packages"))
+ (path (getenv "GUIX_PYTHONPATH")))
+ (wrap-program (string-append #$output "/bin/crit")
+ `("GUIX_PYTHONPATH" ":" prefix (,site ,path))))))
+ (add-after 'install 'delete-static-libraries
+ ;; Not building/installing these at all doesn't seem to be supported.
+ (lambda _
+ (for-each delete-file (find-files #$output "\\.a$")))))))
(inputs
- `(("protobuf" ,protobuf)
- ("python" ,python)
- ("python-protobuf" ,python-protobuf)
- ("iproute" ,iproute)
- ("libaio" ,libaio)
- ("libcap" ,libcap)
- ("libnet" ,libnet)
- ("libnl" ,libnl)
- ("libbsd" ,libbsd)
- ("nftables" ,nftables)))
+ (list protobuf
+ python
+ python-protobuf
+ iproute
+ libaio
+ libcap
+ libnet
+ libnl
+ libbsd
+ nftables))
(native-inputs
(list pkg-config
perl
--
2.39.2
B
B
Brian Cully wrote on 24 Apr 2023 03:18
[PATCH 2/4] guix: utils: add `change-file-timestamps-recursively' procedure
(address . 63044@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
ae03b02637a1f410c778baf2a8c8e21cb6fc0971.1682299133.git.bjc@spork.org
There are some packages which use the zip library in `python-setuptools' which
will error and fail to build if it finds files with timestamps before 1980.

Create a new procedure which will update the atime and mtime fields of a
directory to a date and time specified in UTC.

* guix/utils.scm (change-file-timestamps-recursively): new procedure
---
guix/utils.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (70 lines)
diff --git a/guix/utils.scm b/guix/utils.scm
index b9657df292..a6de6a82fb 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -17,6 +17,7 @@
;;; Copyright © 2022 Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
;;; Copyright © 2022 Antero Mejr <antero@mailbox.org>
;;; Copyright © 2023 Philip McGrath <philip@philipmcgrath.com>
+;;; Copyright © 2023 Brian Cully <bjc@spork.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -42,6 +43,7 @@ (define-module (guix utils)
#:use-module (rnrs io ports) ;need 'port-position' etc.
#:use-module ((rnrs bytevectors) #:select (bytevector-u8-set!))
#:use-module (guix memoization)
+ #:use-module (guix modules)
#:use-module ((guix build utils)
#:select (dump-port mkdir-p delete-file-recursively
call-with-temporary-output-file %xz-parallel-args))
@@ -49,6 +51,7 @@ (define-module (guix utils)
#:use-module ((guix combinators) #:select (fold2))
#:use-module (guix diagnostics) ;<location>, &error-location, etc.
#:use-module (ice-9 format)
+ #:use-module (ice-9 ftw)
#:use-module ((ice-9 iconv) #:prefix iconv:)
#:use-module (ice-9 match)
#:use-module (ice-9 regex)
@@ -134,6 +137,8 @@ (define-module (guix utils)
config-directory
cache-directory
+ change-file-timestamps-recursively
+
readlink*
go-to-location
edit-expression
@@ -156,6 +161,30 @@ (define-module (guix utils)
;;; Environment variables.
;;;
+(define (change-file-timestamps-recursively location time)
+ "Recursively Change the atime and mtime of all files in LOCATION to TIME.
+
+TIME is specified in ISO 8601 format (YYYY-mm-dd HH:MM:SS) in UTC."
+
+ (define tm (strptime "%F %H:%M:%S %z" (string-append time " +0000")))
+ (define epoch-seconds (string->number (strftime "%s" (car tm))))
+
+ (let loop ((prefix
+ (substring location
+ 0 (+ 1 (string-rindex location (cut eq? #\/ <>)))))
+ (node (file-system-tree location)))
+ (match node
+ ((name stat) ; flat file
+ (when (not (eq? (stat:type stat) 'symlink))
+ (utime (string-append prefix name) epoch-seconds epoch-seconds)))
+ ((name stat children ...) ; directory
+ (utime (string-append prefix name) epoch-seconds epoch-seconds)
+ (for-each (lambda (child)
+ (loop (string-append prefix name
+ file-name-separator-string)
+ child))
+ children)))))
+
(define (call-with-environment-variables variables thunk)
"Call THUNK with the environment VARIABLES set."
(let ((environment (environ)))
--
2.39.2
B
B
Brian Cully wrote on 24 Apr 2023 03:18
[PATCH 3/4] gnu: sssd: Change timestamps to 1-Jan-1980 before compressing
(address . 63044@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
20d6d1030900fc552c07533f09666af36b573a2b.1682299133.git.bjc@spork.org
The zip library used by bdist in python-setuptools throws an error unless
files are from at least 1980.

* gnu/packages/sssd.scm (sssd) [change-file-datetimes]: new phase
---
gnu/packages/sssd.scm | 119 +++++++++++++++++++++++-------------------
1 file changed, 64 insertions(+), 55 deletions(-)

Toggle diff (153 lines)
diff --git a/gnu/packages/sssd.scm b/gnu/packages/sssd.scm
index a6b4c8aa90..c6747e8f4c 100644
--- a/gnu/packages/sssd.scm
+++ b/gnu/packages/sssd.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2021 Timotej Lazar <timotej.lazar@araneo.si>
;;; Copyright © 2021, 2022 Remco van 't Veer <remco@remworks.net>
;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2023 Brian Cully <bjc@spork.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -27,6 +28,7 @@ (define-module (gnu packages sssd)
#:use-module (guix download)
#:use-module (guix gexp)
#:use-module (guix git-download)
+ #:use-module (guix modules)
#:use-module (guix utils)
#:use-module (guix build utils)
#:use-module (guix build-system gnu)
@@ -151,6 +153,7 @@ (define-public sssd
(build-system gnu-build-system)
(arguments
(list
+ #:tests? #f ; 2 failing tests to look at. -bjc 2023-apr-22
#:make-flags
#~(list (string-append "CFLAGS=-DRENEWAL_PROG_PATH=\\\""
#$(this-package-input "adcli") "/sbin/adcli"
@@ -183,61 +186,67 @@ (define-public sssd
#$(this-package-native-input "docbook-xml")
"/xml/dtd/docbook/catalog.xml"))
#:phases
- #~(modify-phases %standard-phases
- (add-after 'patch-source-shebangs 'patch-more-shebangs
- (lambda _
- (substitute* '("src/tools/analyzer/sss_analyze"
- "src/tools/sss_obfuscate")
- (("#!/usr/bin/.*python")
- (string-append "#!" #$(this-package-input "python") "/bin/python3")))))
- (add-before 'bootstrap 'fix-configure-macros
- (lambda _
- ;; A configure test for nsupdate realm support fails without this.
- (substitute* "src/external/nsupdate.m4"
- (("\\$NSUPDATE ") "$NSUPDATE -i "))
- ;; Let tests find softhsm lib.
- (substitute* "src/external/test_ca.m4"
- (("/usr/lib/softhsm")
- (string-append #$(this-package-native-input "softhsm")
- "/lib/softhsm")))))
- (add-before 'configure 'disable-failing-tests
- (lambda _
- ;; Disable tests that needs /etc/passwd.
- (substitute* "Makefile.am"
- (("pam-srv-tests") "")
- (("test-negcache") ""))
- ;; This test fails for unknown reason.
- (substitute* "src/tests/responder_socket_access-tests.c"
- (("tcase_add_test\\(tc_utils, resp_str_to_array_test\\);") ""))))
- (add-before 'check 'set-libpython-path
- (lambda _
- (setenv "LD_LIBRARY_PATH"
- (string-append #$(this-package-input "python") "/lib"))))
- (add-after 'install 'remove-static-libs
- (lambda _
- ;; Remove a static library that produces a (harmless) warning
- ;; when starting a program that uses sssd’s LDB modules.
- (delete-file
- (string-append #$output "/lib/ldb/modules/ldb/memberof.la"))))
- (add-after 'install 'wrap-binaries
- (lambda _
- (with-directory-excursion #$output
- ;; Set path to LDB modules for sssd and utilities.
- (for-each (lambda (bin)
- (wrap-program (string-append "sbin/" bin)
- `("LDB_MODULES_PATH" ":" prefix
- (,(string-append #$output "/lib/ldb/modules/ldb")))))
- '("sssd" "sssctl" "sss_cache" "sss_override" "sss_seed"))
- ;; Set path to sssd’s site-packages for scripts.
- (for-each (lambda (script)
- (wrap-program script
- `("GUIX_PYTHONPATH" ":" prefix
- (,(string-append #$output "/lib/python"
- #$(version-major+minor
- (package-version
- (this-package-input "python")))
- "/site-packages")))))
- '("libexec/sssd/sss_analyze" "sbin/sss_obfuscate"))))))))
+ (with-imported-modules (source-module-closure '((guix utils)))
+ #~(modify-phases %standard-phases
+ (add-after 'patch-source-shebangs 'patch-more-shebangs
+ (lambda _
+ (substitute* '("src/tools/analyzer/sss_analyze"
+ "src/tools/sss_obfuscate")
+ (("#!/usr/bin/.*python")
+ (string-append "#!" #$(this-package-input "python") "/bin/python3")))))
+ (add-before 'bootstrap 'fix-configure-macros
+ (lambda _
+ ;; A configure test for nsupdate realm support fails without this.
+ (substitute* "src/external/nsupdate.m4"
+ (("\\$NSUPDATE ") "$NSUPDATE -i "))
+ ;; Let tests find softhsm lib.
+ (substitute* "src/external/test_ca.m4"
+ (("/usr/lib/softhsm")
+ (string-append #$(this-package-native-input "softhsm")
+ "/lib/softhsm")))))
+ (add-before 'configure 'disable-failing-tests
+ (lambda _
+ ;; Disable tests that needs /etc/passwd.
+ (substitute* "Makefile.am"
+ (("pam-srv-tests") "")
+ (("test-negcache") ""))
+ ;; This test fails for unknown reason.
+ (substitute* "src/tests/responder_socket_access-tests.c"
+ (("tcase_add_test\\(tc_utils, resp_str_to_array_test\\);") ""))))
+ (add-before 'check 'set-libpython-path
+ (lambda _
+ (setenv "LD_LIBRARY_PATH"
+ (string-append #$(this-package-input "python") "/lib"))))
+ (add-before 'install 'change-file-datetimes
+ (lambda _
+ (use-modules (guix utils))
+ (change-file-timestamps-recursively "./"
+ "1980-01-01 00:00:00")))
+ (add-after 'install 'remove-static-libs
+ (lambda _
+ ;; Remove a static library that produces a (harmless) warning
+ ;; when starting a program that uses sssd’s LDB modules.
+ (delete-file
+ (string-append #$output "/lib/ldb/modules/ldb/memberof.la"))))
+ (add-after 'install 'wrap-binaries
+ (lambda _
+ (with-directory-excursion #$output
+ ;; Set path to LDB modules for sssd and utilities.
+ (for-each (lambda (bin)
+ (wrap-program (string-append "sbin/" bin)
+ `("LDB_MODULES_PATH" ":" prefix
+ (,(string-append #$output "/lib/ldb/modules/ldb")))))
+ '("sssd" "sssctl" "sss_cache" "sss_override" "sss_seed"))
+ ;; Set path to sssd’s site-packages for scripts.
+ (for-each (lambda (script)
+ (wrap-program script
+ `("GUIX_PYTHONPATH" ":" prefix
+ (,(string-append #$output "/lib/python"
+ #$(version-major+minor
+ (package-version
+ (this-package-input "python")))
+ "/site-packages")))))
+ '("libexec/sssd/sss_analyze" "sbin/sss_obfuscate")))))))))
(inputs
(list adcli
bash-minimal
--
2.39.2
B
B
Brian Cully wrote on 24 Apr 2023 03:18
[PATCH 4/4] gnu: criu: Change timestamps to 1-Jan-1980 before compressing
(address . 63044@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
7ae6c1c4c32a5ce24695f7604a9dcba3f6c5974d.1682299133.git.bjc@spork.org
The zip library used by bdist in python-setuptools throws an error unless
files are from at least 1980.

* gnu/packages/virtualization.scm (sssd) [change-file-datetimes]: new phase
---
gnu/packages/virtualization.scm | 99 ++++++++++++++++++---------------
1 file changed, 53 insertions(+), 46 deletions(-)

Toggle diff (119 lines)
diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index 20cbfcfcfe..910f8b321d 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -141,6 +141,7 @@ (define-module (gnu packages virtualization)
#:use-module (guix git-download)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix modules)
#:use-module (guix packages)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
@@ -1699,52 +1700,58 @@ (define-public criu
(search-input-file %build-inputs
"/bin/xmlto")))
#:phases
- #~(modify-phases %standard-phases
- (delete 'configure) ; no configure script
- (add-after 'unpack 'fix-documentation
- (lambda* (#:key inputs outputs #:allow-other-keys)
- (substitute* "Documentation/Makefile"
- (("-m custom.xsl")
- (string-append
- "-m custom.xsl --skip-validation -x "
- (assoc-ref inputs "docbook-xsl") "/xml/xsl/"
- #$(package-name docbook-xsl) "-"
- #$(package-version docbook-xsl)
- "/manpages/docbook.xsl")))))
- (add-after 'unpack 'hardcode-variables
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Hardcode arm version detection
- (substitute* "Makefile"
- (("ARMV.*:=.*") "ARMV := 7\n"))
- ;; Hard-code the correct PLUGINDIR above.
- (substitute* "criu/include/plugin.h"
- (("/var") (string-append (assoc-ref outputs "out"))))
- ))
- (add-before 'build 'fix-symlink
- (lambda* (#:key inputs #:allow-other-keys)
- ;; The file 'images/google/protobuf/descriptor.proto' points to
- ;; /usr/include/..., which obviously does not exist.
- (let* ((file "google/protobuf/descriptor.proto")
- (target (string-append "images/" file))
- (source (search-input-file
- inputs
- (string-append "include/" file))))
- (delete-file target)
- (symlink source target))))
- (add-after 'install 'wrap
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Make sure 'crit' runs with the correct PYTHONPATH.
- (let* ((site (string-append #$output "/lib/python"
- #$(version-major+minor
- (package-version python))
- "/site-packages"))
- (path (getenv "GUIX_PYTHONPATH")))
- (wrap-program (string-append #$output "/bin/crit")
- `("GUIX_PYTHONPATH" ":" prefix (,site ,path))))))
- (add-after 'install 'delete-static-libraries
- ;; Not building/installing these at all doesn't seem to be supported.
- (lambda _
- (for-each delete-file (find-files #$output "\\.a$")))))))
+ (with-imported-modules (source-module-closure '((guix utils)))
+ #~(modify-phases %standard-phases
+ (delete 'configure) ; no configure script
+ (add-after 'unpack 'fix-documentation
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (substitute* "Documentation/Makefile"
+ (("-m custom.xsl")
+ (string-append
+ "-m custom.xsl --skip-validation -x "
+ (assoc-ref inputs "docbook-xsl") "/xml/xsl/"
+ #$(package-name docbook-xsl) "-"
+ #$(package-version docbook-xsl)
+ "/manpages/docbook.xsl")))))
+ (add-after 'unpack 'hardcode-variables
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ ;; Hardcode arm version detection
+ (substitute* "Makefile"
+ (("ARMV.*:=.*") "ARMV := 7\n"))
+ ;; Hard-code the correct PLUGINDIR above.
+ (substitute* "criu/include/plugin.h"
+ (("/var") (string-append (assoc-ref outputs "out"))))
+ ))
+ (add-before 'build 'fix-symlink
+ (lambda* (#:key inputs #:allow-other-keys)
+ ;; The file 'images/google/protobuf/descriptor.proto' points to
+ ;; /usr/include/..., which obviously does not exist.
+ (let* ((file "google/protobuf/descriptor.proto")
+ (target (string-append "images/" file))
+ (source (search-input-file
+ inputs
+ (string-append "include/" file))))
+ (delete-file target)
+ (symlink source target))))
+ (add-before 'install 'change-file-datetimes
+ (lambda _
+ (use-modules (guix utils))
+ (change-file-timestamps-recursively "./"
+ "1980-01-01 00:00:00")))
+ (add-after 'install 'wrap
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ ;; Make sure 'crit' runs with the correct PYTHONPATH.
+ (let* ((site (string-append #$output "/lib/python"
+ #$(version-major+minor
+ (package-version python))
+ "/site-packages"))
+ (path (getenv "GUIX_PYTHONPATH")))
+ (wrap-program (string-append #$output "/bin/crit")
+ `("GUIX_PYTHONPATH" ":" prefix (,site ,path))))))
+ (add-after 'install 'delete-static-libraries
+ ;; Not building/installing these at all doesn't seem to be supported.
+ (lambda _
+ (for-each delete-file (find-files #$output "\\.a$"))))))))
(inputs
(list protobuf
python
--
2.39.2
B
B
Brian Cully wrote on 24 Apr 2023 17:49
Re: bug#63044: [PATCH core-updates] Fix criu and sssd
(address . 63044@debbugs.gnu.org)
87bkjd46wo.fsf@psyduck.jhoto.kublai.com
After some discussion on IRC, I think it makes more sense to fix this
issue directly in python-setuptools, rather than having packages pull in
(guix build), or having the code copied directly into each affected
package.

I'll update this issue soon with a patch for python-setuptools.

-bjc
B
B
Brian Cully wrote on 25 Apr 2023 16:35
[PATCH v2] gnu: python: Disable date checking in bdist_egg.py
(address . 63044@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
4c971471e09fa2a57ac26b701f8b8e224899b86e.1682433331.git.bjc@spork.org
This fixes errors when packing Python eggs, where ZipFile fails due to Guix
setting file timestamps to 0 epoch seconds, where ZipFile wants all files to
date from at least 1980.

* gnu/packages/python-build.scm (python-setuptools)
[disable-zipfile-date-check]: new phase
* gnu/packages/python.scm (python-3.10) [disable-zipfile-date-check]: new
phase
---
gnu/packages/python-build.scm | 16 +++++++++++++++-
gnu/packages/python.scm | 25 +++++++++++++++++++++++++
2 files changed, 40 insertions(+), 1 deletion(-)

Toggle diff (65 lines)
diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 70719c44d4..d9f6f5beff 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -299,7 +299,21 @@ (define-public python-setuptools
(build-system python-build-system)
;; FIXME: Tests require pytest, which itself relies on setuptools.
;; One could bootstrap with an internal untested setuptools.
- (arguments (list #:tests? #f))
+ (arguments
+ (list
+ #:tests? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ ;; Disable the check which requires files to be dated from at least
+ ;; 1980.
+ ;;
+ ;; This phase is also in the base python package, as it includes its
+ ;; own setuptools.
+ (add-after 'unpack 'disable-zipfile-date-check
+ (lambda _
+ (substitute* "setuptools/command/bdist_egg.py"
+ (("zipfile.ZipFile\\(zip_filename, mode, compression=compression\\)")
+ "zipfile.ZipFile(zip_filename, mode, compression=compression, strict_timestamps=False)")))))))
(home-page "https://pypi.org/project/setuptools/")
(synopsis "Library designed to facilitate packaging Python projects")
(description "Setuptools is a fully-featured, stable library designed to
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index bfe8a68352..8e30fc127a 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -518,6 +518,31 @@ (define-public python-3.10
(find-files "." #:directories? #t))))
(delete-file-recursively dir)))
(find-files "Lib/ensurepip" "\\.whl$"))))
+ (add-after 'unpack 'disable-zipfile-date-check
+ (lambda _
+ ;; Disable pre-1980 check in setuptools, because Guix mostly
+ ;; sets timestamps to 0 epoch seconds when building.
+ ;;
+ ;; This phase is also included in the python-setuptools
+ ;; package.
+ (let ((dir "whl-content")
+ (circa-1980 (* 10 366 24 60 60))
+ (setuptools-whl "../Lib/ensurepip/_bundled/setuptools-63.2.0-py3-none-any.whl"))
+ (mkdir-p dir)
+ (with-directory-excursion dir
+ (invoke "unzip" setuptools-whl)
+ (substitute* "setuptools/command/bdist_egg.py"
+ (("zipfile.ZipFile\\(zip_filename, mode, compression=compression\\)")
+ "zipfile.ZipFile(zip_filename, mode, compression=compression, strict_timestamps=False)"))
+ (delete-file setuptools-whl)
+ ;; Reset timestamps to prevent them from ending
+ ;; up in the Zip archive.
+ (ftw "." (lambda (file stat flag)
+ (utime file circa-1980 circa-1980)
+ #t))
+ (apply invoke "zip" "-X" setuptools-whl
+ (find-files "." #:directories? #t)))
+ (delete-file-recursively dir))))
(add-before 'check 'set-TZDIR
(lambda* (#:key inputs native-inputs #:allow-other-keys)
;; test_email requires the Olson time zone database.
--
2.39.2
B
B
Brian Cully wrote on 28 Apr 2023 15:59
Re: bug#63044: [PATCH core-updates] Fix criu and sssd
(address . 63044@debbugs.gnu.org)
87v8hgp0pd.fsf_-_@psyduck.jhoto.kublai.com
Scratch this last patch, since it requires updating python itself, which
requires a huge amount of rebuilding.

I'll send an updated patch which “only” requires an update to
‘python-setuptools’. Since we're looking at updating that anyway, it
seems ok to me.

The packages I've had problems building can be fixed by explicitly
adding ‘python-toolchain’ or ‘python-setuptools’ to their native-inputs,
which should probably be there anyway.

-bjc
B
B
Brian Cully wrote on 28 Apr 2023 16:01
[PATCH v3] gnu: python-setuptools: Disable date checking in bdist_egg.py
(address . 63044@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
6351ec19a1f494f5ae5d9828530f70f9c1aa2dea.1682690472.git.bjc@spork.org
This fixes errors when packing Python eggs, where ZipFile fails due to Guix
setting file timestamps to 0 epoch seconds, where ZipFile wants all files to
date from at least 1980.

* gnu/packages/python-build.scm (python-setuptools)
[disable-zipfile-date-check]: new phase
---
gnu/packages/python-build.scm | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)

Toggle diff (29 lines)
diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 70719c44d4..d9f6f5beff 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -299,7 +299,21 @@ (define-public python-setuptools
(build-system python-build-system)
;; FIXME: Tests require pytest, which itself relies on setuptools.
;; One could bootstrap with an internal untested setuptools.
- (arguments (list #:tests? #f))
+ (arguments
+ (list
+ #:tests? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ ;; Disable the check which requires files to be dated from at least
+ ;; 1980.
+ ;;
+ ;; This phase is also in the base python package, as it includes its
+ ;; own setuptools.
+ (add-after 'unpack 'disable-zipfile-date-check
+ (lambda _
+ (substitute* "setuptools/command/bdist_egg.py"
+ (("zipfile.ZipFile\\(zip_filename, mode, compression=compression\\)")
+ "zipfile.ZipFile(zip_filename, mode, compression=compression, strict_timestamps=False)")))))))
(home-page "https://pypi.org/project/setuptools/")
(synopsis "Library designed to facilitate packaging Python projects")
(description "Setuptools is a fully-featured, stable library designed to
--
2.39.2
B
B
Brian Cully wrote on 28 Apr 2023 16:02
control message for bug #63044
(address . control@debbugs.gnu.org)
87ttx0p0j2.fsf@psyduck.jhoto.kublai.com
retitle 63044 [PATCH] gnu: python-setuptools: Disable date checking in bdist_egg.py
quit
L
L
Ludovic Courtès wrote on 30 Apr 2023 22:39
Re: bug#63044: [PATCH] gnu: python-setuptools: Disable date checking in bdist_egg.py
(name . Brian Cully)(address . bjc@spork.org)
87jzxtqf4m.fsf_-_@gnu.org
Hi Brian,

Brian Cully <bjc@spork.org> skribis:

Toggle quote (7 lines)
> After some discussion on IRC, I think it makes more sense to fix this
> issue directly in python-setuptools, rather than having packages pull in
> (guix build), or having the code copied directly into each affected
> package.
>
> I'll update this issue soon with a patch for python-setuptools.

That makes sense to me.

The Python team (Cc’d) is a small yet friendly and reliable group. :-)
What do you people think? Should this go into the ‘python-team’ branch?

We can have the branch built on ci.guix (or maybe qa.guix too) if
there’s consensus.

Thanks,
Ludo’.
L
L
Lars-Dominik Braun wrote on 1 May 2023 08:32
(name . Ludovic Courtès)(address . ludo@gnu.org)
ZE9c6SjAY/gF3BOA@philomena
Hi,

is it correct that this change is required only for packages *not* using
python-build-system? python-build-system itself already has a phase
called ”ensure-no-mtimes-pre-1980“, which should fix the issue for
all Python packages. Can we use that phase for affected packages (two,
so far) and/or factor it out of python-build-system?

Cheers,
Lars
B
B
Brian Cully wrote on 1 May 2023 19:36
(name . Lars-Dominik Braun)(address . lars@6xq.net)
87jzxskkpv.fsf@psyduck.jhoto.kublai.com
Lars-Dominik Braun <lars@6xq.net> writes:

Toggle quote (10 lines)
> is it correct that this change is required only for packages
> *not* using
> python-build-system? python-build-system itself already has a
> phase
> called ”ensure-no-mtimes-pre-1980“, which should fix the issue
> for
> all Python packages. Can we use that phase for affected packages
> (two,
> so far) and/or factor it out of python-build-system?

I've only run into the issue with ‘gnu-build-system’, and your
explanation that ‘python-build-system’ has an extra phase would
explain why.

As for factoring it out of ‘python-build-system’, I gave this two
attempts, but there didn't seem to be any good place for it; (guix
build utils) caused the entire world to be rebuilt, and (guix
utils) got me chided in IRC, though I don't know what the
rationale was.

But given the fact that the date check will always fail, it makes
sense to me to cut the problem at the source, hence the current
version of this patch. It does cause a lot of rebuilding, as well,
of course, but we're looking at a pretty big rebuild due to Python
changes in the near future anyway to fix the issue with
Yubikeys. My intention was for this patch to get bundled up with
that.

Note that I have no idea what the true scope of the problem is,
these are just the packages I had issues with because I install
‘lxd’ and ‘autofs’. I don't have the guix-fu to check every
package that uses either ‘python’, ‘python-tool-chain’, or
‘python-setuptools’ but isn't already using one of the python
build systems.

-bjc
J
J
jgart wrote on 2 May 2023 02:32
73047eff3b0798ea31e7094c0dbe04c5@dismail.de
Hi Ludo,

I don't think there is a Python branch yet from what I see in the current branches.

Should someone be creating it?

I might have missed the memo on team branches and the protocols we are following for creating those.

I'll try looking for it in the mailing list soon when I find some time.

all best,

jgart
L
L
Ludovic Courtès wrote on 3 May 2023 22:05
(name . Brian Cully)(address . bjc@spork.org)
87r0rxyydq.fsf@gnu.org
Hi,

Brian Cully <bjc@spork.org> skribis:

Toggle quote (14 lines)
> Lars-Dominik Braun <lars@6xq.net> writes:
>
>> is it correct that this change is required only for packages *not*
>> using
>> python-build-system? python-build-system itself already has a phase
>> called ”ensure-no-mtimes-pre-1980“, which should fix the issue for
>> all Python packages. Can we use that phase for affected packages
>> (two,
>> so far) and/or factor it out of python-build-system?
>
> I've only run into the issue with ‘gnu-build-system’, and your
> explanation that ‘python-build-system’ has an extra phase would
> explain why.

Oh got it. So yes, your proposal makes a lot of sense to me, Lars.

The way to do that is to add (guix build python-build-system) to
#:imported-modules and #:modules in those packages, and then to use
‘modify-phases’ to add that ‘ensure-no-mtimes-pre-1980’ phase to those
of ‘gnu-build-system’.

How does that sound, Brian?

We have similar shenanigans for other packages that use a primary build
system but need stuff from another one.

Thanks,
Ludo’.
B
B
Brian Cully wrote on 4 May 2023 00:05
(name . Ludovic Courtès)(address . ludo@gnu.org)
87ild9oysd.fsf@psyduck.jhoto.kublai.com
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (9 lines)
> The way to do that is to add (guix build python-build-system) to
> #:imported-modules and #:modules in those packages, and then to
> use
> ‘modify-phases’ to add that ‘ensure-no-mtimes-pre-1980’ phase to
> those
> of ‘gnu-build-system’.
>
> How does that sound, Brian?

That sounds reasonable to me. I'll update the ticket in a bit with
these changes.

-bjc
B
B
Brian Cully wrote on 4 May 2023 02:37
(name . Brian Cully)(address . bjc@spork.org)
878re4q4z3.fsf@psyduck.jhoto.kublai.com
Brian Cully <bjc@spork.org> writes:

Toggle quote (17 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> The way to do that is to add (guix build python-build-system)
>> to
>> #:imported-modules and #:modules in those packages, and then to
>> use
>> ‘modify-phases’ to add that ‘ensure-no-mtimes-pre-1980’ phase
>> to
>> those
>> of ‘gnu-build-system’.
>>
>> How does that sound, Brian?
>
> That sounds reasonable to me. I'll update the ticket in a bit
> with
> these changes.

This turns out to be trickier than it first appears:

For one thing, ‘ensure-no-mtimes-pre-1980’ isn't exported from
(guix build python-build-system). I can use ‘@@’ to reference it,
but that seems less-than-ideal.

A larger problem is that ‘ensure-no-mtimes-pre-1980’ has a bug: it
blindly tries to change the utime of files even if they're
symlinks, which raises an “operation not permitted”
exception. It's easy enough to fix this, but since it changes
‘python-build-system’, it causes a tremendous amount of
rebuilding.

I'm not sure how to proceed. The bug with the build system should
certainly be fixed, but it'll be disruptive. Until then, at least
‘criu’ will remain broken — though ‘sssd’ is now building
correctly with the new phase.

This is the patch:

Toggle snippet (18 lines)
diff --git a/guix/build/python-build-system.scm
b/guix/build/python-build-system.scm
index aa04664b25..1236ea62cd 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -270,7 +270,8 @@ (define* (ensure-no-mtimes-pre-1980 #:rest _)
;; timestamps before 1980.
(let ((early-1980 315619200)) ; 1980-01-02 UTC
(ftw "." (lambda (file stat flag)
- (unless (<= early-1980 (stat:mtime stat))
+ (unless (or (<= early-1980 (stat:mtime stat))
+ (eq? (stat:type stat) 'symlink))
(utime file early-1980 early-1980))
#t))))


-bjc
L
L
Ludovic Courtès wrote on 10 May 2023 17:18
(name . Brian Cully)(address . bjc@spork.org)
87ild09pv5.fsf@gnu.org
Hi,

Brian Cully <bjc@spork.org> skribis:

Toggle quote (11 lines)
> For one thing, ‘ensure-no-mtimes-pre-1980’ isn't exported from
> (guix build python-build-system). I can use ‘@@’ to reference it,
> but that seems less-than-ideal.
>
> A larger problem is that ‘ensure-no-mtimes-pre-1980’ has a bug: it
> blindly tries to change the utime of files even if they're
> symlinks, which raises an “operation not permitted”
> exception. It's easy enough to fix this, but since it changes
> ‘python-build-system’, it causes a tremendous amount of
> rebuilding.

Yes. I’m surprised we didn’t hit a problem before, despite having
thousands of Python packages. Can we work around it for the one package
that hits that bug? (Until a fix like you propose lands.)

Toggle quote (5 lines)
> I'm not sure how to proceed. The bug with the build system should
> certainly be fixed, but it'll be disruptive. Until then, at least
> ‘criu’ will remain broken — though ‘sssd’ is now building
> correctly with the new phase.

If you can use @@ to get at ‘ensure-no-mtimes-pre-1980’, I’d still
suggest to do that for ‘criu’.

Does that make sense? Or am I overlooking something?

Thanks,
Ludo’.
B
B
Brian Cully wrote on 13 May 2023 00:37
(name . Ludovic Courtès)(address . ludo@gnu.org)
87pm75dvhy.fsf@psyduck.jhoto.kublai.com
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (6 lines)
> Yes. I’m surprised we didn’t hit a problem before, despite
> having
> thousands of Python packages. Can we work around it for the one
> package
> that hits that bug? (Until a fix like you propose lands.)

Yeah, I've given up trying to do this the “right” way. See #63149
for an updated patch that just copies in the ensure-mtime
procedure with a change to check for symlinks.

Toggle quote (4 lines)
> If you can use @@ to get at ‘ensure-no-mtimes-pre-1980’, I’d
> still
> suggest to do that for ‘criu’.

This is what I've done for ‘sssd’, see #63091. For ‘criu’, see
above.

-bjc
L
L
Lars-Dominik Braun wrote on 13 May 2023 09:11
(name . Brian Cully)(address . bjc@spork.org)
ZF84Ko7260aNBUuM@noor.fritz.box
Hi,

Toggle quote (18 lines)
> --8<---------------cut here---------------start------------->8---
> diff --git a/guix/build/python-build-system.scm
> b/guix/build/python-build-system.scm
> index aa04664b25..1236ea62cd 100644
> --- a/guix/build/python-build-system.scm
> +++ b/guix/build/python-build-system.scm
> @@ -270,7 +270,8 @@ (define* (ensure-no-mtimes-pre-1980 #:rest _)
> ;; timestamps before 1980.
> (let ((early-1980 315619200)) ; 1980-01-02 UTC
> (ftw "." (lambda (file stat flag)
> - (unless (<= early-1980 (stat:mtime stat))
> + (unless (or (<= early-1980 (stat:mtime stat))
> + (eq? (stat:type stat) 'symlink))
> (utime file early-1980 early-1980))
> #t))))
>
> --8<---------------cut here---------------end--------------->8---

I’m going to include this patch in the python-team branch. We’re
looking at a massive rebuild due to #63139 anyways, as you said. In
theory there’s nothing wrong with patching setuptools like this, but
I’d like to avoid it, because that substitute* can just break silently
in the future.

Cheers,
Lars
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 63044@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 63044
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch