[PATCH 0/3] Reproducible `make dist' tarball: Avoid override stamp-N warnings.

  • Done
  • quality assurance status badge
Details
2 participants
  • Janneke Nieuwenhuizen
  • pelzflorian (Florian Pelz)
Owner
unassigned
Submitted by
Janneke Nieuwenhuizen
Severity
normal
J
J
Janneke Nieuwenhuizen wrote on 14 Apr 11:56 +0200
(address . guix-patches@gnu.org)
cover.1713087991.git.janneke@gnu.org
Hi!

Reproducibility is fine and all that but the forteen new make warnings

Toggle snippet (7 lines)
Makefile:7400: warning: overriding recipe for target 'doc/stamp-vti'
Makefile:5117: warning: ignoring old recipe for target 'doc/stamp-vti'
Makefile:7401: warning: overriding recipe for target 'doc/stamp-1'
Makefile:5182: warning: ignoring old recipe for target 'doc/stamp-1'
[..]

already started to annoy me so much that I found another solution: Overwrite
build-aux/mdate-sh with our own, new build-aux/mdate-from-git.scm script.

It comes with a bit of a hack: because of how Automake's stamp-N rules are
written, the new script needs knowledge about where to get the timestamp for
doc/guix.LANG.texi files.

WDYT?

Greetings,
Janneke

Janneke Nieuwenhuizen (3):
maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'.
Revert "maint: Generate 'doc/version-LANG.texi' reproducibly."
Revert "maint: Generate 'doc/version.texi' reproducibly."

bootstrap | 7 +++
build-aux/mdate-from-git.scm | 88 ++++++++++++++++++++++++++++++++++++
doc/local.mk | 32 -------------
3 files changed, 95 insertions(+), 32 deletions(-)
create mode 100755 build-aux/mdate-from-git.scm


base-commit: e5dda412c2e28fb65a549824f492895e72c33813
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 14 Apr 11:59 +0200
[PATCH 1/3] maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'.
(address . 70380@debbugs.gnu.org)
bb7d78763d44b6f6b7b38e72f90db8f7208496d1.1713087991.git.janneke@gnu.org
This replaces Automake's `build-aux/mdate-sh' with our own
`build-aux/mdate-from-git.scm' to use reproducible timestamps from Git
instead.

* build-aux/mdate-from-git.scm: New script.
* bootstrap: Use it to replace build-aux/mdate-sh.

Change-Id: I17d0a7de9ffea397129c0db1728f86e28a4e245f
---
bootstrap | 7 +++
build-aux/mdate-from-git.scm | 88 ++++++++++++++++++++++++++++++++++++
2 files changed, 95 insertions(+)
create mode 100755 build-aux/mdate-from-git.scm

Toggle diff (111 lines)
diff --git a/bootstrap b/bootstrap
index de024aeaa5..abd1ed353e 100755
--- a/bootstrap
+++ b/bootstrap
@@ -25,3 +25,10 @@ for lang in ${langs}; do
done
exec autoreconf -vfi
+
+# Replace Automake's build-aux/mdate-sh with build-aux/mdate-from-git, our
+# own, reproducible version.
+chmod +w build-aux/mdate-sh
+rm -f build-aux/mdate-sh
+cp build-aux/mdate-from-git build-aux/mdate-sh
+chmod -wx build-aux/mdate-sh
diff --git a/build-aux/mdate-from-git.scm b/build-aux/mdate-from-git.scm
new file mode 100755
index 0000000000..0abd580460
--- /dev/null
+++ b/build-aux/mdate-from-git.scm
@@ -0,0 +1,88 @@
+#! /bin/sh
+# -*-scheme-*-
+exec guile --no-auto-compile -L $srcdir -C $srcdir -e '(mdate-from-git)' -s "$0" "$@"
+!#
+
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Janneke Nieuwenhuizen <janneke@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; This program is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; This program is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+;;;; Commentary:
+;;;
+;;; Usage: mdate-from-git FILE
+;;;
+;;; This script is compatible with Automake's `mdate-sh' but uses the timestamp
+;;; from Git instead of from the file system. Also, it can be appended to
+;;; mdate-sh.
+
+;;; As a special exception for Guix, it caters for doc/guix.LANG.texi files that
+;;; are not stored in Git, by using po/doc/guix-manual.LANG.po for the Git
+;;; timestamp. Test doing something like:
+;;;
+;;; build-aux/mdate-from-git.scm doc/guix.de.texi
+;;;
+;;;; Code:
+
+(define-module (mdate-from-git)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 popen)
+ #:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
+ #:export (main))
+
+(define (pipe-command command)
+ (let* ((port (apply open-pipe* OPEN_READ command))
+ (output (read-string port)))
+ (close-port port)
+ output))
+
+(define (guix.LANG.texi->guix-manual.LANG.po file-name)
+ "Translated manuals doc/guix.LANG.texi are not tracked in Git and are
+generated from po/doc/guix-manual.LANG.po. For such an untraced .TEXI file,
+return its .PO counterpart."
+ (let ((m (string-match "doc/guix.([^.]+).texi" file-name)))
+ (if (not m) file-name
+ (let ((lang (match:substring m 1)))
+ (format #f "po/doc/guix-manual.~a.po" lang)))))
+
+
+;;;
+;;; Entry point.
+;;;
+(define (main args)
+ (setenv "LANG" "C")
+ (setenv "LANGUAGE" "C")
+ (setenv "LC_TIME" "C")
+ (setenv "TZ" "UTC0")
+ (match args
+ ((script file-name)
+ (let* ((command `("git" "ls-files" "--error-unmatch" "--" ,file-name))
+ (tracked? (zero? (with-error-to-port (%make-void-port "w")
+ (lambda _ (apply system* command)))))
+ (file-name (if tracked? file-name
+ (guix.LANG.texi->guix-manual.LANG.po file-name)))
+ (command `("git" "log" "--pretty=format:%ct" "-n1" "--" ,file-name))
+ (timestamp (pipe-command command))
+ (source-date-epoch (or (getenv "SOURCE_DATE_EPOCH") "1"))
+ (timestamp (if (string-null? timestamp) source-date-epoch
+ timestamp))
+ (time (gmtime (string->number timestamp)))
+ (d-y-m (strftime "%-d %B %Y" time)))
+ (display d-y-m)))
+ (_
+ (format (current-error-port) "Usage: mdate-from-git.scm FILE\n")
+ (exit 2))))
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 14 Apr 11:59 +0200
[PATCH 3/3] Revert "maint: Generate 'doc/version.texi' reproducibly."
(address . 70380@debbugs.gnu.org)
4a6c25fc18e70051a753c438fa1f5fa5abb50eb0.1713087991.git.janneke@gnu.org
Using `build-aux/mdate-from-git' makes this no longer necessary.

This reverts commit e73ea7bd64f64709c71f89dfb111cf3e8ada3771.

Change-Id: I29d1e36b13d255e5a65b7348e7ae4f2b2c24a518
---
doc/local.mk | 26 --------------------------
1 file changed, 26 deletions(-)

Toggle diff (39 lines)
diff --git a/doc/local.mk b/doc/local.mk
index 77d48902b6..1d94e3c758 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -260,32 +260,6 @@ endif
# Reproducible tarball
-# Generate 'version.texi' reproducibly using metadata from Git rather than
-# using metadata from the filesystem. This is expected to generate warnings:
-#
-# Makefile:7376: warning: overriding recipe for target 'doc/stamp-vti'
-# Makefile:5098: warning: ignoring old recipe for target 'doc/stamp-vti'
-$(srcdir)/doc/stamp-vti: $(srcdir)/doc/guix.texi $(top_srcdir)/configure
- $$(AM_V_GEN)set -e; \
- export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
- export TZ=UTC0; \
- timestamp=$$(git log --pretty=format:%ct -n1 -- $< 2>/dev/null \
- || echo $(SOURCE_DATE_EPOCH)) \
- dmy=$$(date --date="@$$timestamp" "+%-d %B %Y"); \
- my=$$(date --date="@$$timestamp" "+%B %Y"); \
- { echo "@set UPDATED $$dmy"; \
- echo "@set UPDATED-MONTH $$my"; \
- echo "@set EDITION $(VERSION)"; \
- echo "@set VERSION $(VERSION)"; } > $@-t; \
- mv $@-t $@; \
- cp $@ $(srcdir)/doc/version.texi
-
-i:=0
-$(eval $(call version.texi-from-git,vti,doc/guix.texi,))
-$(foreach lang, $(MANUAL_LANGUAGES), \
- $(eval i=$(shell echo $$(($(i)+1)))) \
- $(eval $(call version.texi-from-git,$(i),po/doc/guix-manual.$(lang).po,-$(lang))))
-
DIST_CONFIGURE_FLAGS = \
--localstatedir=/var \
--sysconfdir=/etc
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 14 Apr 11:59 +0200
[PATCH 2/3] Revert "maint: Generate 'doc/version-LANG.texi' reproducibly."
(address . 70380@debbugs.gnu.org)
eeeddd0065ab6c512c9911f61b355beed36d4531.1713087991.git.janneke@gnu.org
Using `build-aux/mdate-from-git' makes this no longer necessary.

This reverts commit 0e4ead187d83a958ca0deb54857c04967e84d68b.

Change-Id: I9177828f90fa7f7e256bc72fdff35a2fab355780
---
doc/local.mk | 40 +++++++++++++++++-----------------------
1 file changed, 17 insertions(+), 23 deletions(-)

Toggle diff (56 lines)
diff --git a/doc/local.mk b/doc/local.mk
index 130f40ece9..77d48902b6 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -260,32 +260,26 @@ endif
# Reproducible tarball
-# Define a rule to build `version[LANG].texi' reproducibly using metadata from
-# Git rather than using metadata from the filesystem.
-define version.texi-from-git
-$(srcdir)/doc/stamp-$(1): $(srcdir)/$(2) $(top_srcdir)/configure
- $$(AM_V_GEN)set -e; \
- export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
- export TZ=UTC0; \
- timestamp="$$$$(git log --pretty=format:%ct -n1 -- "$$<" \
- 2>/dev/null \
- || echo $$(SOURCE_DATE_EPOCH))" \
- dmy=$$$$(date --date="@$$$$timestamp" "+%-d %B %Y"); \
- my=$$$$(date --date="@$$$$timestamp" "+%B %Y"); \
- { echo "@set UPDATED $$$$dmy"; \
- echo "@set UPDATED-MONTH $$$$my"; \
- echo "@set EDITION $$$(VERSION)"; \
- echo "@set VERSION $$$(VERSION)"; } > "$$@-t"; \
- mv "$$@-t" "$$@"; \
- cp -p "$$@" "$$(srcdir)/doc/version$(3).texi"
-endef
-
-# Generate rules for stamp-vti and stamp-N that create version.texi and
-# version-LANG.texi to override the Autotools versions that use timestamps
-# embedded in the file-system. These are expected to generate warnings:
+# Generate 'version.texi' reproducibly using metadata from Git rather than
+# using metadata from the filesystem. This is expected to generate warnings:
#
# Makefile:7376: warning: overriding recipe for target 'doc/stamp-vti'
# Makefile:5098: warning: ignoring old recipe for target 'doc/stamp-vti'
+$(srcdir)/doc/stamp-vti: $(srcdir)/doc/guix.texi $(top_srcdir)/configure
+ $$(AM_V_GEN)set -e; \
+ export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
+ export TZ=UTC0; \
+ timestamp=$$(git log --pretty=format:%ct -n1 -- $< 2>/dev/null \
+ || echo $(SOURCE_DATE_EPOCH)) \
+ dmy=$$(date --date="@$$timestamp" "+%-d %B %Y"); \
+ my=$$(date --date="@$$timestamp" "+%B %Y"); \
+ { echo "@set UPDATED $$dmy"; \
+ echo "@set UPDATED-MONTH $$my"; \
+ echo "@set EDITION $(VERSION)"; \
+ echo "@set VERSION $(VERSION)"; } > $@-t; \
+ mv $@-t $@; \
+ cp $@ $(srcdir)/doc/version.texi
+
i:=0
$(eval $(call version.texi-from-git,vti,doc/guix.texi,))
$(foreach lang, $(MANUAL_LANGUAGES), \
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 15 Apr 16:27 +0200
[PATCH v2 0/3] Reproducible `make dist' tarball: Avoid override stamp-N warnings.
(address . 70380@debbugs.gnu.org)
cover.1713190364.git.janneke@gnu.org
V2 fixes embarrasing typos in 1/3 that should have prevented for this patch to
build a reproducible tarball at all. Most probably I only tested a previous
version where mdate-from-git was a plain shell script.

Janneke Nieuwenhuizen (3):
maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'.
Revert "maint: Generate 'doc/version-LANG.texi' reproducibly."
Revert "maint: Generate 'doc/version.texi' reproducibly."

bootstrap | 8 +++-
build-aux/mdate-from-git.scm | 86 ++++++++++++++++++++++++++++++++++++
doc/local.mk | 32 --------------
3 files changed, 93 insertions(+), 33 deletions(-)
create mode 100755 build-aux/mdate-from-git.scm


base-commit: e5dda412c2e28fb65a549824f492895e72c33813
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 15 Apr 16:27 +0200
[PATCH v2 1/3] maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'.
(address . 70380@debbugs.gnu.org)
5fab9ccaca83003bfcf79baa00dd23cc4179b3ae.1713190364.git.janneke@gnu.org
This replaces Automake's `build-aux/mdate-sh' with our own
`build-aux/mdate-from-git.scm' to use reproducible timestamps from Git
instead.

* build-aux/mdate-from-git.scm: New script.
* bootstrap: Use it to replace build-aux/mdate-sh.

Change-Id: I17d0a7de9ffea397129c0db1728f86e28a4e245f
---
bootstrap | 8 +++-
build-aux/mdate-from-git.scm | 86 ++++++++++++++++++++++++++++++++++++
2 files changed, 93 insertions(+), 1 deletion(-)
create mode 100755 build-aux/mdate-from-git.scm

Toggle diff (110 lines)
diff --git a/bootstrap b/bootstrap
index de024aeaa5..5bf83175e5 100755
--- a/bootstrap
+++ b/bootstrap
@@ -24,4 +24,10 @@ for lang in ${langs}; do
fi
done
-exec autoreconf -vfi
+autoreconf -vfi
+
+# Replace Automake's build-aux/mdate-sh with build-aux/mdate-from-git, our
+# own, reproducible version.
+chmod +w build-aux/mdate-sh
+rm -f build-aux/mdate-sh
+ln -s mdate-from-git.scm build-aux/mdate-sh
diff --git a/build-aux/mdate-from-git.scm b/build-aux/mdate-from-git.scm
new file mode 100755
index 0000000000..abe7e97f32
--- /dev/null
+++ b/build-aux/mdate-from-git.scm
@@ -0,0 +1,86 @@
+#! /bin/sh
+# -*-scheme-*-
+export LANG=C LANGUAGE=C LC_TIME=C
+export TZ=UTC0
+exec guile --no-auto-compile -L $srcdir -C $srcdir -e '(mdate-from-git)' -s "$0" "$@"
+!#
+
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Janneke Nieuwenhuizen <janneke@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; This program is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; This program is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+;;;; Commentary:
+;;;
+;;; Usage: mdate-from-git.scm FILE
+;;;
+;;; This script is compatible with Automake's `mdate-sh' but uses the timestamp
+;;; from Git instead of from the file system. Also, it can be appended to
+;;; mdate-sh.
+
+;;; As a special exception for Guix, it caters for doc/guix.LANG.texi files that
+;;; are not stored in Git, by using po/doc/guix-manual.LANG.po for the Git
+;;; timestamp. Test doing something like:
+;;;
+;;; build-aux/mdate-from-git.scm doc/guix.de.texi
+;;;
+;;;; Code:
+
+(define-module (mdate-from-git)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 popen)
+ #:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
+ #:export (main))
+
+(define (pipe-command command)
+ (let* ((port (apply open-pipe* OPEN_READ command))
+ (output (read-string port)))
+ (close-port port)
+ output))
+
+(define (guix.LANG.texi->guix-manual.LANG.po file-name)
+ "Translated manuals doc/guix.LANG.texi are not tracked in Git and are
+generated from po/doc/guix-manual.LANG.po. For such an untraced .TEXI file,
+return its .PO counterpart."
+ (let ((m (string-match "doc/guix.([^.]+).texi" file-name)))
+ (if (not m) file-name
+ (let ((lang (match:substring m 1)))
+ (format #f "po/doc/guix-manual.~a.po" lang)))))
+
+
+;;;
+;;; Entry point.
+;;;
+(define (main args)
+ (match args
+ ((script file-name)
+ (let* ((command `("git" "ls-files" "--error-unmatch" "--" ,file-name))
+ (tracked? (zero? (with-error-to-port (%make-void-port "w")
+ (lambda _ (apply system* command)))))
+ (file-name (if tracked? file-name
+ (guix.LANG.texi->guix-manual.LANG.po file-name)))
+ (command `("git" "log" "--pretty=format:%ct" "-n1" "--" ,file-name))
+ (timestamp (pipe-command command))
+ (source-date-epoch (or (getenv "SOURCE_DATE_EPOCH") "1"))
+ (timestamp (if (string-null? timestamp) source-date-epoch
+ timestamp))
+ (time (gmtime (string->number timestamp)))
+ (d-m-y (strftime "%-d %B %Y" time)))
+ (display d-m-y)))
+ (_
+ (format (current-error-port) "Usage: mdate-from-git.scm FILE\n")
+ (exit 2))))
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 15 Apr 16:27 +0200
[PATCH v2 3/3] Revert "maint: Generate 'doc/version.texi' reproducibly."
(address . 70380@debbugs.gnu.org)
da11f794cf3449860055cd45a4550c7922aaf76f.1713190364.git.janneke@gnu.org
Using `build-aux/mdate-from-git.scm' makes this no longer necessary.

This reverts commit e73ea7bd64f64709c71f89dfb111cf3e8ada3771.

Change-Id: I29d1e36b13d255e5a65b7348e7ae4f2b2c24a518
---
doc/local.mk | 26 --------------------------
1 file changed, 26 deletions(-)

Toggle diff (39 lines)
diff --git a/doc/local.mk b/doc/local.mk
index 77d48902b6..1d94e3c758 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -260,32 +260,6 @@ endif
# Reproducible tarball
-# Generate 'version.texi' reproducibly using metadata from Git rather than
-# using metadata from the filesystem. This is expected to generate warnings:
-#
-# Makefile:7376: warning: overriding recipe for target 'doc/stamp-vti'
-# Makefile:5098: warning: ignoring old recipe for target 'doc/stamp-vti'
-$(srcdir)/doc/stamp-vti: $(srcdir)/doc/guix.texi $(top_srcdir)/configure
- $$(AM_V_GEN)set -e; \
- export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
- export TZ=UTC0; \
- timestamp=$$(git log --pretty=format:%ct -n1 -- $< 2>/dev/null \
- || echo $(SOURCE_DATE_EPOCH)) \
- dmy=$$(date --date="@$$timestamp" "+%-d %B %Y"); \
- my=$$(date --date="@$$timestamp" "+%B %Y"); \
- { echo "@set UPDATED $$dmy"; \
- echo "@set UPDATED-MONTH $$my"; \
- echo "@set EDITION $(VERSION)"; \
- echo "@set VERSION $(VERSION)"; } > $@-t; \
- mv $@-t $@; \
- cp $@ $(srcdir)/doc/version.texi
-
-i:=0
-$(eval $(call version.texi-from-git,vti,doc/guix.texi,))
-$(foreach lang, $(MANUAL_LANGUAGES), \
- $(eval i=$(shell echo $$(($(i)+1)))) \
- $(eval $(call version.texi-from-git,$(i),po/doc/guix-manual.$(lang).po,-$(lang))))
-
DIST_CONFIGURE_FLAGS = \
--localstatedir=/var \
--sysconfdir=/etc
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 15 Apr 16:27 +0200
[PATCH v2 2/3] Revert "maint: Generate 'doc/version-LANG.texi' reproducibly."
(address . 70380@debbugs.gnu.org)
70a03da06dc3a247d0b91807d0f3b9c160632dda.1713190364.git.janneke@gnu.org
Using `build-aux/mdate-from-git.scm' makes this no longer necessary.

This reverts commit 0e4ead187d83a958ca0deb54857c04967e84d68b.

Change-Id: I9177828f90fa7f7e256bc72fdff35a2fab355780
---
doc/local.mk | 40 +++++++++++++++++-----------------------
1 file changed, 17 insertions(+), 23 deletions(-)

Toggle diff (56 lines)
diff --git a/doc/local.mk b/doc/local.mk
index 130f40ece9..77d48902b6 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -260,32 +260,26 @@ endif
# Reproducible tarball
-# Define a rule to build `version[LANG].texi' reproducibly using metadata from
-# Git rather than using metadata from the filesystem.
-define version.texi-from-git
-$(srcdir)/doc/stamp-$(1): $(srcdir)/$(2) $(top_srcdir)/configure
- $$(AM_V_GEN)set -e; \
- export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
- export TZ=UTC0; \
- timestamp="$$$$(git log --pretty=format:%ct -n1 -- "$$<" \
- 2>/dev/null \
- || echo $$(SOURCE_DATE_EPOCH))" \
- dmy=$$$$(date --date="@$$$$timestamp" "+%-d %B %Y"); \
- my=$$$$(date --date="@$$$$timestamp" "+%B %Y"); \
- { echo "@set UPDATED $$$$dmy"; \
- echo "@set UPDATED-MONTH $$$$my"; \
- echo "@set EDITION $$$(VERSION)"; \
- echo "@set VERSION $$$(VERSION)"; } > "$$@-t"; \
- mv "$$@-t" "$$@"; \
- cp -p "$$@" "$$(srcdir)/doc/version$(3).texi"
-endef
-
-# Generate rules for stamp-vti and stamp-N that create version.texi and
-# version-LANG.texi to override the Autotools versions that use timestamps
-# embedded in the file-system. These are expected to generate warnings:
+# Generate 'version.texi' reproducibly using metadata from Git rather than
+# using metadata from the filesystem. This is expected to generate warnings:
#
# Makefile:7376: warning: overriding recipe for target 'doc/stamp-vti'
# Makefile:5098: warning: ignoring old recipe for target 'doc/stamp-vti'
+$(srcdir)/doc/stamp-vti: $(srcdir)/doc/guix.texi $(top_srcdir)/configure
+ $$(AM_V_GEN)set -e; \
+ export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
+ export TZ=UTC0; \
+ timestamp=$$(git log --pretty=format:%ct -n1 -- $< 2>/dev/null \
+ || echo $(SOURCE_DATE_EPOCH)) \
+ dmy=$$(date --date="@$$timestamp" "+%-d %B %Y"); \
+ my=$$(date --date="@$$timestamp" "+%B %Y"); \
+ { echo "@set UPDATED $$dmy"; \
+ echo "@set UPDATED-MONTH $$my"; \
+ echo "@set EDITION $(VERSION)"; \
+ echo "@set VERSION $(VERSION)"; } > $@-t; \
+ mv $@-t $@; \
+ cp $@ $(srcdir)/doc/version.texi
+
i:=0
$(eval $(call version.texi-from-git,vti,doc/guix.texi,))
$(foreach lang, $(MANUAL_LANGUAGES), \
--
2.41.0
P
P
pelzflorian (Florian Pelz) wrote on 16 Apr 09:33 +0200
Re: [bug#70380] [PATCH v2 0/3] Reproducible `make dist' tarball: Avoid override stamp-N warnings.
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 70380@debbugs.gnu.org)
877cgxkbi7.fsf@pelzflorian.de
Hi again, Jan.

Janneke Nieuwenhuizen <janneke@gnu.org> writes:
Toggle quote (9 lines)
> V2 fixes embarrasing typos in 1/3 that should have prevented for this patch to
> build a reproducible tarball at all. Most probably I only tested a previous
> version where mdate-from-git was a plain shell script.
>
> Janneke Nieuwenhuizen (3):
> maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'.
> Revert "maint: Generate 'doc/version-LANG.texi' reproducibly."
> Revert "maint: Generate 'doc/version.texi' reproducibly."

It all looked great when reading, but sorry to say, there are errors. I
run “make dist” from a “make dist”-generated tarball and get (with
LC_ALL=C):

HELP2MAN doc/guix-style.1
HELP2MAN doc/guix-system.1
HELP2MAN doc/guix-time-machine.1
HELP2MAN doc/guix-weather.1
HELP2MAN doc/guix.1
CXX nix/nix-daemon/guix_daemon-guix-daemon.o
CXXLD guix-daemon
HELP2MAN doc/guix-daemon.1
make[2]: *** No rule to make target 'build-aux/mdate-sh', needed by 'distdir-am'. Stop.
make[2]: Leaving directory '/home/florian/src/guix-1.3.0.57920-346e22'
make[1]: *** [Makefile:6570: distdir] Error 2
make[1]: Leaving directory '/home/florian/src/guix-1.3.0.57920-346e22'
make: *** [Makefile:6679: dist] Error 2


Alarming is also, when I run “make -j6” from a generated tarball, it
fails. I had not tested this previously but should have. Arrgh. The
error message about an error at Makefile:6301 gives me no clue. “make”
runs almost through, though, except for the above error. Strange. Does
it work for you?

Something else less relevant, when running “./bootstrap && ./configure
--localstatedir=/var --sysconfdir=/etc” from a generated tarball, it
prints many harmless errors:

Copying file m4/visibility.m4
Copying file m4/wchar_t.m4
Copying file m4/wint_t.m4
Copying file m4/xsize.m4
Copying file po/guix/Makevars.template
Copying file po/packages/Makevars.template
autoreconf: running: aclocal --force -I m4
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
autoreconf: configure.ac: tracing
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
autoreconf: configure.ac: not using Libtool
autoreconf: running: /gnu/store/4q6xf35c45c2a7xrw8brdgqn20cgb4bx-autoconf-2.69/bin/autoconf --force
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
autoreconf: running: /gnu/store/4q6xf35c45c2a7xrw8brdgqn20cgb4bx-autoconf-2.69/bin/autoheader --force
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
sh: line 1: build-aux/git-version-gen: Datei oder Verzeichnis nicht gefunden
autoreconf: running: automake --add-missing --copy --force-missing
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
sh: line 1: build-aux/git-version-gen: No such file or directory
Makefile.am:922: warning: AM_GNU_GETTEXT used but 'po' not in SUBDIRS
autoreconf: Leaving directory `.'
+ chmod +w build-aux/mdate-sh
+ rm -f build-aux/mdate-sh
+ ln -s mdate-from-git.scm build-aux/mdate-sh
checking for a BSD-compatible install... /gnu/store/gfcvx5bbybpvc5dbq32dra04ncrvdhn9-profile/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /gnu/store/gfcvx5bbybpvc5dbq32dra04ncrvdhn9-profile/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes


(“Datei oder Verzeichnis nicht gefunden” is German for “No such file or
directory”, so it sometimes gets translated and sometimes not. Possibly
it is related to your patches, possibly not; I have not tested without
patches.)

Regards,
Florian
P
P
pelzflorian (Florian Pelz) wrote on 16 Apr 09:38 +0200
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 70380@debbugs.gnu.org)
8734rlkb9n.fsf@pelzflorian.de
I’ve reported wrongly,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> writes:
Toggle quote (12 lines)
> make[2]: *** No rule to make target 'build-aux/mdate-sh', needed by 'distdir-am'. Stop.
> make[2]: Leaving directory '/home/florian/src/guix-1.3.0.57920-346e22'
> make[1]: *** [Makefile:6570: distdir] Error 2
> make[1]: Leaving directory '/home/florian/src/guix-1.3.0.57920-346e22'
> make: *** [Makefile:6679: dist] Error 2
>
>
> Alarming is also, when I run “make -j6” from a generated tarball, it
> fails. I had not tested this previously but should have. Arrgh. The
> error message about an error at Makefile:6301 gives me no clue. “make”
> runs almost through, though, except for the above error.

It was not the above mdate-sh error. The error running make from a
tarball is

[100%] GUILEC guix/scripts/time-machine.go
[100%] GUILEC guix/scripts/copy.go
[100%] GUILEC guix/scripts/discover.go
[100%] GUILEC guix/scripts/offload.go
HELP2MAN doc/guix-daemon.1
make[2]: *** No rule to make target 'etc/git/pre-push', needed by '.git/hooks/pre-push'. Stop.
make[2]: Leaving directory '/home/florian/guix-1.3.0.57920-346e22'
make[1]: *** [Makefile:6301: all-recursive] Error 1
make[1]: Leaving directory '/home/florian/guix-1.3.0.57920-346e22'
make: *** [Makefile:4368: all] Error 2

Regards,
Florian
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 10:10 +0200
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 70380@debbugs.gnu.org)
874jc0z9y7.fsf@gnu.org
pelzflorian (Florian Pelz) writes:

Hi!

Toggle quote (2 lines)
> I’ve reported wrongly,

Ah, phew. And thanks for testing!

Toggle quote (27 lines)
> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> writes:
>> make[2]: *** No rule to make target 'build-aux/mdate-sh', needed by 'distdir-am'. Stop.
>> make[2]: Leaving directory '/home/florian/src/guix-1.3.0.57920-346e22'
>> make[1]: *** [Makefile:6570: distdir] Error 2
>> make[1]: Leaving directory '/home/florian/src/guix-1.3.0.57920-346e22'
>> make: *** [Makefile:6679: dist] Error 2
>>
>>
>> Alarming is also, when I run “make -j6” from a generated tarball, it
>> fails. I had not tested this previously but should have. Arrgh. The
>> error message about an error at Makefile:6301 gives me no clue. “make”
>> runs almost through, though, except for the above error.
>
> It was not the above mdate-sh error. The error running make from a
> tarball is
>
> [100%] GUILEC guix/scripts/time-machine.go
> [100%] GUILEC guix/scripts/copy.go
> [100%] GUILEC guix/scripts/discover.go
> [100%] GUILEC guix/scripts/offload.go
> HELP2MAN doc/guix-daemon.1
> make[2]: *** No rule to make target 'etc/git/pre-push', needed by '.git/hooks/pre-push'. Stop.
> make[2]: Leaving directory '/home/florian/guix-1.3.0.57920-346e22'
> make[1]: *** [Makefile:6301: all-recursive] Error 1
> make[1]: Leaving directory '/home/florian/guix-1.3.0.57920-346e22'
> make: *** [Makefile:4368: all] Error 2

Okay, this reproduces. I didn't think about this scenario test this at
all... V3 comes with an extra commit to cater for this.

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 11:53 +0200
[PATCH v3 0/4] Reproducible `make dist' tarball: Avoid override stamp-N warnings.
(address . 70380@debbugs.gnu.org)
cover.1713340673.git.janneke@gnu.org
Changes in V3

* Resurrect/cater for running `make dist' from tarball by not removing
any Autotools' caches,
* Distribute build-aux/mdate-from-git.scm to support running ./bootstrap
from tarball.

Doing the latter will still generate an unversioned `guix-.tar.gz' tarball,
just like before, as we use Autotools' git-version-gen in configure.ac that
doesn't cater for this scenario, AFAIU.

Janneke Nieuwenhuizen (4):
maint: Cater for running `make dist' from tarball.
maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'.
Revert "maint: Generate 'doc/version-LANG.texi' reproducibly."
Revert "maint: Generate 'doc/version.texi' reproducibly."

Makefile.am | 62 +++++++++++++++-----------
bootstrap | 8 +++-
build-aux/mdate-from-git.scm | 86 ++++++++++++++++++++++++++++++++++++
doc/local.mk | 32 --------------
4 files changed, 130 insertions(+), 58 deletions(-)
create mode 100755 build-aux/mdate-from-git.scm


base-commit: b47ae1ecc43baaf726701ab2d2f810ecfaa75428
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 11:53 +0200
[PATCH v3 1/4] maint: Cater for running `make dist' from tarball.
(address . 70380@debbugs.gnu.org)
9dd6e4436bcc0639cbbae2d468e5bc3760d73aa3.1713340673.git.janneke@gnu.org
* Makefile.am (in_git_p): New variable. Use it to disable Autotools'
cache consistency assert and removal when bulding from tarball.
(dist): Depend on doc-pot-update again when building from tarball.
(dist-hook): Remove dependencies on gen-ChangeLog and gen-AUTHORS when
building from tarball.
(gen-ChangeLog, gen-AUTHORS): Remove guarding for building from tarball.
Use set -e to avoid silently failing.
(gen-tarball-version): Use $(SOURCE_DATE_EPOCH) instead of re-generating it
using git; this also works running from a tarball.

Change-Id: I9ebdd28a70837f6a4db610c4816bb283d176e2d9
---
Makefile.am | 61 +++++++++++++++++++++++++++++++----------------------
1 file changed, 36 insertions(+), 25 deletions(-)

Toggle diff (101 lines)
diff --git a/Makefile.am b/Makefile.am
index 147767ece4..39cf4b4b17 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -953,9 +953,24 @@ guix-binary.%.tar.xz:
guix` ; \
cp "$$tarball" "$@.tmp" ; mv "$@.tmp" "$@"
+# The `dist' target has other dependencies when building from Git
+# to assert and achieve reproducibility. Indented to get past Automake.
+in_git_p = $(shell test -e .git && echo true)
+ ifeq ($(in_git_p),true)
+
+# The dependency on dist-doc-pot-update is to clean possibly stale doc and po
+# files and only then generate the .pot files, which are not checked in.
+dist: dist-doc-pot-update
+dist-doc-pot-update: auto-clean
+ $(MAKE) guile$(EXEEXT)
+ $(MAKE) -C po/guix all
+ $(MAKE) -C po/packages all
+ $(MAKE) doc-pot-update
+
+dist-hook: gen-ChangeLog gen-AUTHORS
# Assert that Autotools cache is up to date with Git, by checking
-# PACKAGE_VERSION against HEAD. Indented to get past Automake.
+# PACKAGE_VERSION against HEAD.
ifeq ($(MAKECMDGOALS),dist)
git_version = $(shell build-aux/git-version-gen .tarball-version)
ifneq ($(PACKAGE_VERSION),$(git_version))
@@ -967,20 +982,18 @@ guix-binary.%.tar.xz:
$(error Cannot create reproducible tarball)
else
$(warning Tarball will be irreproducible; distdir will not get removed!)
- endif
- endif
- endif
+ endif # !GUIX_ALLOW_IRREPRODUCIBLE_TARBALL
+ endif # PACKAGE_VERSION != git_version
+ endif # MAKECMDGOALS dist
-# The dependency on dist-doc-pot-update is to clean possibly stale doc and po
-# files and only then generate the .pot files, which are not checked in.
-dist: dist-doc-pot-update
-dist-doc-pot-update: auto-clean
- $(MAKE) guile$(EXEEXT)
- $(MAKE) -C po/guix all
- $(MAKE) -C po/packages all
- $(MAKE) doc-pot-update
+ else # !in_git_p
-dist-hook: gen-ChangeLog gen-AUTHORS gen-tarball-version
+dist: doc-pot-update
+$(warning Not using Git, tarball will likely be irreproducible!)
+
+ endif # !in_git_p
+
+dist-hook: gen-tarball-version
dist-hook: assert-no-store-file-names
distcheck-hook: assert-binaries-available assert-final-inputs-self-contained
@@ -992,27 +1005,25 @@ $(top_srcdir)/.version: config.status
gen-tarball-version:
echo $(VERSION) > "$(distdir)/.tarball-version"
- git show HEAD --format=%ct --no-patch > $(distdir)/.tarball-timestamp
+ echo $(SOURCE_DATE_EPOCH) > $(distdir)/.tarball-timestamp
gen-ChangeLog:
- $(AM_V_GEN)if test -e .git; then \
- export LC_ALL=en_US.UTF-8; \
- export TZ=UTC0; \
- $(top_srcdir)/build-aux/gitlog-to-changelog \
- > $(distdir)/ChangeLog.tmp; \
- rm -f $(distdir)/ChangeLog; \
- mv $(distdir)/ChangeLog.tmp $(distdir)/ChangeLog; \
- fi
+ $(AM_V_GEN)set -e; \
+ export LC_ALL=en_US.UTF-8; \
+ export TZ=UTC0; \
+ $(top_srcdir)/build-aux/gitlog-to-changelog \
+ > $(distdir)/ChangeLog.tmp; \
+ rm -f $(distdir)/ChangeLog; \
+ mv $(distdir)/ChangeLog.tmp $(distdir)/ChangeLog;
gen-AUTHORS:
- $(AM_V_GEN)if test -e .git; then \
+ $(AM_V_GEN)set -e; \
rm -f "$(distdir)/AUTHORS"; \
export LC_ALL=en_US.UTF-8; \
export TZ=UTC0; \
$(top_builddir)/pre-inst-env "$(GUILE)" \
"$(top_srcdir)/build-aux/generate-authors.scm" \
- "$(top_srcdir)" "$(distdir)/AUTHORS"; \
- fi
+ "$(top_srcdir)" "$(distdir)/AUTHORS";
# Like 'dist', but regenerate 'configure' so we get an up-to-date
# 'PACKAGE_VERSION' string. (In Gnulib, 'GNUmakefile' has a special trick to
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 11:53 +0200
[PATCH v3 4/4] Revert "maint: Generate 'doc/version.texi' reproducibly."
(address . 70380@debbugs.gnu.org)
8de809241c8abd7ec234fcc8da2590074a5fc828.1713340674.git.janneke@gnu.org
Using `build-aux/mdate-from-git.scm' makes this no longer necessary.

This reverts commit e73ea7bd64f64709c71f89dfb111cf3e8ada3771.

Change-Id: I29d1e36b13d255e5a65b7348e7ae4f2b2c24a518
---
doc/local.mk | 26 --------------------------
1 file changed, 26 deletions(-)

Toggle diff (39 lines)
diff --git a/doc/local.mk b/doc/local.mk
index 77d48902b6..1d94e3c758 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -260,32 +260,6 @@ endif
# Reproducible tarball
-# Generate 'version.texi' reproducibly using metadata from Git rather than
-# using metadata from the filesystem. This is expected to generate warnings:
-#
-# Makefile:7376: warning: overriding recipe for target 'doc/stamp-vti'
-# Makefile:5098: warning: ignoring old recipe for target 'doc/stamp-vti'
-$(srcdir)/doc/stamp-vti: $(srcdir)/doc/guix.texi $(top_srcdir)/configure
- $$(AM_V_GEN)set -e; \
- export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
- export TZ=UTC0; \
- timestamp=$$(git log --pretty=format:%ct -n1 -- $< 2>/dev/null \
- || echo $(SOURCE_DATE_EPOCH)) \
- dmy=$$(date --date="@$$timestamp" "+%-d %B %Y"); \
- my=$$(date --date="@$$timestamp" "+%B %Y"); \
- { echo "@set UPDATED $$dmy"; \
- echo "@set UPDATED-MONTH $$my"; \
- echo "@set EDITION $(VERSION)"; \
- echo "@set VERSION $(VERSION)"; } > $@-t; \
- mv $@-t $@; \
- cp $@ $(srcdir)/doc/version.texi
-
-i:=0
-$(eval $(call version.texi-from-git,vti,doc/guix.texi,))
-$(foreach lang, $(MANUAL_LANGUAGES), \
- $(eval i=$(shell echo $$(($(i)+1)))) \
- $(eval $(call version.texi-from-git,$(i),po/doc/guix-manual.$(lang).po,-$(lang))))
-
DIST_CONFIGURE_FLAGS = \
--localstatedir=/var \
--sysconfdir=/etc
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 11:53 +0200
[PATCH v3 3/4] Revert "maint: Generate 'doc/version-LANG.texi' reproducibly."
(address . 70380@debbugs.gnu.org)
99333fca4662a435c68d6a54f5f6e28aecc21103.1713340674.git.janneke@gnu.org
Using `build-aux/mdate-from-git.scm' makes this no longer necessary.

This reverts commit 0e4ead187d83a958ca0deb54857c04967e84d68b.

Change-Id: I9177828f90fa7f7e256bc72fdff35a2fab355780
---
doc/local.mk | 40 +++++++++++++++++-----------------------
1 file changed, 17 insertions(+), 23 deletions(-)

Toggle diff (56 lines)
diff --git a/doc/local.mk b/doc/local.mk
index 130f40ece9..77d48902b6 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -260,32 +260,26 @@ endif
# Reproducible tarball
-# Define a rule to build `version[LANG].texi' reproducibly using metadata from
-# Git rather than using metadata from the filesystem.
-define version.texi-from-git
-$(srcdir)/doc/stamp-$(1): $(srcdir)/$(2) $(top_srcdir)/configure
- $$(AM_V_GEN)set -e; \
- export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
- export TZ=UTC0; \
- timestamp="$$$$(git log --pretty=format:%ct -n1 -- "$$<" \
- 2>/dev/null \
- || echo $$(SOURCE_DATE_EPOCH))" \
- dmy=$$$$(date --date="@$$$$timestamp" "+%-d %B %Y"); \
- my=$$$$(date --date="@$$$$timestamp" "+%B %Y"); \
- { echo "@set UPDATED $$$$dmy"; \
- echo "@set UPDATED-MONTH $$$$my"; \
- echo "@set EDITION $$$(VERSION)"; \
- echo "@set VERSION $$$(VERSION)"; } > "$$@-t"; \
- mv "$$@-t" "$$@"; \
- cp -p "$$@" "$$(srcdir)/doc/version$(3).texi"
-endef
-
-# Generate rules for stamp-vti and stamp-N that create version.texi and
-# version-LANG.texi to override the Autotools versions that use timestamps
-# embedded in the file-system. These are expected to generate warnings:
+# Generate 'version.texi' reproducibly using metadata from Git rather than
+# using metadata from the filesystem. This is expected to generate warnings:
#
# Makefile:7376: warning: overriding recipe for target 'doc/stamp-vti'
# Makefile:5098: warning: ignoring old recipe for target 'doc/stamp-vti'
+$(srcdir)/doc/stamp-vti: $(srcdir)/doc/guix.texi $(top_srcdir)/configure
+ $$(AM_V_GEN)set -e; \
+ export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
+ export TZ=UTC0; \
+ timestamp=$$(git log --pretty=format:%ct -n1 -- $< 2>/dev/null \
+ || echo $(SOURCE_DATE_EPOCH)) \
+ dmy=$$(date --date="@$$timestamp" "+%-d %B %Y"); \
+ my=$$(date --date="@$$timestamp" "+%B %Y"); \
+ { echo "@set UPDATED $$dmy"; \
+ echo "@set UPDATED-MONTH $$my"; \
+ echo "@set EDITION $(VERSION)"; \
+ echo "@set VERSION $(VERSION)"; } > $@-t; \
+ mv $@-t $@; \
+ cp $@ $(srcdir)/doc/version.texi
+
i:=0
$(eval $(call version.texi-from-git,vti,doc/guix.texi,))
$(foreach lang, $(MANUAL_LANGUAGES), \
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 11:53 +0200
[PATCH v3 2/4] maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'.
(address . 70380@debbugs.gnu.org)
93b2d3b1b675dfd3018d0b3005d19bdbdd97b629.1713340673.git.janneke@gnu.org
This replaces Automake's `build-aux/mdate-sh' with our own
`build-aux/mdate-from-git.scm' to use reproducible timestamps from Git
instead.

* build-aux/mdate-from-git.scm: New script.
* bootstrap: Use it to replace build-aux/mdate-sh.
* Makefile.am (EXTRA_DIST): Add it.

Change-Id: I17d0a7de9ffea397129c0db1728f86e28a4e245f
---
Makefile.am | 1 +
bootstrap | 8 +++-
build-aux/mdate-from-git.scm | 86 ++++++++++++++++++++++++++++++++++++
3 files changed, 94 insertions(+), 1 deletion(-)
create mode 100755 build-aux/mdate-from-git.scm

Toggle diff (122 lines)
diff --git a/Makefile.am b/Makefile.am
index 39cf4b4b17..1de8149772 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -723,6 +723,7 @@ EXTRA_DIST += \
build-aux/config.rpath \
build-aux/convert-xref.scm \
build-aux/generate-authors.scm \
+ build-aux/mdate-from-git.scm \
build-aux/test-driver.scm \
build-aux/update-NEWS.scm \
build-aux/update-guix-package.scm \
diff --git a/bootstrap b/bootstrap
index de024aeaa5..5bf83175e5 100755
--- a/bootstrap
+++ b/bootstrap
@@ -24,4 +24,10 @@ for lang in ${langs}; do
fi
done
-exec autoreconf -vfi
+autoreconf -vfi
+
+# Replace Automake's build-aux/mdate-sh with build-aux/mdate-from-git, our
+# own, reproducible version.
+chmod +w build-aux/mdate-sh
+rm -f build-aux/mdate-sh
+ln -s mdate-from-git.scm build-aux/mdate-sh
diff --git a/build-aux/mdate-from-git.scm b/build-aux/mdate-from-git.scm
new file mode 100755
index 0000000000..abe7e97f32
--- /dev/null
+++ b/build-aux/mdate-from-git.scm
@@ -0,0 +1,86 @@
+#! /bin/sh
+# -*-scheme-*-
+export LANG=C LANGUAGE=C LC_TIME=C
+export TZ=UTC0
+exec guile --no-auto-compile -L $srcdir -C $srcdir -e '(mdate-from-git)' -s "$0" "$@"
+!#
+
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Janneke Nieuwenhuizen <janneke@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; This program is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; This program is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+;;;; Commentary:
+;;;
+;;; Usage: mdate-from-git.scm FILE
+;;;
+;;; This script is compatible with Automake's `mdate-sh' but uses the timestamp
+;;; from Git instead of from the file system. Also, it can be appended to
+;;; mdate-sh.
+
+;;; As a special exception for Guix, it caters for doc/guix.LANG.texi files that
+;;; are not stored in Git, by using po/doc/guix-manual.LANG.po for the Git
+;;; timestamp. Test doing something like:
+;;;
+;;; build-aux/mdate-from-git.scm doc/guix.de.texi
+;;;
+;;;; Code:
+
+(define-module (mdate-from-git)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 popen)
+ #:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
+ #:export (main))
+
+(define (pipe-command command)
+ (let* ((port (apply open-pipe* OPEN_READ command))
+ (output (read-string port)))
+ (close-port port)
+ output))
+
+(define (guix.LANG.texi->guix-manual.LANG.po file-name)
+ "Translated manuals doc/guix.LANG.texi are not tracked in Git and are
+generated from po/doc/guix-manual.LANG.po. For such an untraced .TEXI file,
+return its .PO counterpart."
+ (let ((m (string-match "doc/guix.([^.]+).texi" file-name)))
+ (if (not m) file-name
+ (let ((lang (match:substring m 1)))
+ (format #f "po/doc/guix-manual.~a.po" lang)))))
+
+
+;;;
+;;; Entry point.
+;;;
+(define (main args)
+ (match args
+ ((script file-name)
+ (let* ((command `("git" "ls-files" "--error-unmatch" "--" ,file-name))
+ (tracked? (zero? (with-error-to-port (%make-void-port "w")
+ (lambda _ (apply system* command)))))
+ (file-name (if tracked? file-name
+ (guix.LANG.texi->guix-manual.LANG.po file-name)))
+ (command `("git" "log" "--pretty=format:%ct" "-n1" "--" ,file-name))
+ (timestamp (pipe-command command))
+ (source-date-epoch (or (getenv "SOURCE_DATE_EPOCH") "1"))
+ (timestamp (if (string-null? timestamp) source-date-epoch
+ timestamp))
+ (time (gmtime (string->number timestamp)))
+ (d-m-y (strftime "%-d %B %Y" time)))
+ (display d-m-y)))
+ (_
+ (format (current-error-port) "Usage: mdate-from-git.scm FILE\n")
+ (exit 2))))
--
2.41.0
P
P
pelzflorian (Florian Pelz) wrote on 17 Apr 17:37 +0200
Re: [bug#70380] [PATCH v3 1/4] maint: Cater for running `make dist' from tarball.
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 70380@debbugs.gnu.org)
87jzkwov9t.fsf@pelzflorian.de
Hello Jan. Thank you for fixing “make dist” from a tarball, which
admittedly is far-fetched. However, “make” from a tarball is not
actually fixed / catered for and is essential.

[100%] GUILEC guix/scripts/discover.go
[100%] GUILEC guix/scripts/offload.go
HELP2MAN doc/guix-daemon.1
make[2]: *** No rule to make target 'etc/git/pre-push', needed by '.git/hooks/pre-push'. Stop.
make[2]: Leaving directory '/home/florian/guix-1.3.0.58013-5dfeb'
make[1]: *** [Makefile:6304: all-recursive] Error 1
make[1]: Leaving directory '/home/florian/guix-1.3.0.58013-5dfeb'
make: *** [Makefile:4371: all] Error 2

I suppose it is not actually your fault, but since you know how to fix
it, would you fix it?

Note that the non-fatal “./bootstrap” errors from tarball (sh: line 1:
build-aux/git-version-gen: No such file or directory ) are not fixed
either, but they are not fatal.

Janneke Nieuwenhuizen <janneke@gnu.org> writes:
Toggle quote (3 lines)
> +dist: doc-pot-update
> +$(warning Not using Git, tarball will likely be irreproducible!)

I get this warning when running “make” from a tarball, not only “make
dist”. However (!) “make dist” from the same (rebuilt reproducibly from
the git repo with your patches) tarball on another machine is in my case
perfectly reproducible, so could you just drop the warning?


Thank you for making Guix more secure.

Regards,
Florian
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 21:02 +0200
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 70380@debbugs.gnu.org)
87plunyfq9.fsf@gnu.org
pelzflorian (Florian Pelz) writes:

Hello Florian,

Toggle quote (4 lines)
> Hello Jan. Thank you for fixing “make dist” from a tarball, which
> admittedly is far-fetched. However, “make” from a tarball is not
> actually fixed / catered for and is essential.

Ouch, how did I miss that?

Toggle quote (12 lines)
> [100%] GUILEC guix/scripts/discover.go
> [100%] GUILEC guix/scripts/offload.go
> HELP2MAN doc/guix-daemon.1
> make[2]: *** No rule to make target 'etc/git/pre-push', needed by '.git/hooks/pre-push'. Stop.
> make[2]: Leaving directory '/home/florian/guix-1.3.0.58013-5dfeb'
> make[1]: *** [Makefile:6304: all-recursive] Error 1
> make[1]: Leaving directory '/home/florian/guix-1.3.0.58013-5dfeb'
> make: *** [Makefile:4371: all] Error 2
>
> I suppose it is not actually your fault, but since you know how to fix
> it, would you fix it?

Ah, indeed. It was broken almost a year ago. Your suggestion makes
much sense to me, so I'm including a new patch in V4 to address this.

It required moving in_git_p from GNU make to to configure.ac and
introducing it earlier.

Toggle quote (4 lines)
> Note that the non-fatal “./bootstrap” errors from tarball (sh: line 1:
> build-aux/git-version-gen: No such file or directory ) are not fixed
> either, but they are not fatal.

Yes, I don't know what to do about it. Looks like an Autotools bug to
me.

Toggle quote (7 lines)
> Janneke Nieuwenhuizen <janneke@gnu.org> writes:
>> +dist: doc-pot-update
>> +$(warning Not using Git, tarball will likely be irreproducible!)
>
> I get this warning when running “make” from a tarball, not only “make
> dist”.

Oops, fixed in V4.

Toggle quote (4 lines)
> However (!) “make dist” from the same (rebuilt reproducibly from
> the git repo with your patches) tarball on another machine is in my case
> perfectly reproducible, so could you just drop the warning?

Hmm. You're probably rightt. I tried touching files, run update-po,
but reproducibility seems pretty resilient to such changes. I've chosen
to keep it anyway just to discourage maintainers from doing such a
thing.

Toggle quote (2 lines)
> Thank you for making Guix more secure.

Most happy to. Thanks for your reviews!

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 21:08 +0200
[PATCH v4 1/6] maint: Resurrect running `make' from a tarball.
(address . 70380@debbugs.gnu.org)
543d5a5e035e6a6dc68013393cc470dcf1dc0772.1713379858.git.janneke@gnu.org
This is a follow-up to commit
8b972da068708a8b17f3ab153ea940690ca49ca9
Makefile.am: Auto-configure Git on 'make'.

* configure.ac (in_git_p): New conditional.
* Makefile.am (nodist_noinst_DATA): Use it to only enable this when building
from Git.

Change-Id: I09a90a59a4933a8cdb04124467d38209171f2a57
---
Makefile.am | 5 +++++
configure.ac | 10 ++++++++++
2 files changed, 15 insertions(+)

Toggle diff (40 lines)
diff --git a/Makefile.am b/Makefile.am
index 25b3eb3378..98008c528d 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1241,7 +1241,12 @@ COMMIT_MSG_MAGIC = VGhpcyBpcyB0aGUgY29tbWl0LW1zZyBob29rIG9mIEd1aXg=
cp etc/git/commit-msg $@; \
fi
+# The etc/git/ config files are not distributed and have no use when building
+# from a tarball. Do not add dependencies on these to *_DATA when building
+# from a tarball, as that breaks the build.
+if in_git_p
nodist_noinst_DATA = .git/hooks/pre-push .git/config .git/hooks/commit-msg
+endif
# Downloading up-to-date PO files.
diff --git a/configure.ac b/configure.ac
index ecbd596a34..8c3a06da37 100644
--- a/configure.ac
+++ b/configure.ac
@@ -265,6 +265,16 @@ dnl Documentation translation.
AM_MISSING_PROG([PO4A_TRANSLATE], [po4a-translate])
AM_MISSING_PROG([PO4A_UPDATEPO], [po4a-updatepo])
+AC_MSG_CHECKING([if building from git])
+if test -e .git; then
+ in_git_p=yes
+else
+ in_git_p=no
+fi
+AC_MSG_RESULT([$in_git_p])
+AM_CONDITIONAL([in_git_p],
+ [test "x$in_git_p" = "xyes"])
+
case "$storedir" in
/gnu/store)
;;
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 21:08 +0200
[PATCH v4 0/6] Reproducible `make dist' tarball: Avoid override stamp-N warnings.
(address . 70380@debbugs.gnu.org)
cover.1713379858.git.janneke@gnu.org
New in V4:

* Move in_git_p check from GNU make to configure.ac, in order to,
* Ressurrect running `make' from a tarball,
* Have xgettext.scm support being run from a tarball, and,
* Distribute xgettext.scm, in order to,
* Support running update-po from a tarball.

Note that running `make' from a tarball has been broken for almost a year now
and has nothing to do with any of the reproducible tarball work. Apparently
that bug has a low priority at the moment and including it here creeps the
scope maybe a little but probably helps more with the review.

Greetings,
Janneke

Janneke Nieuwenhuizen (6):
maint: Resurrect running `make' from a tarball.
maint: Support `make doc-po-update' from tarball.
maint: Cater for running `make dist' from tarball.
maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'.
Revert "maint: Generate 'doc/version-LANG.texi' reproducibly."
Revert "maint: Generate 'doc/version.texi' reproducibly."

Makefile.am | 67 +++++++++++++++++----------
bootstrap | 8 +++-
build-aux/mdate-from-git.scm | 87 ++++++++++++++++++++++++++++++++++++
build-aux/xgettext.scm | 10 ++---
configure.ac | 10 +++++
doc/local.mk | 32 -------------
6 files changed, 152 insertions(+), 62 deletions(-)
create mode 100755 build-aux/mdate-from-git.scm


base-commit: 2126dab4cd81db4cbde4566d8c638e45a4c0077c
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 21:08 +0200
[PATCH v4 2/6] maint: Support `make doc-po-update' from tarball.
(address . 70380@debbugs.gnu.org)
8c9f63715e8d7613c4a23fcccf828ccb5b323dae.1713379858.git.janneke@gnu.org
* build-aux/xgettext.scm: Move setting of environment variables to shell
header.
(main): Use SOURCE_DATE_EPOCH as fallback for timestamp. This fixes running
from a tarball.
* Makefile.am (EXTRA_DIST): Add it.

Change-Id: Ic487587b22495868fd2a21545a13dc9e3458299c
---
Makefile.am | 1 +
build-aux/xgettext.scm | 10 +++++-----
2 files changed, 6 insertions(+), 5 deletions(-)

Toggle diff (50 lines)
diff --git a/Makefile.am b/Makefile.am
index 98008c528d..ca3fa0a693 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -727,6 +727,7 @@ EXTRA_DIST += \
build-aux/test-driver.scm \
build-aux/update-NEWS.scm \
build-aux/update-guix-package.scm \
+ build-aux/xgettext.scm \
doc/build.scm \
etc/disarchive-manifest.scm \
etc/guix-install.sh \
diff --git a/build-aux/xgettext.scm b/build-aux/xgettext.scm
index 44d30b8149..7142336a9d 100755
--- a/build-aux/xgettext.scm
+++ b/build-aux/xgettext.scm
@@ -2,6 +2,8 @@
# -*-scheme-*-
build_aux=$(dirname $0)
srcdir=$build_aux/..
+export LC_ALL=en_US.UTF-8
+export TZ=UTC0
exec guile --no-auto-compile -L $srcdir -C $srcdir -e main -s "$0" "$@"
!#
@@ -59,9 +61,6 @@ (define (pipe-command command)
;;; Entry point.
;;;
(define (main args)
- ;; Cater for being run in a container.
- (setenv "LC_ALL" "en_US.UTF-8")
- (setenv "TZ" "UTC0")
(fluid-set! %default-port-encoding #f)
(let* ((files-from (get-option args "--files-from="))
(default-domain (get-option args "--default-domain="))
@@ -82,9 +81,10 @@ (define (main args)
(files (map (cute string-append directory "/" <>) files))
(git-command `("git" "log" "--pretty=format:%ci" "-n1" ,@files))
(timestamp (pipe-command git-command))
+ (source-date-epoch (or (getenv "SOURCE_DATE_EPOCH") "1"))
+ (timestamp (if (string-null? timestamp) source-date-epoch
+ timestamp))
(po-file (string-append default-domain ".po")))
- (when (string-null? timestamp)
- (exit 1))
(substitute* po-file
(("(\"POT-Creation-Date: )[^\\]*" all header)
(string-append header timestamp)))))))
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 21:08 +0200
[PATCH v4 3/6] maint: Cater for running `make dist' from tarball.
(address . 70380@debbugs.gnu.org)
b08c1af94ea36b341ba2ccb91d5a6341a48a4745.1713379858.git.janneke@gnu.org
* Makefile.am: Use in_git_p conditional to disable Autotools' cache
consistency assert and removal when bulding from tarball.
(dist): Depend on doc-pot-update again when building from tarball.
(dist-hook): Remove dependencies on gen-ChangeLog and gen-AUTHORS when
building from tarball.
(gen-ChangeLog, gen-AUTHORS): Remove guarding for building from tarball.
Use set -e to avoid silently failing.
(gen-tarball-version): Use $(SOURCE_DATE_EPOCH) instead of re-generating it
using git; this also works running from a tarball.

Change-Id: I9ebdd28a70837f6a4db610c4816bb283d176e2d9
---
Makefile.am | 60 ++++++++++++++++++++++++++++++++---------------------
1 file changed, 36 insertions(+), 24 deletions(-)

Toggle diff (98 lines)
diff --git a/Makefile.am b/Makefile.am
index ca3fa0a693..af08bc546f 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -955,6 +955,20 @@ guix-binary.%.tar.xz:
guix` ; \
cp "$$tarball" "$@.tmp" ; mv "$@.tmp" "$@"
+# The `dist' target has other dependencies when building from Git
+# to assert and achieve reproducibility.
+if in_git_p
+
+# The dependency on dist-doc-pot-update is to clean possibly stale doc and po
+# files and only then generate the .pot files, which are not checked in.
+dist: dist-doc-pot-update
+dist-doc-pot-update: auto-clean
+ $(MAKE) guile$(EXEEXT)
+ $(MAKE) -C po/guix all
+ $(MAKE) -C po/packages all
+ $(MAKE) doc-pot-update
+
+dist-hook: gen-ChangeLog gen-AUTHORS
# Assert that Autotools cache is up to date with Git, by checking
# PACKAGE_VERSION against HEAD. Indented to get past Automake.
@@ -969,20 +983,20 @@ guix-binary.%.tar.xz:
$(error Cannot create reproducible tarball)
else
$(warning Tarball will be irreproducible; distdir will not get removed!)
- endif
- endif
- endif
+ endif # !GUIX_ALLOW_IRREPRODUCIBLE_TARBALL
+ endif # PACKAGE_VERSION != git_version
+ endif # MAKECMDGOALS dist
-# The dependency on dist-doc-pot-update is to clean possibly stale doc and po
-# files and only then generate the .pot files, which are not checked in.
-dist: dist-doc-pot-update
-dist-doc-pot-update: auto-clean
- $(MAKE) guile$(EXEEXT)
- $(MAKE) -C po/guix all
- $(MAKE) -C po/packages all
- $(MAKE) doc-pot-update
+else # !in_git_p
+
+dist: doc-pot-update
-dist-hook: gen-ChangeLog gen-AUTHORS gen-tarball-version
+ ifeq ($(MAKECMDGOALS),dist)
+$(warning Not using Git, tarball will likely be irreproducible!)
+ endif # MAKECMDGOALS dist
+endif # !in_git_p
+
+dist-hook: gen-tarball-version
dist-hook: assert-no-store-file-names
distcheck-hook: assert-binaries-available assert-final-inputs-self-contained
@@ -994,27 +1008,25 @@ $(top_srcdir)/.version: config.status
gen-tarball-version:
echo $(VERSION) > "$(distdir)/.tarball-version"
- git show HEAD --format=%ct --no-patch > $(distdir)/.tarball-timestamp
+ echo $(SOURCE_DATE_EPOCH) > $(distdir)/.tarball-timestamp
gen-ChangeLog:
- $(AM_V_GEN)if test -e .git; then \
- export LC_ALL=en_US.UTF-8; \
- export TZ=UTC0; \
- $(top_srcdir)/build-aux/gitlog-to-changelog \
- > $(distdir)/ChangeLog.tmp; \
- rm -f $(distdir)/ChangeLog; \
- mv $(distdir)/ChangeLog.tmp $(distdir)/ChangeLog; \
- fi
+ $(AM_V_GEN)set -e; \
+ export LC_ALL=en_US.UTF-8; \
+ export TZ=UTC0; \
+ $(top_srcdir)/build-aux/gitlog-to-changelog \
+ > $(distdir)/ChangeLog.tmp; \
+ rm -f $(distdir)/ChangeLog; \
+ mv $(distdir)/ChangeLog.tmp $(distdir)/ChangeLog;
gen-AUTHORS:
- $(AM_V_GEN)if test -e .git; then \
+ $(AM_V_GEN)set -e; \
rm -f "$(distdir)/AUTHORS"; \
export LC_ALL=en_US.UTF-8; \
export TZ=UTC0; \
$(top_builddir)/pre-inst-env "$(GUILE)" \
"$(top_srcdir)/build-aux/generate-authors.scm" \
- "$(top_srcdir)" "$(distdir)/AUTHORS"; \
- fi
+ "$(top_srcdir)" "$(distdir)/AUTHORS";
# Like 'dist', but regenerate 'configure' so we get an up-to-date
# 'PACKAGE_VERSION' string. (In Gnulib, 'GNUmakefile' has a special trick to
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 21:08 +0200
[PATCH v4 4/6] maint: Generate doc/version[-LANG].texi using `mdate-from-git.scm'.
(address . 70380@debbugs.gnu.org)
5ab984e73310715a4b1d8b671c84eddcd5ccd699.1713379858.git.janneke@gnu.org
This replaces Automake's `build-aux/mdate-sh' with our own
`build-aux/mdate-from-git.scm' to use reproducible timestamps from Git
instead.

* build-aux/mdate-from-git.scm: New script.
* bootstrap: Use it to replace build-aux/mdate-sh.
* Makefile.am (EXTRA_DIST): Add it.

Change-Id: I17d0a7de9ffea397129c0db1728f86e28a4e245f
---
Makefile.am | 1 +
bootstrap | 8 +++-
build-aux/mdate-from-git.scm | 87 ++++++++++++++++++++++++++++++++++++
3 files changed, 95 insertions(+), 1 deletion(-)
create mode 100755 build-aux/mdate-from-git.scm

Toggle diff (123 lines)
diff --git a/Makefile.am b/Makefile.am
index af08bc546f..92a7a85a01 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -724,6 +724,7 @@ EXTRA_DIST += \
build-aux/config.rpath \
build-aux/convert-xref.scm \
build-aux/generate-authors.scm \
+ build-aux/mdate-from-git.scm \
build-aux/test-driver.scm \
build-aux/update-NEWS.scm \
build-aux/update-guix-package.scm \
diff --git a/bootstrap b/bootstrap
index de024aeaa5..5bf83175e5 100755
--- a/bootstrap
+++ b/bootstrap
@@ -24,4 +24,10 @@ for lang in ${langs}; do
fi
done
-exec autoreconf -vfi
+autoreconf -vfi
+
+# Replace Automake's build-aux/mdate-sh with build-aux/mdate-from-git, our
+# own, reproducible version.
+chmod +w build-aux/mdate-sh
+rm -f build-aux/mdate-sh
+ln -s mdate-from-git.scm build-aux/mdate-sh
diff --git a/build-aux/mdate-from-git.scm b/build-aux/mdate-from-git.scm
new file mode 100755
index 0000000000..a9e36031b4
--- /dev/null
+++ b/build-aux/mdate-from-git.scm
@@ -0,0 +1,87 @@
+#! /bin/sh
+# -*-scheme-*-
+export LANG=C LANGUAGE=C LC_TIME=C
+export TZ=UTC0
+exec guile --no-auto-compile -L $srcdir -C $srcdir -e '(mdate-from-git)' -s "$0" "$@"
+!#
+
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Janneke Nieuwenhuizen <janneke@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; This program is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; This program is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+;;;; Commentary:
+;;;
+;;; Usage: mdate-from-git.scm FILE
+;;;
+;;; This script is compatible with Automake's `mdate-sh' but uses the timestamp
+;;; from Git instead of from the file system. Also, it can be appended to
+;;; mdate-sh.
+
+;;; As a special exception for Guix, it caters for doc/guix.LANG.texi files that
+;;; are not stored in Git, by using po/doc/guix-manual.LANG.po for the Git
+;;; timestamp. Test doing something like:
+;;;
+;;; build-aux/mdate-from-git.scm doc/guix.de.texi
+;;;
+;;;; Code:
+
+(define-module (mdate-from-git)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 popen)
+ #:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
+ #:export (main))
+
+(define (pipe-command command)
+ (let* ((port (apply open-pipe* OPEN_READ command))
+ (output (read-string port)))
+ (close-port port)
+ output))
+
+(define (guix.LANG.texi->guix-manual.LANG.po file-name)
+ "Translated manuals doc/guix.LANG.texi are not tracked in Git and are
+generated from po/doc/guix-manual.LANG.po. For such an untraced .TEXI file,
+return its .PO counterpart."
+ (let ((m (string-match "doc/guix.([^.]+).texi" file-name)))
+ (if (not m) file-name
+ (let ((lang (match:substring m 1)))
+ (format #f "po/doc/guix-manual.~a.po" lang)))))
+
+
+;;;
+;;; Entry point.
+;;;
+(define (main args)
+ (match args
+ ((script file-name)
+ (let* ((command `("git" "ls-files" "--error-unmatch" "--" ,file-name))
+ (tracked? (zero? (with-error-to-port (%make-void-port "w")
+ (lambda _ (apply system* command)))))
+ (file-name (if tracked? file-name
+ (guix.LANG.texi->guix-manual.LANG.po file-name)))
+ (command `("git" "log" "--pretty=format:%ct" "-n1" "--" ,file-name))
+ (timestamp (with-error-to-port (%make-void-port "w")
+ (lambda _ (pipe-command command))))
+ (source-date-epoch (or (getenv "SOURCE_DATE_EPOCH") "1"))
+ (timestamp (if (string-null? timestamp) source-date-epoch
+ timestamp))
+ (time (gmtime (string->number timestamp)))
+ (d-m-y (strftime "%-d %B %Y" time)))
+ (display d-m-y)))
+ (_
+ (format (current-error-port) "Usage: mdate-from-git.scm FILE\n")
+ (exit 2))))
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 21:08 +0200
[PATCH v4 5/6] Revert "maint: Generate 'doc/version-LANG.texi' reproducibly."
(address . 70380@debbugs.gnu.org)
cd3be58d80633491d6e1ca74e27756ac461a3e8d.1713379858.git.janneke@gnu.org
Using `build-aux/mdate-from-git.scm' makes this no longer necessary.

This reverts commit 0e4ead187d83a958ca0deb54857c04967e84d68b.

Change-Id: I9177828f90fa7f7e256bc72fdff35a2fab355780
---
doc/local.mk | 40 +++++++++++++++++-----------------------
1 file changed, 17 insertions(+), 23 deletions(-)

Toggle diff (56 lines)
diff --git a/doc/local.mk b/doc/local.mk
index 130f40ece9..77d48902b6 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -260,32 +260,26 @@ endif
# Reproducible tarball
-# Define a rule to build `version[LANG].texi' reproducibly using metadata from
-# Git rather than using metadata from the filesystem.
-define version.texi-from-git
-$(srcdir)/doc/stamp-$(1): $(srcdir)/$(2) $(top_srcdir)/configure
- $$(AM_V_GEN)set -e; \
- export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
- export TZ=UTC0; \
- timestamp="$$$$(git log --pretty=format:%ct -n1 -- "$$<" \
- 2>/dev/null \
- || echo $$(SOURCE_DATE_EPOCH))" \
- dmy=$$$$(date --date="@$$$$timestamp" "+%-d %B %Y"); \
- my=$$$$(date --date="@$$$$timestamp" "+%B %Y"); \
- { echo "@set UPDATED $$$$dmy"; \
- echo "@set UPDATED-MONTH $$$$my"; \
- echo "@set EDITION $$$(VERSION)"; \
- echo "@set VERSION $$$(VERSION)"; } > "$$@-t"; \
- mv "$$@-t" "$$@"; \
- cp -p "$$@" "$$(srcdir)/doc/version$(3).texi"
-endef
-
-# Generate rules for stamp-vti and stamp-N that create version.texi and
-# version-LANG.texi to override the Autotools versions that use timestamps
-# embedded in the file-system. These are expected to generate warnings:
+# Generate 'version.texi' reproducibly using metadata from Git rather than
+# using metadata from the filesystem. This is expected to generate warnings:
#
# Makefile:7376: warning: overriding recipe for target 'doc/stamp-vti'
# Makefile:5098: warning: ignoring old recipe for target 'doc/stamp-vti'
+$(srcdir)/doc/stamp-vti: $(srcdir)/doc/guix.texi $(top_srcdir)/configure
+ $$(AM_V_GEN)set -e; \
+ export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
+ export TZ=UTC0; \
+ timestamp=$$(git log --pretty=format:%ct -n1 -- $< 2>/dev/null \
+ || echo $(SOURCE_DATE_EPOCH)) \
+ dmy=$$(date --date="@$$timestamp" "+%-d %B %Y"); \
+ my=$$(date --date="@$$timestamp" "+%B %Y"); \
+ { echo "@set UPDATED $$dmy"; \
+ echo "@set UPDATED-MONTH $$my"; \
+ echo "@set EDITION $(VERSION)"; \
+ echo "@set VERSION $(VERSION)"; } > $@-t; \
+ mv $@-t $@; \
+ cp $@ $(srcdir)/doc/version.texi
+
i:=0
$(eval $(call version.texi-from-git,vti,doc/guix.texi,))
$(foreach lang, $(MANUAL_LANGUAGES), \
--
2.41.0
J
J
Janneke Nieuwenhuizen wrote on 17 Apr 21:08 +0200
[PATCH v4 6/6] Revert "maint: Generate 'doc/version.texi' reproducibly."
(address . 70380@debbugs.gnu.org)
1d8520a4b963a6e9c0d638c5a1eb43de122b20eb.1713379858.git.janneke@gnu.org
Using `build-aux/mdate-from-git.scm' makes this no longer necessary.

This reverts commit e73ea7bd64f64709c71f89dfb111cf3e8ada3771.

Change-Id: I29d1e36b13d255e5a65b7348e7ae4f2b2c24a518
---
doc/local.mk | 26 --------------------------
1 file changed, 26 deletions(-)

Toggle diff (39 lines)
diff --git a/doc/local.mk b/doc/local.mk
index 77d48902b6..1d94e3c758 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -260,32 +260,6 @@ endif
# Reproducible tarball
-# Generate 'version.texi' reproducibly using metadata from Git rather than
-# using metadata from the filesystem. This is expected to generate warnings:
-#
-# Makefile:7376: warning: overriding recipe for target 'doc/stamp-vti'
-# Makefile:5098: warning: ignoring old recipe for target 'doc/stamp-vti'
-$(srcdir)/doc/stamp-vti: $(srcdir)/doc/guix.texi $(top_srcdir)/configure
- $$(AM_V_GEN)set -e; \
- export LANG=C LANGUAGE=C LC_ALL=C LC_TIME=C; \
- export TZ=UTC0; \
- timestamp=$$(git log --pretty=format:%ct -n1 -- $< 2>/dev/null \
- || echo $(SOURCE_DATE_EPOCH)) \
- dmy=$$(date --date="@$$timestamp" "+%-d %B %Y"); \
- my=$$(date --date="@$$timestamp" "+%B %Y"); \
- { echo "@set UPDATED $$dmy"; \
- echo "@set UPDATED-MONTH $$my"; \
- echo "@set EDITION $(VERSION)"; \
- echo "@set VERSION $(VERSION)"; } > $@-t; \
- mv $@-t $@; \
- cp $@ $(srcdir)/doc/version.texi
-
-i:=0
-$(eval $(call version.texi-from-git,vti,doc/guix.texi,))
-$(foreach lang, $(MANUAL_LANGUAGES), \
- $(eval i=$(shell echo $$(($(i)+1)))) \
- $(eval $(call version.texi-from-git,$(i),po/doc/guix-manual.$(lang).po,-$(lang))))
-
DIST_CONFIGURE_FLAGS = \
--localstatedir=/var \
--sysconfdir=/etc
--
2.41.0
P
P
pelzflorian (Florian Pelz) wrote on 18 Apr 14:01 +0200
Re: [bug#70380] [PATCH v4 2/6] maint: Support `make doc-po-update' from tarball.
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 70380@debbugs.gnu.org)
877cguubgd.fsf@pelzflorian.de
Hi Jan. There is a typo in the 2/6 top commit message, where `make
doc-po-update' must be `make doc-pot-update' with a t. Which I believe
is also what you meant when in your non-patch V4 0/6 response, you spoke
of update-po.

Also yet one more problem (perhaps) I’ve discovered that “info
doc/guix.info” now says:

This document describes GNU Guix version , a functional package

The version number is missing, but the official 1.4.0 tarball’s
doc/guix.info says:

This document describes GNU Guix version 1.4.0, a functional package

Could you fix this, too? Or do I make a mistake and would a real
release display properly?


Otherwise, all LGTM now and again, thank you!

Regards,
Florian
J
J
Janneke Nieuwenhuizen wrote on 18 Apr 20:50 +0200
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 70380@debbugs.gnu.org)
87le5asdy6.fsf@gnu.org
pelzflorian (Florian Pelz) writes:

Hi Florian,

Toggle quote (3 lines)
> Hi Jan. There is a typo in the 2/6 top commit message, where `make
> doc-po-update' must be `make doc-pot-update' with a t.

Thanks, fixed.

Toggle quote (3 lines)
> Which I believe is also what you meant when in your non-patch V4 0/6
> response, you spoke of update-po.

In V4 0/6 I was thinking about make -C po/{guix,packages} update-po,
but yeah. Same difference I guess ;)

Toggle quote (4 lines)
> Also yet one more problem (perhaps) I’ve discovered that “info
> doc/guix.info” now says:
>
> This document describes GNU Guix version , a functional package
[..]
Toggle quote (3 lines)
> Could you fix this, too? Or do I make a mistake and would a real
> release display properly?

If you can help me reproduce this problem, sure. I tried several things
but haven't been able to reproduce it yet. Every time I get

Toggle snippet (4 lines)
This document describes GNU Guix version 1.3.0.58022-1d8520, a
functional package management tool written for the GNU system.

What recipe did you use? I'm probably doing the same thing to test and
might well be missing something.

Toggle quote (2 lines)
> Otherwise, all LGTM now and again, thank you!

Yay!

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
P
P
pelzflorian (Florian Pelz) wrote on 19 Apr 13:34 +0200
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)(address . 70380@debbugs.gnu.org)
87il0dy4a5.fsf@pelzflorian.de
Hi,

Janneke Nieuwenhuizen <janneke@gnu.org> writes:
Toggle quote (18 lines)
> pelzflorian (Florian Pelz) writes:
>> Also yet one more problem (perhaps) I’ve discovered that “info
>> doc/guix.info” now says:
>>
>> This document describes GNU Guix version , a functional package
> [..]
>> Could you fix this, too? Or do I make a mistake and would a real
>> release display properly?
>
> If you can help me reproduce this problem, sure. I tried several things
> but haven't been able to reproduce it yet. Every time I get
>
> This document describes GNU Guix version 1.3.0.58022-1d8520, a
> functional package management tool written for the GNU system.
>
> What recipe did you use? I'm probably doing the same thing to test and
> might well be missing something.

From a “make dist”-generated tarball, I ran “./bootstrap”, “./configure
--sysconfdir=/etc --localstatedir=/var” and “make”.

From the 1.4.0 source tarball, this leads to “info doc/guix.info”
displaying 1.4.0, but not from a “make dist”-generated tarball.

Thank you for trying reproducing the error.

Regards,
Florian
J
J
Janneke Nieuwenhuizen wrote on 19 Apr 16:47 +0200
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 70380-done@debbugs.gnu.org)
87ttjxwgrp.fsf@gnu.org
pelzflorian (Florian Pelz) writes:

Hi Florian,

Toggle quote (4 lines)
> Janneke Nieuwenhuizen <janneke@gnu.org> writes:
>> pelzflorian (Florian Pelz) writes:
>>> Also yet one more problem (perhaps) I’ve discovered that “info
>>> doc/guix.info” now says:
[..]
Toggle quote (6 lines)
>> What recipe did you use? I'm probably doing the same thing to test and
>> might well be missing something.
>
> From a “make dist”-generated tarball, I ran “./bootstrap”, “./configure
> --sysconfdir=/etc --localstatedir=/var” and “make”.

Ah! I said before that `git-version-gen' was broken when not run from a
tarball...but it turns out to work fine...as long as you distribute it ;)

I've added a patch to distribute it and that also fixes all the

Toggle snippet (3 lines)
sh: line 1: build-aux/git-version-gen: No such file or directory

noise.

Toggle quote (3 lines)
> From the 1.4.0 source tarball, this leads to “info doc/guix.info”
> displaying 1.4.0, but not from a “make dist”-generated tarball.

Yeah, I can see that now. AFAICS that was more of a "happy
coincidence". We didn't distribute git-version-gen, so after running
./bootstrap, also in the 1.4.0 tarball, ./configure has

Toggle snippet (5 lines)
# Define the identity of the package.
PACKAGE='guix'
VERSION=''

I cannot say that I completely understand why in 1.4.0 guix.info still
has a non-empty version after running make. It must have something to
do with timestamp or dependency magic causing doc/version.texi not to be
regerenated, even after running ./bootstrap (although it depends on
configure

Toggle snippet (4 lines)
$(srcdir)/doc/version.texi: $(srcdir)/doc/stamp-vti
$(srcdir)/doc/stamp-vti: doc/guix.texi $(top_srcdir)/configure

which we just regenerated. Oh well, we have a proper fix now.

Thanks a lot for your reviews!

Pushed to master as ba52975ea91af49e8e6a436438a578589a209ecc

Greetings,
Janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
Closed
?