PETSc is not reproducible

  • Open
  • quality assurance status badge
Details
3 participants
  • Andreas Enge
  • Lars Bilke
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 5 Aug 2022 16:51
(address . bug-guix@gnu.org)
87fsiag2vm.fsf@inria.fr
PETSc 3.16.1 is not reproducible as it captures the kernel version and
number of processors of the machine it’s built on:

Toggle snippet (50 lines)
$ guix describe
Generation 223 Aug 01 2022 00:30:24 (current)
guix ab59155
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: ab59155c5a38dda7efaceb47c7528578fcf0def4
$ guix challenge petsc --diff="$(type -P diff) -ru"
/gnu/store/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1 contents differ:
no local build for '/gnu/store/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1'
https://ci.guix.gnu.org/nar/lzip/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1: 1fqg24x0qq0vww8a21a4ifyxlwy0l7bg8zzrzvg024lk76n54kxh
https://bordeaux.guix.gnu.org/nar/lzip/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1: 162fp2wsxxjjl1pd42522jb3i37hpq8lnfs12qaipzhkzqda5h34
bordeaux.guix.gnu.org 10.3MiB 9.7MiB/s 00:01 [##################] 100.0%diff -ru /tmp/guix-directory.4kiUDI/include/petscmachineinfo.h /tmp/guix-directory.0i2wFi/include/petscmachineinfo.h
--- /tmp/guix-directory.4kiUDI/include/petscmachineinfo.h 1970-01-01 01:00:01.000000000 +0100
+++ /tmp/guix-directory.0i2wFi/include/petscmachineinfo.h 1970-01-01 01:00:01.000000000 +0100
@@ -1,6 +1,6 @@
static const char *petscmachineinfo = "\n"
"-----------------------------------------\n"
-"Machine characteristics: Linux-5.15.10-gnu-x86_64-with-glibc2.33\n"
+"Machine characteristics: Linux-5.15.11-gnu-x86_64-with-glibc2.33\n"
"Using PETSc directory: /gnu/store/4dii69ddwxjiyf2pnay82l4w64s03n60-petsc-3.16.1\n"
"Using PETSc arch: \n"
"-----------------------------------------\n";
Binary files /tmp/guix-directory.4kiUDI/lib/libpetsc.so and /tmp/guix-directory.0i2wFi/lib/libpetsc.so differ
Binary files /tmp/guix-directory.4kiUDI/lib/libpetsc.so.3.16 and /tmp/guix-directory.0i2wFi/lib/libpetsc.so.3.16 differ
Binary files /tmp/guix-directory.4kiUDI/lib/libpetsc.so.3.16.1 and /tmp/guix-directory.0i2wFi/lib/libpetsc.so.3.16.1 differ
diff -ru /tmp/guix-directory.4kiUDI/lib/petsc/conf/petscvariables /tmp/guix-directory.0i2wFi/lib/petsc/conf/petscvariables
--- /tmp/guix-directory.4kiUDI/lib/petsc/conf/petscvariables 1970-01-01 01:00:01.000000000 +0100
+++ /tmp/guix-directory.0i2wFi/lib/petsc/conf/petscvariables 1970-01-01 01:00:01.000000000 +0100
@@ -107,10 +107,10 @@
OMAKE_PRINTDIR = /gnu/store/55cbpsi18mahg131nmiya6km5b4mscfa-make-4.3/bin/make --print-directory
OMAKE = /gnu/store/55cbpsi18mahg131nmiya6km5b4mscfa-make-4.3/bin/make --no-print-directory
MAKE_PAR_OUT_FLG = --output-sync=recurse
-MAKE_NP = 40
-MAKE_TEST_NP = 40
-MAKE_LOAD = 96.0
-NPMAX = 64
+MAKE_NP = 24
+MAKE_TEST_NP = 24
+MAKE_LOAD = 48.0
+NPMAX = 32
SONAME_FUNCTION = $(1).$(SL_LINKER_SUFFIX).$(2)
SL_LINKER_FUNCTION = -shared -fPIC -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -fstack-protector -fvisibility=hidden -g3 -O0 -Wl,-soname,$(call SONAME_FUNCTION,$(notdir $(1)),$(2))
BUILDSHAREDLIB = yes

1 store items were analyzed:
- 0 (0.0%) were identical
- 1 (100.0%) differed
- 0 (0.0%) were inconclusive

Ludo’.
L
L
Lars Bilke wrote on 29 Feb 10:46 +0100
[PATCH v1] gnu: petsc: scrub build machine specifics
(address . 57005@debbugs.gnu.org)(name . Lars Bilke)(address . lars.bilke@ufz.de)
de4e8fff610a489bb8c14b1d260a5bd542dfc7d6.1709200010.git.lars.bilke@ufz.de
'Libraries compiled on' does not have to be removed since PETSc 3.9:

https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f

Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68

---
gnu/packages/maths.scm | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)

Toggle diff (35 lines)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 1f61ee0583..f09cbee48b 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -3455,18 +3455,21 @@ (define-public petsc
(substitute* "config/example_template.py"
(("#!/usr/bin/env bash")
(string-append "#!" (which "bash")))))))
- (add-after 'configure 'clean-local-references
+ (add-after 'install 'clean-local-references
(lambda* (#:key outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out")))
(substitute* (find-files "." "^petsc(conf|machineinfo).h$")
;; Prevent build directory from leaking into compiled code
(((getcwd)) out)
- ;; Scrub timestamp for reproducibility
- ((".*Libraries compiled on.*") ""))
+ ;; Scrub build machine characteristic
+ (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
+ "Machine characteristics: Linux-x.x.x"))
(substitute* (find-files "." "petscvariables")
;; Do not expose build machine characteristics, set to defaults.
- (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
- (("NPMAX = [:digit:]+") "NPMAX = 2")))))
+ (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
+ (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
+ (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
+ (("NPMAX = [[:digit:]]+") "NPMAX = 2")))))
(add-after 'install 'clean-install
;; Try to keep installed files from leaking build directory names.
(lambda* (#:key inputs outputs #:allow-other-keys)

base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
--
2.43.0
L
L
Lars Bilke wrote on 29 Feb 11:00 +0100
(address . 57005@debbugs.gnu.org)
44D266D4-7B4A-4486-AA7E-24CC9E4622B0@ufz.de
This patch makes these files deterministic:

- petscmachineinfo.h
- petscvariables

Unfortunately libpetsc.so.3.16.1 still differs between builds, see e.g. this diffoscope output:


Maybe someone has an idea how to also fix the library?

Sincerely,
Lars

On 29 Feb 2024, at 10:46, Lars Bilke wrote:

Toggle quote (45 lines)
> 'Libraries compiled on' does not have to be removed since PETSc 3.9:
>
> https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
>
> Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
>
> ---
> gnu/packages/maths.scm | 13 ++++++++-----
> 1 file changed, 8 insertions(+), 5 deletions(-)
>
> diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
> index 1f61ee0583..f09cbee48b 100644
> --- a/gnu/packages/maths.scm
> +++ b/gnu/packages/maths.scm
> @@ -3455,18 +3455,21 @@ (define-public petsc
> (substitute* "config/example_template.py"
> (("#!/usr/bin/env bash")
> (string-append "#!" (which "bash")))))))
> - (add-after 'configure 'clean-local-references
> + (add-after 'install 'clean-local-references
> (lambda* (#:key outputs #:allow-other-keys)
> (let ((out (assoc-ref outputs "out")))
> (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
> ;; Prevent build directory from leaking into compiled code
> (((getcwd)) out)
> - ;; Scrub timestamp for reproducibility
> - ((".*Libraries compiled on.*") ""))
> + ;; Scrub build machine characteristic
> + (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
> + "Machine characteristics: Linux-x.x.x"))
> (substitute* (find-files "." "petscvariables")
> ;; Do not expose build machine characteristics, set to defaults.
> - (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
> - (("NPMAX = [:digit:]+") "NPMAX = 2")))))
> + (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
> + (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
> + (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
> + (("NPMAX = [[:digit:]]+") "NPMAX = 2")))))
> (add-after 'install 'clean-install
> ;; Try to keep installed files from leaking build directory names.
> (lambda* (#:key inputs outputs #:allow-other-keys)
>
> base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
> --
> 2.43.0
Attachment: file
Attachment: smime.p7s
A
A
Andreas Enge wrote on 29 Feb 18:48 +0100
Suggestion
(address . 57005@debbugs.gnu.org)
ZeDDeFwO98E7pi1g@jurong
Hello Lars,

thanks for the patch! Concerning "Machine characteristics", I see it only
in petscmachineinfo.h, so we can probably drop looking for petscconf.h
in find-files.

There is already the cleaning phase 'clean-install after 'install.
So I would either suggest to keep the phase 'clean-local-references
where it is, or to merge its content with 'clean-install (I do not see
why one should be preferred over the other).

Andreas
L
L
Lars Bilke wrote on 1 Mar 09:27 +0100
[PATCH v2] gnu: petsc: scrub build machine specifics
(address . 57005@debbugs.gnu.org)(name . Lars Bilke)(address . lars.bilke@ufz.de)
18f29b40d908e9c965f49bc124a505a6278b2a50.1709281622.git.lars.bilke@ufz.de
'Libraries compiled on' does not have to be removed since PETSc 3.9:

https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f

Also merged all add-after 'install phases into `clean-install.

Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
---
gnu/packages/maths.scm | 43 ++++++++++++++++++------------------------
1 file changed, 18 insertions(+), 25 deletions(-)

Toggle diff (75 lines)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 1f61ee0583..4b4d033506 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -3455,30 +3455,34 @@ (define-public petsc
(substitute* "config/example_template.py"
(("#!/usr/bin/env bash")
(string-append "#!" (which "bash")))))))
- (add-after 'configure 'clean-local-references
- (lambda* (#:key outputs #:allow-other-keys)
+ (add-after 'install 'clean-install
+ ;; Try to keep installed files from leaking build directory names.
+ (lambda* (#:key inputs outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out")))
- (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
+ (substitute* (find-files "." "petscmachineinfo.h")
;; Prevent build directory from leaking into compiled code
(((getcwd)) out)
- ;; Scrub timestamp for reproducibility
- ((".*Libraries compiled on.*") ""))
+ ;; Scrub build machine characteristic
+ (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
+ "Machine characteristics: Linux-x.x.x"))
(substitute* (find-files "." "petscvariables")
;; Do not expose build machine characteristics, set to defaults.
- (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
- (("NPMAX = [:digit:]+") "NPMAX = 2")))))
- (add-after 'install 'clean-install
- ;; Try to keep installed files from leaking build directory names.
- (lambda* (#:key inputs outputs #:allow-other-keys)
- (let ((out (assoc-ref outputs "out")))
+ (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
+ (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
+ (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
+ (("NPMAX = [[:digit:]]+") "NPMAX = 2"))
(substitute* (map (lambda (file)
(string-append out "/lib/petsc/conf/" file))
'("petscvariables"))
(((getcwd)) out))
- ;; Make compiler references point to the store
(substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("= (gcc|g\\+\\+|gfortran)" _ compiler)
- (string-append "= " (which compiler))))
+ ;; Do not retain a reference to GCC and other build only inputs.
+ (("([[:graph:]]+)/bin/gcc") "gcc")
+ (("([[:graph:]]+)/bin/g\\+\\+") "g++")
+ (("([[:graph:]]+)/bin/make") "make")
+ (("([[:graph:]]+)/bin/diff") "diff")
+ (("([[:graph:]]+)/bin/sed") "sed")
+ (("([[:graph:]]+)/bin/gfortran") "gfortran"))
;; PETSc installs some build logs, which aren't necessary.
(for-each (lambda (file)
(let ((f (string-append out "/lib/petsc/conf/" file)))
@@ -3490,17 +3494,6 @@ (define-public petsc
"configure-hash"
;; Once installed, should uninstall with Guix
"uninstall.py")))))
- (add-after 'clean-install 'clear-reference-to-compiler
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Do not retain a reference to GCC and other build only inputs.
- (let ((out (assoc-ref outputs "out")))
- (substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("([[:graph:]]+)/bin/gcc") "gcc")
- (("([[:graph:]]+)/bin/g\\+\\+") "g++")
- (("([[:graph:]]+)/bin/make") "make")
- (("([[:graph:]]+)/bin/diff") "diff")
- (("([[:graph:]]+)/bin/sed") "sed")
- (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
(add-after 'install 'move-examples
(lambda* (#:key outputs #:allow-other-keys)
(let* ((out (assoc-ref outputs "out"))

base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
--
2.43.0
L
L
Lars Bilke wrote on 5 Mar 15:58 +0100
[PATCH v3] gnu: petsc: scrub build machine specifics
(address . 57005@debbugs.gnu.org)(name . Lars Bilke)(address . lars.bilke@ufz.de)
65e5bfc8e0e0580eb9c86e1cc0dc8bb89b286a7a.1709650693.git.lars.bilke@ufz.de
'Libraries compiled on' does not have to be removed since PETSc 3.9:

https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f

Also merged all add-after 'install phases into `clean-install.

Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68

sort source files in generated makefiles for reproducibility

Change-Id: I86e5b9cd0d7184019617bd8ae9a23eaf459891d7
---
gnu/packages/maths.scm | 46 +++++++++++++++++++-----------------------
1 file changed, 21 insertions(+), 25 deletions(-)

Toggle diff (85 lines)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 1f61ee0583..b828d5d5cb 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -3447,6 +3447,9 @@ (define-public petsc
(let* ((prefix (assoc-ref outputs "out"))
(flags `(,(string-append "--prefix=" prefix)
,@configure-flags)))
+ ;; sort source files in configure (for reproducibility)
+ (substitute* "config/gmakegen.py"
+ (("join\\(srcs\\[lang\\]\\)") "join(sorted(srcs[lang]))"))
(format #t "build directory: ~s~%" (getcwd))
(format #t "configure flags: ~s~%" flags)
(apply invoke "./configure" flags)
@@ -3455,30 +3458,34 @@ (define-public petsc
(substitute* "config/example_template.py"
(("#!/usr/bin/env bash")
(string-append "#!" (which "bash")))))))
- (add-after 'configure 'clean-local-references
- (lambda* (#:key outputs #:allow-other-keys)
+ (add-after 'install 'clean-install
+ ;; Try to keep installed files from leaking build directory names.
+ (lambda* (#:key inputs outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out")))
- (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
+ (substitute* (find-files "." "petscmachineinfo.h")
;; Prevent build directory from leaking into compiled code
(((getcwd)) out)
- ;; Scrub timestamp for reproducibility
- ((".*Libraries compiled on.*") ""))
+ ;; Scrub build machine characteristic
+ (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
+ "Machine characteristics: Linux-x.x.x"))
(substitute* (find-files "." "petscvariables")
;; Do not expose build machine characteristics, set to defaults.
- (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
- (("NPMAX = [:digit:]+") "NPMAX = 2")))))
- (add-after 'install 'clean-install
- ;; Try to keep installed files from leaking build directory names.
- (lambda* (#:key inputs outputs #:allow-other-keys)
- (let ((out (assoc-ref outputs "out")))
+ (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
+ (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
+ (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
+ (("NPMAX = [[:digit:]]+") "NPMAX = 2"))
(substitute* (map (lambda (file)
(string-append out "/lib/petsc/conf/" file))
'("petscvariables"))
(((getcwd)) out))
- ;; Make compiler references point to the store
(substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("= (gcc|g\\+\\+|gfortran)" _ compiler)
- (string-append "= " (which compiler))))
+ ;; Do not retain a reference to GCC and other build only inputs.
+ (("([[:graph:]]+)/bin/gcc") "gcc")
+ (("([[:graph:]]+)/bin/g\\+\\+") "g++")
+ (("([[:graph:]]+)/bin/make") "make")
+ (("([[:graph:]]+)/bin/diff") "diff")
+ (("([[:graph:]]+)/bin/sed") "sed")
+ (("([[:graph:]]+)/bin/gfortran") "gfortran"))
;; PETSc installs some build logs, which aren't necessary.
(for-each (lambda (file)
(let ((f (string-append out "/lib/petsc/conf/" file)))
@@ -3490,17 +3497,6 @@ (define-public petsc
"configure-hash"
;; Once installed, should uninstall with Guix
"uninstall.py")))))
- (add-after 'clean-install 'clear-reference-to-compiler
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Do not retain a reference to GCC and other build only inputs.
- (let ((out (assoc-ref outputs "out")))
- (substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("([[:graph:]]+)/bin/gcc") "gcc")
- (("([[:graph:]]+)/bin/g\\+\\+") "g++")
- (("([[:graph:]]+)/bin/make") "make")
- (("([[:graph:]]+)/bin/diff") "diff")
- (("([[:graph:]]+)/bin/sed") "sed")
- (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
(add-after 'install 'move-examples
(lambda* (#:key outputs #:allow-other-keys)
(let* ((out (assoc-ref outputs "out"))

base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
--
2.43.0
L
L
Lars Bilke wrote on 5 Mar 16:01 +0100
(address . 57005@debbugs.gnu.org)
AB996BF0-5CDD-4BE5-9715-16C586C24784@ufz.de
v3 of this patch finally makes PETSc fully reproducible!

On 5 Mar 2024, at 15:58, Lars Bilke wrote:

Toggle quote (100 lines)
> 'Libraries compiled on' does not have to be removed since PETSc 3.9:
>
> https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
>
> Also merged all add-after 'install phases into `clean-install.
>
> Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
>
> sort source files in generated makefiles for reproducibility
>
> Change-Id: I86e5b9cd0d7184019617bd8ae9a23eaf459891d7
> ---
> gnu/packages/maths.scm | 46 +++++++++++++++++++-----------------------
> 1 file changed, 21 insertions(+), 25 deletions(-)
>
> diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
> index 1f61ee0583..b828d5d5cb 100644
> --- a/gnu/packages/maths.scm
> +++ b/gnu/packages/maths.scm
> @@ -3447,6 +3447,9 @@ (define-public petsc
> (let* ((prefix (assoc-ref outputs "out"))
> (flags `(,(string-append "--prefix=" prefix)
> ,@configure-flags)))
> + ;; sort source files in configure (for reproducibility)
> + (substitute* "config/gmakegen.py"
> + (("join\\(srcs\\[lang\\]\\)") "join(sorted(srcs[lang]))"))
> (format #t "build directory: ~s~%" (getcwd))
> (format #t "configure flags: ~s~%" flags)
> (apply invoke "./configure" flags)
> @@ -3455,30 +3458,34 @@ (define-public petsc
> (substitute* "config/example_template.py"
> (("#!/usr/bin/env bash")
> (string-append "#!" (which "bash")))))))
> - (add-after 'configure 'clean-local-references
> - (lambda* (#:key outputs #:allow-other-keys)
> + (add-after 'install 'clean-install
> + ;; Try to keep installed files from leaking build directory names.
> + (lambda* (#:key inputs outputs #:allow-other-keys)
> (let ((out (assoc-ref outputs "out")))
> - (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
> + (substitute* (find-files "." "petscmachineinfo.h")
> ;; Prevent build directory from leaking into compiled code
> (((getcwd)) out)
> - ;; Scrub timestamp for reproducibility
> - ((".*Libraries compiled on.*") ""))
> + ;; Scrub build machine characteristic
> + (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
> + "Machine characteristics: Linux-x.x.x"))
> (substitute* (find-files "." "petscvariables")
> ;; Do not expose build machine characteristics, set to defaults.
> - (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
> - (("NPMAX = [:digit:]+") "NPMAX = 2")))))
> - (add-after 'install 'clean-install
> - ;; Try to keep installed files from leaking build directory names.
> - (lambda* (#:key inputs outputs #:allow-other-keys)
> - (let ((out (assoc-ref outputs "out")))
> + (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
> + (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
> + (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
> + (("NPMAX = [[:digit:]]+") "NPMAX = 2"))
> (substitute* (map (lambda (file)
> (string-append out "/lib/petsc/conf/" file))
> '("petscvariables"))
> (((getcwd)) out))
> - ;; Make compiler references point to the store
> (substitute* (string-append out "/lib/petsc/conf/petscvariables")
> - (("= (gcc|g\\+\\+|gfortran)" _ compiler)
> - (string-append "= " (which compiler))))
> + ;; Do not retain a reference to GCC and other build only inputs.
> + (("([[:graph:]]+)/bin/gcc") "gcc")
> + (("([[:graph:]]+)/bin/g\\+\\+") "g++")
> + (("([[:graph:]]+)/bin/make") "make")
> + (("([[:graph:]]+)/bin/diff") "diff")
> + (("([[:graph:]]+)/bin/sed") "sed")
> + (("([[:graph:]]+)/bin/gfortran") "gfortran"))
> ;; PETSc installs some build logs, which aren't necessary.
> (for-each (lambda (file)
> (let ((f (string-append out "/lib/petsc/conf/" file)))
> @@ -3490,17 +3497,6 @@ (define-public petsc
> "configure-hash"
> ;; Once installed, should uninstall with Guix
> "uninstall.py")))))
> - (add-after 'clean-install 'clear-reference-to-compiler
> - (lambda* (#:key inputs outputs #:allow-other-keys)
> - ;; Do not retain a reference to GCC and other build only inputs.
> - (let ((out (assoc-ref outputs "out")))
> - (substitute* (string-append out "/lib/petsc/conf/petscvariables")
> - (("([[:graph:]]+)/bin/gcc") "gcc")
> - (("([[:graph:]]+)/bin/g\\+\\+") "g++")
> - (("([[:graph:]]+)/bin/make") "make")
> - (("([[:graph:]]+)/bin/diff") "diff")
> - (("([[:graph:]]+)/bin/sed") "sed")
> - (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
> (add-after 'install 'move-examples
> (lambda* (#:key outputs #:allow-other-keys)
> (let* ((out (assoc-ref outputs "out"))
>
> base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
> --
> 2.43.0
Attachment: smime.p7s
L
L
Lars Bilke wrote on 6 Mar 12:51 +0100
[PATCH v4] gnu: petsc: scrub build machine specifics
(address . 57005@debbugs.gnu.org)(name . Lars Bilke)(address . lars.bilke@ufz.de)
8d1edc3769b3994fc2a7be34470509e7f45aee31.1709725874.git.lars.bilke@ufz.de
'Libraries compiled on' does not have to be removed since PETSc 3.9:

https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f

Also merged all add-after 'install phases into `clean-install.

Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68

sort source files in generated makefiles for reproducibility

Change-Id: I86e5b9cd0d7184019617bd8ae9a23eaf459891d7
---
gnu/packages/maths.scm | 56 ++++++++++++++++++------------------------
1 file changed, 24 insertions(+), 32 deletions(-)

Toggle diff (89 lines)
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 1f61ee0583..1ea5ccee74 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -3447,38 +3447,41 @@ (define-public petsc
(let* ((prefix (assoc-ref outputs "out"))
(flags `(,(string-append "--prefix=" prefix)
,@configure-flags)))
+ ;; sort source files in configure (for reproducibility)
+ (substitute* "config/gmakegen.py"
+ (("join\\(srcs\\[lang\\]\\)") "join(sorted(srcs[lang]))"))
(format #t "build directory: ~s~%" (getcwd))
(format #t "configure flags: ~s~%" flags)
(apply invoke "./configure" flags)
-
+ ;; Prevent build directory Linux version from leaking into
+ ;; compiled code
+ (substitute* (find-files "." "petscmachineinfo.h")
+ (((getcwd)) out)
+ ;; Scrub build machine characteristic
+ (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
+ "Machine characteristics: Linux-x.x.x"))
;; Generate test scripts with the right shebang.
(substitute* "config/example_template.py"
(("#!/usr/bin/env bash")
(string-append "#!" (which "bash")))))))
- (add-after 'configure 'clean-local-references
- (lambda* (#:key outputs #:allow-other-keys)
- (let ((out (assoc-ref outputs "out")))
- (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
- ;; Prevent build directory from leaking into compiled code
- (((getcwd)) out)
- ;; Scrub timestamp for reproducibility
- ((".*Libraries compiled on.*") ""))
- (substitute* (find-files "." "petscvariables")
- ;; Do not expose build machine characteristics, set to defaults.
- (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
- (("NPMAX = [:digit:]+") "NPMAX = 2")))))
(add-after 'install 'clean-install
- ;; Try to keep installed files from leaking build directory names.
(lambda* (#:key inputs outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out")))
- (substitute* (map (lambda (file)
- (string-append out "/lib/petsc/conf/" file))
- '("petscvariables"))
- (((getcwd)) out))
- ;; Make compiler references point to the store
(substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("= (gcc|g\\+\\+|gfortran)" _ compiler)
- (string-append "= " (which compiler))))
+ ;; Try to keep installed files from leaking build directory names.
+ (((getcwd)) out)
+ ;; Do not expose build machine characteristics, set to defaults.
+ (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
+ (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
+ (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
+ (("NPMAX = [[:digit:]]+") "NPMAX = 2")
+ ;; Do not retain a reference to GCC and other build only inputs.
+ (("([[:graph:]]+)/bin/gcc") "gcc")
+ (("([[:graph:]]+)/bin/g\\+\\+") "g++")
+ (("([[:graph:]]+)/bin/make") "make")
+ (("([[:graph:]]+)/bin/diff") "diff")
+ (("([[:graph:]]+)/bin/sed") "sed")
+ (("([[:graph:]]+)/bin/gfortran") "gfortran"))
;; PETSc installs some build logs, which aren't necessary.
(for-each (lambda (file)
(let ((f (string-append out "/lib/petsc/conf/" file)))
@@ -3490,17 +3493,6 @@ (define-public petsc
"configure-hash"
;; Once installed, should uninstall with Guix
"uninstall.py")))))
- (add-after 'clean-install 'clear-reference-to-compiler
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Do not retain a reference to GCC and other build only inputs.
- (let ((out (assoc-ref outputs "out")))
- (substitute* (string-append out "/lib/petsc/conf/petscvariables")
- (("([[:graph:]]+)/bin/gcc") "gcc")
- (("([[:graph:]]+)/bin/g\\+\\+") "g++")
- (("([[:graph:]]+)/bin/make") "make")
- (("([[:graph:]]+)/bin/diff") "diff")
- (("([[:graph:]]+)/bin/sed") "sed")
- (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
(add-after 'install 'move-examples
(lambda* (#:key outputs #:allow-other-keys)
(let* ((out (assoc-ref outputs "out"))

base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
--
2.43.0
L
L
Lars Bilke wrote on 6 Mar 12:54 +0100
(address . 57005@debbugs.gnu.org)
8412A210-E386-46B9-A3E1-10D9240AA484@ufz.de
v4 should (hopefully) fix all issues regarding reprocuibility of petsc:

I have tested v4 on multiple machines with exact identical output (ran `guix hash -S nar ...` on the packages) and also tested on one machine with `guix build --check`

On 6 Mar 2024, at 12:51, Lars Bilke wrote:

Toggle quote (104 lines)
> 'Libraries compiled on' does not have to be removed since PETSc 3.9:
>
> https: //gitlab.com/petsc/petsc/-/commit/a970bd748978a4e12790be7c36b10e832ec0149f
>
> Also merged all add-after 'install phases into `clean-install.
>
> Change-Id: I41d82339e43187d7dc7e3fc189a338784c363c68
>
> sort source files in generated makefiles for reproducibility
>
> Change-Id: I86e5b9cd0d7184019617bd8ae9a23eaf459891d7
> ---
> gnu/packages/maths.scm | 56 ++++++++++++++++++------------------------
> 1 file changed, 24 insertions(+), 32 deletions(-)
>
> diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
> index 1f61ee0583..1ea5ccee74 100644
> --- a/gnu/packages/maths.scm
> +++ b/gnu/packages/maths.scm
> @@ -3447,38 +3447,41 @@ (define-public petsc
> (let* ((prefix (assoc-ref outputs "out"))
> (flags `(,(string-append "--prefix=" prefix)
> ,@configure-flags)))
> + ;; sort source files in configure (for reproducibility)
> + (substitute* "config/gmakegen.py"
> + (("join\\(srcs\\[lang\\]\\)") "join(sorted(srcs[lang]))"))
> (format #t "build directory: ~s~%" (getcwd))
> (format #t "configure flags: ~s~%" flags)
> (apply invoke "./configure" flags)
> -
> + ;; Prevent build directory Linux version from leaking into
> + ;; compiled code
> + (substitute* (find-files "." "petscmachineinfo.h")
> + (((getcwd)) out)
> + ;; Scrub build machine characteristic
> + (("Machine characteristics: Linux-[0-9]+\\.[0-9]+\\.[0-9]+")
> + "Machine characteristics: Linux-x.x.x"))
> ;; Generate test scripts with the right shebang.
> (substitute* "config/example_template.py"
> (("#!/usr/bin/env bash")
> (string-append "#!" (which "bash")))))))
> - (add-after 'configure 'clean-local-references
> - (lambda* (#:key outputs #:allow-other-keys)
> - (let ((out (assoc-ref outputs "out")))
> - (substitute* (find-files "." "^petsc(conf|machineinfo).h$")
> - ;; Prevent build directory from leaking into compiled code
> - (((getcwd)) out)
> - ;; Scrub timestamp for reproducibility
> - ((".*Libraries compiled on.*") ""))
> - (substitute* (find-files "." "petscvariables")
> - ;; Do not expose build machine characteristics, set to defaults.
> - (("MAKE_NP = [:digit:]+") "MAKE_NP = 2")
> - (("NPMAX = [:digit:]+") "NPMAX = 2")))))
> (add-after 'install 'clean-install
> - ;; Try to keep installed files from leaking build directory names.
> (lambda* (#:key inputs outputs #:allow-other-keys)
> (let ((out (assoc-ref outputs "out")))
> - (substitute* (map (lambda (file)
> - (string-append out "/lib/petsc/conf/" file))
> - '("petscvariables"))
> - (((getcwd)) out))
> - ;; Make compiler references point to the store
> (substitute* (string-append out "/lib/petsc/conf/petscvariables")
> - (("= (gcc|g\\+\\+|gfortran)" _ compiler)
> - (string-append "= " (which compiler))))
> + ;; Try to keep installed files from leaking build directory names.
> + (((getcwd)) out)
> + ;; Do not expose build machine characteristics, set to defaults.
> + (("MAKE_NP = [[:digit:]]+") "MAKE_NP = 2")
> + (("MAKE_TEST_NP = [[:digit:]]+") "MAKE_TEST_NP = 2")
> + (("MAKE_LOAD = [[:digit:]]+\\.[[:digit:]]+") "MAKE_LOAD = 2.0")
> + (("NPMAX = [[:digit:]]+") "NPMAX = 2")
> + ;; Do not retain a reference to GCC and other build only inputs.
> + (("([[:graph:]]+)/bin/gcc") "gcc")
> + (("([[:graph:]]+)/bin/g\\+\\+") "g++")
> + (("([[:graph:]]+)/bin/make") "make")
> + (("([[:graph:]]+)/bin/diff") "diff")
> + (("([[:graph:]]+)/bin/sed") "sed")
> + (("([[:graph:]]+)/bin/gfortran") "gfortran"))
> ;; PETSc installs some build logs, which aren't necessary.
> (for-each (lambda (file)
> (let ((f (string-append out "/lib/petsc/conf/" file)))
> @@ -3490,17 +3493,6 @@ (define-public petsc
> "configure-hash"
> ;; Once installed, should uninstall with Guix
> "uninstall.py")))))
> - (add-after 'clean-install 'clear-reference-to-compiler
> - (lambda* (#:key inputs outputs #:allow-other-keys)
> - ;; Do not retain a reference to GCC and other build only inputs.
> - (let ((out (assoc-ref outputs "out")))
> - (substitute* (string-append out "/lib/petsc/conf/petscvariables")
> - (("([[:graph:]]+)/bin/gcc") "gcc")
> - (("([[:graph:]]+)/bin/g\\+\\+") "g++")
> - (("([[:graph:]]+)/bin/make") "make")
> - (("([[:graph:]]+)/bin/diff") "diff")
> - (("([[:graph:]]+)/bin/sed") "sed")
> - (("([[:graph:]]+)/bin/gfortran") "gfortran")))))
> (add-after 'install 'move-examples
> (lambda* (#:key outputs #:allow-other-keys)
> (let* ((out (assoc-ref outputs "out"))
>
> base-commit: 75bad75367fcf2c289fae3b40dbcc850f92177be
> --
> 2.43.0
Attachment: file
Attachment: smime.p7s
?