[PATCH core-updates 1/?] build: substitute: Don't fail silently.

OpenSubmitted by Jakub Kądziołka.
Details
One participant
  • Jakub Kądziołka
Owner
unassigned
Severity
normal
J
J
Jakub Kądziołka wrote 5 days ago
(address . guix-patches@gnu.org)(address . mbakke@fastmail.com)
20200630220913.14665-1-kuba@kadziolka.net
* guix/build/utils.scm (substitute, substitute*)[require-matches?]: New argument.---Of course, this will require some changes in the build recipes. However,many of the bugs I have seen before could've been prevented by thisbehavior, so I believe it is worth it.
I am currently running 'guix build hello' with these changes; I willsend any follow-up fixes needed to this bug#.
guix/build/utils.scm | 47 +++++++++++++++++++++++++++++++++----------- 1 file changed, 35 insertions(+), 12 deletions(-)
Toggle diff (96 lines)diff --git a/guix/build/utils.scm b/guix/build/utils.scmindex dc55c6745d..1bfb774c60 100644--- a/guix/build/utils.scm+++ b/guix/build/utils.scm@@ -6,6 +6,7 @@ ;;; Copyright © 2018 Arun Isaac <arunisaac@systemreboot.net> ;;; Copyright © 2018, 2019 Ricardo Wurmus <rekado@elephly.net> ;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>+;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net> ;;; ;;; This file is part of GNU Guix. ;;;@@ -747,29 +748,42 @@ PROC's result is returned." (lambda (key . args) (false-if-exception (delete-file template)))))) -(define (substitute file pattern+procs)+(define* (substitute file pattern+procs #:key (require-matches? #t)) "PATTERN+PROCS is a list of regexp/two-argument-procedure pairs. For each line of FILE, and for each PATTERN that it matches, call the corresponding PROC as (PROC LINE MATCHES); PROC must return the line that will be written as a substitution of the original line. Be careful about using '$' to match the-end of a line; by itself it won't match the terminating newline of a line."- (let ((rx+proc (map (match-lambda- (((? regexp? pattern) . proc)- (cons pattern proc))- ((pattern . proc)- (cons (make-regexp pattern regexp/extended)- proc)))- pattern+procs)))+end of a line; by itself it won't match the terminating newline of a line.++By default, SUBSTITUTE will raise a &message condition if one of the patterns+fails to match. If a lack of matches is acceptable, pass #:require-matches? #f+to disable this check."+ (let ((rx+proc (map (match-lambda+ (((? regexp? pattern) . proc)+ (cons* #f "<unknown>" pattern proc))+ ((pattern . proc)+ (cons* #f pattern (make-regexp pattern regexp/extended)+ proc)))+ pattern+procs))) (with-atomic-file-replacement file (lambda (in out) (let loop ((line (read-line in 'concat))) (if (eof-object? line)- #t+ (when require-matches?+ (for-each+ (match-lambda+ ((#f pat . _)+ (raise (condition+ (&message+ (message (format #f "substitute: ~a: pattern failed to match: ~a" file pat))))))+ ((#t . _) #t))+ rx+proc)) (let ((line (fold (lambda (r+p line) (match r+p- ((regexp . proc)+ ((_ _ regexp . proc) (match (list-matches regexp line) ((and m+ (_ _ ...))+ (set-car! r+p #t) (proc line m+)) (_ line))))) line@@ -814,9 +828,17 @@ match substring. Alternatively, FILE may be a list of file names, in which case they are all subject to the substitutions. +By default, SUBSTITUTE* will raise a &message condition if one of the patterns+fails to match on one of the files. If a lack of matches is acceptable,+add #:require-matches? #f after FILE to disable this check.+ Be careful about using '$' to match the end of a line; by itself it won't match the terminating newline of a line." ((substitute* file ((regexp match-var ...) body ...) ...)+ (substitute* file #:require-matches? #t+ ((regexp match-var ...) body ...) ...))+ ((substitute* file #:require-matches? require-matches?+ ((regexp match-var ...) body ...) ...) (let () (define (substitute-one-file file-name) (substitute@@ -840,7 +862,8 @@ match the terminating newline of a line." (begin body ...) (substring l o (match:start m)) r))))))))- ...)))+ ...)+ #:require-matches? require-matches?)) (match file ((files (... ...))-- 2.26.2
J
J
Jakub Kądziołka wrote 5 days ago
[PATCH 2/?] build: bootstrap-configure: Allow lack of matches in substitute.
(address . 42146@debbugs.gnu.org)
20200630221121.16086-1-kuba@kadziolka.net
Matches are not required here, as not every file will use everyvariable.
* guix/build/gnu-bootstrap.scm (bootstrap-configure): Pass #:require-matches? #f to substitute*.--- guix/build/gnu-bootstrap.scm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Toggle diff (31 lines)diff --git a/guix/build/gnu-bootstrap.scm b/guix/build/gnu-bootstrap.scmindex 1cb9dc5512..6ee520e301 100644--- a/guix/build/gnu-bootstrap.scm+++ b/guix/build/gnu-bootstrap.scm@@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2020 Timothy Sample <samplet@ngyro.com>+;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net> ;;; ;;; This file is part of GNU Guix. ;;;@@ -49,7 +50,7 @@ and object directories." (format #t "Configuring ~a~%" template) (let ((target (string-drop-right template 3))) (copy-file template target)- (substitute* target+ (substitute* target #:require-matches? #f (("@VERSION@") version)))) (find-files modules (lambda (fn st)@@ -58,7 +59,7 @@ and object directories." (format #t "Configuring ~a~%" template) (let ((target (string-drop-right template 3))) (copy-file template target)- (substitute* target+ (substitute* target #:require-matches? #f (("@GUILE@") guile) (("@MODDIR@") moddir) (("@GODIR@") godir))-- 2.26.2
J
J
Jakub Kądziołka wrote 5 days ago
[PATCH 3/?] gnu: commencement: Build fix for %bootstrap-mes-rewired.
(address . 42146@debbugs.gnu.org)
20200630221134.16350-1-kuba@kadziolka.net
Not every file has to refer to all dependencies.
* gnu/packages/commencement.scm (%bootstrap-mes-rewired)[builder]: Allow lack of matches in substitute*.--- gnu/packages/commencement.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Toggle diff (23 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scmindex ef250e037b..1c7cf3d810 100644--- a/gnu/packages/commencement.scm+++ b/gnu/packages/commencement.scm@@ -8,6 +8,7 @@ ;;; Copyright © 2018, 2019, 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> ;;; Copyright © 2019, 2020 Marius Bakke <mbakke@fastmail.com> ;;; Copyright © 2020 Timothy Sample <samplet@ngyro.com>+;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net> ;;; ;;; This file is part of GNU Guix. ;;;@@ -283,7 +284,7 @@ pure Scheme to Tar and decompression in one easy step.") (mescc (string-append bin "/mescc")) (module (string-append out "/share/mes/module"))) (define (rewire file)- (substitute* file+ (substitute* file #:require-matches? #f ((mes) out) (("/gnu/store[^ ]+mes-minimal-[^/)}\"]*") out) (("/gnu/store[^ ]+guile-[^/]*/bin/guile") guile)-- 2.26.2
J
J
Jakub Kądziołka wrote 5 days ago
Re: [PATCH core-updates 1/?] build: substitute: Don't fail silently.
(address . 42146@debbugs.gnu.org)
20200630225007.tw2rcfbmzh3wvthf@gravity
The error messages I'm getting from this are quite indirect - there arewrapped in quite a few layers. Is there a better way to report errorsinside a build utility?
ice-9/eval.scm:387:11: In procedure eval:ice-9/eval.scm:387:11: Throw to key `srfi-34' with args `(#<condition &message [message: "substit ute: install.sh: pattern failed to match: -xf"] 11aa480>)'.builder for `/gnu/store/wdyzzh7rkg47hfp434w72ly9nay1yva1-mes-boot-0.22.drv' failed with exit code 1
Regards,Jakub Kądziołka
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE5Xa/ss9usT31cTO54xWnWEYTFWQFAl77wZ8ACgkQ4xWnWEYTFWRIhxAAhBBY1dtp5/7VfFJ2cKcDK8X1xTJucuTGpM5AflHMVarCBdba/7pofenb/cpRR52LfhwQU2HMornopoRIeBnm7DR81vSE4A/V4XO7Z6+BhG2AXAU8biQ6Mzqu1SH0ykadjyxZSOC+JZ4XfqSc/SDb79aIUh5O4Mw22xCir+2ed1zJcHrhjm4jOhMWe3ZPoL1+WDhe26LYdhcjUXciBTf0TEhkb4ZCmHgdbe9cVaLq97gA25MsJRiV8Q1dIoSgf7Fg5FUktl4xCZdMSQ/N3f5mkpS94P6X0EGZQRu3V2KaQH4Yut661W8X00YrF2OtlTu4RHivHk/sdkGWETb9W/EqAqquoSg+KENRzkbbASn2upS79/fG74kPooIqULNcSW0qCqbSUuELSBiRvP/G+AvSlrQ6w0Tfsur8KF2Rkh4f8m4hzCHkccUSP0FZDRsF1sjILlBDjjtb/dgm4uuWr7NOBUddp9s0DvCKpSdHVq9ms7Q9QNyRyZ7sEgzreqFkmF2Mv4IZWISy5K1XITWmL+ypctm8chceoQmwrhSU2YQ+2peiLykB7GxcymrBUkkeULOJ2OZDE5Nl0Mgrz5WOxzN7AuH5C9FAE8JnNJ32mgZsgtPJ1GN4YpL4tmBvKxwHdl55Rbc/PV3+oiVFnsZ6UOt/bL4wF9DqmPV25KXog63pNQ0==BRqx-----END PGP SIGNATURE-----

J
J
Jakub Kądziołka wrote 5 days ago
[PATCH 4/?] gnu: mes-boot: Use verbosity settings integrated into buildsystem
(address . 42146@debbugs.gnu.org)
20200701004234.18050-1-kuba@kadziolka.net
* gnu/packages/commencement.scm (mes-boot): Set the V environment variable. Don't substitute in the install script.--- gnu/packages/commencement.scm | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
Toggle diff (26 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scmindex 1c7cf3d810..295a378d11 100644--- a/gnu/packages/commencement.scm+++ b/gnu/packages/commencement.scm@@ -407,6 +407,7 @@ $MES -e '(mescc)' module/mescc.scm -- \"$@\" (string-append mes "/share/mes/module" ":" dir "/nyacc-0.99.0/module"))+ (setenv "V" "1") (invoke "gash" "configure.sh" (string-append "--prefix=" out) (string-append "--host=i686-linux-gnu")))))@@ -416,11 +417,6 @@ $MES -e '(mescc)' module/mescc.scm -- \"$@\" (delete 'check) (replace 'install (lambda _- (substitute* "install.sh" ; show some progress- ((" -xf") " -xvf")- (("^( *)((cp|mkdir|tar) [^']*[^\\])\n" all space cmd)- (string-append space "echo '" cmd "'\n"- space cmd "\n"))) (invoke "sh" "install.sh") ;; Keep ASCII output, for friendlier comparison and bisection (let* ((out (assoc-ref %outputs "out"))-- 2.26.2
J
J
Jakub Kądziołka wrote 5 days ago
[PATCH 5/?] build-system/gnu: Allow lack of matches in substitution phases.
(address . 42146@debbugs.gnu.org)
20200701004242.18245-1-kuba@kadziolka.net
* guix/build/utils.scm (patch-makefile-SHELL, patch-/usr/bin/file), guix/build/gnu-build-system.scm (patch-dot-desktop-files): Allow lack of matches in substitute*.--- guix/build/gnu-build-system.scm | 3 ++- guix/build/utils.scm | 4 ++-- 2 files changed, 4 insertions(+), 3 deletions(-)
Toggle diff (45 lines)diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scmindex 2e7dff2034..4b96761233 100644--- a/guix/build/gnu-build-system.scm+++ b/guix/build/gnu-build-system.scm@@ -2,6 +2,7 @@ ;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès <ludo@gnu.org> ;;; Copyright © 2018 Mark H Weaver <mhw@netris.org> ;;; Copyright © 2020 Brendan Tildesley <mail@brendan.scot>+;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net> ;;; ;;; This file is part of GNU Guix. ;;;@@ -724,7 +725,7 @@ which cannot be found~%" ;; '.desktop' files contain translations and are always ;; UTF-8-encoded. (with-fluids ((%default-port-encoding "UTF-8"))- (substitute* files+ (substitute* files #:require-matches? #f (("^Exec=([^/[:blank:]\r\n]*)(.*)$" _ binary rest) (string-append "Exec=" (which binary) rest)) (("^TryExec=([^/[:blank:]\r\n]*)(.*)$" _ binary rest)diff --git a/guix/build/utils.scm b/guix/build/utils.scmindex 1bfb774c60..a8218a7743 100644--- a/guix/build/utils.scm+++ b/guix/build/utils.scm@@ -1005,7 +1005,7 @@ When KEEP-MTIME? is true, the atime/mtime of FILE are kept unchanged." (let ((st (stat file))) ;; Consider FILE is using an 8-bit encoding to avoid errors. (with-fluids ((%default-port-encoding #f))- (substitute* file+ (substitute* file #:require-matches? #f (("^ *SHELL[[:blank:]]*:?=[[:blank:]]*([[:graph:]]*/)([[:graph:]]+)(.*)$" _ dir shell args) (let* ((old (string-append dir shell))@@ -1033,7 +1033,7 @@ no replacement 'file' command, doing nothing~%") (let ((st (stat file))) ;; Consider FILE is using an 8-bit encoding to avoid errors. (with-fluids ((%default-port-encoding #f))- (substitute* file+ (substitute* file #:require-matches? #f (("/usr/bin/file") (begin (format (current-error-port)-- 2.26.2
J
J
Jakub Kądziołka wrote 5 days ago
[PATCH 6/6] gnu: glibc-mesboot0: Fixup the fixup-configure phase.
(address . 42146@debbugs.gnu.org)
20200701095523.9501-1-kuba@kadziolka.net
* gnu/packages/commencement.scm (glibc-mesboot0)[fixup-configure]: Don't repeat substitution. Fix indentation.---This is an interesting cleanup commit. I believe this concludes a nicesampling of the consequences of this patch, and thus subsequent fixupsare unlikely to benefit from a review. I will be pushing my work to
https://github.com/NieDzejkob/guix/tree/core-updates
gnu/packages/commencement.scm | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)
Toggle diff (37 lines)diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scmindex 295a378d11..837f4e2ecc 100644--- a/gnu/packages/commencement.scm+++ b/gnu/packages/commencement.scm@@ -1418,19 +1418,17 @@ ac_cv_c_float_format='IEEE (little-endian)' (format (current-error-port) "running ./configure ~a\n" (string-join configure-flags)) (apply invoke "./configure" configure-flags)))- (add-after 'configure 'fixup-configure- (lambda _- (let* ((out (assoc-ref %outputs "out"))- (bash (assoc-ref %build-inputs "bash"))- (shell (string-append bash "/bin/bash")))- (substitute* "config.make"- (("INSTALL = scripts/") "INSTALL = $(..)./scripts/"))- (substitute* "config.make"- (("INSTALL = scripts/") "INSTALL = $(..)./scripts/")- (("BASH = ") (string-append- "SHELL = " shell "- BASH = ")))- #t))))))))+ (add-after 'configure 'fixup-configure+ (lambda _+ (let* ((out (assoc-ref %outputs "out"))+ (bash (assoc-ref %build-inputs "bash"))+ (shell (string-append bash "/bin/bash")))+ (substitute* "config.make"+ (("INSTALL = scripts/") "INSTALL = $(..)./scripts/")+ (("BASH = ") (string-append+ "SHELL = " shell "+BASH = ")))+ #t)))))))) (define gcc-mesboot0 (package-- 2.26.2
?