[PATCH] monad-repl: Add REPL commands to inspect package arguments.

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Simon Tournier
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 27 Nov 2023 18:19
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
46b62be786928db845547a02f54d16b2635727f9.1701105272.git.ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>

* guix/monad-repl.scm (keyword-argument-value, package-argument-command):
New procedures.
(phases, configure-flags, make-flags): New REPL commands.
* doc/guix.texi (package Reference): Link to “Using Guix Interactively”.
(Defining Package Variants): Add “Tips” quotation.
(Build Phases): Add “Tip” quotation.
(Using Guix Interactively): Document the new REPL commands.

Change-Id: I7049c1d8aa9241e07d7c921aa396e578a1b4ef16
---
doc/guix.texi | 61 +++++++++++++++++++++++++++++++++++++
guix/monad-repl.scm | 74 ++++++++++++++++++++++++++++++++++++++++++---
2 files changed, 131 insertions(+), 4 deletions(-)

Hello!

Someone recently suggested we should have better debugging facilities
for packagers, in particular when dealing with package variants,
inheritance, and so on, which make it harder to understand what’s
going on.

This change partially addresses that problem: new REPL commands
let you inspect the code of #:phases, #:configure-flags, and #:make-flags.

Toggle snippet (33 lines)
scheme@(guix-user)> ,use(guix)
scheme@(guix-user)> ,use(gnu packages mpi)
scheme@(guix-user)> ,use(gnu packages guile)
scheme@(guix-user)> ,phases guile-3.0
$1 = (modify-phases (modify-phases %standard-phases
(add-before 'configure 'pre-configure
(lambda* (#:key inputs #:allow-other-keys)
(let ((bash (assoc-ref inputs "bash")))
(substitute* "module/ice-9/popen.scm"
(("/bin/sh")
(string-append bash "/bin/bash"))) #t))))
(add-before 'check 'disable-stack-overflow-test
(lambda _
(substitute* "test-suite/standalone/test-stack-overflow"
(("!#")
"!#\n(exit 77)\n")))))
scheme@(guix-user)> ,configure-flags openmpi
$2 = `("--enable-mpi-ext=affinity" "--with-sge"
"--enable-memchecker"
"--with-valgrind"
"--with-hwloc=external"
"--with-libevent"
"--enable-mpirun-prefix-by-default"
"--enable-openib-control-hdr-padding"
"--enable-openib-dynamic-sl"
"--enable-openib-udcm"
"--enable-openib-rdmacm"
"--enable-openib-rdmacm-ibaddr"
"--with-pmix=internal"
,(string-append "--with-pmi="
"/gnu/store/h3vmqdgfnfqyk84vc1p3z57dqhrccx7b-slurm-23.02.6"))

Thoughts?

Ludo’.

Toggle diff (203 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 1fd2e21608..7dde9b727b 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -7975,6 +7975,10 @@ package Reference
@code{#:phases}. The @code{#:phases} keyword in particular lets you
modify the set of build phases for your package (@pxref{Build Phases}).
+The REPL has dedicated commands to interactively inspect values of some
+of these arguments, as a convenient debugging aid (@pxref{Using Guix
+Interactively}).
+
@quotation Compatibility Note
Until version 1.3.0, the @code{arguments} field would typically use
@code{quote} (@code{'}) or @code{quasiquote} (@code{`}) and no
@@ -8774,6 +8778,23 @@ Defining Package Variants
applied to implicit inputs as well.
@end deffn
+@quotation Tips
+Understanding what a variant really looks like can be difficult as one
+starts combining the tools shown above. There are several ways to
+inspect a package before attempting to build it that can prove handy:
+
+@itemize
+@item
+You can inspect the package interactively at the REPL, for instance to
+view its inputs, the code of its build phases, or its configure flags
+(@pxref{Using Guix Interactively}).
+
+@item
+When rewriting dependencies, @command{guix graph} can often help
+visualize the changes that are made (@pxref{Invoking guix graph}).
+@end itemize
+@end quotation
+
@node Writing Manifests
@section Writing Manifests
@@ -10585,6 +10606,11 @@ Build Phases
the helpers used by this phase, and for more examples of
@code{modify-phases}.
+@quotation Tip
+You can inspect the code associated with a package's @code{#:phases}
+argument interactively, at the REPL (@pxref{Using Guix Interactively}).
+@end quotation
+
@cindex code staging
@cindex staging, of code
Keep in mind that build phases are code evaluated at the time the
@@ -12763,6 +12789,30 @@ Using Guix Interactively
$5 = ("." ".." "egrep" "fgrep" "grep")
@end example
+As a packager, you may be willing to inspect the build phases or flags
+of a given package; this is particularly useful when relying a lot on
+inheritance to define package variants (@pxref{Defining Package
+Variants}) or when package arguments are a result of some computation,
+both of which can make it harder to foresee what ends up in the package
+arguments. Additional commands let you inspect those package arguments:
+
+@example
+scheme@@(guix-user)> ,phases grep
+$1 = (modify-phases %standard-phases
+ (add-after 'install 'fix-egrep-and-fgrep
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin")))
+ (substitute* (list (string-append bin "/egrep")
+ (string-append bin "/fgrep"))
+ (("^exec grep")
+ (string-append "exec " bin "/grep")))))))
+scheme@@(guix-user)> ,configure-flags findutils
+$2 = (list "--localstatedir=/var")
+scheme@@(guix-user)> ,make-flags binutils
+$3 = '("MAKEINFO=true")
+@end example
+
At a lower-level, a useful command is @code{lower}: it takes a file-like
object and ``lowers'' it into a derivation (@pxref{Derivations}) or a
store file:
@@ -12794,6 +12844,17 @@ Using Guix Interactively
shows build events only, and higher levels print build logs.
@end deffn
+@deffn {REPL command} phases @var{package}
+@deffnx {REPL command} configure-flags @var{package}
+@deffnx {REPL command} make-flags @var{package}
+These REPL commands return the value of one element of the
+@code{arguments} field of @var{package} (@pxref{package Reference}): the
+first one show the staged code associated with @code{#:phases}
+(@pxref{Build Phases}), the second shows the code for
+@code{#:configure-flags}, and @code{,make-flags} returns the code for
+@code{#:make-flags}.
+@end deffn
+
@deffn {REPL command} run-in-store @var{exp}
Run @var{exp}, a monadic expression, through the store monad.
@xref{The Store Monad}, for more information.
diff --git a/guix/monad-repl.scm b/guix/monad-repl.scm
index 8a6053edd5..d6b39112b7 100644
--- a/guix/monad-repl.scm
+++ b/guix/monad-repl.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2014, 2015, 2016, 2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2014-2016, 2022-2023 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -21,13 +21,15 @@ (define-module (guix monad-repl)
#:use-module (guix monads)
#:use-module (guix utils)
#:use-module (guix packages)
+ #:autoload (guix build-system) (bag)
#:use-module (guix status)
- #:autoload (guix gexp) (lower-object)
+ #:autoload (guix gexp) (gexp gexp? lower-gexp lowered-gexp-sexp lower-object)
#:use-module ((guix derivations)
#:select (derivation?
derivation->output-paths built-derivations))
+ #:autoload (guix read-print) (pretty-print-with-comments)
#:use-module (ice-9 match)
- #:use-module (ice-9 pretty-print)
+ #:autoload (ice-9 pretty-print) (pretty-print)
#:use-module (system repl repl)
#:use-module (system repl common)
#:use-module (system repl command)
@@ -138,4 +140,68 @@ (define-meta-command ((enter-store-monad guix) repl)
(repl-option-set! new 'interp #t)
(run-repl new))))
-;;; monad-repl.scm ends here
+
+;;;
+;;; Viewing package arguments.
+;;;
+
+(define (keyword-argument-value args keyword default)
+ "Return the value associated with KEYWORD in ARGS, a keyword/value sequence,
+or DEFAULT if KEYWORD is missing from ARGS."
+ (let loop ((args args))
+ (match args
+ (()
+ default)
+ ((kw value rest ...)
+ (if (eq? kw keyword)
+ value
+ (loop rest))))))
+
+(define (package-argument-command repl form keyword default)
+ "Implement a command that display KEYWORD, a keyword such as #:phases, in
+the arguments of the package FORM evaluates to. Return DEFAULT is KEYWORD is
+missing from those arguments."
+ (match (repl-eval repl form)
+ ((? package? package)
+ (let* ((bag* (bag
+ (inherit (package->bag package))
+ (build (lambda* (name inputs #:rest args)
+ (with-monad %store-monad
+ (return (keyword-argument-value args keyword
+ default))))))))
+ (define phases
+ (parameterize ((%graft? #f))
+ (with-store store
+ (set-build-options store
+ #:print-build-trace #t
+ #:print-extended-build-trace? #t
+ #:multiplexed-build-output? #t)
+ (run-with-store store
+ (mlet %store-monad ((exp (bag->derivation bag*)))
+ (if (gexp? exp)
+ (mlet %store-monad ((gexp (lower-gexp exp)))
+ (return (lowered-gexp-sexp gexp)))
+ (return exp)))))))
+
+ (run-hook before-print-hook phases)
+ (let ((column (port-column (current-output-port))))
+ (pretty-print-with-comments (current-output-port) phases
+ #:indent column)
+ (newline (current-output-port)))))
+ (_
+ (format #t ";; ERROR: This command only accepts package records.~%"))))
+
+(define-meta-command ((phases guix) repl (form))
+ "phases
+Return the build phases of the package defined by FORM."
+ (package-argument-command repl form #:phases #~%standard-phases))
+
+(define-meta-command ((configure-flags guix) repl (form))
+ "configure-flags
+Return the configure flags of the package defined by FORM."
+ (package-argument-command repl form #:configure-flags #~'()))
+
+(define-meta-command ((make-flags guix) repl (form))
+ "make-flags
+Return the make flags of the package defined by FORM."
+ (package-argument-command repl form #:make-flags #~'()))

base-commit: f3173bb901c0eee6c71d066cffab115d27052fc1
--
2.41.0
S
S
Simon Tournier wrote on 28 Nov 2023 11:49
87a5qycfuk.fsf@gmail.com
Hi,

TLDR: All LGTM. :-)

Minor comments.


On Mon, 27 Nov 2023 at 18:19, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (5 lines)
> Someone recently suggested we should have better debugging facilities
> for packagers, in particular when dealing with package variants,
> inheritance, and so on, which make it harder to understand what’s
> going on.

From a quick test:

Toggle snippet (28 lines)
scheme@(guix-user)> ,phases cava
$1 = (modify-phases %standard-phases
(replace 'bootstrap
(lambda _
(setenv "HOME"
(getcwd))
(invoke "sh" "autogen.sh")))
(add-before 'build 'make-cava-ldflags
(lambda _
(mkdir-p (string-append ((@ (guile) getenv)
"out") "/lib"))))
(add-after 'install 'data
(lambda _
(for-each (lambda (file)
(install-file file
(string-append ((@ (guile) getenv)
"out")
"/share/doc/examples")))
(find-files "example_files")))))
scheme@(guix-user)> ,configure-flags cava
$2 = (list (string-append "PREFIX="
((@ (guile) getenv)
"out"))
(string-append "FONT_DIR="
((@ (guile) getenv)
"out") "/share/consolefonts"))

Cool! Very helpful. Much better than, something like:

Toggle snippet (4 lines)
scheme@(guix-user)> ,pp (cdr (memq #:phases (package-arguments cava)))
$3 = (#<gexp (modify-phases %standard-phases (replace (quote bootstrap) (lambda _ (setenv "HOME" (getcwd)) (invoke "sh" "autogen.sh"))) (add-before (quote build) (quote make-cava-ldflags) (lambda _ (mkdir-p (string-append #<gexp-output out> "/lib")))) (add-after (quote install) (quote data) (lambda _ (for-each (lambda (file) (install-file file (string-append #<gexp-output out> "/share/doc/examples"))) (find-files "example_files"))))) gnu/packages/audio.scm:5161:11 7f7258403a50>)

Without drifting, why not the meta-command ,arguments? For example, the
package could also have other specific keywords that maybe I would like
to inspect from the REPL. For example, some packages have their own
keywords as #:julia-package-name.

Well, package-arguments partially does the job:

Toggle snippet (9 lines)
scheme@(guix-user)> ,pp (package-arguments julia-uris)
$5 = (#:julia-package-name
"URIs"
#:julia-package-uuid
"5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
#:phases
#<gexp (modify-phases %standard-phases (add-before (quote check) (quote change-dir) (lambda* (#:key source outputs #:allow-other-keys) (let ((out (assoc-ref outputs "out"))) (chdir (string-append out "/share/julia/loadpath/URIs/test")))))) gnu/packages/julia-xyz.scm:6482:6 7f725a441120>)

Other said, it could be nice to:

1. pretty-print package-arguments by default,
2. and compile the G-expressions.

Hence ,arguments. :-)


Toggle quote (25 lines)
> +As a packager, you may be willing to inspect the build phases or flags
> +of a given package; this is particularly useful when relying a lot on
> +inheritance to define package variants (@pxref{Defining Package
> +Variants}) or when package arguments are a result of some computation,
> +both of which can make it harder to foresee what ends up in the package
> +arguments. Additional commands let you inspect those package arguments:
> +
> +@example
> +scheme@@(guix-user)> ,phases grep
> +$1 = (modify-phases %standard-phases
> + (add-after 'install 'fix-egrep-and-fgrep
> + (lambda* (#:key outputs #:allow-other-keys)
> + (let* ((out (assoc-ref outputs "out"))
> + (bin (string-append out "/bin")))
> + (substitute* (list (string-append bin "/egrep")
> + (string-append bin "/fgrep"))
> + (("^exec grep")
> + (string-append "exec " bin "/grep")))))))
> +scheme@@(guix-user)> ,configure-flags findutils
> +$2 = (list "--localstatedir=/var")
> +scheme@@(guix-user)> ,make-flags binutils
> +$3 = '("MAKEINFO=true")
> +@end example
> +

Here all is clear. What can be confusing is the expansion of #$output
as ((@ (guile) getenv) "out"). Maybe it could be worth to document this
under G-expression section. Or provide a similar example using
G-expression under the section G-expression.

Well, that’s unrelated to this patch and could be done separately. :-)



Toggle quote (22 lines)
> ;;; GNU Guix --- Functional package management for GNU
> -;;; Copyright © 2014, 2015, 2016, 2022 Ludovic Courtès <ludo@gnu.org>
> +;;; Copyright © 2014-2016, 2022-2023 Ludovic Courtès <ludo@gnu.org>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -21,13 +21,15 @@ (define-module (guix monad-repl)
> #:use-module (guix monads)
> #:use-module (guix utils)
> #:use-module (guix packages)
> + #:autoload (guix build-system) (bag)
> #:use-module (guix status)
> - #:autoload (guix gexp) (lower-object)
> + #:autoload (guix gexp) (gexp gexp? lower-gexp lowered-gexp-sexp lower-object)
> #:use-module ((guix derivations)
> #:select (derivation?
> derivation->output-paths built-derivations))
> + #:autoload (guix read-print) (pretty-print-with-comments)
> #:use-module (ice-9 match)
> - #:use-module (ice-9 pretty-print)
> + #:autoload (ice-9 pretty-print) (pretty-print)

Out of curiosity, what is the advantage for #:autoload versus
#:use-module + #:select?


Toggle quote (12 lines)
> +(define (keyword-argument-value args keyword default)
> + "Return the value associated with KEYWORD in ARGS, a keyword/value sequence,
> +or DEFAULT if KEYWORD is missing from ARGS."
> + (let loop ((args args))
> + (match args
> + (()
> + default)
> + ((kw value rest ...)
> + (if (eq? kw keyword)
> + value
> + (loop rest))))))

( Aside. Each time, I am surprised that common helpers are not
Guile builtins, here for extracting keyword value. It would help
Guile to provide such builtin, by default or via some ice-9
module.

My 2 cents. ;-) )

Cheers,
simon
L
L
Ludovic Courtès wrote on 28 Nov 2023 17:20
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
871qc9svc5.fsf@gnu.org
Hello!

Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (2 lines)
> TLDR: All LGTM. :-)

Thanks. :-)

Toggle quote (6 lines)
> scheme@(guix-user)> ,pp (cdr (memq #:phases (package-arguments cava)))
> $3 = (#<gexp (modify-phases %standard-phases (replace (quote bootstrap) (lambda _ (setenv "HOME" (getcwd)) (invoke "sh" "autogen.sh"))) (add-before (quote build) (quote make-cava-ldflags) (lambda _ (mkdir-p (string-append #<gexp-output out> "/lib")))) (add-after (quote install) (quote data) (lambda _ (for-each (lambda (file) (install-file file (string-append #<gexp-output out> "/share/doc/examples"))) (find-files "example_files"))))) gnu/packages/audio.scm:5161:11 7f7258403a50>)
>
>
> Without drifting, why not the meta-command ,arguments?

One can already do “,pp (package-arguments package)”, so I wanted to
have something short and to-the-point that would improve on that.

In particular, we know by convention that certain arguments can be
interpreted as code (gexps or sexps) so we can process them do display
them in a nice way (lowering gexps, applying the pretty printer). We
cannot necessarily do that for other arguments where there’s no
established convention.

These REPL commands are really tailored for packagers.

Toggle quote (7 lines)
> Here all is clear. What can be confusing is the expansion of #$output
> as ((@ (guile) getenv) "out"). Maybe it could be worth to document this
> under G-expression section. Or provide a similar example using
> G-expression under the section G-expression.
>
> Well, that’s unrelated to this patch and could be done separately. :-)

Yeah, we could/should document the implementation of #$output.

Toggle quote (3 lines)
> Out of curiosity, what is the advantage for #:autoload versus
> #:use-module + #:select?

#:autoload allows modules to be loaded lazily, the first time one of the
selected symbols is used. It’s a way to reduce startup time (equivalent
to Emacs autoloads).

Toggle quote (17 lines)
>> +(define (keyword-argument-value args keyword default)
>> + "Return the value associated with KEYWORD in ARGS, a keyword/value sequence,
>> +or DEFAULT if KEYWORD is missing from ARGS."
>> + (let loop ((args args))
>> + (match args
>> + (()
>> + default)
>> + ((kw value rest ...)
>> + (if (eq? kw keyword)
>> + value
>> + (loop rest))))))
>
> ( Aside. Each time, I am surprised that common helpers are not
> Guile builtins, here for extracting keyword value. It would help
> Guile to provide such builtin, by default or via some ice-9
> module.

Yeah, I was disappointed too. :-) (ice-9 optargs) has helpers that
weren’t a good match.

Thanks for your feedback!

Ludo’.
S
S
Simon Tournier wrote on 28 Nov 2023 17:39
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAJ3okZ2dryn0ZyxJAB9719kesOizUODnYnnWuFiUcqA8bkPdDQ@mail.gmail.com
Re,

On Tue, 28 Nov 2023 at 17:20, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (11 lines)
> > Without drifting, why not the meta-command ,arguments?
>
> One can already do “,pp (package-arguments package)”, so I wanted to
> have something short and to-the-point that would improve on that.
>
> In particular, we know by convention that certain arguments can be
> interpreted as code (gexps or sexps) so we can process them do display
> them in a nice way (lowering gexps, applying the pretty printer). We
> cannot necessarily do that for other arguments where there’s no
> established convention.

Unrelated to this patch series, maybe an opportunity for establishing
some of these conventions, no?

BTW, because this potential meta ,arguments targets package developer,
I would find nice to apply what it is done with ,phases to
'package-arguments'. As I showed in my previous message, “,pp
(package-arguments package)” is not really practical. Well, anyway...
let as a exercise for the curious reader. ;-)

Cheers,
simon
L
L
Ludovic Courtès wrote on 4 Dec 2023 22:51
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87wmttljqh.fsf@gnu.org
Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (2 lines)
> TLDR: All LGTM. :-)

Pushed as 3178b1a442c2f2eeeec1ff73bb852d2837b8aa3d!

Ludo’.
Closed
?