[PATCH 00/10] Add 'guix shell' to subsume 'guix environment'

OpenSubmitted by Ludovic Courtès.
Details
15 participants
  • Nicolò Balzarotti
  • Katherine Cox-Buday
  • Christine Lemmer-Webber
  • Thompson, David
  • Jelle Licht
  • Konrad Hinsen
  • Leo Famulari
  • Liliana Marie Prikler
  • Ludovic Courtès
  • Ludovic Courtès
  • Maxime Devos
  • pelzflorian (Florian Pelz)
  • Stefan
  • Vagrant Cascadian
  • zimoun
Owner
unassigned
Severity
normal
L
L
Ludovic Courtès wrote on 2 Oct 12:21 +0200
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211002102116.27726-1-ludo@gnu.org
Hello Guix!
Here comes ‘guix shell’, a proposed replacement for ‘guix environment’!‘guix environment’ would stay around though, at least for some time,probably for a long time.
The differences to ‘guix environment’ are:
1. ‘--ad-hoc’ is the default.
‘guix shell hello’ ≍ ‘guix environment --ad-hoc hello’ ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’
2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ from the current directory or one of its ancestors.
3. ‘--load’/‘-l’ is not ‘-f’/‘--install-from-file’ for consistency with ‘guix package’.
4. ‘guix shell’ without arguments maintains a cache, such that, the second time you run it, it runs in ~0.1s (it does not even need to connect to the daemon).
If you run ‘guix pull’ and run again ‘guix shell’, it recomputes the environment, as is currently the case with ‘guix environment’.
Here’s a summary of previous proposals:
- Dave Thompson: https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html - [X] --ad-hoc is the default - [X] caching - [X] behavior with no arguments - [ ] --load accepts <environment> - [ ] Shepherd services - [ ] 'guix environment --update' to explicitly update - make --ad-hoc the default: https://issues.guix.gnu.org/38529 - [X] https://issues.guix.gnu.org/38529#17:proposal for a new subcommand deprecation of ‘guix environment’
I think <environment> records and Shepherd services could come later.As for ‘--update’, I prefer the behavior implemented here because it’sstateless and thus more predictable.
Thoughts? Are there other changes people would like to see?
If there’s rough consensus I can work on v2 with documentation. Pleaselet’s keep the discussion focused. :-)
As for deprecation, I think there’s no rush. I imagine there could beseveral phases, like: initially we only mention deprecation in the manual,later on ‘guix environment’ starts emitting a warning, and later (I guessat least two years later, probably more) we ask ourselves whether toremove ‘guix environment’. At this point keeping it doesn’t cost us much.
Thanks,Ludo’.
Ludovic Courtès (10): packages: Add 'package-development-inputs'. profiles: Add 'package->development-manifest'. DRAFT Add 'guix shell'. DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file. environment: Add tests for '--profile'. environment: Skip derivation computation when '--profile' is used. environment: Do not connect to the daemon when '--profile' is used. environment: Autoload some modules. cache: Gracefully handle non-existent cache. shell: Maintain a profile cache.
Makefile.am | 2 + doc/guix.texi | 52 ++++++ guix/cache.scm | 10 +- guix/packages.scm | 10 ++ guix/profiles.scm | 19 ++ guix/scripts/environment.scm | 260 +++++++++++++++------------- guix/scripts/shell.scm | 254 +++++++++++++++++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-environment-container.sh | 8 + tests/guix-environment.sh | 7 + tests/guix-shell.sh | 70 ++++++++ tests/packages.scm | 14 ++ tests/profiles.scm | 7 + 13 files changed, 594 insertions(+), 120 deletions(-) create mode 100644 guix/scripts/shell.scm create mode 100644 tests/guix-shell.sh
-- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 01/10] packages: Add 'package-development-inputs'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211002102240.27815-1-ludo@gnu.org
* guix/packages.scm (package-development-inputs): New procedure.* guix/scripts/environment.scm (package-environment-inputs): Use it.* tests/packages.scm ("package-development-inputs")("package-development-inputs, cross-compilation"): New tests.* doc/guix.texi (package Reference): Document it.--- doc/guix.texi | 41 ++++++++++++++++++++++++++++++++++++ guix/packages.scm | 10 +++++++++ guix/scripts/environment.scm | 2 +- tests/packages.scm | 14 ++++++++++++ 4 files changed, 66 insertions(+), 1 deletion(-)
Toggle diff (120 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex a72a726b54..49399e792b 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -6847,6 +6847,47 @@ cross-compiling: It is an error to refer to @code{this-package} outside a package definition. @end deffn +@cindex development inputs, of a package+@cindex implicit inputs, of a package+Sometimes you will want to obtain the list of inputs needed to+@emph{develop} a package---all the inputs that are visible when the+package is compiled. This is what the @code{package-development-inputs}+procedure returns.++@deffn {Scheme Procedure} package-development-inputs @var{package} @+ [@var{system}] [#:target #f]+Return the list of inputs required by @var{package} for development+purposes on @var{system}. When @var{target} is true, return the inputs+needed to cross-compile @var{package} from @var{system} to+@var{triplet}, where @var{triplet} is a triplet such as+@code{"aarch64-linux-gnu"}.++Note that the result includes both explicit inputs and implicit+inputs---inputs automatically added by the build system (@pxref{Build+Systems}). Let us take the @code{hello} package to illustrate that:++@lisp+(use-modules (gnu packages base) (guix packages))++hello+@result{} #<package hello@@2.10 gnu/packages/base.scm:79 7f585d4f6790>++(package-direct-inputs hello)+@result{} ()++(package-development-inputs hello)+@result{} (("source" @dots{}) ("tar" #<package tar@@1.32 @dots{}>) @dots{})+@end lisp++In this example, @code{package-direct-inputs} returns the empty list,+because @code{hello} has zero explicit dependencies. Conversely,+@code{package-development-inputs} includes inputs implicitly added by+@code{gnu-build-system} that are required to build @code{hello}: tar,+gzip, GCC, libc, Bash, and more. To visualize it, @command{guix graph+hello} would show you explicit inputs, whereas @command{guix graph -t+bag hello} would include implicit inputs (@pxref{Invoking guix graph}).+@end deffn+ Because packages are regular Scheme objects that capture a complete dependency graph and associated build procedures, it is often useful to write procedures that take a package and return a modified versiondiff --git a/guix/packages.scm b/guix/packages.scmindex 8c3a0b0b7b..43e0130793 100644--- a/guix/packages.scm+++ b/guix/packages.scm@@ -153,6 +153,7 @@ bag-transitive-host-inputs bag-transitive-build-inputs bag-transitive-target-inputs+ package-development-inputs package-closure default-guile@@ -1070,6 +1071,15 @@ dependencies are known to build on SYSTEM." (%current-system (bag-system bag))) (transitive-inputs (bag-target-inputs bag)))) +(define* (package-development-inputs package+ #:optional (system (%current-system))+ #:key target)+ "Return the list of inputs required by PACKAGE for development purposes on+SYSTEM. When TARGET is true, return the inputs needed to cross-compile+PACKAGE from SYSTEM to TRIPLET, where TRIPLET is a triplet such as+\"aarch64-linux-gnu\"."+ (bag-transitive-inputs (package->bag package system target)))+ (define* (package-closure packages #:key (system (%current-system))) "Return the closure of PACKAGES on SYSTEM--i.e., PACKAGES and the list of packages they depend on, recursively."diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 6958bd6238..d555969b27 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -82,7 +82,7 @@ package." packages for PACKAGE." ;; Remove non-package inputs such as origin records. (filter-map input->manifest-entry- (bag-transitive-inputs (package->bag package))))+ (package-development-inputs package system))) (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]diff --git a/tests/packages.scm b/tests/packages.scmindex 3756877270..266b5aeb7a 100644--- a/tests/packages.scm+++ b/tests/packages.scm@@ -353,6 +353,20 @@ (package-transitive-supported-systems d) (package-transitive-supported-systems e)))) +(test-assert "package-development-inputs"+ ;; Note: Due to propagated inputs, 'package-development-inputs' returns a+ ;; couple more inputs, such as 'linux-libre-headers'.+ (lset<= equal?+ `(("source" ,(package-source hello)) ,@(standard-packages))+ (package-development-inputs hello)))++(test-assert "package-development-inputs, cross-compilation"+ (lset<= equal?+ `(("source" ,(package-source hello))+ ,@(standard-cross-packages "mips64el-linux-gnu" 'host)+ ,@(standard-cross-packages "mips64el-linux-gnu" 'target))+ (package-development-inputs hello #:target "mips64el-linux-gnu")))+ (test-assert "package-closure" (let-syntax ((dummy-package/no-implicit (syntax-rules ()-- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 02/10] profiles: Add 'package->development-manifest'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20211002102240.27815-2-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>
* guix/profiles.scm (package->development-manifest): New procedure.* guix/scripts/environment.scm (input->manifest-entry)(package-environment-inputs): Remove.* guix/scripts/environment.scm (options/resolve-packages): Use'package->development-manifest' instead of 'package-environment-inputs'.* tests/profiles.scm ("package->development-manifest"): New test.--- doc/guix.texi | 11 +++++++++++ guix/profiles.scm | 19 +++++++++++++++++++ guix/scripts/environment.scm | 27 +++++---------------------- tests/profiles.scm | 7 +++++++ 4 files changed, 42 insertions(+), 22 deletions(-)
Toggle diff (134 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 49399e792b..bc3f5a537b 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -3340,6 +3340,17 @@ objects, like this: '("emacs" "guile@@2.2" "guile@@2.2:debug")) @end lisp +@findex package->development-manifest+You might also want to create a manifest for all the dependencies of a+package, rather than the package itself:++@lisp+(package->development-manifest (specification->package "emacs"))+@end lisp++The example above gives you all the software required to develop Emacs,+similar to what @command{guix environment emacs} provides.+ @xref{export-manifest, @option{--export-manifest}}, to learn how to obtain a manifest file from an existing profile. diff --git a/guix/profiles.scm b/guix/profiles.scmindex 2486f91d09..9f30349c69 100644--- a/guix/profiles.scm+++ b/guix/profiles.scm@@ -124,6 +124,7 @@ profile-manifest package->manifest-entry+ package->development-manifest packages->manifest ca-certificate-bundle %default-profile-hooks@@ -400,6 +401,24 @@ file name." (properties properties)))) entry)) +(define* (package->development-manifest package+ #:optional+ (system (%current-system))+ #:key target)+ "Return a manifest for the \"development inputs\" of PACKAGE for SYSTEM,+optionally when cross-compiling to TARGET. Development inputs include both+explicit and implicit inputs of PACKAGE."+ (manifest+ (filter-map (match-lambda+ ((label (? package? package))+ (package->manifest-entry package))+ ((label (? package? package) output)+ (package->manifest-entry package output))+ ;; TODO: Support <inferior-package>.+ (_+ #f))+ (package-development-inputs package system #:target target))))+ (define (packages->manifest packages) "Return a list of manifest entries, one for each item listed in PACKAGES. Elements of PACKAGES can be either package objects or package/string tuplesdiff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex d555969b27..54f48a7482 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -66,24 +66,6 @@ do not augment existing environment variables with additional search paths." (newline))) (profile-search-paths profile manifest))) -(define (input->manifest-entry input)- "Return a manifest entry for INPUT, or #f if INPUT does not correspond to a-package."- (match input- ((_ (? package? package))- (package->manifest-entry package))- ((_ (? package? package) output)- (package->manifest-entry package output))- (_- #f)))--(define (package-environment-inputs package)- "Return a list of manifest entries corresponding to the transitive input-packages for PACKAGE."- ;; Remove non-package inputs such as origin records.- (filter-map input->manifest-entry- (package-development-inputs package system)))- (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] Build an environment that includes the dependencies of PACKAGE and execute@@ -297,11 +279,11 @@ for the corresponding packages." ((? package? package) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package))- (package-environment-inputs package)))+ (manifest-entries (package->development-manifest package)))) (((? package? package) (? string? output)) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package output))- (package-environment-inputs package)))+ (manifest-entries (package->development-manifest package)))) ((lst ...) (append-map (cut packages->outputs <> mode) lst)))) @@ -313,8 +295,9 @@ for the corresponding packages." (specification->package+output spec))) (list (package->manifest-entry* package output)))) (('package 'package (? string? spec))- (package-environment-inputs- (transform (specification->package+output spec))))+ (manifest-entries+ (package->development-manifest+ (transform (specification->package+output spec))))) (('expression mode str) ;; Add all the outputs of the package STR evaluates to. (packages->outputs (read/eval str) mode))diff --git a/tests/profiles.scm b/tests/profiles.scmindex 06a0387221..cac5b73347 100644--- a/tests/profiles.scm+++ b/tests/profiles.scm@@ -265,6 +265,13 @@ (manifest-transaction-removal-candidate? guile-2.0.9 t) (null? install) (null? downgrade) (null? upgrade))))) +(test-assert "package->development-manifest"+ (let ((manifest (package->development-manifest packages:hello)))+ (every (lambda (name)+ (manifest-installed? manifest+ (manifest-pattern (name name))))+ '("gcc" "binutils" "glibc" "coreutils" "grep" "sed"))))+ (test-assertm "profile-derivation" (mlet* %store-monad ((entry -> (package->manifest-entry %bootstrap-guile))-- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 03/10] DRAFT Add 'guix shell'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20211002102240.27815-3-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>
DRAFT: Add doc. Print deprecation warning for 'guix environment'?
* guix/scripts/shell.scm, tests/guix-shell.sh: New files.* Makefile.am (MODULES): Add 'shell.scm'.(SH_TESTS): Add 'tests/guix-shell.sh'.* guix/scripts/environment.scm (show-environment-options-help): Newprocedure.(show-help): Use it.(guix-environment*): New procedure.(guix-environment): Use it.* po/guix/POTFILES.in: Add it.--- Makefile.am | 2 + guix/scripts/environment.scm | 52 +++++++++----- guix/scripts/shell.scm | 136 +++++++++++++++++++++++++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-shell.sh | 54 ++++++++++++++ 5 files changed, 228 insertions(+), 17 deletions(-) create mode 100644 guix/scripts/shell.scm create mode 100644 tests/guix-shell.sh
Toggle diff (336 lines)diff --git a/Makefile.am b/Makefile.amindex b66789fa0b..c28c8799ec 100644--- a/Makefile.am+++ b/Makefile.am@@ -315,6 +315,7 @@ MODULES = \ guix/scripts/import/stackage.scm \ guix/scripts/import/texlive.scm \ guix/scripts/environment.scm \+ guix/scripts/shell.scm \ guix/scripts/publish.scm \ guix/scripts/edit.scm \ guix/scripts/size.scm \@@ -550,6 +551,7 @@ SH_TESTS = \ tests/guix-authenticate.sh \ tests/guix-environment.sh \ tests/guix-environment-container.sh \+ tests/guix-shell.sh \ tests/guix-graph.sh \ tests/guix-describe.sh \ tests/guix-repl.sh \diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 54f48a7482..77956fc018 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -50,7 +50,11 @@ #:use-module (srfi srfi-37) #:use-module (srfi srfi-98) #:export (assert-container-features- guix-environment))+ guix-environment+ guix-environment*+ show-environment-options-help+ (%options . %environment-options)+ (%default-options . %environment-default-options))) (define %default-shell (or (getenv "SHELL") "/bin/sh"))@@ -66,23 +70,16 @@ do not augment existing environment variables with additional search paths." (newline))) (profile-search-paths profile manifest))) -(define (show-help)- (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]-Build an environment that includes the dependencies of PACKAGE and execute-COMMAND or an interactive shell in that environment.\n"))+(define (show-environment-options-help)+ "Print help about options shared between 'guix environment' and 'guix+shell'." (display (G_ " -e, --expression=EXPR create environment for the package that EXPR evaluates to")) (display (G_ "- -l, --load=FILE create environment for the package that the code within- FILE evaluates to"))- (display (G_ " -m, --manifest=FILE create environment with the manifest from FILE")) (display (G_ " -p, --profile=PATH create environment from profile at PATH"))- (display (G_ "- --ad-hoc include all specified packages in the environment instead- of only their inputs")) (display (G_ " --pure unset existing environment variables")) (display (G_ "@@ -118,7 +115,24 @@ COMMAND or an interactive shell in that environment.\n")) (display (G_ " -v, --verbosity=LEVEL use the given verbosity LEVEL")) (display (G_ "- --bootstrap use bootstrap binaries to build the environment"))+ --bootstrap use bootstrap binaries to build the environment")))++(define (show-help)+ (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]+Build an environment that includes the dependencies of PACKAGE and execute+COMMAND or an interactive shell in that environment.\n"))+ (warning (G_ "This command is deprecated in favor of 'guix shell'.\n"))+ (newline)++ ;; These two options are left out in 'guix shell'.+ (display (G_ "+ -l, --load=FILE create environment for the package that the code within+ FILE evaluates to"))+ (display (G_ "+ --ad-hoc include all specified packages in the environment instead+ of only their inputs"))++ (show-environment-options-help) (newline) (show-build-options-help) (newline)@@ -649,11 +663,15 @@ message if any test fails." (define-command (guix-environment . args) (category development)- (synopsis "spawn one-off software environments")+ (synopsis "spawn one-off software environments (deprecated)") + (guix-environment* (parse-args args)))++(define (guix-environment* opts)+ "Run the 'guix environment' command on OPTS, an alist resulting for+command-line option processing with 'parse-command-line'." (with-error-handling- (let* ((opts (parse-args args))- (pure? (assoc-ref opts 'pure))+ (let* ((pure? (assoc-ref opts 'pure)) (container? (assoc-ref opts 'container?)) (link-prof? (assoc-ref opts 'link-profile?)) (network? (assoc-ref opts 'network?))@@ -724,8 +742,8 @@ message if any test fails." (prof-drv (manifest->derivation manifest system bootstrap?)) (profile -> (if profile- (readlink* profile)- (derivation->output-path prof-drv)))+ (readlink* profile)+ (derivation->output-path prof-drv))) (gc-root -> (assoc-ref opts 'gc-root))) ;; First build the inputs. This is necessary even fordiff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmnew file mode 100644index 0000000000..6a4b7a5092--- /dev/null+++ b/guix/scripts/shell.scm@@ -0,0 +1,136 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 Ludovic Courtès <ludo@gnu.org>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (guix scripts shell)+ #:use-module (guix ui)+ #:use-module (guix scripts environment)+ #:autoload (guix scripts build) (show-build-options-help)+ #:autoload (guix transformations) (show-transformation-options-help)+ #:use-module (guix scripts)+ #:use-module (srfi srfi-1)+ #:use-module (srfi srfi-26)+ #:use-module (srfi srfi-37)+ #:use-module (srfi srfi-71)+ #:use-module (ice-9 match)+ #:export (guix-shell))++(define (show-help)+ (display (G_ "Usage: guix shell [OPTION] PACKAGES... [-- COMMAND...]+Build an environment that includes PACKAGES and execute COMMAND or an+interactive shell in that environment.\n"))+ (newline)++ ;; These two options differ from 'guix environment'.+ (display (G_ "+ -D, --development include the development inputs of the next package"))+ (display (G_ "+ -f, --install-from-file=FILE+ install the package that the code within FILE+ evaluates to"))++ (show-environment-options-help)+ (newline)+ (show-build-options-help)+ (newline)+ (show-transformation-options-help)+ (newline)+ (display (G_ "+ -h, --help display this help and exit"))+ (display (G_ "+ -V, --version display version information and exit"))+ (newline)+ (show-bug-report-information))++(define (tag-package-arg opts arg)+ "Return a two-element list with the form (TAG ARG) that tags ARG with either+'ad-hoc' in OPTS has the 'ad-hoc?' key set to #t, or 'inputs' otherwise."+ (if (assoc-ref opts 'ad-hoc?)+ `(ad-hoc-package ,arg)+ `(package ,arg)))++(define (ensure-ad-hoc alist)+ (if (assq-ref alist 'ad-hoc?)+ alist+ `((ad-hoc? . #t) ,@alist)))++(define (wrapped-option opt)+ "Wrap OPT, a SRFI-37 option, such that its processor always adds the+'ad-hoc?' flag to the resulting alist."+ (option (option-names opt)+ (option-required-arg? opt)+ (option-optional-arg? opt)+ (compose ensure-ad-hoc (option-processor opt))))++(define %options+ ;; Specification of the command-line options.+ (let ((to-remove '("ad-hoc" "inherit" "load" "help" "version")))+ (append+ (list (option '(#\h "help") #f #f+ (lambda args+ (show-help)+ (exit 0)))+ (option '(#\V "version") #f #f+ (lambda args+ (show-version-and-exit "guix shell")))++ (option '(#\D "development") #f #f+ (lambda (opt name arg result)+ ;; Temporarily remove the 'ad-hoc?' flag from result.+ ;; The next option will put it back thanks to+ ;; 'wrapped-option'.+ (alist-delete 'ad-hoc? result)))++ ;; For consistency with 'guix package', support '-f' rather than+ ;; '-l' like 'guix environment' does.+ (option '(#\f "install-from-file") #t #f+ (lambda (opt name arg result)+ (alist-cons 'load (tag-package-arg result arg)+ result))))+ (filter-map (lambda (opt)+ (and (not (any (lambda (name)+ (member name to-remove))+ (option-names opt)))+ (wrapped-option opt)))+ %environment-options))))++(define %default-options+ `((ad-hoc? . #t) ;always true+ ,@%environment-default-options))++(define (parse-args args)+ "Parse the list of command line arguments ARGS."+ (define (handle-argument arg result)+ (alist-cons 'package (tag-package-arg result arg)+ (ensure-ad-hoc result)))++ ;; The '--' token is used to separate the command to run from the rest of+ ;; the operands.+ (let ((args command (break (cut string=? "--" <>) args)))+ (let ((opts (parse-command-line args %options (list %default-options)+ #:argument-handler handle-argument)))+ (match command+ (() opts)+ (("--") opts)+ (("--" command ...) (alist-cons 'exec command opts))))))++ +(define-command (guix-shell . args)+ (category development)+ (synopsis "spawn one-off software environments")++ (guix-environment* (parse-args args)))diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.inindex f5b76bf582..f8abeb2d38 100644--- a/po/guix/POTFILES.in+++ b/po/guix/POTFILES.in@@ -99,6 +99,7 @@ guix/derivations.scm guix/scripts/archive.scm guix/scripts/build.scm guix/scripts/environment.scm+guix/scripts/shell.scm guix/scripts/time-machine.scm guix/scripts/import/cpan.scm guix/scripts/import/crate.scmdiff --git a/tests/guix-shell.sh b/tests/guix-shell.shnew file mode 100644index 0000000000..f08637f7ff--- /dev/null+++ b/tests/guix-shell.sh@@ -0,0 +1,54 @@+# GNU Guix --- Functional package management for GNU+# Copyright © 2021 Ludovic Courtès <ludo@gnu.org>+#+# This file is part of GNU Guix.+#+# GNU Guix is free software; you can redistribute it and/or modify it+# under the terms of the GNU General Public License as published by+# the Free Software Foundation; either version 3 of the License, or (at+# your option) any later version.+#+# GNU Guix is distributed in the hope that it will be useful, but+# WITHOUT ANY WARRANTY; without even the implied warranty of+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+# GNU General Public License for more details.+#+# You should have received a copy of the GNU General Public License+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++#+# Test the 'guix shell' alias.+#++guix shell --version++tmpdir="t-guix-shell-$$"+trap 'rm -r "$tmpdir"' EXIT+mkdir "$tmpdir"++guix shell --bootstrap --pure guile-bootstrap -- guile --version++# '--ad-hoc' is a thing of the past.+! guix shell --ad-hoc guile-bootstrap++if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null+then+ # Compute the build environment for the initial GNU Make.+ guix shell --bootstrap --no-substitutes --search-paths --pure \+ -D -e '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/a"++ # Make sure bootstrap binaries are in the profile.+ profile=`grep "^export PATH" "$tmpdir/a" | sed -r 's|^.*="(.*)/bin"|\1|'`++ # Make sure the bootstrap binaries are all listed where they belong.+ grep -E "^export PATH=\"$profile/bin\"" "$tmpdir/a"+ grep -E "^export CPATH=\"$profile/include\"" "$tmpdir/a"+ grep -E "^export LIBRARY_PATH=\"$profile/lib\"" "$tmpdir/a"+ for dep in bootstrap-binaries-0 gcc-bootstrap-0 glibc-bootstrap-0+ do+ guix gc --references "$profile" | grep "$dep"+ done++ # 'make-boot0' itself must not be listed.+ ! guix gc --references "$profile" | grep make-boot0+fi-- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 04/10] DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211002102240.27815-4-ludo@gnu.org
DRAFT: Add doc.
* guix/scripts/shell.scm (parse-args): Add call to 'auto-detect-manifest'.(find-file-in-parent-directories, auto-detect-manifest): New procedures.* tests/guix-shell.sh: Add test.--- guix/scripts/shell.scm | 44 ++++++++++++++++++++++++++++++++++++++++-- tests/guix-shell.sh | 16 +++++++++++++++ 2 files changed, 58 insertions(+), 2 deletions(-)
Toggle diff (101 lines)diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmindex 6a4b7a5092..2f15befbd3 100644--- a/guix/scripts/shell.scm+++ b/guix/scripts/shell.scm@@ -22,6 +22,8 @@ #:autoload (guix scripts build) (show-build-options-help) #:autoload (guix transformations) (show-transformation-options-help) #:use-module (guix scripts)+ #:use-module (guix packages)+ #:use-module (guix profiles) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) #:use-module (srfi srfi-37)@@ -121,13 +123,51 @@ interactive shell in that environment.\n")) ;; The '--' token is used to separate the command to run from the rest of ;; the operands. (let ((args command (break (cut string=? "--" <>) args)))- (let ((opts (parse-command-line args %options (list %default-options)- #:argument-handler handle-argument)))+ (let ((opts (auto-detect-manifest+ (parse-command-line args %options (list %default-options)+ #:argument-handler handle-argument)))) (match command (() opts) (("--") opts) (("--" command ...) (alist-cons 'exec command opts)))))) +(define (find-file-in-parent-directories candidates)+ "Find one of CANDIDATES in the current directory or one of its ancestors."+ (let loop ((directory (getcwd)))+ (and (= (stat:uid (stat directory)) (getuid))+ (or (any (lambda (candidate)+ (let ((candidate (string-append directory "/" candidate)))+ (and (file-exists? candidate) candidate)))+ candidates)+ (loop (string-append directory "/..")))))) ;Unix ".." resolution++(define (auto-detect-manifest opts)+ "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or+\"manifest.scm\" file from the current directory or one of its ancestors.+Return the modified OPTS."+ (define (options-contain-payload? opts)+ (match opts+ (() #f)+ ((('package . _) . _) #t)+ ((('load . _) . _) #t)+ ((('manifest . _) . _) #t)+ ((('expression . _) . _) #t)+ ((_ . rest) (options-contain-payload? rest))))++ (if (options-contain-payload? opts)+ opts+ (match (find-file-in-parent-directories '("guix.scm" "manifest.scm"))+ (#f+ (warning (G_ "no packages specified; creating an empty environment~%"))+ opts)+ (file+ (info (G_ "loading environment from '~a'...~%") file)+ (match (basename file)+ ("guix.scm"+ (alist-cons 'load `(package ,file) opts))+ ("manifest.scm"+ (alist-cons 'manifest file opts)))))))+ (define-command (guix-shell . args) (category development)diff --git a/tests/guix-shell.sh b/tests/guix-shell.shindex f08637f7ff..498c1c5515 100644--- a/tests/guix-shell.sh+++ b/tests/guix-shell.sh@@ -31,6 +31,16 @@ guix shell --bootstrap --pure guile-bootstrap -- guile --version # '--ad-hoc' is a thing of the past. ! guix shell --ad-hoc guile-bootstrap +# Honoring the local 'manifest.scm' file.+cat > "$tmpdir/manifest.scm" <<EOF+(specifications->manifest '("guile-bootstrap"))+EOF+profile1="$(cd "$tmpdir"; guix shell --bootstrap -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT')"+profile2="$(guix shell --bootstrap guile-bootstrap -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT')"+test -n "$profile1"+test "$profile1" = "$profile2"+rm "$tmpdir/manifest.scm"+ if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null then # Compute the build environment for the initial GNU Make.@@ -51,4 +61,10 @@ then # 'make-boot0' itself must not be listed. ! guix gc --references "$profile" | grep make-boot0++ # Honoring the local 'guix.scm' file.+ echo '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/guix.scm"+ (cd "$tmpdir"; guix shell --bootstrap --search-paths --pure > "b")+ cmp "$tmpdir/a" "$tmpdir/b"+ rm "$tmpdir/guix.scm" fi-- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 06/10] environment: Skip derivation computation when '--profile' is used.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211002102240.27815-6-ludo@gnu.org
* guix/scripts/environment.scm (guix-environment*): Bypass calls to'package-derivation' and to 'manifest->derivation' when PROFILE istrue.--- guix/scripts/environment.scm | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-)
Toggle diff (48 lines)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 77956fc018..32f376fdd2 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -729,18 +729,21 @@ command-line option processing with 'parse-command-line'." ;; Use the bootstrap Guile when requested. (parameterize ((%graft? (assoc-ref opts 'graft?)) (%guile-for-build- (package-derivation- store- (if bootstrap?- %bootstrap-guile- (default-guile)))))+ (and (or container? (not profile))+ (package-derivation+ store+ (if bootstrap?+ %bootstrap-guile+ (default-guile)))))) (run-with-store store ;; Containers need a Bourne shell at /bin/sh. (mlet* %store-monad ((bash (environment-bash container? bootstrap? system))- (prof-drv (manifest->derivation- manifest system bootstrap?))+ (prof-drv (if profile+ (return #f)+ (manifest->derivation+ manifest system bootstrap?))) (profile -> (if profile (readlink* profile) (derivation->output-path prof-drv)))@@ -750,9 +753,9 @@ command-line option processing with 'parse-command-line'." ;; --search-paths. Additionally, we might need to build bash for ;; a container. (mbegin %store-monad- (built-derivations (if (derivation? bash)- (list prof-drv bash)- (list prof-drv)))+ (built-derivations (append+ (if prof-drv (list prof-drv) '())+ (if (derivation? bash) (list bash) '()))) (mwhen gc-root (register-gc-root profile gc-root)) -- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 07/10] environment: Do not connect to the daemon when '--profile' is used.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211002102240.27815-7-ludo@gnu.org
This further speeds up the 'guix environment -p PROFILE' case.
* guix/scripts/environment.scm (guix-environment*)[store-needed?]: Newvariable.[with-store/maybe]: New macro.Use it instead of 'with-store', and remove 'with-build-handler' form.--- guix/scripts/environment.scm | 169 +++++++++++++++++++---------------- 1 file changed, 93 insertions(+), 76 deletions(-)
Toggle diff (195 lines)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 32f376fdd2..e23d52df39 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -691,6 +691,26 @@ command-line option processing with 'parse-command-line'." (mappings (pick-all opts 'file-system-mapping)) (white-list (pick-all opts 'inherit-regexp))) + (define store-needed?+ ;; Whether connecting to the daemon is needed.+ (or container? (not profile)))++ (define-syntax-rule (with-store/maybe store exp ...)+ ;; Evaluate EXP... with STORE bound to a connection, unless+ ;; STORE-NEEDED? is false, in which case STORE is bound to #f.+ (let ((proc (lambda (store) exp ...)))+ (if store-needed?+ (with-store s+ (set-build-options-from-command-line s opts)+ (with-build-handler (build-notifier #:use-substitutes?+ (assoc-ref opts 'substitutes?)+ #:verbosity+ (assoc-ref opts 'verbosity)+ #:dry-run?+ (assoc-ref opts 'dry-run?))+ (proc s)))+ (proc #f))))+ (when container? (assert-container-features)) (when (and (not container?) link-prof?)@@ -701,88 +721,85 @@ command-line option processing with 'parse-command-line'." (leave (G_ "--no-cwd cannot be used without --container~%"))) - (with-store store- (with-build-handler (build-notifier #:use-substitutes?- (assoc-ref opts 'substitutes?)- #:verbosity- (assoc-ref opts 'verbosity)- #:dry-run?- (assoc-ref opts 'dry-run?))- (with-status-verbosity (assoc-ref opts 'verbosity)- (define manifest-from-opts- (options/resolve-packages store opts))+ (with-store/maybe store+ (with-status-verbosity (assoc-ref opts 'verbosity)+ (define manifest-from-opts+ (options/resolve-packages store opts)) - (define manifest- (if profile- (profile-manifest profile)- manifest-from-opts))+ (define manifest+ (if profile+ (profile-manifest profile)+ manifest-from-opts)) - (when (and profile- (> (length (manifest-entries manifest-from-opts)) 0))- (leave (G_ "'--profile' cannot be used with package options~%")))+ (when (and profile+ (> (length (manifest-entries manifest-from-opts)) 0))+ (leave (G_ "'--profile' cannot be used with package options~%"))) - (when (null? (manifest-entries manifest))- (warning (G_ "no packages specified; creating an empty environment~%")))+ (when (null? (manifest-entries manifest))+ (warning (G_ "no packages specified; creating an empty environment~%"))) - (set-build-options-from-command-line store opts)+ ;; Use the bootstrap Guile when requested.+ (parameterize ((%graft? (assoc-ref opts 'graft?))+ (%guile-for-build+ (and store-needed?+ (package-derivation+ store+ (if bootstrap?+ %bootstrap-guile+ (default-guile))))))+ (run-with-store store+ ;; Containers need a Bourne shell at /bin/sh.+ (mlet* %store-monad ((bash (environment-bash container?+ bootstrap?+ system))+ (prof-drv (if profile+ (return #f)+ (manifest->derivation+ manifest system bootstrap?)))+ (profile -> (if profile+ (readlink* profile)+ (derivation->output-path prof-drv)))+ (gc-root -> (assoc-ref opts 'gc-root))) - ;; Use the bootstrap Guile when requested.- (parameterize ((%graft? (assoc-ref opts 'graft?))- (%guile-for-build- (and (or container? (not profile))- (package-derivation- store- (if bootstrap?- %bootstrap-guile- (default-guile))))))- (run-with-store store- ;; Containers need a Bourne shell at /bin/sh.- (mlet* %store-monad ((bash (environment-bash container?- bootstrap?- system))- (prof-drv (if profile- (return #f)- (manifest->derivation- manifest system bootstrap?)))- (profile -> (if profile- (readlink* profile)- (derivation->output-path prof-drv)))- (gc-root -> (assoc-ref opts 'gc-root)))-- ;; First build the inputs. This is necessary even for- ;; --search-paths. Additionally, we might need to build bash for- ;; a container.- (mbegin %store-monad+ ;; First build the inputs. This is necessary even for+ ;; --search-paths. Additionally, we might need to build bash for+ ;; a container.+ (mbegin %store-monad+ (mwhen store-needed? (built-derivations (append (if prof-drv (list prof-drv) '())- (if (derivation? bash) (list bash) '())))- (mwhen gc-root- (register-gc-root profile gc-root))+ (if (derivation? bash) (list bash) '()))))+ (mwhen gc-root+ (register-gc-root profile gc-root)) - (cond- ((assoc-ref opts 'search-paths)- (show-search-paths profile manifest #:pure? pure?)- (return #t))- (container?- (let ((bash-binary- (if bootstrap?- (derivation->output-path bash)- (string-append (derivation->output-path bash)- "/bin/sh"))))- (launch-environment/container #:command command- #:bash bash-binary- #:user user- #:user-mappings mappings- #:profile profile- #:manifest manifest- #:white-list white-list- #:link-profile? link-prof?- #:network? network?- #:map-cwd? (not no-cwd?))))+ (cond+ ((assoc-ref opts 'search-paths)+ (show-search-paths profile manifest #:pure? pure?)+ (return #t))+ (container?+ (let ((bash-binary+ (if bootstrap?+ (derivation->output-path bash)+ (string-append (derivation->output-path bash)+ "/bin/sh"))))+ (launch-environment/container #:command command+ #:bash bash-binary+ #:user user+ #:user-mappings mappings+ #:profile profile+ #:manifest manifest+ #:white-list white-list+ #:link-profile? link-prof?+ #:network? network?+ #:map-cwd? (not no-cwd?)))) - (else- (return- (exit/status- (launch-environment/fork command profile manifest- #:white-list white-list- #:pure? pure?)))))))))))))))+ (else+ (return+ (exit/status+ (launch-environment/fork command profile manifest+ #:white-list white-list+ #:pure? pure?))))))))))))))++;;; Local Variables:+;;; (put 'with-store/maybe 'scheme-indent-function 1)+;;; End:-- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 08/10] environment: Autoload some modules.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211002102240.27815-8-ludo@gnu.org
This further speeds up the 'guix environment -p PROFILE' case.
* guix/scripts/environment.scm: Autoload a bunch of modules.--- guix/scripts/environment.scm | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-)
Toggle diff (33 lines)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex e23d52df39..05a43659da 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -34,15 +34,18 @@ #:use-module (guix scripts) #:use-module (guix scripts build) #:use-module (guix transformations)- #:use-module (gnu build linux-container)- #:use-module (gnu build accounts)- #:use-module ((guix build syscalls) #:select (set-network-interface-up))- #:use-module (gnu system linux-container)+ #:autoload (gnu build linux-container) (call-with-container %namespaces+ user-namespace-supported?+ unprivileged-user-namespace-supported?+ setgroups-supported?)+ #:autoload (gnu build accounts) (password-entry group-entry+ password-entry-name password-entry-directory+ write-passwd write-group)+ #:autoload (guix build syscalls) (set-network-interface-up) #:use-module (gnu system file-systems)- #:use-module (gnu packages)- #:use-module (gnu packages bash)- #:use-module ((gnu packages bootstrap)- #:select (bootstrap-executable %bootstrap-guile))+ #:autoload (gnu packages) (specification->package+output)+ #:autoload (gnu packages bash) (bash)+ #:autoload (gnu packages bootstrap) (bootstrap-executable %bootstrap-guile) #:use-module (ice-9 match) #:use-module (srfi srfi-1) #:use-module (srfi srfi-11)-- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 09/10] cache: Gracefully handle non-existent cache.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211002102240.27815-9-ludo@gnu.org
* guix/cache.scm (maybe-remove-expired-cache-entries): Ignore ENOENTwhen writing EXPIRY-FILE.--- guix/cache.scm | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
Toggle diff (22 lines)diff --git a/guix/cache.scm b/guix/cache.scmindex 0401a9d428..51009809bd 100644--- a/guix/cache.scm+++ b/guix/cache.scm@@ -101,7 +101,13 @@ CLEANUP-PERIOD denotes the minimum time between two cache cleanups." #:now now #:entry-expiration entry-expiration #:delete-entry delete-entry)- (call-with-output-file expiry-file- (cute write (time-second now) <>))))+ (catch 'system-error+ (lambda ()+ (call-with-output-file expiry-file+ (cute write (time-second now) <>)))+ (lambda args+ ;; ENOENT means CACHE does not exist.+ (unless (= ENOENT (system-error-errno args))+ (apply throw args)))))) ;;; cache.scm ends here-- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 10/10] shell: Maintain a profile cache.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211002102240.27815-10-ludo@gnu.org
With this change, running "guix shell" (no arguments) is equivalent to:
guix environment -r ~/.cache/guix/profiles/some-root -l guix.scm
This is the cache miss. On cache hit, it's equivalent to:
guix environment -p ~/.cache/guix/profiles/some-root
... which can run in 0.1s.
* guix/scripts/shell.scm (auto-detect-manifest): Looked for a cached GCroot to the profile and use it.(%profile-cache-directory): New variable.(profile-cache-key, profile-cached-gc-root): New procedures.(guix-shell)[cache-entries, entry-expiration]: New procedures.Add call to 'maybe-remove-expired-cache-entries'.--- guix/scripts/shell.scm | 90 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 84 insertions(+), 6 deletions(-)
Toggle diff (114 lines)diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmindex 2f15befbd3..7c116cc770 100644--- a/guix/scripts/shell.scm+++ b/guix/scripts/shell.scm@@ -29,6 +29,15 @@ #:use-module (srfi srfi-37) #:use-module (srfi srfi-71) #:use-module (ice-9 match)+ #:autoload (guix base32) (bytevector->base32-string)+ #:autoload (rnrs bytevectors) (string->utf8)+ #:autoload (guix utils) (cache-directory)+ #:autoload (guix describe) (current-channels)+ #:autoload (guix channels) (channel-commit)+ #:autoload (gcrypt hash) (sha256)+ #:use-module ((guix build utils) #:select (mkdir-p))+ #:use-module (guix cache)+ #:use-module ((ice-9 ftw) #:select (scandir)) #:export (guix-shell)) (define (show-help)@@ -161,16 +170,85 @@ Return the modified OPTS." (warning (G_ "no packages specified; creating an empty environment~%")) opts) (file+ ;; Load environment from FILE; if possible, use/maintain a GC root to+ ;; the corresponding profile in cache. (info (G_ "loading environment from '~a'...~%") file)- (match (basename file)- ("guix.scm"- (alist-cons 'load `(package ,file) opts))- ("manifest.scm"- (alist-cons 'manifest file opts)))))))+ (let* ((root (profile-cached-gc-root file))+ (stat (and root (false-if-exception (lstat root)))))+ (if (and stat+ (<= (stat:mtime ((@ (guile) stat) file))+ (stat:mtime stat)))+ (let ((now (current-time)))+ ;; Update the atime on ROOT to reflect usage.+ (utime root+ now (stat:mtime stat)+ 0 (stat:mtimensec stat)+ AT_SYMLINK_NOFOLLOW)+ (alist-cons 'profile root opts)) ;load right away+ (let ((opts (match (basename file)+ ("guix.scm"+ (alist-cons 'load `(package ,file) opts))+ ("manifest.scm"+ (alist-cons 'manifest file opts)))))+ (if (and root (not (assq-ref opts 'gc-root)))+ (begin+ (if stat+ (delete-file root)+ (mkdir-p (dirname root)))+ (alist-cons 'gc-root root opts))+ opts))))))))++ +;;;+;;; Profile cache.+;;;++(define %profile-cache-directory+ ;; Directory where profiles created by 'guix shell' alone (without extra+ ;; options) are cached.+ (make-parameter (string-append (cache-directory #:ensure? #f)+ "/profiles")))++(define (profile-cache-key file)+ "Return the cache key for the profile corresponding to FILE, a 'guix.scm' or+'manifest.scm' file, or #f if we lack channel information."+ (match (current-channels)+ (() #f)+ (((= channel-commit commits) ...)+ (let ((stat (stat file)))+ (bytevector->base32-string+ (sha256 (string->utf8+ (string-append (string-join commits) ":"+ (basename file) ":"+ (number->string (stat:dev stat)) ":"+ (number->string (stat:ino stat))))))))))++(define (profile-cached-gc-root file)+ "Return the cached GC root for FILE, a 'guix.scm' or 'manifest.scm' file, or+#f if we lack information to cache it."+ (match (profile-cache-key file)+ (#f #f)+ (key (string-append (%profile-cache-directory) "/" key)))) (define-command (guix-shell . args) (category development) (synopsis "spawn one-off software environments") - (guix-environment* (parse-args args)))+ (define (cache-entries directory)+ (filter-map (match-lambda+ ((or "." "..") #f)+ (file (string-append directory "/" file)))+ (or (scandir directory) '())))++ (define* (entry-expiration file)+ ;; Return the time at which FILE, a cached profile, is considered expired.+ (match (false-if-exception (lstat file))+ (#f 0) ;FILE may have been deleted in the meantime+ (st (+ (stat:atime st) (* 60 60 24 7)))))++ (let ((result (guix-environment* (parse-args args))))+ (maybe-remove-expired-cache-entries (%profile-cache-directory)+ cache-entries+ #:entry-expiration entry-expiration)+ result))-- 2.33.0
L
L
Ludovic Courtès wrote on 2 Oct 12:22 +0200
[PATCH 05/10] environment: Add tests for '--profile'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211002102240.27815-5-ludo@gnu.org
This is a followup to a643deac2de81755a1843a3b41dd53857678bebc.
* tests/guix-environment-container.sh, tests/guix-environment.sh: Addtests for '--profile'.--- tests/guix-environment-container.sh | 8 ++++++++ tests/guix-environment.sh | 7 +++++++ 2 files changed, 15 insertions(+)
Toggle diff (39 lines)diff --git a/tests/guix-environment-container.sh b/tests/guix-environment-container.shindex f2d15c8d0c..2e238c501d 100644--- a/tests/guix-environment-container.sh+++ b/tests/guix-environment-container.sh@@ -44,6 +44,14 @@ else test $? = 42 fi +# Try '--root' and '--profile'.+root="$tmpdir/root"+guix environment -C --ad-hoc --bootstrap guile-bootstrap -r "$root" -- guile --version+guix environment -C -p "$root" --bootstrap -- guile --version+path1=$(guix environment -C -p "$root" --bootstrap -- guile -c '(display (getenv "PATH"))')+path2=$(guix environment -C --ad-hoc --bootstrap guile-bootstrap -- guile -c '(display (getenv "PATH"))')+test "$path1" = "$path2"+ # Make sure "localhost" resolves. guix environment --container --ad-hoc --bootstrap guile-bootstrap \ -- guile -c '(exit (pair? (getaddrinfo "localhost" "80")))'diff --git a/tests/guix-environment.sh b/tests/guix-environment.shindex afadcbe195..f4fc2e39ed 100644--- a/tests/guix-environment.sh+++ b/tests/guix-environment.sh@@ -119,6 +119,13 @@ test `readlink "$gcroot"` = "$expected" guix environment --bootstrap -r "$gcroot" --ad-hoc guile-bootstrap \ -- guile -c 1 test `readlink "$gcroot"` = "$expected"++# Make sure '-p' works as expected.+test $(guix environment -p "$gcroot" -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT') = "$expected"+paths1="$(guix environment -p "$gcroot" --search-paths)"+paths2="$(guix environment --bootstrap --ad-hoc guile-bootstrap --search-paths)"+test "$paths1" = "$paths2"+ rm "$gcroot" # Try '-r' with a relative file name.-- 2.33.0
J
J
Jelle Licht wrote on 2 Oct 12:50 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)
86mtnrpuoa.fsf@fsfe.org
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (3 lines)> Hello Guix!>> Here comes ‘guix shell’, a proposed replacement for ‘guix environment’!
Suddenly thousands(/dozens?) of shell scripts fear for their continuedexistence :-)
Toggle quote (2 lines)> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> from the current directory or one of its ancestors.
<snip>
Toggle quote (6 lines)> 4. ‘guix shell’ without arguments maintains a cache, such that, the> second time you run it, it runs in ~0.1s (it does not even need to> connect to the daemon).>> If you run ‘guix pull’ and run again ‘guix shell’, it recomputes> the environment, as is currently the case with ‘guix environment’.
<snip>
Toggle quote (5 lines)> Thoughts? Are there other changes people would like to see?>> If there’s rough consensus I can work on v2 with documentation. Please> let’s keep the discussion focused. :-)
At the risk of doing exactly not that; since it will(/might) alreadyprovide some automagic conveniences, perhaps it makes sense toadditionally load a guix-channels.scm.
I understand that there would be some duplication of functionalityw.r.t. ‘guix time-machine -C guix-channels.scm -- shell ...’, but if wealready go the DWIW-route, why not go all the way? I think this shouldonly apply when running ‘guix shell’ without arguments, if that wasunclear.
Love the proposal- Jelle
L
L
Liliana Marie Prikler wrote on 2 Oct 13:39 +0200
Re: [PATCH 06/10] environment: Skip derivation computation when '--profile' is used.
c2b770e4c418bd113e39f089e2a118422ef25cc4.camel@gmail.com
Am Samstag, den 02.10.2021, 12:22 +0200 schrieb Ludovic Courtès:
Toggle quote (3 lines)> * guix/scripts/environment.scm (guix-environment*): Bypass calls to> 'package-derivation' and to 'manifest->derivation' when PROFILE is> true.
This only affects `guix shell' and not `guix environment', right? Ifnot, does the outward behaviour of `guix environment' stay the sameconsidering this patch and 07/10? There might be people relying on theway `guix environment' *currently* works, who would need to be informedabout that change.
Then again, if the following holds
Toggle quote (2 lines)> If you run ‘guix pull’ and run again ‘guix shell’, it recomputes> the environment, as is currently the case with ‘guix environment’.
then the behaviour of guix environment should also be consistent withwhat it did before, but with the added cache of guix shell. Am Ireading this correctly?
Regards,Liliana
L
L
Liliana Marie Prikler wrote on 2 Oct 13:52 +0200
Re: [PATCH 04/10] DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file.
1c5d7a3d62da1b495a14b850a1b56b2a410b2405.camel@gmail.com
Hi,
Am Samstag, den 02.10.2021, 12:22 +0200 schrieb Ludovic Courtès:
Toggle quote (32 lines)> [...]> +(define (auto-detect-manifest opts)> + "If OPTS do not specify packages or a manifest, load a> \"guix.scm\" or> +\"manifest.scm\" file from the current directory or one of its> ancestors.> +Return the modified OPTS."> + (define (options-contain-payload? opts)> + (match opts> + (() #f)> + ((('package . _) . _) #t)> + ((('load . _) . _) #t)> + ((('manifest . _) . _) #t)> + ((('expression . _) . _) #t)> + ((_ . rest) (options-contain-payload? rest))))> +> + (if (options-contain-payload? opts)> + opts> + (match (find-file-in-parent-directories '("guix.scm"> "manifest.scm"))> + (#f> + (warning (G_ "no packages specified; creating an empty> environment~%"))> + opts)> + (file> + (info (G_ "loading environment from '~a'...~%") file)> + (match (basename file)> + ("guix.scm"> + (alist-cons 'load `(package ,file) opts))> + ("manifest.scm"> + (alist-cons 'manifest file opts)))))))> [...]
What would happen on the top-level of the Guix source tree or deepinside the tree of a guile package that deals with manifests, thataren't necessarily related to Guix? I think we should try searchingfor something less ambiguous first (".guix-shell/manifest" perhaps?)and maybe also provide further options after manifest.scm (e.g. build-aux/guix.scm or etc/guix.scm)
WDYT?
P
P
pelzflorian (Florian Pelz) wrote on 2 Oct 14:10 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
20211002115958.rzupz4afq7elehq4@pelzflorian.localdomain
guix shell is very interesting, but:
On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote:
Toggle quote (3 lines)> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> from the current directory or one of its ancestors.
This however is concerning. Users will not expect guix to executearbitrary code. Maybe print a suggestion to maybe --file the fileinstead.
Maybe I should not have such expectations but IMHO Guix is not like`haunt build`, which is expected to load haunt.scm.
Regards,Florian
M
M
Maxime Devos wrote on 2 Oct 15:28 +0200
Re: [bug#50960] [PATCH 09/10] cache: Gracefully handle non-existent cache.
5686bf529c9ea7fe5e8a34d9116dfc317c7f1745.camel@telenet.be
Ludovic Courtès schreef op za 02-10-2021 om 12:22 [+0200]:
Toggle quote (24 lines)> * guix/cache.scm (maybe-remove-expired-cache-entries): Ignore ENOENT> when writing EXPIRY-FILE.> ---> guix/cache.scm | 10 ++++++++--> 1 file changed, 8 insertions(+), 2 deletions(-)> > diff --git a/guix/cache.scm b/guix/cache.scm> index 0401a9d428..51009809bd 100644> --- a/guix/cache.scm> +++ b/guix/cache.scm> @@ -101,7 +101,13 @@ CLEANUP-PERIOD denotes the minimum time between two cache cleanups."> #:now now> #:entry-expiration entry-expiration> #:delete-entry delete-entry)> - (call-with-output-file expiry-file> - (cute write (time-second now) <>))))> + (catch 'system-error> + (lambda ()> + (call-with-output-file expiry-file> + (cute write (time-second now) <>)))> + (lambda args> + ;; ENOENT means CACHE does not exist.> + (unless (= ENOENT (system-error-errno args))
And EROFS perhaps, such that "guix shell" works even if the root file system wasremounted read-only for some reason and all the necessary derivations have alreadybeen built.
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVhekxccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7nAEAPwKOGXBXLXxya7HiaIWjRAUbp0Kb1b/0zFzzMF4Hn2e+QEA72Q2UB9KwoJfTVA3rKwqH7wa4BmQLmEhvgucBUxZRAw==HxkF-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 2 Oct 15:40 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 50960@debbugs.gnu.org)
87sfxjzgsv.fsf_-_@gnu.org
Hi,
"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:
Toggle quote (11 lines)> On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote:>> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’>> from the current directory or one of its ancestors.>> This however is concerning. Users will not expect guix to execute> arbitrary code. Maybe print a suggestion to maybe --file the file> instead.>> Maybe I should not have such expectations but IMHO Guix is not like> `haunt build`, which is expected to load haunt.scm.
I think it’s fine as long as, as in the case of ‘haunt build’ or ‘make’or ‘git’, it’s properly documented. Also, ‘guix shell’ unconditionallywrites a message.
Ludo’.
L
L
Ludovic Courtès wrote on 2 Oct 15:43 +0200
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)(address . 50960@debbugs.gnu.org)
87mtnrzgn9.fsf_-_@gnu.org
Hi,
Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
Toggle quote (37 lines)> Am Samstag, den 02.10.2021, 12:22 +0200 schrieb Ludovic Courtès:>> [...]>> +(define (auto-detect-manifest opts)>> + "If OPTS do not specify packages or a manifest, load a>> \"guix.scm\" or>> +\"manifest.scm\" file from the current directory or one of its>> ancestors.>> +Return the modified OPTS.">> + (define (options-contain-payload? opts)>> + (match opts>> + (() #f)>> + ((('package . _) . _) #t)>> + ((('load . _) . _) #t)>> + ((('manifest . _) . _) #t)>> + ((('expression . _) . _) #t)>> + ((_ . rest) (options-contain-payload? rest))))>> +>> + (if (options-contain-payload? opts)>> + opts>> + (match (find-file-in-parent-directories '("guix.scm">> "manifest.scm"))>> + (#f>> + (warning (G_ "no packages specified; creating an empty>> environment~%"))>> + opts)>> + (file>> + (info (G_ "loading environment from '~a'...~%") file)>> + (match (basename file)>> + ("guix.scm">> + (alist-cons 'load `(package ,file) opts))>> + ("manifest.scm">> + (alist-cons 'manifest file opts)))))))>> [...]> What would happen on the top-level of the Guix source tree or deep> inside the tree of a guile package that deals with manifests, that> aren't necessarily related to Guix?
You mean a directory that contains a file named ‘guix.scm’ or‘manifest.scm’ but that happens to do something completely unrelated?
We can never rule this out, but I’d say it’s unlikely (these twoconventions are rather well established) and it’s up to the user to payattention.
WDYT?
Thanks,Ludo’.
M
M
Maxime Devos wrote on 2 Oct 15:43 +0200
Re: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache.
91012c32e4f8b04beae71378d44787a81055391a.camel@telenet.be
Ludovic Courtès schreef op za 02-10-2021 om 12:22 [+0200]:
Toggle quote (9 lines)> With this change, running "guix shell" (no arguments) is equivalent to:> > guix environment -r ~/.cache/guix/profiles/some-root -l guix.scm> > This is the cache miss. On cache hit, it's equivalent to:> > guix environment -p ~/.cache/guix/profiles/some-root>
What if guix.scm is something like
;; Load custom package definitions(include "a-package.scm")(include "b-package.scm")(include "c-package.scm")(list a-package b-package c-package bash ...)
and a-package.scm, b-package.scm or c-package.scm is modified?Then the cached profile should be rebuild, no?
I use something like that for my operating system definition(though with -L and use-modules). It would be nice if this workedfor "guix shell" as well.
So I think the cache should also check if these dependencies have been modified.To keep track of the dependencies, something like the ‘compile-all,compile:Keep track of dependencies of compiled modules.’ patch fromhttps://issues.guix.gnu.org/50384 could be used, though the 'include', 'load','include-from-path' and maybe 'use-modules' (if something like "guix shell -Lextra-modules-directory ..."is done) macros would need to be replaced.
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVhh/xccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7iVYAQCEq4K0E44VpESXGHkGswfRcBbGMxm6kspIO3UOyPqV+wEAp26SJiJ651QxY4UGqeGQRXf0hS95YF8Bn4Dk25SZbgk==5gE1-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 2 Oct 15:46 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)(address . 50960@debbugs.gnu.org)
87h7dzzghi.fsf_-_@gnu.org
Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
Toggle quote (6 lines)> Am Samstag, den 02.10.2021, 12:22 +0200 schrieb Ludovic Courtès:>> * guix/scripts/environment.scm (guix-environment*): Bypass calls to>> 'package-derivation' and to 'manifest->derivation' when PROFILE is>> true.> This only affects `guix shell' and not `guix environment', right?
No, it affects ‘guix environment’ (it’s in environment.scm). It’s anoptimization of ‘guix environment -p’, but its observable behavior isunchanged; it’s just faster.
Toggle quote (7 lines)> Then again, if the following holds>> If you run ‘guix pull’ and run again ‘guix shell’, it recomputes>> the environment, as is currently the case with ‘guix environment’.> then the behaviour of guix environment should also be consistent with> what it did before, but with the added cache of guix shell. Am I> reading this correctly?
The cache itself is only in ‘guix shell’, in the no-argument case:
https://issues.guix.gnu.org/50960#9
Ludo’.
L
L
Ludovic Courtès wrote on 2 Oct 15:52 +0200
(name . Jelle Licht)(address . jlicht@fsfe.org)(address . 50960@debbugs.gnu.org)
87bl47zg8n.fsf_-_@gnu.org
Hi!
Jelle Licht <jlicht@fsfe.org> skribis:
Toggle quote (10 lines)> At the risk of doing exactly not that; since it will(/might) already> provide some automagic conveniences, perhaps it makes sense to> additionally load a guix-channels.scm.>> I understand that there would be some duplication of functionality> w.r.t. ‘guix time-machine -C guix-channels.scm -- shell ...’, but if we> already go the DWIW-route, why not go all the way? I think this should> only apply when running ‘guix shell’ without arguments, if that was> unclear.
Oh, it’s tempting yeah, but… (1) only ‘time-machine’ and ‘pull’ dealwith channels currently, so it’d feel kind of weird to take care of thathere, and (2) pulling a specific channel is resource-intensive as youknow, much more than anything else, so I’d rather not have that happenautomatically.
But yeah, I agree that it could be useful. Maybe a first step we couldmake is have ‘time-machine’ load ‘channels.scm’, such that those whowant it can type:
guix time-machine -- shell
?
Ludo’.
M
M
Maxime Devos wrote on 2 Oct 15:52 +0200
Re: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache.
6c199526d88150c31461e3b7f9e76909217e658c.camel@telenet.be
Ludovic Courtès schreef op za 02-10-2021 om 12:22 [+0200]:
Toggle quote (14 lines)> +(define (profile-cache-key file)> + "Return the cache key for the profile corresponding to FILE, a 'guix.scm' or> +'manifest.scm' file, or #f if we lack channel information."> + (match (current-channels)> + (() #f)> + (((= channel-commit commits) ...)> + (let ((stat (stat file)))> + (bytevector->base32-string> + (sha256 (string->utf8> + (string-append (string-join commits) ":"> + (basename file) ":"> + (number->string (stat:dev stat)) ":"> + (number->string (stat:ino stat))))))))))
Why only use the 'basename' of a file name instead of the full name?(Consider the case where a user has multiple "guix.scm" or "manifest.scm".)This turns out to be unproblematic, because stat:dev and stat:ino is includedas well, though including (a part of) the file name is superfluous becausestat:dev and stat:ino are included.
Could you document the rationale for including the file name, and why onlythe basename is included instead of the full file name?
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVhkIxccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7ibCAPsGdx+zzlusF1H+ukK1zNmhBuVVdMZ9ckNiSz2XOoQxfAD+JkzSnybdAHlIxyOm/ARE4P8tJDEgxXsKOnX6tYqrlwE==BuKD-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 2 Oct 16:00 +0200
‘guix shell’ shebangs
(address . 50960@debbugs.gnu.org)
87zgrry1at.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:
Toggle quote (20 lines)> The differences to ‘guix environment’ are:>> 1. ‘--ad-hoc’ is the default.>> ‘guix shell hello’ ≍ ‘guix environment --ad-hoc hello’> ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’>> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> from the current directory or one of its ancestors.>> 3. ‘--load’/‘-l’ is not ‘-f’/‘--install-from-file’ for consistency with> ‘guix package’.>> 4. ‘guix shell’ without arguments maintains a cache, such that, the> second time you run it, it runs in ~0.1s (it does not even need to> connect to the daemon).>> If you run ‘guix pull’ and run again ‘guix shell’, it recomputes> the environment, as is currently the case with ‘guix environment’.
Oh, I had another goal in mind, which was to make it easy to use ‘guixshell’ in shebangs.
Problem is, since it’s called ‘guix shell’ and not ‘guix-shell’, we haveto use ‘/usr/bin/env -S’ anyway as the shebang so it splits arguments.And with ‘-S’, we can already do pretty much everything we want. Here’sa shell script:
Toggle snippet (6 lines)$ cat t.sh#!/usr/bin/env -S guix shell hello bash -- shtype -P hellohello
Should we simply document it or should we do something more?
For the record, ‘nix-shell’ has shebang support where it can interpret“special” lines as arguments:
https://nixos.org/manual/nix/stable/#sec-nix-shell
Ludo’.
L
L
Ludovic Courtès wrote on 2 Oct 16:12 +0200
Re: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache.
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 50960@debbugs.gnu.org)
87ilyfy0q4.fsf@gnu.org
Maxime Devos <maximedevos@telenet.be> skribis:
Toggle quote (21 lines)> Ludovic Courtès schreef op za 02-10-2021 om 12:22 [+0200]:>> With this change, running "guix shell" (no arguments) is equivalent to:>> >> guix environment -r ~/.cache/guix/profiles/some-root -l guix.scm>> >> This is the cache miss. On cache hit, it's equivalent to:>> >> guix environment -p ~/.cache/guix/profiles/some-root>> >> What if guix.scm is something like>> ;; Load custom package definitions> (include "a-package.scm")> (include "b-package.scm")> (include "c-package.scm")> (list a-package b-package c-package bash ...)>> and a-package.scm, b-package.scm or c-package.scm is modified?> Then the cached profile should be rebuild, no?
It should, but it won’t; that’s a weakness of this mtime-based cachingstrategy.
I’m not sure how to address it. We’d need a cache key that’s moreprecise than inode/mtime, yet cheap to compute.
Perhaps we need to live with that limitation, document it, and provide aflag to force a rebuild?
Toggle quote (7 lines)> So I think the cache should also check if these dependencies have been modified.> To keep track of the dependencies, something like the ‘compile-all,compile:> Keep track of dependencies of compiled modules.’ patch from> <https://issues.guix.gnu.org/50384> could be used, though the 'include', 'load',> 'include-from-path' and maybe 'use-modules' (if something like "guix shell -Lextra-modules-directory ..."> is done) macros would need to be replaced.
Problem is that any attempt to keep track of dependencies is always anapproximation because macros can do anything—I can have my own macrothat reads files at expansion time. So I’m inclined to not even try.
Thanks,Ludo’.
L
L
Ludovic Courtès wrote on 2 Oct 16:14 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 50960@debbugs.gnu.org)
87ee93y0n9.fsf@gnu.org
Maxime Devos <maximedevos@telenet.be> skribis:
Toggle quote (24 lines)> Ludovic Courtès schreef op za 02-10-2021 om 12:22 [+0200]:>> +(define (profile-cache-key file)>> + "Return the cache key for the profile corresponding to FILE, a 'guix.scm' or>> +'manifest.scm' file, or #f if we lack channel information.">> + (match (current-channels)>> + (() #f)>> + (((= channel-commit commits) ...)>> + (let ((stat (stat file)))>> + (bytevector->base32-string>> + (sha256 (string->utf8>> + (string-append (string-join commits) ":">> + (basename file) ":">> + (number->string (stat:dev stat)) ":">> + (number->string (stat:ino stat))))))))))>> Why only use the 'basename' of a file name instead of the full name?> (Consider the case where a user has multiple "guix.scm" or "manifest.scm".)> This turns out to be unproblematic, because stat:dev and stat:ino is included> as well, though including (a part of) the file name is superfluous because> stat:dev and stat:ino are included.>> Could you document the rationale for including the file name, and why only> the basename is included instead of the full file name?
Actually it’s probably not useful to include the file (base)name. Ithink initially I thought about distinguishing between guix.scm andmanifest.scm, since they return different kinds of objects, but dev/inois probably enough. WDYT?
Ludo’.
M
M
Maxime Devos wrote on 2 Oct 16:15 +0200
Re: [bug#50960] [PATCH 04/10] DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file.
80360a349abc0eb00a8645fe3e7b5f8008f33ec8.camel@telenet.be
Ludovic Courtès schreef op za 02-10-2021 om 12:22 [+0200]:
Toggle quote (10 lines)> +(define (find-file-in-parent-directories candidates)> + "Find one of CANDIDATES in the current directory or one of its ancestors."> + (let loop ((directory (getcwd)))> + (and (= (stat:uid (stat directory)) (getuid))> + (or (any (lambda (candidate)> + (let ((candidate (string-append directory "/" candidate)))> + (and (file-exists? candidate) candidate)))> + candidates)> + (loop (string-append directory "/..")))))) ;Unix ".." resolution
I do not recommend this. What would happen if someone creates a temporary directory"/tmp/stuff" do things in to throw away later (setting permissions appropriately),tries to create a guix.scm in that directory but misspells it as, say, guix.sm, and runs"guix shell" from within /tmp/stuff? Then find-file-in-parent-directories wouldload /tmp/guix.scm (possibly created by a local attacker, assuming a multi-user system),-- if it weren't for the (= (stat:uid (stat directory)) (getuid)).
Because of the (= (stat:uid ...) (getuid)), this attack method is not possible.However, it causes other issues. Now it isn't possible for two users (that trusteach other), to set up a directory writable by both (e.g. with ACLs, or by makingthe directory group-writable and placing the two users in the same group), forworking together, with a guix.scm usable by both.
These can be two users on the same machine, or remotely via something like NFS,or a single person having multiple user accounts used for different purposes.
(I once created multiple user accounts on Debian: one regular purpose, one for readingand games, and one for school, and made the ‘for-reading’ and ‘school’ home directoryreadable by the ‘regular-purpose’ account. It was occasionally useful.)
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVhpehccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7h14AP4oGokYlwTAj7FIFC7JO93S/qm/bYoetp8tXwpua6omJwEA4mbiRD/NwWir0VBFRneNqIBtnDu3yqHhHXCNtX2CewI==9g2s-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 2 Oct 16:22 +0200
Re: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
7c07bacb08ead4ee3f36b7ff034d3eff82a814ab.camel@telenet.be
Ludovic Courtès schreef op za 02-10-2021 om 16:14 [+0200]:
Toggle quote (14 lines)> > Why only use the 'basename' of a file name instead of the full name?> > (Consider the case where a user has multiple "guix.scm" or "manifest.scm".)> > This turns out to be unproblematic, because stat:dev and stat:ino is included> > as well, though including (a part of) the file name is superfluous because> > stat:dev and stat:ino are included.> > > > Could you document the rationale for including the file name, and why only> > the basename is included instead of the full file name?> > Actually it’s probably not useful to include the file (base)name. I> think initially I thought about distinguishing between guix.scm and> manifest.scm, since they return different kinds of objects, but dev/ino> is probably enough. WDYT?
Looking at https://lwn.net/Articles/866582/,it appears when BTRFS and NFS arecombined, it is possible for two dev/ino pairs to be the same for different files.It appears to be considered a bug but non-trivial to fix. Thus, dev/ino apparentlyis not always sufficient, so it may be reasonable to include the file name.
I would include the full file name, because the basename is often rather generic("guix.scm", "manifest.scm"), though this depends on the habits of the user.
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVhrMhccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7nKHAQD6BCZGjPM4qeIqRrFG+WMLPnQVPtHseP+LHLl7P3OKnQD/Q97JX9o16rY4wMSZajmE4shFgl73moiQmbT/XHJD9gc==rU4o-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 2 Oct 16:47 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
d169e493187d1de26c09dd1503e55cb5db982384.camel@telenet.be
Ludovic Courtès schreef op za 02-10-2021 om 16:12 [+0200]:[reordered]
Toggle quote (11 lines)> > So I think the cache should also check if these dependencies have been modified.> > To keep track of the dependencies, something like the ‘compile-all,compile:> > Keep track of dependencies of compiled modules.’ patch from> > <https://issues.guix.gnu.org/50384> could be used, though the 'include', 'load',> > 'include-from-path' and maybe 'use-modules' (if something like "guix shell -Lextra-modules-directory ..."> > is done) macros would need to be replaced.> > Problem is that any attempt to keep track of dependencies is always an> approximation because macros can do anything—I can have my own macro> that reads files at expansion time. So I’m inclined to not even try.
I expect most people use 'include', 'load', 'include-from-path' or 'load-from-path'instead of writing their own macro reading files at expansion time, and if they dowrite their own macro, I expect it would be implemented in terms of the former anyway,so I expect replacing these macros and 'use-modules' to be sufficient, especiallyif the flag proposed below is available as ‘escape hatch’ for when the dependencytracking is insufficient.
This is not merely an expansion-time problem, files loaded at 'load' or 'eval' timeare important as well. E.g., consider the case where the manifest is something like
(list (package (inherit stuff) (source (local-file "stuff" #:recursive? #t))) other-packages ...)
then all files inside 'stuff' are important as well. I don't see how dependenciescould be tracked here without an excessive amount of 'stat' calls, maybe guix shouldignore these dependencies (possibly with a warning, and a reference to the manualdocumenting which dependencies are tracked and which are not?).
Ludovic Courtès schreef op za 02-10-2021 om 16:12 [+0200]:
Toggle quote (3 lines)> I’m not sure how to address it. We’d need a cache key that’s more> precise than inode/mtime, yet cheap to compute.
The mtime of the file and the mtime of every dependency (ignoring modules fromGuile, Guix and channels to reduce the number of calls to 'stat').
Toggle quote (3 lines)> Perhaps we need to live with that limitation, document it, and provide a> flag to force a rebuild?
A documented flag to always consider the cache stale seems good, though I thinkat least the dependencies made with the common macros and procedures 'include','load', 'include-from-path', 'load-from-path', 'use-modules' and non-recursive'local-file' could be tracked, though this could be left as a TODO for laterI suppose.
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVhw+BccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7q74AP4r7cPOVy4SvOcyF2gpDs/kVWL0J/RNunvEiCSBtLoTNgD+IbIxStM2PdCQH2WbA9iN27BJ9cfJ3dHkzrDPuYG4/wk==nG4D-----END PGP SIGNATURE-----

P
P
pelzflorian (Florian Pelz) wrote on 2 Oct 17:08 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
20211002150851.7xa2wdb225dbjtpi@pelzflorian.localdomain
On Sat, Oct 02, 2021 at 03:40:00PM +0200, Ludovic Courtès wrote:
Toggle quote (11 lines)> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:> > On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote:> >> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> >> from the current directory or one of its ancestors.> > This however is concerning. Users will not expect guix to execute> > arbitrary code. Maybe print a suggestion to maybe --file the file> > instead.> I think it’s fine as long as, as in the case of ‘haunt build’ or ‘make’> or ‘git’, it’s properly documented. Also, ‘guix shell’ unconditionally> writes a message.
Let’s say I have downloaded undesirable code to a file/home/florian/Downloads/guix.scm and am hacking on source code in/home/florian/Downloads/something/ where I run `guix shell`, but/home/florian/Downloads/something/ does not in fact contain aguix.scm file. Now I’d have accidentally run the other guix.scm.
Also `make` is typically used without arguments, but a novice `guixshell` user might know `guix shell program-a program-b` but issurprised when running `guix shell` without arguments in an untrusteddirectory.
But yes, git hooks are dangerous too.
Regards,Florian
C
C
Christine Lemmer-Webber wrote on 2 Oct 15:03 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)
87k0iv37eu.fsf@dustycloud.org
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (48 lines)> Hello Guix!>> Here comes ‘guix shell’, a proposed replacement for ‘guix environment’!> ‘guix environment’ would stay around though, at least for some time,> probably for a long time.>> The differences to ‘guix environment’ are:>> 1. ‘--ad-hoc’ is the default.>> ‘guix shell hello’ ≍ ‘guix environment --ad-hoc hello’> ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’>> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> from the current directory or one of its ancestors.>> 3. ‘--load’/‘-l’ is not ‘-f’/‘--install-from-file’ for consistency with> ‘guix package’.>> 4. ‘guix shell’ without arguments maintains a cache, such that, the> second time you run it, it runs in ~0.1s (it does not even need to> connect to the daemon).>> If you run ‘guix pull’ and run again ‘guix shell’, it recomputes> the environment, as is currently the case with ‘guix environment’.>> Here’s a summary of previous proposals:>> - Dave Thompson: https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html> - [X] --ad-hoc is the default> - [X] caching> - [X] behavior with no arguments> - [ ] --load accepts <environment>> - [ ] Shepherd services> - [ ] 'guix environment --update' to explicitly update> - make --ad-hoc the default: https://issues.guix.gnu.org/38529> - [X] https://issues.guix.gnu.org/38529#17: proposal for a new subcommand> deprecation of ‘guix environment’>> I think <environment> records and Shepherd services could come later.> As for ‘--update’, I prefer the behavior implemented here because it’s> stateless and thus more predictable.>> Thoughts? Are there other changes people would like to see?>> If there’s rough consensus I can work on v2 with documentation. Please> let’s keep the discussion focused. :-)
You have my massive thumbs up. Exciting stuff.
Toggle quote (38 lines)> As for deprecation, I think there’s no rush. I imagine there could be> several phases, like: initially we only mention deprecation in the manual,> later on ‘guix environment’ starts emitting a warning, and later (I guess> at least two years later, probably more) we ask ourselves whether to> remove ‘guix environment’. At this point keeping it doesn’t cost us much.>> Thanks,> Ludo’.>> Ludovic Courtès (10):> packages: Add 'package-development-inputs'.> profiles: Add 'package->development-manifest'.> DRAFT Add 'guix shell'.> DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm'> file.> environment: Add tests for '--profile'.> environment: Skip derivation computation when '--profile' is used.> environment: Do not connect to the daemon when '--profile' is used.> environment: Autoload some modules.> cache: Gracefully handle non-existent cache.> shell: Maintain a profile cache.>> Makefile.am | 2 +> doc/guix.texi | 52 ++++++> guix/cache.scm | 10 +-> guix/packages.scm | 10 ++> guix/profiles.scm | 19 ++> guix/scripts/environment.scm | 260 +++++++++++++++-------------> guix/scripts/shell.scm | 254 +++++++++++++++++++++++++++> po/guix/POTFILES.in | 1 +> tests/guix-environment-container.sh | 8 +> tests/guix-environment.sh | 7 +> tests/guix-shell.sh | 70 ++++++++> tests/packages.scm | 14 ++> tests/profiles.scm | 7 +> 13 files changed, 594 insertions(+), 120 deletions(-)> create mode 100644 guix/scripts/shell.scm> create mode 100644 tests/guix-shell.sh
V
V
Vagrant Cascadian wrote on 3 Oct 01:57 +0200
871r53htdv.fsf@yucca
On 2021-10-02, Ludovic Courtès wrote:
Toggle quote (2 lines)> Here comes ‘guix shell’, a proposed replacement for ‘guix environment’!
Yay!
Toggle quote (4 lines)> ‘guix environment’ would stay around though, at least for some time,> probably for a long time.>> The differences to ‘guix environment’ are:
...
Toggle quote (3 lines)> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> from the current directory or one of its ancestors.
This sounds a little scary to me, just implicitly importing whateverhappens to be lying around doesn't sound very guixy...
Wouldn't it be better to:
guix shell guix.scm
or
guix shell ./guix.scm
or
guix shell --some-argument guix.scm

Or maybe I'm not understanding the idea all that well...

live well, vagrant
-----BEGIN PGP SIGNATURE-----
iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYVjx/QAKCRDcUY/If5cWqkqbAP4ztPWcB+iWaF4iGBwCG6KT26xZORGVFLuLSq3AKd6FBgD+JpB/6mlgkmLz6eSG3kP4InjWlUdSL0908jVO6zz2xgs==a9aj-----END PGP SIGNATURE-----
N
N
Nicolò Balzarotti wrote on 3 Oct 10:36 +0200
87zgrqijxj.fsf@guix.i-did-not-set--mail-host-address--so-tickle-me
Hi!
Vagrant Cascadian <vagrant@debian.org> writes:
Toggle quote (17 lines)> On 2021-10-02, Ludovic Courtès wrote:>> Here comes ‘guix shell’, a proposed replacement for ‘guix environment’!>> Yay!>>> ‘guix environment’ would stay around though, at least for some time,>> probably for a long time.>>>> The differences to ‘guix environment’ are:> ...>> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’>> from the current directory or one of its ancestors.>> This sounds a little scary to me, just implicitly importing whatever> happens to be lying around doesn't sound very guixy...> [...]
What about doing something like what direnv[fn:1] does?
Quoting the website:"direnv checks for the existence of a .envrc file in the current andparent directories. If the file exists (and is authorized), it is loadedinto a bash sub-shell and all exported variables are then captured bydirenv and then made available to the current shell."
The difference between direnv and the current approach is that if thefile has never been "authorized", before it being imported you need torun a command (direnv allow) to authorize it. There's the~/.config/direnv/allow dir which stores files named with the hash of thecontent of the config, and whose content is just the path of the file(don't know why this is needed).
This allows for automatic environment ({manifest,guix}.scm) fileselection AND a it's a bit more secure (it won't run arbitrary coderesiding anywhere in the directory structure).
Except for this, I'd love to see guix shell merged, it will be a majorimprovement over guix environment for my use cases.

Thanks!Nicolò
[fn:1] direnv.net
K
K
Katherine Cox-Buday wrote on 4 Oct 00:50 +0200
Re: [bug#50960] ‘guix shell’ shebangs
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
87k0itbu50.fsf@gmail.com
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (15 lines)> Oh, I had another goal in mind, which was to make it easy to use ‘guix> shell’ in shebangs.>> Problem is, since it’s called ‘guix shell’ and not ‘guix-shell’, we have> to use ‘/usr/bin/env -S’ anyway as the shebang so it splits arguments.> And with ‘-S’, we can already do pretty much everything we want. Here’s> a shell script:>> --8<---------------cut here---------------start------------->8---> $ cat t.sh> #!/usr/bin/env -S guix shell hello bash -- sh> type -P hello> hello> --8<---------------cut here---------------end--------------->8---
I was already very excited about this patch series, and now I see this!
I wanted to point out that I don't think Guix's shebang substitution phases know how to deal with ~env -S~, and I wish they would (I recently ran into this). I can't really think of any use-cases for a Guix package which when built contains a ~guix shell~ script, but I wanted to raise the possibility in case someone else can.
-- Katherine
Z
Z
zimoun wrote on 4 Oct 08:56 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)
868rz9ffck.fsf@gmail.com
Hi,
All looks good to me; from the surface, I have not looked at thedetails.

On Sat, 02 Oct 2021 at 12:21, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (3 lines)> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> from the current directory or one of its ancestors.
Personally, I do not like this default behaviour because explicit isbetter than implicit and in the face of ambiguity, refuse the temptationto guess; as any good Zen says. :-)
Idem with the proposal ’channels.scm’ and time-machine.

Toggle quote (6 lines)> As for deprecation, I think there’s no rush. I imagine there could be> several phases, like: initially we only mention deprecation in the manual,> later on ‘guix environment’ starts emitting a warning, and later (I guess> at least two years later, probably more) we ask ourselves whether to> remove ‘guix environment’. At this point keeping it doesn’t cost us much.
Concretely, I propose this plan:
- v1.4: mention deprecation in the manual and remove from “guix help” - v1.5: emit a warning - v1.6: remove the command
Well, I do not see why the removal should be an issue, because there is“guix time-machine”. To me, the real issue is to let people knowingsuch change.
Moreover, I am doubtful that the exact same command-line from v1.0 isproducing the same result with v1.3 for instance. Not because of Guixbut because of the underlying packages.
For example, http://issues.guix.gnu.org/47097.

All the best,simon
L
L
Ludovic Courtès wrote on 4 Oct 10:07 +0200
Re: [bug#50960] [PATCH 04/10] DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file.
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 50960@debbugs.gnu.org)
87bl45tdpn.fsf@gnu.org
Hi Maxime,
Maxime Devos <maximedevos@telenet.be> skribis:
Toggle quote (20 lines)> Ludovic Courtès schreef op za 02-10-2021 om 12:22 [+0200]:>> +(define (find-file-in-parent-directories candidates)>> + "Find one of CANDIDATES in the current directory or one of its ancestors.">> + (let loop ((directory (getcwd)))>> + (and (= (stat:uid (stat directory)) (getuid))>> + (or (any (lambda (candidate)>> + (let ((candidate (string-append directory "/" candidate)))>> + (and (file-exists? candidate) candidate)))>> + candidates)>> + (loop (string-append directory "/..")))))) ;Unix ".." resolution>> I do not recommend this. What would happen if someone creates a temporary directory> "/tmp/stuff" do things in to throw away later (setting permissions appropriately),> tries to create a guix.scm in that directory but misspells it as, say, guix.sm, and runs> "guix shell" from within /tmp/stuff? Then find-file-in-parent-directories would> load /tmp/guix.scm (possibly created by a local attacker, assuming a multi-user system),> -- if it weren't for the (= (stat:uid (stat directory)) (getuid)).>> Because of the (= (stat:uid ...) (getuid)), this attack method is not possible.
Right. :-)
In libgit2, ‘find_repo’ (called by ‘git_repository_discover’) stops atdevice boundaries, which is wise. But it doesn’t stop when the parenthas a different owner (!).
Unlike the code above, it does lexical “..” resolution after firstcalling realpath(3) on the directory name; not sure what to think aboutthis. (The code of Git itself is harder to read for me.)
Toggle quote (8 lines)> However, it causes other issues. Now it isn't possible for two users (that trust> each other), to set up a directory writable by both (e.g. with ACLs, or by making> the directory group-writable and placing the two users in the same group), for> working together, with a guix.scm usable by both.>> These can be two users on the same machine, or remotely via something like NFS,> or a single person having multiple user accounts used for different purposes.
Well, sure, but that’s a very uncommon scenario, isn’t it?
I was actually hesitant about this find-in-parent behavior. I find itconvenient that ‘git’ does that, for instance, so I thought it might benice as well.
Thoughts?
Ludo’.
L
L
Ludovic Courtès wrote on 4 Oct 10:08 +0200
Re: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache.
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 50960@debbugs.gnu.org)
877dettdns.fsf@gnu.org
Maxime Devos <maximedevos@telenet.be> skribis:
Toggle quote (23 lines)> Ludovic Courtès schreef op za 02-10-2021 om 16:14 [+0200]:>> > Why only use the 'basename' of a file name instead of the full name?>> > (Consider the case where a user has multiple "guix.scm" or "manifest.scm".)>> > This turns out to be unproblematic, because stat:dev and stat:ino is included>> > as well, though including (a part of) the file name is superfluous because>> > stat:dev and stat:ino are included.>> > >> > Could you document the rationale for including the file name, and why only>> > the basename is included instead of the full file name?>> >> Actually it’s probably not useful to include the file (base)name. I>> think initially I thought about distinguishing between guix.scm and>> manifest.scm, since they return different kinds of objects, but dev/ino>> is probably enough. WDYT?>> Looking at https://lwn.net/Articles/866582/, it appears when BTRFS and NFS are> combined, it is possible for two dev/ino pairs to be the same for different files.> It appears to be considered a bug but non-trivial to fix. Thus, dev/ino apparently> is not always sufficient, so it may be reasonable to include the file name.>> I would include the full file name, because the basename is often rather generic> ("guix.scm", "manifest.scm"), though this depends on the habits of the user.
OK, that makes sense to me, let’s do that.
Ludo’.
L
L
Ludovic Courtès wrote on 4 Oct 10:19 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 50960@debbugs.gnu.org)
87lf39ryle.fsf@gnu.org
Maxime Devos <maximedevos@telenet.be> skribis:
Toggle quote (29 lines)> Ludovic Courtès schreef op za 02-10-2021 om 16:12 [+0200]:> [reordered]>> > So I think the cache should also check if these dependencies have been modified.>> > To keep track of the dependencies, something like the ‘compile-all,compile:>> > Keep track of dependencies of compiled modules.’ patch from>> > <https://issues.guix.gnu.org/50384> could be used, though the 'include', 'load',>> > 'include-from-path' and maybe 'use-modules' (if something like "guix shell -Lextra-modules-directory ...">> > is done) macros would need to be replaced.>> >> Problem is that any attempt to keep track of dependencies is always an>> approximation because macros can do anything—I can have my own macro>> that reads files at expansion time. So I’m inclined to not even try.>> I expect most people use 'include', 'load', 'include-from-path' or 'load-from-path'> instead of writing their own macro reading files at expansion time, and if they do> write their own macro, I expect it would be implemented in terms of the former anyway,> so I expect replacing these macros and 'use-modules' to be sufficient, especially> if the flag proposed below is available as ‘escape hatch’ for when the dependency> tracking is insufficient.>> This is not merely an expansion-time problem, files loaded at 'load' or 'eval' time> are important as well. E.g., consider the case where the manifest is something like>> (list> (package> (inherit stuff)> (source (local-file "stuff" #:recursive? #t)))> other-packages ...)
In a ‘guix.scm’ file, ‘source’ doesn’t matter. It does matter in amanifest, but that sounds a bit less common.
Toggle quote (5 lines)> then all files inside 'stuff' are important as well. I don't see how dependencies> could be tracked here without an excessive amount of 'stat' calls, maybe guix should> ignore these dependencies (possibly with a warning, and a reference to the manual> documenting which dependencies are tracked and which are not?).
[...]
Toggle quote (6 lines)> A documented flag to always consider the cache stale seems good, though I think> at least the dependencies made with the common macros and procedures 'include',> 'load', 'include-from-path', 'load-from-path', 'use-modules' and non-recursive> 'local-file' could be tracked, though this could be left as a TODO for later> I suppose.
Tracking those uses reliably is impossible: there could be same-namedbindings that do other things, there could be custom macros, there couldbe “dynamic arguments” (whose value is not known statically), etc. Youhave to expand + evaluate the code to get better results, and even then,there might be different paths in the code so you can’t be sure you gotit right.
We could get an approximation for common uses by recognizing specialforms as you suggest. But it’s just that, an approximation.
In such situations, I err on the side of not even trying. The addedcomplexity for a flaky result doesn’t pay off to me. I prefer to beupfront, document limitations, and let users handle them as they seefit.
WDYT?
A similar problem occurs with system provenance tracking, which saves‘configuration.scm’ but leaves it up to the user to preserve additionalfiles if needed (info "(guix) Service Reference").
Thanks,Ludo’.
L
L
Ludovic Courtès wrote on 4 Oct 10:22 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 50960@debbugs.gnu.org)
87ee91ryg1.fsf@gnu.org
Hi,
"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:
Toggle quote (18 lines)> On Sat, Oct 02, 2021 at 03:40:00PM +0200, Ludovic Courtès wrote:>> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:>> > On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote:>> >> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’>> >> from the current directory or one of its ancestors.>> > This however is concerning. Users will not expect guix to execute>> > arbitrary code. Maybe print a suggestion to maybe --file the file>> > instead.>> I think it’s fine as long as, as in the case of ‘haunt build’ or ‘make’>> or ‘git’, it’s properly documented. Also, ‘guix shell’ unconditionally>> writes a message.>> Let’s say I have downloaded undesirable code to a file> /home/florian/Downloads/guix.scm and am hacking on source code in> /home/florian/Downloads/something/ where I run `guix shell`, but> /home/florian/Downloads/something/ does not in fact contain a> guix.scm file. Now I’d have accidentally run the other guix.scm.
Sure, but it’s all under your control; it’s not very different fromsomeone knowingly running “guix build -f guix.scm” on an untrusted file,is it?
Toggle quote (5 lines)> Also `make` is typically used without arguments, but a novice `guix> shell` user might know `guix shell program-a program-b` but is> surprised when running `guix shell` without arguments in an untrusted> directory.
We have the advantage that ‘guix shell’ is a new command, so we candocument it from the start as behaving this way without arguments.
WDYT?
Thanks,Ludo’.
L
L
Ludovic Courtès wrote on 4 Oct 10:34 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Vagrant Cascadian)(address . vagrant@debian.org)(address . 50960@debbugs.gnu.org)
874k9xrxxj.fsf@gnu.org
Hello!
Vagrant Cascadian <vagrant@debian.org> skribis:
Toggle quote (2 lines)> On 2021-10-02, Ludovic Courtès wrote:
[...]
Toggle quote (6 lines)>> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’>> from the current directory or one of its ancestors.>> This sounds a little scary to me, just implicitly importing whatever> happens to be lying around doesn't sound very guixy...
Right, it would be the first command that does that.
I became quite convinced that conventions and, thus, implicit argumentscan occasionally improve usability. We use tools that operate this waydaily: ‘make’, ‘git’, etc. Dave nicely argued about it:
https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html
But yeah, it is a departure from what we’ve done so far, so we shouldtake the time to think through it.
Thanks,Ludo’.
L
L
Ludovic Courtès wrote on 4 Oct 10:39 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 50960@debbugs.gnu.org)
87r1d1qj3v.fsf@gnu.org
Hi,
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (9 lines)> On Sat, 02 Oct 2021 at 12:21, Ludovic Courtès <ludo@gnu.org> wrote:>>> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’>> from the current directory or one of its ancestors.>> Personally, I do not like this default behaviour because explicit is> better than implicit and in the face of ambiguity, refuse the temptation> to guess; as any good Zen says. :-)
It should be clear from the code that I generally prefer explicit overimplicit :-), but I think Dave has a point when talking aboutconventions for this kind of developer tool.
Toggle quote (12 lines)>> As for deprecation, I think there’s no rush. I imagine there could be>> several phases, like: initially we only mention deprecation in the manual,>> later on ‘guix environment’ starts emitting a warning, and later (I guess>> at least two years later, probably more) we ask ourselves whether to>> remove ‘guix environment’. At this point keeping it doesn’t cost us much.>> Concretely, I propose this plan:>> - v1.4: mention deprecation in the manual and remove from “guix help”> - v1.5: emit a warning> - v1.6: remove the command
Could be like this. I guess we could also slow down the plan if weobserve that ‘guix environment’ sticks around longer than we thought.
Toggle quote (4 lines)> Well, I do not see why the removal should be an issue, because there is> “guix time-machine”. To me, the real issue is to let people knowing> such change.
As discussed in https://issues.guix.gnu.org/38529#17, removal is anissue because of existing scripts but also because of learning materialaround: MOOCs, articles, tutorials, etc. These won’t be updatedovernight, and we owe our users stability.
Ludo’.
P
P
pelzflorian (Florian Pelz) wrote on 4 Oct 11:23 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
20211004092317.576ffl52qm2g2utr@pelzflorian.localdomain
On Mon, Oct 04, 2021 at 10:22:54AM +0200, Ludovic Courtès wrote:
Toggle quote (11 lines)> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:> > Let’s say I have downloaded undesirable code to a file> > /home/florian/Downloads/guix.scm and am hacking on source code in> > /home/florian/Downloads/something/ where I run `guix shell`, but> > /home/florian/Downloads/something/ does not in fact contain a> > guix.scm file. Now I’d have accidentally run the other guix.scm.> > Sure, but it’s all under your control; it’s not very different from> someone knowingly running “guix build -f guix.scm” on an untrusted file,> is it?
What I meant is that I may wrongly expect a guix.scm file in/home/florian/Downloads/something/, but it is not there, so things goawry.
`guix shell` loading files by default would mean one would have to payattention to what one is doing, unlike `guix environment`. Forexample, not save unrelated (not even malicious) code by the nameguix.scm, and not run guix commands without inspecting what they’d do.This I don’t like.


Toggle quote (3 lines)> We have the advantage that ‘guix shell’ is a new command, so we can> document it from the start as behaving this way without arguments.
Many people don’t read manuals.
The probability of an accident is low, but it feels not robust.I can live with either (and am very happy you and others keepimproving Guix), I just don’t think loading by default is a good idea.
Regards,Florian
Z
Z
zimoun wrote on 4 Oct 12:40 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
86a6jpdqf2.fsf@gmail.com
Hi,
On Mon, 04 Oct 2021 at 10:39, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (4 lines)> It should be clear from the code that I generally prefer explicit over> implicit :-), but I think Dave has a point when talking about> conventions for this kind of developer tool.
Well, if you speak about this thread [1], the point is others are doingso and it is an usability improvement. :-)
I am fine since somehow it is already the case with ’make’ or all theconfig files or many of us seem to simplify their workflow using direnvor etc. so yes it is probably handy to have conventions and automaticload by default. :-)
1: https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html

Toggle quote (15 lines)>>> As for deprecation, I think there’s no rush. I imagine there could be>>> several phases, like: initially we only mention deprecation in the manual,>>> later on ‘guix environment’ starts emitting a warning, and later (I guess>>> at least two years later, probably more) we ask ourselves whether to>>> remove ‘guix environment’. At this point keeping it doesn’t cost us much.>>>> Concretely, I propose this plan:>>>> - v1.4: mention deprecation in the manual and remove from “guix help”>> - v1.5: emit a warning>> - v1.6: remove the command>> Could be like this. I guess we could also slow down the plan if we> observe that ‘guix environment’ sticks around longer than we thought.
Bah, let reconsider this when preparing v1.6. ;-) At the currentrelease rate, probably 2024. :-)
From my point of view, we should clearly document at v1.4 that thiscommand will be removed soon or later.

Toggle quote (9 lines)>> Well, I do not see why the removal should be an issue, because there is>> “guix time-machine”. To me, the real issue is to let people knowing>> such change.>> As discussed in <https://issues.guix.gnu.org/38529#17>, removal is an> issue because of existing scripts but also because of learning material> around: MOOCs, articles, tutorials, etc. These won’t be updated> overnight, and we owe our users stability.
For sure, I agree «we owe our users stability». That’s I translate into«the real issue is to let people knowing such change». :-)
As I am trying to explain, this scenario,
guix environment <options> <packages> … wait several months or years … guix pull guix environment <options> <packages>
will break because <packages>. Therefore, one will have to fix thesevery same scripts. Even us, we are not able to maintain working scriptsfor different points in time. The example using “guix environment” isbroken for the manual released at v1.3, as pointed here [2].
However, “guix environment” will still work using “guix time-machine”.Nothing is really broken, IMHO.
About learning materials, we could remove the command “guix environment”but keep a section in the manual explaining how to switch to the new“guix shell”.
Well, let discuss all that when preparing v1.5 or v1.6. Something asone or two years from now. ;-) Which i
2: http://issues.guix.gnu.org/47097

Cheers,simon
L
L
Ludovic Courtès wrote on 4 Oct 14:23 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 50960@debbugs.gnu.org)
878rz9q8q1.fsf@gnu.org
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (2 lines)> On Mon, 04 Oct 2021 at 10:39, Ludovic Courtès <ludo@gnu.org> wrote:
[...]
Toggle quote (8 lines)>> As discussed in https://issues.guix.gnu.org/38529#17, removal is an>> issue because of existing scripts but also because of learning material>> around: MOOCs, articles, tutorials, etc. These won’t be updated>> overnight, and we owe our users stability.>> For sure, I agree «we owe our users stability». That’s I translate into> «the real issue is to let people knowing such change». :-)
No no. The problem is that there *will* be material out thereexplaining Guix in terms of ‘guix environment’, people *will* stumble onit, and it’s important to keep it working long enough so people canupdate their material.
Ludo’.
Z
Z
zimoun wrote on 4 Oct 15:42 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
87zgrosy8k.fsf_-_@gmail.com
On Mon, 4 Oct 2021 at 14:23, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (10 lines)> zimoun <zimon.toutoune@gmail.com> skribis:
> > For sure, I agree «we owe our users stability». That’s I translate into> > «the real issue is to let people knowing such change». :-)>> No no. The problem is that there *will* be material out there> explaining Guix in terms of ‘guix environment’, people *will* stumble on> it, and it’s important to keep it working long enough so people can> update their material.
We are saying the same, no?
Except that for me "people" and "material" are vague so I propose to adda section to deal with this vagueness at removal time (maybe 2023 or2024?) explaining how to switch from "guix enviornment" to "guix shell".Therefore, people will stumble on this not-updated material, then theywill be able to run it with "guix shell" using the proposed section.One could image:
$ guix enviornment <old> <example> <how> <to> -- <use> <it> error: environment had be removed by commit: xxxxx. hint: Try `guix time-machine --commit=yyyyy -- environment <old> <example> <how> <to> -- <use> <it>' Please read section "guix environment" in the manual.
where yyyy is that last commit right before the removal. Or somethingalong this idea.

Well, we agree to speak about that when preparing v1.5 or v1.6 and weare far from that. ;-)
Cheers,simon
Z
Z
zimoun wrote on 4 Oct 16:20 +0200
Re: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache.
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAJ3okZ01u41eV+w=CjE_jVT1bqkmFZVQTwmzq+8g0ObM7RxqqQ@mail.gmail.com
Hi,
On Mon, 4 Oct 2021 at 10:52, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (5 lines)> In such situations, I err on the side of not even trying. The added> complexity for a flaky result doesn’t pay off to me. I prefer to be> upfront, document limitations, and let users handle them as they see> fit.
I agree. Just a flag for forcing seems be worth here. For thisdependencies use-case or simply for debugging.
Cheers,simon
M
M
Maxime Devos wrote on 4 Oct 17:58 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
f821bfcc64304e467575a26c4473ddc6863eb42e.camel@telenet.be
Ludovic Courtès schreef op ma 04-10-2021 om 10:19 [+0200]:
Toggle quote (13 lines)> > A documented flag to always consider the cache stale seems good, though I think> > at least the dependencies made with the common macros and procedures 'include',> > 'load', 'include-from-path', 'load-from-path', 'use-modules' and non-recursive> > 'local-file' could be tracked, though this could be left as a TODO for later> > I suppose.> > Tracking those uses reliably is impossible: there could be same-named> bindings that do other things, there could be custom macros, there could> be “dynamic arguments” (whose value is not known statically), etc. You> have to expand + evaluate the code to get better results, and even then,> there might be different paths in the code so you can’t be sure you got> it right.
I think there's a miscommunication here. From what I'm reading, what you havein mind is that, to determine the dependency information, "guix shell" wouldopen "guix.scm", read it with the procedure 'read' and sniff for 'load','include-from-path', 'load-from-path', 'use-modules' and 'local-file' form-- something like 'module-file-dependencies' and 'extract-dependencies', butmore general.
However, my idea is to replace these macros, such that, when "guix shell"loads "guix.scm" or "manifest.scm", these macros inform "guix shell"that "guix.scm" or "manifest.scm" depend on certain files referred toby 'load', 'include-from-path', etc. forms, using a mechanism like the'notice-dependency' defined in https://issues.guix.gnu.org/50384.
Then, when "guix shell" puts the resulting profile in the cache,it includes the generated list of files. And when "guix shell" findsan entry in the cache, it will check if the files in this list (and guix.scmor manifest.scm of course) have been modified. If some are (or the forcingflag is set), guix.scm needs to be loaded and a new profile needs to begenerated. If they are all unchanged, guix.scm will _not_ be read: the cachedprofile can be reused.
It's not 100% reliable (e.g. the list of packages in the manifest coulddepend on the phase of the moon if (current-time) is used) (is that whatyou mean by ‘different paths’ and ‘dynamic arguments’?), but it shouldcover the vast majority of cases.
I don't know a non-artifical situation where ‘custom macros’ are a problem-- do you know an example in the wild where this dependency tracking schemewould fail to work?
Toggle quote (3 lines)> We could get an approximation for common uses by recognizing special> forms as you suggest. But it’s just that, an approximation.
It's an approximation, sure, but it seems to be a quite accurate approximationto me. And it's not really recognising special forms that I'm suggesting,but rather modifying the macros behind these forms to inform "guix shell"of what the dependencies are.
Toggle quote (5 lines)> In such situations, I err on the side of not even trying. The added> complexity for a flaky result doesn’t pay off to me. I prefer to be> upfront, document limitations, and let users handle them as they see> fit.
About complexity: there's some extra code, sure, but it doesn't seem complexto me. To track dependencies in https://issues.guix.gnu.org/50384,I only needed to add 'notice-dependency' and some other code to (guix build compile),and some code to build-aux/compile-all.scm to save/load the dependency informationto/from the builddir and to also check the mtime of dependencies.
Does it still seem flaky to you, after my explanation on how the dependencyinformation would be determined?
Being upfront, documenting limitations, and having a ‘force rebuild flag’(is that what you mean by ‘letting users handle them as they see fit’?)(possibly also documenting 'notice-dependency') is not incompatible withthe automatic dependency tracking.
More abstractly, this seems more like a ‘Perfect is the enemy of the good’ situation.While I would prefer 'perfect' above 'good', and the automatic dependency trackingcertainly isn't 'perfect', it does seem 'good' to me, and perfection appears tobe impossible and there's the ‘force rebuild flag’ as an escape hatch, so I believe'good-but-not-perfect' to be acceptable here, as long as it is documented in the manualthat there are some limitation on what dependencies are automatically tracked.
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVskshccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7o9RAQCMHGDE6veWG0GNgcj9MUOlBs/9G5xDCZkVsqfM7KsIpQEA5QWvEXT6O9lcGYZUU6VByqjg/T2t45wdRTDSOeThzAc==dE78-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 4 Oct 18:50 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(address . 50960@debbugs.gnu.org)
991f4d26de32b3b9c93b1112920f11427a54f4fd.camel@telenet.be
Ludovic Courtès schreef op ma 04-10-2021 om 10:22 [+0200]:
Toggle quote (26 lines)> Hi,> > "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:> > > On Sat, Oct 02, 2021 at 03:40:00PM +0200, Ludovic Courtès wrote:> > > "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:> > > > On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote:> > > > > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> > > > > from the current directory or one of its ancestors.> > > > This however is concerning. Users will not expect guix to execute> > > > arbitrary code. Maybe print a suggestion to maybe --file the file> > > > instead.> > > I think it’s fine as long as, as in the case of ‘haunt build’ or ‘make’> > > or ‘git’, it’s properly documented. Also, ‘guix shell’ unconditionally> > > writes a message.> > > > Let’s say I have downloaded undesirable code to a file> > /home/florian/Downloads/guix.scm and am hacking on source code in> > /home/florian/Downloads/something/ where I run `guix shell`, but> > /home/florian/Downloads/something/ does not in fact contain a> > guix.scm file. Now I’d have accidentally run the other guix.scm.> > Sure, but it’s all under your control; it’s not very different from> someone knowingly running “guix build -f guix.scm” on an untrusted file,> is it?
Consider the following situation:
1. I browse the web and find some rando's website. It has a link to a "guix.scm" to download. 2. I'd like to know how people are using guix, so I tell IceCat to download it. IceCat downloads it to ~/Downloads/guix.scm. 3. I forget about the guix.scm and didn't look at it. 4. I download some tarball, verify it (with gpg or something), unpack it, and run "guix shell" without arguments from within the directory (e.g. ~/Downloads/some-source-code). 5. It turns out the tarball didn't actually have a guix.scm, so the ~/Downloads/guix.scm from the rando is loaded. 6. It turns out the rando's guix.scm uploads my secret keys, passwords, all e-mails, installs a keylogger ... Oops!
Toggle quote (8 lines)> > Also `make` is typically used without arguments, but a novice `guix> > shell` user might know `guix shell program-a program-b` but is> > surprised when running `guix shell` without arguments in an untrusted> > directory.> > We have the advantage that ‘guix shell’ is a new command, so we can> document it from the start as behaving this way without arguments.
Sure, this behaviour can be documented, but it's very easy to forget a pieceof documentation, especially if the behaviour is inconsistent between "guix environment"and "guix shell", and an attacker only needs an attack to function once.
I'd prefer not be constantly kept on my toes, so if "guix shell" will automaticallyload guix.scm in the current directory or parent directories, I think I'llkeep using "guix environment" to avoid any opportunities for fatal mistakes.
Greetings,Maximes.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVsw1BccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7iNvAP9w6uhC55k8rLsfc6FuAyUHJKqwQ+4aU5maKG+kj1U+zQD7BTf7JP1DbslnAKWsTztizW/ALYUzCn15gP4y0KfE/Ao==WfW1-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 4 Oct 19:12 +0200
(address . 50960@debbugs.gnu.org)
9fedb2b3797673ad7f6f0cbd5731cbc632b1b587.camel@telenet.be
Ludovic Courtès schreef op ma 04-10-2021 om 10:34 [+0200]:
Toggle quote (20 lines)> Hello!> > Vagrant Cascadian <vagrant@debian.org> skribis:> > > On 2021-10-02, Ludovic Courtès wrote:> > [...]> > > > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> > > from the current directory or one of its ancestors.> > > > This sounds a little scary to me, just implicitly importing whatever> > happens to be lying around doesn't sound very guixy...> > Right, it would be the first command that does that.> > I became quite convinced that conventions and, thus, implicit arguments> can occasionally improve usability. We use tools that operate this way> daily: ‘make’, ‘git’, etc. Dave nicely argued about it:
'git' doesn't run binaries in the repository, unless configured otherwise(in .git/config I think). ‘make’ and ‘bundle’ are verbs and are for buildingsource code, which needs to be checked for backdoors anyway, so those programsimplicitely reading code from the current directory seems acceptable.
"guix sh" seems to be useful outside software development.E.g. I sometimes do"guix environment --pure --ad-hoc minetest various-minetest-mods-... -- minetest",which would become"guix shell --pure minetest various-minetest-mods-... -- minetest".I could very easily accidentally press the enter key after typing "shell"(I write from personal experience), and this could easily happen from within,say, a ~/Downloads directory with an untrusted guix.scm (e.g. downloaded fromsome rando's site to look at later).
Conventions are nice, but loading arbitrary code from the current directoryby default is an exploit waiting to happen. This situation seem like including "."in PATH by default to me.
Greetings,Maxime
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVs2CBccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7ju2APwMhHDa5IZLEuzUpci5cznfcmvrqvJYcqAZ1o9bTBTsvQEA+mwVlhVJkbqa2R/eEqAvAesOY3jR1LQlh2w5IDUQOg8==L6bZ-----END PGP SIGNATURE-----

L
L
Leo Famulari wrote on 4 Oct 19:38 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
YVs8CBpBBnw6gFzZ@jasmine.lan
On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote:
Toggle quote (4 lines)> Here comes ‘guix shell’, a proposed replacement for ‘guix environment’!> ‘guix environment’ would stay around though, at least for some time,> probably for a long time.
Very nice!
Toggle quote (2 lines)> ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’
Is my understanding correct, that arguments to `guix environment` arepositional, whereas with `guix shell` arguments are not positional?
T
T
Thompson, David wrote on 4 Oct 23:29 +0200
Re: [EXT] [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
CAJ=RwfZJqZg9K1bH0fdZbC+LaRmfueb9=3tBU5Y=vhrmYKKp3g@mail.gmail.com
Hey Ludo,
First of all: Thanks for picking up where I left off all those yearsago. You remember mailing list posts I wrote 4 years ago much betterthan I do. ;)
On Sat, Oct 2, 2021 at 6:22 AM Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (46 lines)>> Hello Guix!>> Here comes ‘guix shell’, a proposed replacement for ‘guix environment’!> ‘guix environment’ would stay around though, at least for some time,> probably for a long time.>> The differences to ‘guix environment’ are:>> 1. ‘--ad-hoc’ is the default.>> ‘guix shell hello’ ≍ ‘guix environment --ad-hoc hello’> ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’>> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’> from the current directory or one of its ancestors.>> 3. ‘--load’/‘-l’ is not ‘-f’/‘--install-from-file’ for consistency with> ‘guix package’.>> 4. ‘guix shell’ without arguments maintains a cache, such that, the> second time you run it, it runs in ~0.1s (it does not even need to> connect to the daemon).>> If you run ‘guix pull’ and run again ‘guix shell’, it recomputes> the environment, as is currently the case with ‘guix environment’.>> Here’s a summary of previous proposals:>> - Dave Thompson: https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html> - [X] --ad-hoc is the default> - [X] caching> - [X] behavior with no arguments> - [ ] --load accepts <environment>> - [ ] Shepherd services> - [ ] 'guix environment --update' to explicitly update> - make --ad-hoc the default: https://issues.guix.gnu.org/38529> - [X] https://issues.guix.gnu.org/38529#17: proposal for a new subcommand> deprecation of ‘guix environment’>> I think <environment> records and Shepherd services could come later.> As for ‘--update’, I prefer the behavior implemented here because it’s> stateless and thus more predictable.>> Thoughts? Are there other changes people would like to see?
A few thoughts:
1) I can't be the only one that thinks it's a pain to rebuild thewhole environment just because I updated the Guix client. It'sespecially frustrating when there is a regression that breaks theproject, or the substitute servers are having a bad day.
2) One important use-case for tools like Bundler, npm, etc. is toexactly (within their limited ability to do so, of course) reproducethe environment for all developers working on the project. With 'guixenvironment', and now with this proposed 'guix shell', the resultingdevelopment environment depends upon the version of Guix being used.So, to make 'guix shell' a true universal replacement for theselanguage-specific tools, there should be a way to "lock" to a specificversion of Guix. Bundler has Gemfile.lock, npm has package-lock.json,etc. Guix has good support for using older versions of Guix already,so it seems feasible. I spend a non-trivial amount of time sortingout development environment issues for a couple dozen devs,so a tool that took most of the variables out of the equation would bereally great.
3) I haven't looked at the patches to see if it has already beenimplemented, but when running 'guix shell' with no arguments, I thinkthe search for the manifest file should follow the establishedconventions of Git, Bundler, etc. and search not only the currentdirectory, but the parent directory and so on until it finds it orgets to / and gives up. This will make the tool much more usable whenworking inside a subdirectory within a project.
Just some things to think about. I just don't want to see Guix committo an interface that slams the door on future improvement due tocompatibility reasons. I mean, what would you even name the nexttool? guix... biome?
Anyway, great work! Back to lurking.
- Dave
M
M
Maxime Devos wrote on 5 Oct 09:50 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(address . 50960@debbugs.gnu.org)
51388f2a0e841af6f1b4441d3dc34e38996e7209.camel@telenet.be
Ludovic Courtès schreef op za 02-10-2021 om 15:43 [+0200]:
Toggle quote (50 lines)> Hi,> > Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:> > > Am Samstag, den 02.10.2021, 12:22 +0200 schrieb Ludovic Courtès:> > > [...]> > > +(define (auto-detect-manifest opts)> > > + "If OPTS do not specify packages or a manifest, load a> > > \"guix.scm\" or> > > +\"manifest.scm\" file from the current directory or one of its> > > ancestors.> > > +Return the modified OPTS."> > > + (define (options-contain-payload? opts)> > > + (match opts> > > + (() #f)> > > + ((('package . _) . _) #t)> > > + ((('load . _) . _) #t)> > > + ((('manifest . _) . _) #t)> > > + ((('expression . _) . _) #t)> > > + ((_ . rest) (options-contain-payload? rest))))> > > +> > > + (if (options-contain-payload? opts)> > > + opts> > > + (match (find-file-in-parent-directories '("guix.scm"> > > "manifest.scm"))> > > + (#f> > > + (warning (G_ "no packages specified; creating an empty> > > environment~%"))> > > + opts)> > > + (file> > > + (info (G_ "loading environment from '~a'...~%") file)> > > + (match (basename file)> > > + ("guix.scm"> > > + (alist-cons 'load `(package ,file) opts))> > > + ("manifest.scm"> > > + (alist-cons 'manifest file opts)))))))> > > [...]> > What would happen on the top-level of the Guix source tree or deep> > inside the tree of a guile package that deals with manifests, that> > aren't necessarily related to Guix?> > You mean a directory that contains a file named ‘guix.scm’ or> ‘manifest.scm’ but that happens to do something completely unrelated?> > We can never rule this out, but I’d say it’s unlikely (these two> conventions are rather well established) and it’s up to the user to pay> attention.> > WDYT?
Guix itself doesn't follow this convention: the guix source tree has an unrelated"guix.scm" file, that doesn't evaluate to a package. I'd expect that running"guix shell" within the guix source tree would be equivalent to"guix environment guix", but currently this doesn't seem to be the case.
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVwDqxccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7ka0AQDnU7a+eWbslkX7fyYYmApXdwoxf30IDVhPdf7QlCRmiQEA/ISOlBj627fvuxi78KBfTj9oA+4xX5WqI6+fcRHVQgU==8DCn-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 5 Oct 09:51 +0200
Re: [bug#50960] [PATCH 04/10] DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file.
1e4e9ace6c74c37f2f0aaba354ad89d642f81e2b.camel@telenet.be
Ludovic Courtès schreef op za 02-10-2021 om 12:22 [+0200]:
Toggle quote (69 lines)> DRAFT: Add doc.> > * guix/scripts/shell.scm (parse-args): Add call to 'auto-detect-manifest'.> (find-file-in-parent-directories, auto-detect-manifest): New procedures.> * tests/guix-shell.sh: Add test.> ---> guix/scripts/shell.scm | 44 ++++++++++++++++++++++++++++++++++++++++--> tests/guix-shell.sh | 16 +++++++++++++++> 2 files changed, 58 insertions(+), 2 deletions(-)> > diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm> index 6a4b7a5092..2f15befbd3 100644> --- a/guix/scripts/shell.scm> +++ b/guix/scripts/shell.scm> @@ -22,6 +22,8 @@> #:autoload (guix scripts build) (show-build-options-help)> #:autoload (guix transformations) (show-transformation-options-help)> #:use-module (guix scripts)> + #:use-module (guix packages)> + #:use-module (guix profiles)> #:use-module (srfi srfi-1)> #:use-module (srfi srfi-26)> #:use-module (srfi srfi-37)> @@ -121,13 +123,51 @@ interactive shell in that environment.\n"))> ;; The '--' token is used to separate the command to run from the rest of> ;; the operands.> (let ((args command (break (cut string=? "--" <>) args)))> - (let ((opts (parse-command-line args %options (list %default-options)> - #:argument-handler handle-argument)))> + (let ((opts (auto-detect-manifest> + (parse-command-line args %options (list %default-options)> + #:argument-handler handle-argument))))> (match command> (() opts)> (("--") opts)> (("--" command ...) (alist-cons 'exec command opts))))))> > +(define (find-file-in-parent-directories candidates)> + "Find one of CANDIDATES in the current directory or one of its ancestors."> + (let loop ((directory (getcwd)))> + (and (= (stat:uid (stat directory)) (getuid))> + (or (any (lambda (candidate)> + (let ((candidate (string-append directory "/" candidate)))> + (and (file-exists? candidate) candidate)))> + candidates)> + (loop (string-append directory "/..")))))) ;Unix ".." resolution> +> +(define (auto-detect-manifest opts)> + "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or> +\"manifest.scm\" file from the current directory or one of its ancestors.> +Return the modified OPTS."> + (define (options-contain-payload? opts)> + (match opts> + (() #f)> + ((('package . _) . _) #t)> + ((('load . _) . _) #t)> + ((('manifest . _) . _) #t)> + ((('expression . _) . _) #t)> + ((_ . rest) (options-contain-payload? rest))))> +> + (if (options-contain-payload? opts)> + opts> + (match (find-file-in-parent-directories '("guix.scm" "manifest.scm"))> + (#f> + (warning (G_ "no packages specified; creating an empty environment~%"))> + opts)> + (file> + (info (G_ "loading environment from '~a'...~%") file)
Could we have nice ‘curly quotes’ here:
(info (G_ "loading environment from ‘~a’...~%") file)
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVwEBBccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7oKkAPwKaNzKVGx+AL+ncrNdaJSSur75elAgSG/m8u9LFrNqsQEAuFU3chzE9XCEQIHbcx3qVuHyGj6PyQ2Cok7OqcsjYQA==LxUM-----END PGP SIGNATURE-----

K
K
Konrad Hinsen wrote on 6 Oct 10:12 +0200
Re: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(address . 50960@debbugs.gnu.org)
m1fste4lnp.fsf@fastmail.net
Hi everyone,
I just discovered this discussion today... a lot to read...
First of all, I very much like this proposal. It provides manyimprovements to today's "guix environment" for which I have immediateuse cases.
Looking at the proposal and then the discussion, I wonder if the mainissue with "guix shell" is that it's two very different things wrappedinto one:
1) A convenience utility for interactive work, in particular software development, implemented as "guix shell" with no arguments.
2) A modernized "guix environment" for both interactive and scripting use.
The behavior of "guix shell" with and without arguments is sufficientlydifferent, which will make it a challenge to document, and then achallenge for users to understand. And part of the criticism related tothe tacit execution of files is about the convenience for 1) vs. therisk in 2).
So... how about making this two different commands? They could of courseshare most of the implementation.
I'd use "guix shell" for the scenario that actually starts a shell, andsomething different, perhaps "guix process", for the infrastructurecommand for use in scripts.
Cheers, Konrad
L
L
Ludovic Courtès wrote on 7 Oct 10:34 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Konrad Hinsen)(address . konrad.hinsen@fastmail.net)(address . 50960@debbugs.gnu.org)
87mtnlfd2n.fsf_-_@gnu.org
Hi Konrad,
Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:
Toggle quote (10 lines)> Looking at the proposal and then the discussion, I wonder if the main> issue with "guix shell" is that it's two very different things wrapped> into one:>> 1) A convenience utility for interactive work, in particular software> development, implemented as "guix shell" with no arguments.>> 2) A modernized "guix environment" for both interactive and scripting> use.
To me it’s the same as ‘guix environment’, but with a slightly differentcommand-line interface; ‘guix environment’ was already serves these twouse cases.
Toggle quote (6 lines)> The behavior of "guix shell" with and without arguments is sufficiently> different, which will make it a challenge to document, and then a> challenge for users to understand. And part of the criticism related to> the tacit execution of files is about the convenience for 1) vs. the> risk in 2).
‘guix shell’ without arguments is equivalent to ‘guix environment -fguix.scm’ or ‘guix environment -m manifest.scm’.
Toggle quote (7 lines)> So... how about making this two different commands? They could of course> share most of the implementation.>> I'd use "guix shell" for the scenario that actually starts a shell, and> something different, perhaps "guix process", for the infrastructure> command for use in scripts.
I don’t think there’s enough to warrant two different commands (andperhaps we could leave out the auto-detection of ‘guix.scm’ or‘manifest.scm’ is that proves to be too controversial).
Maybe you’re hinting at the name and the fact that it suggests it’sstarting a shell. I’m not fond of it, notably for that reason. For therecord, it came out as the only proposal that seemed viable to me:
https://issues.guix.gnu.org/38529#17
Thoughts?
Thanks for your feedback,Ludo’.
L
L
Liliana Marie Prikler wrote on 7 Oct 11:15 +0200
(address . 50960@debbugs.gnu.org)
dbd60e1f57398800a215020032d9ebfba478e0cb.camel@gmail.com
Hi Ludo,
Am Donnerstag, den 07.10.2021, 10:34 +0200 schrieb Ludovic Courtès:
Toggle quote (19 lines)> > The behavior of "guix shell" with and without arguments is> > sufficiently different, which will make it a challenge to document,> > and then a challenge for users to understand. And part of the> > criticism related to the tacit execution of files is about the> > convenience for 1) vs. the risk in 2).> > ‘guix shell’ without arguments is equivalent to ‘guix environment -f> guix.scm’ or ‘guix environment -m manifest.scm’.> > > So... how about making this two different commands? They could of> > course share most of the implementation.> > > > I'd use "guix shell" for the scenario that actually starts a shell,> > and something different, perhaps "guix process", for the> > infrastructure command for use in scripts.> > I don’t think there’s enough to warrant two different commands (and> perhaps we could leave out the auto-detection of ‘guix.scm’ or> ‘manifest.scm’ is that proves to be too controversial).
Leaving out the auto-detection of ‘guix.scm’ and ‘manifest.scm’ for nowis a good idea, given that Guix itself would be an edge-case for that.
Perhaps to avoid this trouble, guix shell could read a file that simplyspecifies which arguments to add one line at a time e.g.
;; .guix-shell-rc --manifest=manifest.scm
or
;; .guix-shell-rc --load=guix.scm
Similar behaviour is used in other (GNU) tools with lots of availablearguments, which operate on a directory basis, e.g. stow.
Toggle quote (8 lines)> Maybe you’re hinting at the name and the fact that it suggests it’s> starting a shell. I’m not fond of it, notably for that reason. For> the record, it came out as the only proposal that seemed viable to> me:> > https://issues.guix.gnu.org/38529#17> > Thoughts?
I think “guix shell” is fine as-is. People typically don't complainthat “sh somefile.sh” doesn't launch a shell, so the same reasoning canbe applied to Guix tools in my opinion. We would still have the alwaysnon-intuitive naming choices of “do”, “start” and “run” at our disposalif we ever need to discard the shell.
Regards,Liliana
L
L
Ludovic Courtès wrote on 7 Oct 11:26 +0200
(name . Thompson, David)(address . dthompson2@worcester.edu)(address . 50960@debbugs.gnu.org)
877depdw3i.fsf_-_@gnu.org
Hello!
"Thompson, David" <dthompson2@worcester.edu> skribis:
Toggle quote (5 lines)> 1) I can't be the only one that thinks it's a pain to rebuild the> whole environment just because I updated the Guix client. It's> especially frustrating when there is a regression that breaks the> project, or the substitute servers are having a bad day.
The way I see it, I update Guix precisely to get a newer environment.Otherwise, why would I update it for?
Toggle quote (14 lines)> 2) One important use-case for tools like Bundler, npm, etc. is to> exactly (within their limited ability to do so, of course) reproduce> the environment for all developers working on the project. With 'guix> environment', and now with this proposed 'guix shell', the resulting> development environment depends upon the version of Guix being used.> So, to make 'guix shell' a true universal replacement for these> language-specific tools, there should be a way to "lock" to a specific> version of Guix. Bundler has Gemfile.lock, npm has package-lock.json,> etc. Guix has good support for using older versions of Guix already,> so it seems feasible. I spend a non-trivial amount of time sorting> out development environment issues for a couple dozen devs,> so a tool that took most of the variables out of the equation would be> really great.
This part is currently addressed by ‘guix describe -f channels’ (to pinthe Guix revision) and ‘guix time-machine’ (to spin up a specificrevision). The other tools are totally unaware of revisions andchannels.
As it stands, developers would use the pinned version by running:
guix time-machine --commit=XYZ -- shell
or:
guix time-machine -C channels.scm -- shell
Perhaps ‘guix time-machine’, too, could automatically detect‘channels.scm’.
The temptation to add revision data to tools that don’t otherwise dealwith it is real; I think it’s better kept separate though, becausefundamentally, each revision lives in its own world.
Toggle quote (8 lines)> 3) I haven't looked at the patches to see if it has already been> implemented, but when running 'guix shell' with no arguments, I think> the search for the manifest file should follow the established> conventions of Git, Bundler, etc. and search not only the current> directory, but the parent directory and so on until it finds it or> gets to / and gives up. This will make the tool much more usable when> working inside a subdirectory within a project.
It’s implemented (and controversial).
Toggle quote (5 lines)> Just some things to think about. I just don't want to see Guix commit> to an interface that slams the door on future improvement due to> compatibility reasons. I mean, what would you even name the next> tool? guix... biome?
Good point. :-)
Thanks,Ludo’.
P
P
pelzflorian (Florian Pelz) wrote on 7 Oct 12:52 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)
20211007105236.xzkq5tij3g3tcdob@pelzflorian.localdomain
I’m not a direnv user and frankly don’t really understand the gainfrom loading by default the guix.scm file nor manifest.scm.
My fear is accidentally running code, possibly malicious, possiblyjust a backup script one happened to call guix.scm. In German we havethe word DAU for “dumbest assumable user”. Guix without shell isDAU-prove.
Both
- Konrad Hinsen’s suggestion of two different commands and
- Nicolò Balzarotti’s suggestion of having to explicitly allow a file
would take away my paranoia of accidentally running code. (By justnot using Konrad Hinsen’s other command.)
- Hidden .-rc files in the current directory would not take away my paranoia of malicious code.
- I think printing a suggestion to use --file=../guix.scm is fine.
Bash is different from Guix Shell. It does not load code from . orarbitrary parent directories.
Regards,Florian
T
T
Thompson, David wrote on 7 Oct 13:17 +0200
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)
CAJ=Rwfb7fSEuWYxnnPE_miqGF5_O_d8qJdi3XdmwSQsFE=8jXA@mail.gmail.com
On Thu, Oct 7, 2021 at 6:52 AM pelzflorian (Florian Pelz)<pelzflorian@pelzflorian.de> wrote:
Toggle quote (9 lines)>> I’m not a direnv user and frankly don’t really understand the gain> from loading by default the guix.scm file nor manifest.scm.>> My fear is accidentally running code, possibly malicious, possibly> just a backup script one happened to call guix.scm. In German we have> the word DAU for “dumbest assumable user”. Guix without shell is> DAU-prove.
I don't think your fear should outweigh the immense user friendlinessof having some established conventions to ease development. Everymainstream developer tool I use in the professional word hasconventions like this, and the result is that it is quick and easy tojump into a new project and get the software you need to beproductive. Guix needs to embrace this kind of usability if it hopesto offer a real alternative to these other tools. 'guix shell' withno other arguments should "just work" in the context of whateverproject I am in.
- Dave
P
P
pelzflorian (Florian Pelz) wrote on 7 Oct 14:01 +0200
(name . Thompson, David)(address . dthompson2@worcester.edu)
20211007120119.sdhip6yvj2doslsr@pelzflorian.localdomain
On Thu, Oct 07, 2021 at 07:17:43AM -0400, Thompson, David wrote:
Toggle quote (4 lines)> 'guix shell' with> no other arguments should "just work" in the context of whatever> project I am in.
If you only care about projects: Searching only guix.scm andmanifest.scm files when a .git subdirectory is present would indeed bebetter than going through all parent directories. I do not know gitwell enough to judge if that could interfere with git worktrees andgit submodules.
The only remaining problem would be novice userswho know “guix shell program-a program-b” and who do not expect “guixshell” without argument instead loads code from a file.
I’m also afraid of git hooks btw, which is why I clone old gitcheckouts from the backup instead of copying, but at least they staylocal to the .git checkout.
I still don’t understand why not having to type --file is usefulthough.
Regards,Florian
L
L
Ludovic Courtès wrote on 8 Oct 09:37 +0200
Re: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache.
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 50960@debbugs.gnu.org)
8735pc6k6k.fsf@gnu.org
Hi Maxime,
Maxime Devos <maximedevos@telenet.be> skribis:
Toggle quote (21 lines)> Ludovic Courtès schreef op ma 04-10-2021 om 10:19 [+0200]:>> > A documented flag to always consider the cache stale seems good, though I think>> > at least the dependencies made with the common macros and procedures 'include',>> > 'load', 'include-from-path', 'load-from-path', 'use-modules' and non-recursive>> > 'local-file' could be tracked, though this could be left as a TODO for later>> > I suppose.>> >> Tracking those uses reliably is impossible: there could be same-named>> bindings that do other things, there could be custom macros, there could>> be “dynamic arguments” (whose value is not known statically), etc. You>> have to expand + evaluate the code to get better results, and even then,>> there might be different paths in the code so you can’t be sure you got>> it right.>> I think there's a miscommunication here. From what I'm reading, what you have> in mind is that, to determine the dependency information, "guix shell" would> open "guix.scm", read it with the procedure 'read' and sniff for 'load',> 'include-from-path', 'load-from-path', 'use-modules' and 'local-file' form> -- something like 'module-file-dependencies' and 'extract-dependencies', but> more general.
Yes, that’s roughly what I thought you had in mind, sorry!
Toggle quote (23 lines)> However, my idea is to replace these macros, such that, when "guix shell"> loads "guix.scm" or "manifest.scm", these macros inform "guix shell"> that "guix.scm" or "manifest.scm" depend on certain files referred to> by 'load', 'include-from-path', etc. forms, using a mechanism like the> 'notice-dependency' defined in <https://issues.guix.gnu.org/50384>.>> Then, when "guix shell" puts the resulting profile in the cache,> it includes the generated list of files. And when "guix shell" finds> an entry in the cache, it will check if the files in this list (and guix.scm> or manifest.scm of course) have been modified. If some are (or the forcing> flag is set), guix.scm needs to be loaded and a new profile needs to be> generated. If they are all unchanged, guix.scm will _not_ be read: the cached> profile can be reused.>> It's not 100% reliable (e.g. the list of packages in the manifest could> depend on the phase of the moon if (current-time) is used) (is that what> you mean by ‘different paths’ and ‘dynamic arguments’?), but it should> cover the vast majority of cases.>> I don't know a non-artifical situation where ‘custom macros’ are a problem> -- do you know an example in the wild where this dependency tracking scheme> would fail to work?
I think we have to look at the cost and at the benefits. The cost:either we spread ‘notice-dependency’ calls all over the place, or weinterpose on ‘open-file’ altogether, assuming that even works. Thebenefit: we’d have an 80% solution. There would still be edge cases nothandled correctly; a realistic example is a ‘guix.scm’ whose executiondiffers based on the presence of a file or environment variable. Peoplewould have to know about this pitfall and pass the right flag.
[...]
Toggle quote (26 lines)>> In such situations, I err on the side of not even trying. The added>> complexity for a flaky result doesn’t pay off to me. I prefer to be>> upfront, document limitations, and let users handle them as they see>> fit.>> About complexity: there's some extra code, sure, but it doesn't seem complex> to me. To track dependencies in <https://issues.guix.gnu.org/50384>,> I only needed to add 'notice-dependency' and some other code to (guix build compile),> and some code to build-aux/compile-all.scm to save/load the dependency information> to/from the builddir and to also check the mtime of dependencies.>> Does it still seem flaky to you, after my explanation on how the dependency> information would be determined?>> Being upfront, documenting limitations, and having a ‘force rebuild flag’> (is that what you mean by ‘letting users handle them as they see fit’?)> (possibly also documenting 'notice-dependency') is not incompatible with> the automatic dependency tracking.>> More abstractly, this seems more like a ‘Perfect is the enemy of the good’ situation.> While I would prefer 'perfect' above 'good', and the automatic dependency tracking> certainly isn't 'perfect', it does seem 'good' to me, and perfection appears to> be impossible and there's the ‘force rebuild flag’ as an escape hatch, so I believe> 'good-but-not-perfect' to be acceptable here, as long as it is documented in the manual> that there are some limitation on what dependencies are automatically tracked.
Yeah, I can relate to that sentiment, even if I’m all too often on the“perfectionist” side of things. ;-)
I guess I could live with an 80% dependency tracking solution. However,my criteria for it would be that (1) it should be orthogonal (e.g., notrequiring explicit calls to ‘notice-dependency’ in many places), and (2)it should be self-contained and reasonably small. Perhaps having‘load*’ or similar instrument ‘open-file’ or similar could help achievethat?
I would rather not block ‘guix shell’ on that though.
WDYT?
Thanks for taking the time to clarify what you had in mind!
Ludo’.
L
L
Ludovic Courtès wrote on 8 Oct 09:43 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Leo Famulari)(address . leo@famulari.name)(address . 50960@debbugs.gnu.org)
87o88055d4.fsf@gnu.org
Hi,
Leo Famulari <leo@famulari.name> skribis:
Toggle quote (2 lines)> On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote:
[...]
Toggle quote (5 lines)>> ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’>> Is my understanding correct, that arguments to `guix environment` are> positional, whereas with `guix shell` arguments are not positional?
No, it’s the same. However, ‘-D’ here affects only the immediatelyfollowing package, whereas ‘--ad-hoc’ would affect every package thatfollows.
Ludo’.
L
L
Ludovic Courtès wrote on 8 Oct 09:44 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)
87k0io55b1.fsf@gnu.org
Maxime Devos <maximedevos@telenet.be> skribis:
Toggle quote (3 lines)> Guix itself doesn't follow this convention: the guix source tree has an unrelated> "guix.scm" file, that doesn't evaluate to a package.
Indeed, but that’s because Guix predates the convention. :-)
Toggle quote (4 lines)> I'd expect that running "guix shell" within the guix source tree would> be equivalent to "guix environment guix", but currently this doesn't> seem to be the case.
Right, it would fail poorly in this particular case.
Ludo’.
K
K
Katherine Cox-Buday wrote on 8 Oct 16:24 +0200
(name . Thompson, David)(address . dthompson2@worcester.edu)
878rz3lhln.fsf@gmail.com
"Thompson, David" <dthompson2@worcester.edu> writes:
Toggle quote (12 lines)> I don't think your fear should outweigh the immense user friendliness> of having some established conventions to ease development. Every> mainstream developer tool I use in the professional word has> conventions like this, and the result is that it is quick and easy to> jump into a new project and get the software you need to be> productive. Guix needs to embrace this kind of usability if it hopes> to offer a real alternative to these other tools. 'guix shell' with> no other arguments should "just work" in the context of whatever> project I am in.>> - Dave
I agree with this.
-- Katherine
K
K
Konrad Hinsen wrote on 8 Oct 17:45 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)(address . 50960@debbugs.gnu.org)
m1pmsffrky.fsf@fastmail.net
Hi Ludo and Liliana,
Toggle quote (4 lines)> To me it’s the same as ‘guix environment’, but with a slightly different> command-line interface; ‘guix environment’ was already serves these two> use cases.
Indeed, but the new proposal goes much further with supportinginteractive convenience, in particular via the controversial reading ofguix.scm and/or manifest.scm. That's a convenience I'd like to haveinteractively, but not something I expect from an infrastructure toolthat ends up being used in lengthy shell script. For the latter, the"Zen of Python wisdom" of "explicit is better than implicit" matters alot. On the command line, it makes life harder, in particular for badtypists such as myself.
Toggle quote (6 lines)> Perhaps to avoid this trouble, guix shell could read a file that simply> specifies which arguments to add one line at a time e.g.>> ;; .guix-shell-rc> --manifest=manifest.scm
That's a nice idea, but again something I wouldn't want aninfrastructure tool to do. When I use "guix environment" or "guix shell"in a script or workflow, I want whatever it does to be as independent aspossible from my machine or account, except when I explicitly say thecontrary.
Cheers, Konrad
L
L
Leo Famulari wrote on 8 Oct 18:16 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
YWBuwMzkvzKnTr3Z@jasmine.lan
On Fri, Oct 08, 2021 at 09:43:19AM +0200, Ludovic Courtès wrote:
Toggle quote (4 lines)> No, it’s the same. However, ‘-D’ here affects only the immediately> following package, whereas ‘--ad-hoc’ would affect every package that> follows.
I must not know the correct terminology; that's what I was trying toask.
Can we keep the old behavior? Or do we think this new way of doing itis better?
L
L
Liliana Marie Prikler wrote on 9 Oct 09:45 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
b263ffa2e4a1a13ba83aec6bafe2e03410e42009.camel@gmail.com
Am Freitag, den 08.10.2021, 17:45 +0200 schrieb Konrad Hinsen:
Toggle quote (26 lines)> Hi Ludo and Liliana,> > > To me it’s the same as ‘guix environment’, but with a slightly> > different command-line interface; ‘guix environment’ was already> > serves these two use cases.> > Indeed, but the new proposal goes much further with supporting> interactive convenience, in particular via the controversial reading> of guix.scm and/or manifest.scm. That's a convenience I'd like to> have interactively, but not something I expect from an infrastructure> tool that ends up being used in lengthy shell script. For the latter,> the "Zen of Python wisdom" of "explicit is better than implicit"> matters a lot. On the command line, it makes life harder, in> particular for bad typists such as myself.> > > Perhaps to avoid this trouble, guix shell could read a file that> > simply specifies which arguments to add one line at a time e.g.> > > > ;; .guix-shell-rc> > --manifest=manifest.scm> > That's a nice idea, but again something I wouldn't want an> infrastructure tool to do. When I use "guix environment" or "guix> shell" in a script or workflow, I want whatever it does to be as> independent as possible from my machine or account, except when I> explicitly say the contrary.
People do use “emacs -Q” so there's no hard “one way or the other”requirement. All we would need to do is offer a similar flag toprevent reading the rc file, no?
Cheers,Liliana
S
S
Stefan wrote on 9 Oct 10:07 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(address . liliana.prikler@gmail.com)
F72DB966-D292-4169-AFD2-0FBD9C9B7BAC@vodafonemail.de
Hi!
How about a directory argument to load some standard file? And using a special ... to search all parent directories?
guix shell . ;# ./guix.scmguix shell .. ;# ../guix.scmguix shell ... ;# ./(../)*guix.scm

Bye
Stefan
L
L
Ludovic Courtès wrote on 9 Oct 15:38 +0200
(name . Leo Famulari)(address . leo@famulari.name)(address . 50960@debbugs.gnu.org)
87y272xqq8.fsf@gnu.org
Hi,
Leo Famulari <leo@famulari.name> skribis:
Toggle quote (11 lines)> On Fri, Oct 08, 2021 at 09:43:19AM +0200, Ludovic Courtès wrote:>> No, it’s the same. However, ‘-D’ here affects only the immediately>> following package, whereas ‘--ad-hoc’ would affect every package that>> follows.>> I must not know the correct terminology; that's what I was trying to> ask.>> Can we keep the old behavior? Or do we think this new way of doing it> is better?
What do you mean by “old behavior”?
I think there’s large consensus that ‘--ad-hoc’ should be the default,and that’s the main motivation behind this patch series.
As for ‘-D’: we could arrange so that it has effect on all the followingpackages. However, if we did that, we’d need to reintroduce ‘--ad-hoc’to cancel the effect of ‘-D’.
More importantly, I think having ‘-D’ affect only the immediatelyfollowing package is more consistent with typical usage—I don’t thinkI’ve ever run ‘guix environment foo bar baz’ to get the mergeddevelopment environments of all three packages.
Thanks,Ludo’.
L
L
Leo Famulari wrote on 11 Oct 02:29 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
YWOFaa1uC94fvRC4@jasmine.lan
On Sat, Oct 09, 2021 at 03:38:55PM +0200, Ludovic Courtès wrote:
Toggle quote (2 lines)> What do you mean by “old behavior”?
I mean that all the packages listed after the option are affected by it.
Toggle quote (5 lines)> More importantly, I think having ‘-D’ affect only the immediately> following package is more consistent with typical usage—I don’t think> I’ve ever run ‘guix environment foo bar baz’ to get the merged> development environments of all three packages.
I see what you mean: that is a weird behaviour. Let's disregard thequestion I brought up.
L
L
Ludovic Courtès wrote on 11 Oct 10:32 +0200
Re: bug#50960: [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Konrad Hinsen)(address . konrad.hinsen@fastmail.net)
87bl3w0xo3.fsf@inria.fr
Hi,
Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:
Toggle quote (13 lines)>> To me it’s the same as ‘guix environment’, but with a slightly different>> command-line interface; ‘guix environment’ was already serves these two>> use cases.>> Indeed, but the new proposal goes much further with supporting> interactive convenience, in particular via the controversial reading of> guix.scm and/or manifest.scm. That's a convenience I'd like to have> interactively, but not something I expect from an infrastructure tool> that ends up being used in lengthy shell script. For the latter, the> "Zen of Python wisdom" of "explicit is better than implicit" matters a> lot. On the command line, it makes life harder, in particular for bad> typists such as myself.
Agreed. The automatic reading of guix.scm/manifest.scm, if we keep it,should only happen in interactive use; I’ll double-check and make surethis is the case.
Thanks,Ludo’.
Z
Z
zimoun wrote on 11 Oct 11:13 +0200
Re: [bug#50960] [PATCH 00/10] Add 'guix shell' to subsume 'guix environment'
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAJ3okZ3ZpujpsCamLAQ315LvZEizKA-4RaVv=Ys-WpR783ngLQ@mail.gmail.com
Hi Ludo,
On Thu, 7 Oct 2021 at 11:27, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (10 lines)> "Thompson, David" <dthompson2@worcester.edu> skribis:>> > 1) I can't be the only one that thinks it's a pain to rebuild the> > whole environment just because I updated the Guix client. It's> > especially frustrating when there is a regression that breaks the> > project, or the substitute servers are having a bad day.>> The way I see it, I update Guix precisely to get a newer environment.> Otherwise, why would I update it for?
I think there is a kind of misunderstanding because one updates Guixbecause of new project B but then rebuilds the whole environment forproject A. And this can be unpleasant.
The solution is to use "guix time-machine -C channels.scm --environment" but again this can be unpleasant because it builds a lotof things; although before one runs "guix environment".
Well, you should tell me that the solution is to create a profile.However, I often use "guix environment" with the options --pure or--container, and a profile cannot, IIUC.

Toggle quote (3 lines)> Perhaps ‘guix time-machine’, too, could automatically detect> ‘channels.scm’.
Well, I am not convince it helps for readibility. At the end, it would read:
guix time-machine -- shell
but then to know what this command does exactly, one should open themanual check the "guix time-machine" part, check the presence of the'channels.scm' file, then read the "guix shell" part, then check if ifis 'manifests.scm' or 'guix.scm'. Maybe, it is just habits, but Ifind much simpler:
guix time-machine -C channels -- shell -m manifest.scm guix time-machine -C channels -- shell -f guix.scm
where I just have at the command-line type "guix time-machine --help"or "guix shell --help" to remember the options and if I do not knowwhat manifest is, then I go to the manual.
With completion, it is not longer to type. "Explicit is better thanimplicit" as a good Zen says!

Toggle quote (4 lines)> The temptation to add revision data to tools that don’t otherwise deal> with it is real; I think it’s better kept separate though, because> fundamentally, each revision lives in its own world.
I agree.

All the best,simon
L
L
Ludovic Courtès wrote on 11 Oct 23:37 +0200
[PATCH v2 01/11] packages: Add 'package-development-inputs'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-2-ludo@gnu.org
* guix/packages.scm (package-development-inputs): New procedure.* guix/scripts/environment.scm (package-environment-inputs): Use it.* tests/packages.scm ("package-development-inputs")("package-development-inputs, cross-compilation"): New tests.* doc/guix.texi (package Reference): Document it.--- doc/guix.texi | 41 ++++++++++++++++++++++++++++++++++++ guix/packages.scm | 10 +++++++++ guix/scripts/environment.scm | 2 +- tests/packages.scm | 14 ++++++++++++ 4 files changed, 66 insertions(+), 1 deletion(-)
Toggle diff (120 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex a72a726b54..49399e792b 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -6847,6 +6847,47 @@ cross-compiling: It is an error to refer to @code{this-package} outside a package definition. @end deffn +@cindex development inputs, of a package+@cindex implicit inputs, of a package+Sometimes you will want to obtain the list of inputs needed to+@emph{develop} a package---all the inputs that are visible when the+package is compiled. This is what the @code{package-development-inputs}+procedure returns.++@deffn {Scheme Procedure} package-development-inputs @var{package} @+ [@var{system}] [#:target #f]+Return the list of inputs required by @var{package} for development+purposes on @var{system}. When @var{target} is true, return the inputs+needed to cross-compile @var{package} from @var{system} to+@var{triplet}, where @var{triplet} is a triplet such as+@code{"aarch64-linux-gnu"}.++Note that the result includes both explicit inputs and implicit+inputs---inputs automatically added by the build system (@pxref{Build+Systems}). Let us take the @code{hello} package to illustrate that:++@lisp+(use-modules (gnu packages base) (guix packages))++hello+@result{} #<package hello@@2.10 gnu/packages/base.scm:79 7f585d4f6790>++(package-direct-inputs hello)+@result{} ()++(package-development-inputs hello)+@result{} (("source" @dots{}) ("tar" #<package tar@@1.32 @dots{}>) @dots{})+@end lisp++In this example, @code{package-direct-inputs} returns the empty list,+because @code{hello} has zero explicit dependencies. Conversely,+@code{package-development-inputs} includes inputs implicitly added by+@code{gnu-build-system} that are required to build @code{hello}: tar,+gzip, GCC, libc, Bash, and more. To visualize it, @command{guix graph+hello} would show you explicit inputs, whereas @command{guix graph -t+bag hello} would include implicit inputs (@pxref{Invoking guix graph}).+@end deffn+ Because packages are regular Scheme objects that capture a complete dependency graph and associated build procedures, it is often useful to write procedures that take a package and return a modified versiondiff --git a/guix/packages.scm b/guix/packages.scmindex 8c3a0b0b7b..43e0130793 100644--- a/guix/packages.scm+++ b/guix/packages.scm@@ -153,6 +153,7 @@ (define-module (guix packages) bag-transitive-host-inputs bag-transitive-build-inputs bag-transitive-target-inputs+ package-development-inputs package-closure default-guile@@ -1070,6 +1071,15 @@ (define (bag-transitive-target-inputs bag) (%current-system (bag-system bag))) (transitive-inputs (bag-target-inputs bag)))) +(define* (package-development-inputs package+ #:optional (system (%current-system))+ #:key target)+ "Return the list of inputs required by PACKAGE for development purposes on+SYSTEM. When TARGET is true, return the inputs needed to cross-compile+PACKAGE from SYSTEM to TRIPLET, where TRIPLET is a triplet such as+\"aarch64-linux-gnu\"."+ (bag-transitive-inputs (package->bag package system target)))+ (define* (package-closure packages #:key (system (%current-system))) "Return the closure of PACKAGES on SYSTEM--i.e., PACKAGES and the list of packages they depend on, recursively."diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 6958bd6238..d555969b27 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -82,7 +82,7 @@ (define (package-environment-inputs package) packages for PACKAGE." ;; Remove non-package inputs such as origin records. (filter-map input->manifest-entry- (bag-transitive-inputs (package->bag package))))+ (package-development-inputs package system))) (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]diff --git a/tests/packages.scm b/tests/packages.scmindex 3756877270..266b5aeb7a 100644--- a/tests/packages.scm+++ b/tests/packages.scm@@ -353,6 +353,20 @@ (define read-at (package-transitive-supported-systems d) (package-transitive-supported-systems e)))) +(test-assert "package-development-inputs"+ ;; Note: Due to propagated inputs, 'package-development-inputs' returns a+ ;; couple more inputs, such as 'linux-libre-headers'.+ (lset<= equal?+ `(("source" ,(package-source hello)) ,@(standard-packages))+ (package-development-inputs hello)))++(test-assert "package-development-inputs, cross-compilation"+ (lset<= equal?+ `(("source" ,(package-source hello))+ ,@(standard-cross-packages "mips64el-linux-gnu" 'host)+ ,@(standard-cross-packages "mips64el-linux-gnu" 'target))+ (package-development-inputs hello #:target "mips64el-linux-gnu")))+ (test-assert "package-closure" (let-syntax ((dummy-package/no-implicit (syntax-rules ()-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:37 +0200
[PATCH v2 00/11] 'guix shell' strikes again
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-1-ludo@gnu.org
Hello Guix!
Here’s a v2 with the following changes:
• ‘guix.scm’ and ‘manifest.scm’ only loaded in interactive mode, as suggested by Konrad.
• Auto-loading happens if and only if the containing directory is listed in ~/.config/guix/shell-authorized-directories, as suggested by Florian. When a file is found but not authorized, a hint is displayed explaining what to do.
• ‘-q’ inhibits auto-loading.
• Auto-detection of ‘guix.scm’ and ‘manifest.scm’ uses lexical “..” resolution and does not cross device boundaries.
• Long version of ‘-f’ is now ‘--file’, like for ‘guix build’.
• Caching also works with files explicitly specified with ‘-m’ or ‘-f’. That way, ‘guix shell -f guix.scm -- make’ benefits from it.
• Cache key does not include file name, to avoid cache misses given that the file name is not canonicalized (see discussion with Maxime).
• Documentation, including a tentative EOL date for ‘guix environment’ (“supported until at least May 1st, 2023”). The intention is to give an incentive to switch while reassuring that it won’t disappear overnight.
How does that sound? :-)
Ludo’.
Ludovic Courtès (11): packages: Add 'package-development-inputs'. profiles: Add 'package->development-manifest'. Add 'guix shell'. DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file. DRAFT shell: Honor in ~/.config/guix/shell-authorized-directories. environment: Add tests for '--profile'. environment: Skip derivation computation when '--profile' is used. environment: Do not connect to the daemon when '--profile' is used. environment: Autoload some modules. cache: Gracefully handle non-existent cache. shell: Maintain a profile cache.
Makefile.am | 2 + doc/contributing.texi | 8 +- doc/guix.texi | 443 ++++++++++++++++++++++++++-- guix/cache.scm | 10 +- guix/packages.scm | 10 + guix/profiles.scm | 19 ++ guix/scripts/environment.scm | 260 ++++++++-------- guix/scripts/shell.scm | 355 ++++++++++++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-environment-container.sh | 8 + tests/guix-environment.sh | 7 + tests/guix-shell.sh | 100 +++++++ tests/packages.scm | 14 + tests/profiles.scm | 7 + 14 files changed, 1098 insertions(+), 146 deletions(-) create mode 100644 guix/scripts/shell.scm create mode 100644 tests/guix-shell.sh
-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 02/11] profiles: Add 'package->development-manifest'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20211011213809.17482-3-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>
* guix/profiles.scm (package->development-manifest): New procedure.* guix/scripts/environment.scm (input->manifest-entry)(package-environment-inputs): Remove.* guix/scripts/environment.scm (options/resolve-packages): Use'package->development-manifest' instead of 'package-environment-inputs'.* tests/profiles.scm ("package->development-manifest"): New test.--- doc/guix.texi | 11 +++++++++++ guix/profiles.scm | 19 +++++++++++++++++++ guix/scripts/environment.scm | 27 +++++---------------------- tests/profiles.scm | 7 +++++++ 4 files changed, 42 insertions(+), 22 deletions(-)
Toggle diff (134 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 49399e792b..bc3f5a537b 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -3340,6 +3340,17 @@ objects, like this: '("emacs" "guile@@2.2" "guile@@2.2:debug")) @end lisp +@findex package->development-manifest+You might also want to create a manifest for all the dependencies of a+package, rather than the package itself:++@lisp+(package->development-manifest (specification->package "emacs"))+@end lisp++The example above gives you all the software required to develop Emacs,+similar to what @command{guix environment emacs} provides.+ @xref{export-manifest, @option{--export-manifest}}, to learn how to obtain a manifest file from an existing profile. diff --git a/guix/profiles.scm b/guix/profiles.scmindex 2486f91d09..9f30349c69 100644--- a/guix/profiles.scm+++ b/guix/profiles.scm@@ -124,6 +124,7 @@ (define-module (guix profiles) profile-manifest package->manifest-entry+ package->development-manifest packages->manifest ca-certificate-bundle %default-profile-hooks@@ -400,6 +401,24 @@ (define* (package->manifest-entry package #:optional (output "out") (properties properties)))) entry)) +(define* (package->development-manifest package+ #:optional+ (system (%current-system))+ #:key target)+ "Return a manifest for the \"development inputs\" of PACKAGE for SYSTEM,+optionally when cross-compiling to TARGET. Development inputs include both+explicit and implicit inputs of PACKAGE."+ (manifest+ (filter-map (match-lambda+ ((label (? package? package))+ (package->manifest-entry package))+ ((label (? package? package) output)+ (package->manifest-entry package output))+ ;; TODO: Support <inferior-package>.+ (_+ #f))+ (package-development-inputs package system #:target target))))+ (define (packages->manifest packages) "Return a list of manifest entries, one for each item listed in PACKAGES. Elements of PACKAGES can be either package objects or package/string tuplesdiff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex d555969b27..54f48a7482 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -66,24 +66,6 @@ (define* (show-search-paths profile manifest #:key pure?) (newline))) (profile-search-paths profile manifest))) -(define (input->manifest-entry input)- "Return a manifest entry for INPUT, or #f if INPUT does not correspond to a-package."- (match input- ((_ (? package? package))- (package->manifest-entry package))- ((_ (? package? package) output)- (package->manifest-entry package output))- (_- #f)))--(define (package-environment-inputs package)- "Return a list of manifest entries corresponding to the transitive input-packages for PACKAGE."- ;; Remove non-package inputs such as origin records.- (filter-map input->manifest-entry- (package-development-inputs package system)))- (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] Build an environment that includes the dependencies of PACKAGE and execute@@ -297,11 +279,11 @@ (define (packages->outputs packages mode) ((? package? package) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package))- (package-environment-inputs package)))+ (manifest-entries (package->development-manifest package)))) (((? package? package) (? string? output)) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package output))- (package-environment-inputs package)))+ (manifest-entries (package->development-manifest package)))) ((lst ...) (append-map (cut packages->outputs <> mode) lst)))) @@ -313,8 +295,9 @@ (define (packages->outputs packages mode) (specification->package+output spec))) (list (package->manifest-entry* package output)))) (('package 'package (? string? spec))- (package-environment-inputs- (transform (specification->package+output spec))))+ (manifest-entries+ (package->development-manifest+ (transform (specification->package+output spec))))) (('expression mode str) ;; Add all the outputs of the package STR evaluates to. (packages->outputs (read/eval str) mode))diff --git a/tests/profiles.scm b/tests/profiles.scmindex 06a0387221..cac5b73347 100644--- a/tests/profiles.scm+++ b/tests/profiles.scm@@ -265,6 +265,13 @@ (define transform1 (manifest-transaction-removal-candidate? guile-2.0.9 t) (null? install) (null? downgrade) (null? upgrade))))) +(test-assert "package->development-manifest"+ (let ((manifest (package->development-manifest packages:hello)))+ (every (lambda (name)+ (manifest-installed? manifest+ (manifest-pattern (name name))))+ '("gcc" "binutils" "glibc" "coreutils" "grep" "sed"))))+ (test-assertm "profile-derivation" (mlet* %store-monad ((entry -> (package->manifest-entry %bootstrap-guile))-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 04/11] DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-5-ludo@gnu.org
DRAFT: Add doc.
* guix/scripts/shell.scm (parse-args): Add call to 'auto-detect-manifest'.(find-file-in-parent-directories, auto-detect-manifest): New procedures.* tests/guix-shell.sh: Add test.--- guix/scripts/shell.scm | 67 ++++++++++++++++++++++++++++++++++++++---- tests/guix-shell.sh | 36 +++++++++++++++++++++++ 2 files changed, 98 insertions(+), 5 deletions(-)
Toggle diff (153 lines)diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmindex 190dd8837d..39d843bde7 100644--- a/guix/scripts/shell.scm+++ b/guix/scripts/shell.scm@@ -22,6 +22,8 @@ (define-module (guix scripts shell) #:autoload (guix scripts build) (show-build-options-help) #:autoload (guix transformations) (show-transformation-options-help) #:use-module (guix scripts)+ #:use-module (guix packages)+ #:use-module (guix profiles) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) #:use-module (srfi srfi-37)@@ -41,6 +43,8 @@ (define (show-help) (display (G_ " -f, --file=FILE create environment for the package that the code within FILE evaluates to"))+ (display (G_ "+ -q inhibit loading of 'guix.scm' and 'manifest.scm'")) (show-environment-options-help) (newline)@@ -99,7 +103,10 @@ (define %options (option '(#\f "file") #t #f (lambda (opt name arg result) (alist-cons 'load (tag-package-arg result arg)- result))))+ result)))+ (option '(#\q) #f #f+ (lambda (opt name arg result)+ (alist-cons 'explicit-loading? #t result)))) (filter-map (lambda (opt) (and (not (any (lambda (name) (member name to-remove))@@ -122,10 +129,60 @@ (define (handle-argument arg result) (let ((args command (break (cut string=? "--" <>) args))) (let ((opts (parse-command-line args %options (list %default-options) #:argument-handler handle-argument)))- (match command- (() opts)- (("--") opts)- (("--" command ...) (alist-cons 'exec command opts))))))+ (auto-detect-manifest+ (match command+ (() opts)+ (("--") opts)+ (("--" command ...) (alist-cons 'exec command opts)))))))++(define (find-file-in-parent-directories candidates)+ "Find one of CANDIDATES in the current directory or one of its ancestors."+ (define start (getcwd))+ (define device (stat:dev (stat start)))++ (let loop ((directory start))+ (let ((stat (stat directory)))+ (and (= (stat:uid stat) (getuid))+ (= (stat:dev stat) device)+ (or (any (lambda (candidate)+ (let ((candidate (string-append directory "/" candidate)))+ (and (file-exists? candidate) candidate)))+ candidates)+ (and (not (string=? directory "/"))+ (loop (dirname directory)))))))) ;lexical ".." resolution++(define (auto-detect-manifest opts)+ "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or+\"manifest.scm\" file from the current directory or one of its ancestors.+Return the modified OPTS."+ (define (options-contain-payload? opts)+ (match opts+ (() #f)+ ((('package . _) . _) #t)+ ((('load . _) . _) #t)+ ((('manifest . _) . _) #t)+ ((('expression . _) . _) #t)+ ((_ . rest) (options-contain-payload? rest))))++ (define interactive?+ (not (assoc-ref opts 'exec)))++ (define disallow-implicit-load?+ (assoc-ref opts 'explicit-loading?))++ (if (or (not interactive?)+ disallow-implicit-load?+ (options-contain-payload? opts))+ opts+ (match (find-file-in-parent-directories '("guix.scm" "manifest.scm"))+ (#f+ (warning (G_ "no packages specified; creating an empty environment~%"))+ opts)+ (file+ (info (G_ "loading environment from '~a'...~%") file)+ (match (basename file)+ ("guix.scm" (alist-cons 'load `(package ,file) opts))+ ("manifest.scm" (alist-cons 'manifest file opts))))))) (define-command (guix-shell . args)diff --git a/tests/guix-shell.sh b/tests/guix-shell.shindex f08637f7ff..0988ca0a75 100644--- a/tests/guix-shell.sh+++ b/tests/guix-shell.sh@@ -31,6 +31,36 @@ guix shell --bootstrap --pure guile-bootstrap -- guile --version # '--ad-hoc' is a thing of the past. ! guix shell --ad-hoc guile-bootstrap +# Ignoring 'manifest.scm' and 'guix.scm' in non-interactive use.+cat > "$tmpdir/guix.scm" <<EOF+This is a broken guix.scm file.+EOF+(cd "$tmpdir"; guix shell --bootstrap -- true)+mv "$tmpdir/guix.scm" "$tmpdir/manifest.scm"+(cd "$tmpdir"; guix shell --bootstrap -- true)+rm "$tmpdir/manifest.scm"++# Honoring the local 'manifest.scm' file.+cat > "$tmpdir/manifest.scm" <<EOF+(specifications->manifest '("guile-bootstrap"))+EOF+cat > "$tmpdir/fake-shell.sh" <<EOF+#!$SHELL+# This fake shell allows us to test interactive use.+exec echo "\$GUIX_ENVIRONMENT"+EOF+chmod +x "$tmpdir/fake-shell.sh"+profile1="$(cd "$tmpdir"; SHELL="$(realpath fake-shell.sh)" guix shell --bootstrap)"+profile2="$(guix shell --bootstrap guile-bootstrap -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT')"+test -n "$profile1"+test "$profile1" = "$profile2"+rm "$tmpdir/manifest.scm"++# Do not read manifest when passed '-q'.+echo "Broken manifest." > "$tmpdir/manifest.scm"+(cd "$tmpdir"; SHELL="$(realpath fake-shell.sh)" guix shell --bootstrap -q)+rm "$tmpdir/manifest.scm"+ if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null then # Compute the build environment for the initial GNU Make.@@ -51,4 +81,10 @@ then # 'make-boot0' itself must not be listed. ! guix gc --references "$profile" | grep make-boot0++ # Honoring the local 'guix.scm' file.+ echo '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/guix.scm"+ (cd "$tmpdir"; guix shell --bootstrap --search-paths --pure > "b")+ cmp "$tmpdir/a" "$tmpdir/b"+ rm "$tmpdir/guix.scm" fi-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 05/11] DRAFT shell: Honor in ~/.config/guix/shell-authorized-directories.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-6-ludo@gnu.org
DRAFT: Squeeze with previous commit, or instead implement "guix shell ."convention?
* guix/scripts/shell.scm (authorized-directory-file)(authorized-shell-directory?): New procedure.(auto-detect-manifest): Use it.* doc/guix.texi (Invoking guix shell): Document it.--- doc/guix.texi | 14 ++++++++++ guix/scripts/shell.scm | 60 +++++++++++++++++++++++++++++++++++++++--- tests/guix-shell.sh | 16 ++++++++--- 3 files changed, 83 insertions(+), 7 deletions(-)
Toggle diff (156 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex b0d745b9e3..b95025a39f 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -5620,6 +5620,20 @@ before @command{guix shell} was invoked. The next garbage collection (@pxref{Invoking guix gc}) may clean up packages that were installed in the environment and that are no longer used outside of it. +As an added convenience, when running from a directory that contains a+@file{guix.scm} or a @file{manifest.scm} file, possibly in a parent+directory, @command{guix shell} automatically loads the file---provided+the directory is listed in+@file{~/.config/guix/shell-authorized-directories}, and only for+interactive use:++@example+guix shell+@end example++This provides an easy way to define, share, and enter development+environments.+ By default, the shell session or command runs in an @emph{augmented} environment, where the new packages are added to search path environment variables such as @code{PATH}. You can, instead, choose to create andiff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmindex 39d843bde7..45fd536145 100644--- a/guix/scripts/shell.scm+++ b/guix/scripts/shell.scm@@ -18,6 +18,7 @@ (define-module (guix scripts shell) #:use-module (guix ui)+ #:use-module ((guix diagnostics) #:select (location)) #:use-module (guix scripts environment) #:autoload (guix scripts build) (show-build-options-help) #:autoload (guix transformations) (show-transformation-options-help)@@ -29,6 +30,8 @@ (define-module (guix scripts shell) #:use-module (srfi srfi-37) #:use-module (srfi srfi-71) #:use-module (ice-9 match)+ #:autoload (ice-9 rdelim) (read-line)+ #:autoload (guix utils) (config-directory) #:export (guix-shell)) (define (show-help)@@ -151,6 +154,39 @@ (define device (stat:dev (stat start))) (and (not (string=? directory "/")) (loop (dirname directory)))))))) ;lexical ".." resolution +(define (authorized-directory-file)+ "Return the name of the file listing directories for which 'guix shell' may+automatically load 'guix.scm' or 'manifest.scm' files."+ (string-append (config-directory) "/shell-authorized-directories"))++(define (authorized-shell-directory? directory)+ "Return true if DIRECTORY is among the authorized directories for automatic+loading. The list of authorized directories is read from+'authorized-directory-file'; each line must be either: an absolute file name,+a hash-prefixed comment, or a blank line."+ (catch 'system-error+ (lambda ()+ (call-with-input-file (authorized-directory-file)+ (lambda (port)+ (let loop ()+ (match (read-line port)+ ((? eof-object?) #f)+ ((= string-trim line)+ (cond ((string-prefix? "#" line) ;comment+ (loop))+ ((string-prefix? "/" line) ;absolute file name+ (or (string=? line directory)+ (loop)))+ ((string-null? (string-trim-right line)) ;blank line+ (loop))+ (else ;bogus line+ (let ((loc (location (port-filename port)+ (port-line port)+ (port-column port))))+ (warning loc (G_ "ignoring invalid file name: '~a'~%")+ line))))))))))+ (const #f)))+ (define (auto-detect-manifest opts) "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or \"manifest.scm\" file from the current directory or one of its ancestors.@@ -179,10 +215,26 @@ (define disallow-implicit-load? (warning (G_ "no packages specified; creating an empty environment~%")) opts) (file- (info (G_ "loading environment from '~a'...~%") file)- (match (basename file)- ("guix.scm" (alist-cons 'load `(package ,file) opts))- ("manifest.scm" (alist-cons 'manifest file opts)))))))+ (if (authorized-shell-directory? (dirname file))+ (begin+ (info (G_ "loading environment from '~a'...~%") file)+ (match (basename file)+ ("guix.scm" (alist-cons 'load `(package ,file) opts))+ ("manifest.scm" (alist-cons 'manifest file opts))))+ (begin+ (warning (G_ "not loading '~a' because not authorized to do so~%")+ file)+ (display-hint (format #f (G_ "To allow automatic loading of+@file{~a} when running @command{guix shell}, you must explicitly authorize its+directory, like so:++@example+echo ~a >> ~a+@end example\n")+ file+ (dirname file)+ (authorized-directory-file)))+ opts)))))) (define-command (guix-shell . args)diff --git a/tests/guix-shell.sh b/tests/guix-shell.shindex 0988ca0a75..95725cba2d 100644--- a/tests/guix-shell.sh+++ b/tests/guix-shell.sh@@ -22,19 +22,29 @@ guix shell --version +configdir="t-guix-shell-config-$$" tmpdir="t-guix-shell-$$"-trap 'rm -r "$tmpdir"' EXIT-mkdir "$tmpdir"+trap 'rm -r "$tmpdir" "$configdir"' EXIT+mkdir "$tmpdir" "$configdir" "$configdir/guix"++XDG_CONFIG_HOME="$(realpath $configdir)"+export XDG_CONFIG_HOME guix shell --bootstrap --pure guile-bootstrap -- guile --version # '--ad-hoc' is a thing of the past. ! guix shell --ad-hoc guile-bootstrap -# Ignoring 'manifest.scm' and 'guix.scm' in non-interactive use.+# Ignoring unauthorized files. cat > "$tmpdir/guix.scm" <<EOF This is a broken guix.scm file. EOF+(cd "$tmpdir"; SHELL="$(type -P true)" guix shell --bootstrap)++# Authorize the directory.+echo "$(realpath "$tmpdir")" > "$configdir/guix/shell-authorized-directories"++# Ignoring 'manifest.scm' and 'guix.scm' in non-interactive use. (cd "$tmpdir"; guix shell --bootstrap -- true) mv "$tmpdir/guix.scm" "$tmpdir/manifest.scm" (cd "$tmpdir"; guix shell --bootstrap -- true)-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 06/11] environment: Add tests for '--profile'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-7-ludo@gnu.org
This is a followup to a643deac2de81755a1843a3b41dd53857678bebc.
* tests/guix-environment-container.sh, tests/guix-environment.sh: Addtests for '--profile'.--- tests/guix-environment-container.sh | 8 ++++++++ tests/guix-environment.sh | 7 +++++++ 2 files changed, 15 insertions(+)
Toggle diff (39 lines)diff --git a/tests/guix-environment-container.sh b/tests/guix-environment-container.shindex f2d15c8d0c..2e238c501d 100644--- a/tests/guix-environment-container.sh+++ b/tests/guix-environment-container.sh@@ -44,6 +44,14 @@ else test $? = 42 fi +# Try '--root' and '--profile'.+root="$tmpdir/root"+guix environment -C --ad-hoc --bootstrap guile-bootstrap -r "$root" -- guile --version+guix environment -C -p "$root" --bootstrap -- guile --version+path1=$(guix environment -C -p "$root" --bootstrap -- guile -c '(display (getenv "PATH"))')+path2=$(guix environment -C --ad-hoc --bootstrap guile-bootstrap -- guile -c '(display (getenv "PATH"))')+test "$path1" = "$path2"+ # Make sure "localhost" resolves. guix environment --container --ad-hoc --bootstrap guile-bootstrap \ -- guile -c '(exit (pair? (getaddrinfo "localhost" "80")))'diff --git a/tests/guix-environment.sh b/tests/guix-environment.shindex afadcbe195..f4fc2e39ed 100644--- a/tests/guix-environment.sh+++ b/tests/guix-environment.sh@@ -119,6 +119,13 @@ test `readlink "$gcroot"` = "$expected" guix environment --bootstrap -r "$gcroot" --ad-hoc guile-bootstrap \ -- guile -c 1 test `readlink "$gcroot"` = "$expected"++# Make sure '-p' works as expected.+test $(guix environment -p "$gcroot" -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT') = "$expected"+paths1="$(guix environment -p "$gcroot" --search-paths)"+paths2="$(guix environment --bootstrap --ad-hoc guile-bootstrap --search-paths)"+test "$paths1" = "$paths2"+ rm "$gcroot" # Try '-r' with a relative file name.-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 07/11] environment: Skip derivation computation when '--profile' is used.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-8-ludo@gnu.org
* guix/scripts/environment.scm (guix-environment*): Bypass calls to'package-derivation' and to 'manifest->derivation' when PROFILE istrue.--- guix/scripts/environment.scm | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-)
Toggle diff (48 lines)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 77956fc018..32f376fdd2 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -729,18 +729,21 @@ (define manifest ;; Use the bootstrap Guile when requested. (parameterize ((%graft? (assoc-ref opts 'graft?)) (%guile-for-build- (package-derivation- store- (if bootstrap?- %bootstrap-guile- (default-guile)))))+ (and (or container? (not profile))+ (package-derivation+ store+ (if bootstrap?+ %bootstrap-guile+ (default-guile)))))) (run-with-store store ;; Containers need a Bourne shell at /bin/sh. (mlet* %store-monad ((bash (environment-bash container? bootstrap? system))- (prof-drv (manifest->derivation- manifest system bootstrap?))+ (prof-drv (if profile+ (return #f)+ (manifest->derivation+ manifest system bootstrap?))) (profile -> (if profile (readlink* profile) (derivation->output-path prof-drv)))@@ -750,9 +753,9 @@ (define manifest ;; --search-paths. Additionally, we might need to build bash for ;; a container. (mbegin %store-monad- (built-derivations (if (derivation? bash)- (list prof-drv bash)- (list prof-drv)))+ (built-derivations (append+ (if prof-drv (list prof-drv) '())+ (if (derivation? bash) (list bash) '()))) (mwhen gc-root (register-gc-root profile gc-root)) -- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 03/11] Add 'guix shell'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20211011213809.17482-4-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>
* guix/scripts/shell.scm, tests/guix-shell.sh: New files.* Makefile.am (MODULES): Add 'shell.scm'.(SH_TESTS): Add 'tests/guix-shell.sh'.* guix/scripts/environment.scm (show-environment-options-help): Newprocedure.(show-help): Use it.(guix-environment*): New procedure.(guix-environment): Use it.* po/guix/POTFILES.in: Add it.* doc/guix.texi (Features): Refer to "guix shell"(Invoking guix package): Likewise.(Development): Likewise.(Invoking guix shell): New node.(Invoking guix environment): Add deprecation warning.(Debugging Build Failures): Use 'guix shell' in examples.(Invoking guix container): Refer to 'guix shell'.(Invoking guix processes, Virtualization Services): Adjust examples touse 'guix shell'.* doc/contributing.texi (Building from Git): Refer to 'guix shell'.--- Makefile.am | 2 + doc/contributing.texi | 8 +- doc/guix.texi | 366 ++++++++++++++++++++++++++++++++--- guix/scripts/environment.scm | 52 +++-- guix/scripts/shell.scm | 135 +++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-shell.sh | 54 ++++++ 7 files changed, 575 insertions(+), 43 deletions(-) create mode 100644 guix/scripts/shell.scm create mode 100644 tests/guix-shell.sh
Toggle diff (827 lines)diff --git a/Makefile.am b/Makefile.amindex b66789fa0b..c28c8799ec 100644--- a/Makefile.am+++ b/Makefile.am@@ -315,6 +315,7 @@ MODULES = \ guix/scripts/import/stackage.scm \ guix/scripts/import/texlive.scm \ guix/scripts/environment.scm \+ guix/scripts/shell.scm \ guix/scripts/publish.scm \ guix/scripts/edit.scm \ guix/scripts/size.scm \@@ -550,6 +551,7 @@ SH_TESTS = \ tests/guix-authenticate.sh \ tests/guix-environment.sh \ tests/guix-environment-container.sh \+ tests/guix-shell.sh \ tests/guix-graph.sh \ tests/guix-describe.sh \ tests/guix-repl.sh \diff --git a/doc/contributing.texi b/doc/contributing.texiindex fbb3c47c78..363bbe9278 100644--- a/doc/contributing.texi+++ b/doc/contributing.texi@@ -73,10 +73,10 @@ all the dependencies and appropriate environment variables are set up to hack on Guix: @example-guix environment guix --pure+guix shell -D guix --pure @end example -@xref{Invoking guix environment}, for more information on that command.+@xref{Invoking guix shell}, for more information on that command. If you are unable to use Guix when building Guix from a checkout, the following are the required packages in addition to those mentioned in the@@ -92,10 +92,10 @@ installation instructions (@pxref{Requirements}). @end itemize On Guix, extra dependencies can be added by instead running @command{guix-environment} with @option{--ad-hoc}:+shell}: @example-guix environment guix --pure --ad-hoc help2man git strace+guix shell -D guix help2man git strace --pure @end example Run @command{./bootstrap} to generate the build system infrastructurediff --git a/doc/guix.texi b/doc/guix.texiindex bc3f5a537b..b0d745b9e3 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -118,6 +118,7 @@ Documentation License''. @dircategory Software development @direntry+* guix shell: (guix)Invoking guix shell. Creating software environments. * guix environment: (guix)Invoking guix environment. Building development environments with Guix. * guix build: (guix)Invoking guix build. Building packages. * guix pack: (guix)Invoking guix pack. Creating binary bundles.@@ -261,6 +262,7 @@ Channels Development +* Invoking guix shell:: Spawning one-off software environments. * Invoking guix environment:: Setting up development environments. * Invoking guix pack:: Creating software bundles. * The GCC toolchain:: Working with languages supported by GCC.@@ -3066,10 +3068,10 @@ substitutes: they can force a local build and @emph{challenge} providers (@pxref{Invoking guix challenge}). Control over the build environment is a feature that is also useful for-developers. The @command{guix environment} command allows developers of+developers. The @command{guix shell} command allows developers of a package to quickly set up the right development environment for their package, without having to manually install the dependencies of the-package into their profile (@pxref{Invoking guix environment}).+package into their profile (@pxref{Invoking guix shell}). @cindex replication, of software environments @cindex provenance tracking, of software artifacts@@ -3233,7 +3235,7 @@ As an example, @var{file} might contain a definition like this Developers may find it useful to include such a @file{guix.scm} file in the root of their project source tree that can be used to test development snapshots and create reproducible development environments-(@pxref{Invoking guix environment}).+(@pxref{Invoking guix shell}). The @var{file} may also contain a JSON representation of one or more package definitions. Running @code{guix package -f} on@@ -5558,31 +5560,352 @@ If you are a software developer, Guix provides tools that you should find helpful---independently of the language you're developing in. This is what this chapter is about. -The @command{guix environment} command provides a convenient way to set up-@dfn{development environments} containing all the dependencies and tools-necessary to work on the software package of your choice. The @command{guix+The @command{guix shell} command provides a convenient way to set up+one-off software environments, be it for development purposes or to run+a command without installing it in your profile. The @command{guix pack} command allows you to create @dfn{application bundles} that can be easily distributed to users who do not run Guix. @menu+* Invoking guix shell:: Spawning one-off software environments. * Invoking guix environment:: Setting up development environments. * Invoking guix pack:: Creating software bundles. * The GCC toolchain:: Working with languages supported by GCC. * Invoking guix git authenticate:: Authenticating Git repositories. @end menu -@node Invoking guix environment-@section Invoking @command{guix environment}+@node Invoking guix shell+@section Invoking @command{guix shell} @cindex reproducible build environments @cindex development environments @cindex @command{guix environment} @cindex environment, package build environment-The purpose of @command{guix environment} is to assist hackers in-creating reproducible development environments without polluting their-package profile. The @command{guix environment} tool takes one or more-packages, builds all of their inputs, and creates a shell-environment to use them.+The purpose of @command{guix shell} is to make it easy to create one-off+software environments, without changing one's profile. It is typically+used to create development environments; it is also a convenient way to+run applications without ``polluting'' your profile.++@quotation Note+The @command{guix shell} command was recently introduced to supersede+@command{guix environment} (@pxref{Invoking guix environment}). If you+are familiar with @command{guix environment}, you will notice that it is+similar but also---we hope!---more convenient.+@end quotation++The general syntax is:++@example+guix shell [@var{options}] [@var{package}@dots{}]+@end example++The following example creates a environment containing Python and NumPy,+building and downloading any missing package, and runs the+@command{python3} command in that environment:++@example+guix shell python python-numpy -- python3+@end example++Development environments can be created as in the example below, which+spawns an interactive shell containing all the dependencies and+environment variables needed to work on Inkscape:++@example+guix shell --development inkscape+@end example++Exiting the shell places the user back in the original environment+before @command{guix shell} was invoked. The next garbage collection+(@pxref{Invoking guix gc}) may clean up packages that were installed in+the environment and that are no longer used outside of it.++By default, the shell session or command runs in an @emph{augmented}+environment, where the new packages are added to search path environment+variables such as @code{PATH}. You can, instead, choose to create an+@emph{isolated} environment containing nothing but the packages you+asked for. Passing the @option{--pure} option clears environment+variable definitions found in the parent environment@footnote{Users+sometimes wrongfully augment environment variables such as @env{PATH} in+their @file{~/.bashrc} file. As a consequence, when @command{guix+environment} launches it, Bash may read @file{~/.bashrc}, thereby+introducing ``impurities'' in these environment variables. It is an+error to define such environment variables in @file{.bashrc}; instead,+they should be defined in @file{.bash_profile}, which is sourced only by+log-in shells. @xref{Bash Startup Files,,, bash, The GNU Bash Reference+Manual}, for details on Bash start-up files.}; passing+@option{--container} goes one step further by spawning a @dfn{container}+isolated from the rest of the system:++@example+guix shell --container emacs gcc-toolchain+@end example++The command above spawns an interactive shell in a container when+nothing but @code{emacs}, @code{gcc-toolchain}, and their dependencies+is available. The container lacks network access and shares no files+other than the current working directory with the surrounding+environment. This is useful to prevent access to system-wide resources+such as @file{/usr/bin} on foreign distros.++This @option{--container} option can also prove useful if you wish to+run a security-sensitive application, such as a web browser, in an+isolated environment. For example, the command below launches+Ungoogled-Chromium in an isolated environment, this time sharing network+access with the host and preserving its @code{DISPLAY} environment+variable, but without even sharing the current directory:++@example+guix shell --container --network --no-cwd ungoogled-chromium \+ --preserve='^DISPLAY$' -- chromium+@end example++@vindex GUIX_ENVIRONMENT+@command{guix shell} defines the @env{GUIX_ENVIRONMENT}+variable in the shell it spawns; its value is the file name of the+profile of this environment. This allows users to, say, define a+specific prompt for development environments in their @file{.bashrc}+(@pxref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}):++@example+if [ -n "$GUIX_ENVIRONMENT" ]+then+ export PS1="\u@@\h \w [dev]\$ "+fi+@end example++@noindent+...@: or to browse the profile:++@example+$ ls "$GUIX_ENVIRONMENT/bin"+@end example++The available options are summarized below.++@table @code+@item --development+@itemx -D+Cause @command{guix shell} to include in the environment the+dependencies of the following package rather than the package itself.+This can be combined with other packages. For instance, the command+below starts an interactive shell containing the build-time dependencies+of GNU@tie{}Guile, plus Autoconf, Automake, and Libtool:++@example+guix shell -D guile autoconf automake libtool+@end example++@item --expression=@var{expr}+@itemx -e @var{expr}+Create an environment for the package or list of packages that+@var{expr} evaluates to.++For example, running:++@example+guix shell -D -e '(@@ (gnu packages maths) petsc-openmpi)'+@end example++starts a shell with the environment for this specific variant of the+PETSc package.++Running:++@example+guix shell -e '(@@ (gnu) %base-packages)'+@end example++starts a shell with all the base system packages available.++The above commands only use the default output of the given packages.+To select other outputs, two element tuples can be specified:++@example+guix shell -e '(list (@@ (gnu packages bash) bash) "include")'+@end example++@item --file=@var{file}+@itemx -f @var{file}+Create an environment containing the package or list of packages that+the code within @var{file} evaluates to.++As an example, @var{file} might contain a definition like this+(@pxref{Defining Packages}):++@lisp+@verbatiminclude environment-gdb.scm+@end lisp++With the file above, you can enter a development environment for GDB by+running:++@example+guix shell -D -f gdb-devel.scm+@end example++@item --manifest=@var{file}+@itemx -m @var{file}+Create an environment for the packages contained in the manifest object+returned by the Scheme code in @var{file}. This option can be repeated+several times, in which case the manifests are concatenated.++This is similar to the same-named option in @command{guix package}+(@pxref{profile-manifest, @option{--manifest}}) and uses the same+manifest files.++@item --pure+Unset existing environment variables when building the new environment, except+those specified with @option{--preserve} (see below). This has the effect of+creating an environment in which search paths only contain package inputs.++@item --preserve=@var{regexp}+@itemx -E @var{regexp}+When used alongside @option{--pure}, preserve the environment variables+matching @var{regexp}---in other words, put them on a ``white list'' of+environment variables that must be preserved. This option can be repeated+several times.++@example+guix shell --pure --preserve=^SLURM openmpi @dots{} \+ -- mpirun @dots{}+@end example++This example runs @command{mpirun} in a context where the only environment+variables defined are @env{PATH}, environment variables whose name starts+with @samp{SLURM}, as well as the usual ``precious'' variables (@env{HOME},+@env{USER}, etc.).++@item --search-paths+Display the environment variable definitions that make up the+environment.++@item --system=@var{system}+@itemx -s @var{system}+Attempt to build for @var{system}---e.g., @code{i686-linux}.++@item --container+@itemx -C+@cindex container+Run @var{command} within an isolated container. The current working+directory outside the container is mapped inside the container.+Additionally, unless overridden with @option{--user}, a dummy home+directory is created that matches the current user's home directory, and+@file{/etc/passwd} is configured accordingly.++The spawned process runs as the current user outside the container. Inside+the container, it has the same UID and GID as the current user, unless+@option{--user} is passed (see below).++@item --network+@itemx -N+For containers, share the network namespace with the host system.+Containers created without this flag only have access to the loopback+device.++@item --link-profile+@itemx -P+For containers, link the environment profile to @file{~/.guix-profile}+within the container and set @code{GUIX_ENVIRONMENT} to that.+This is equivalent to making @file{~/.guix-profile} a symlink to the+actual profile within the container.+Linking will fail and abort the environment if the directory already+exists, which will certainly be the case if @command{guix shell}+was invoked in the user's home directory.++Certain packages are configured to look in @file{~/.guix-profile} for+configuration files and data;@footnote{For example, the+@code{fontconfig} package inspects @file{~/.guix-profile/share/fonts}+for additional fonts.} @option{--link-profile} allows these programs to+behave as expected within the environment.++@item --user=@var{user}+@itemx -u @var{user}+For containers, use the username @var{user} in place of the current+user. The generated @file{/etc/passwd} entry within the container will+contain the name @var{user}, the home directory will be+@file{/home/@var{user}}, and no user GECOS data will be copied. Furthermore,+the UID and GID inside the container are 1000. @var{user}+need not exist on the system.++Additionally, any shared or exposed path (see @option{--share} and+@option{--expose} respectively) whose target is within the current user's+home directory will be remapped relative to @file{/home/USER}; this+includes the automatic mapping of the current working directory.++@example+# will expose paths as /home/foo/wd, /home/foo/test, and /home/foo/target+cd $HOME/wd+guix shell --container --user=foo \+ --expose=$HOME/test \+ --expose=/tmp/target=$HOME/target+@end example++While this will limit the leaking of user identity through home paths+and each of the user fields, this is only one useful component of a+broader privacy/anonymity solution---not one in and of itself.++@item --no-cwd+For containers, the default behavior is to share the current working+directory with the isolated container and immediately change to that+directory within the container. If this is undesirable,+@option{--no-cwd} will cause the current working directory to @emph{not}+be automatically shared and will change to the user's home directory+within the container instead. See also @option{--user}.++@item --expose=@var{source}[=@var{target}]+@itemx --share=@var{source}[=@var{target}]+For containers, @option{--expose} (resp. @option{--share}) exposes the+file system @var{source} from the host system as the read-only+(resp. writable) file system @var{target} within the container. If+@var{target} is not specified, @var{source} is used as the target mount+point in the container.++The example below spawns a Guile REPL in a container in which the user's+home directory is accessible read-only via the @file{/exchange}+directory:++@example+guix shell --container --expose=$HOME=/exchange guile -- guile+@end example++@item --root=@var{file}+@itemx -r @var{file}+@cindex persistent environment+@cindex garbage collector root, for environments+Make @var{file} a symlink to the profile for this environment, and+register it as a garbage collector root.++This is useful if you want to protect your environment from garbage+collection, to make it ``persistent''.++When this option is omitted, the environment is protected from garbage+collection only for the duration of the @command{guix shell}+session. This means that next time you recreate the same environment,+you could have to rebuild or re-download packages. @xref{Invoking guix+gc}, for more on GC roots.+@end table++@command{guix shell} also supports all of the common build options that+@command{guix build} supports (@pxref{Common Build Options}) as well as+package transformation options (@pxref{Package Transformation Options}).++@node Invoking guix environment+@section Invoking @command{guix environment}++The purpose of @command{guix environment} is to assists in creating+development environments.++@quotation Deprecation warning+The @command{guix environment} command is deprecated in favor of+@command{guix shell}, which performs similar functions but is more+convenient to use. @xref{Invoking guix shell}.++Being deprecated, @command{guix environment} is slated for eventual+removal, but the Guix project is committed to keeping it until May 1st,+2023. Please get in touch with us at @email{guix-devel@@gnu.org} if you+would like to discuss it.+@end quotation The general syntax is: @@ -11098,14 +11421,14 @@ a container similar to the one the build daemon creates: $ guix build -K foo @dots{} $ cd /tmp/guix-build-foo.drv-0-$ guix environment --no-grafts -C foo --ad-hoc strace gdb+$ guix shell --no-grafts -C foo strace gdb [env]# source ./environment-variables [env]# cd foo-1.2 @end example Here, @command{guix environment -C} creates a container and spawns a new-shell in it (@pxref{Invoking guix environment}). The @command{--ad-hoc-strace gdb} part adds the @command{strace} and @command{gdb} commands to+shell in it (@pxref{Invoking guix shell}). The @command{strace gdb}+part adds the @command{strace} and @command{gdb} commands to the container, which you may find handy while debugging. The @option{--no-grafts} option makes sure we get the exact same environment, with ungrafted packages (@pxref{Security Updates}, for more@@ -11119,7 +11442,7 @@ remove @file{/bin/sh}: @end example (Don't worry, this is harmless: this is all happening in the throw-away-container created by @command{guix environment}.)+container created by @command{guix shell}.) The @command{strace} command is probably not in the search path, but we can run:@@ -13315,8 +13638,8 @@ is subject to radical change in the future. The purpose of @command{guix container} is to manipulate processes running within an isolated environment, commonly known as a-``container'', typically created by the @command{guix environment}-(@pxref{Invoking guix environment}) and @command{guix system container}+``container'', typically created by the @command{guix shell}+(@pxref{Invoking guix shell}) and @command{guix system container} (@pxref{Invoking guix system}) commands. The general syntax is:@@ -13502,7 +13825,7 @@ listed.}. Here's an example of the information it returns: $ sudo guix processes SessionPID: 19002 ClientPID: 19090-ClientCommand: guix environment --ad-hoc python+ClientCommand: guix shell python SessionPID: 19402 ClientPID: 19367@@ -29697,8 +30020,7 @@ When the service is running, you can view its console by connecting to it with a VNC client, for example with: @example-guix environment --ad-hoc tigervnc-client -- \- vncviewer localhost:5900+guix shell tigervnc-client -- vncviewer localhost:5900 @end example The default configuration (see @code{hurd-vm-configuration} below)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 54f48a7482..77956fc018 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -50,7 +50,11 @@ (define-module (guix scripts environment) #:use-module (srfi srfi-37) #:use-module (srfi srfi-98) #:export (assert-container-features- guix-environment))+ guix-environment+ guix-environment*+ show-environment-options-help+ (%options . %environment-options)+ (%default-options . %environment-default-options))) (define %default-shell (or (getenv "SHELL") "/bin/sh"))@@ -66,23 +70,16 @@ (define* (show-search-paths profile manifest #:key pure?) (newline))) (profile-search-paths profile manifest))) -(define (show-help)- (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]-Build an environment that includes the dependencies of PACKAGE and execute-COMMAND or an interactive shell in that environment.\n"))+(define (show-environment-options-help)+ "Print help about options shared between 'guix environment' and 'guix+shell'." (display (G_ " -e, --expression=EXPR create environment for the package that EXPR evaluates to")) (display (G_ "- -l, --load=FILE create environment for the package that the code within- FILE evaluates to"))- (display (G_ " -m, --manifest=FILE create environment with the manifest from FILE")) (display (G_ " -p, --profile=PATH create environment from profile at PATH"))- (display (G_ "- --ad-hoc include all specified packages in the environment instead- of only their inputs")) (display (G_ " --pure unset existing environment variables")) (display (G_ "@@ -118,7 +115,24 @@ (define (show-help) (display (G_ " -v, --verbosity=LEVEL use the given verbosity LEVEL")) (display (G_ "- --bootstrap use bootstrap binaries to build the environment"))+ --bootstrap use bootstrap binaries to build the environment")))++(define (show-help)+ (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]+Build an environment that includes the dependencies of PACKAGE and execute+COMMAND or an interactive shell in that environment.\n"))+ (warning (G_ "This command is deprecated in favor of 'guix shell'.\n"))+ (newline)++ ;; These two options are left out in 'guix shell'.+ (display (G_ "+ -l, --load=FILE create environment for the package that the code within+ FILE evaluates to"))+ (display (G_ "+ --ad-hoc include all specified packages in the environment instead+ of only their inputs"))++ (show-environment-options-help) (newline) (show-build-options-help) (newline)@@ -649,11 +663,15 @@ (define (register-gc-root target root) (define-command (guix-environment . args) (category development)- (synopsis "spawn one-off software environments")+ (synopsis "spawn one-off software environments (deprecated)") + (guix-environment* (parse-args args)))++(define (guix-environment* opts)+ "Run the 'guix environment' command on OPTS, an alist resulting for+command-line option processing with 'parse-command-line'." (with-error-handling- (let* ((opts (parse-args args))- (pure? (assoc-ref opts 'pure))+ (let* ((pure? (assoc-ref opts 'pure)) (container? (assoc-ref opts 'container?)) (link-prof? (assoc-ref opts 'link-profile?)) (network? (assoc-ref opts 'network?))@@ -724,8 +742,8 @@ (define manifest (prof-drv (manifest->derivation manifest system bootstrap?)) (profile -> (if profile- (readlink* profile)- (derivation->output-path prof-drv)))+ (readlink* profile)+ (derivation->output-path prof-drv))) (gc-root -> (assoc-ref opts 'gc-root))) ;; First build the inputs. This is necessary even fordiff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmnew file mode 100644index 0000000000..190dd8837d--- /dev/null+++ b/guix/scripts/shell.scm@@ -0,0 +1,135 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 Ludovic Courtès <ludo@gnu.org>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (guix scripts shell)+ #:use-module (guix ui)+ #:use-module (guix scripts environment)+ #:autoload (guix scripts build) (show-build-options-help)+ #:autoload (guix transformations) (show-transformation-options-help)+ #:use-module (guix scripts)+ #:use-module (srfi srfi-1)+ #:use-module (srfi srfi-26)+ #:use-module (srfi srfi-37)+ #:use-module (srfi srfi-71)+ #:use-module (ice-9 match)+ #:export (guix-shell))++(define (show-help)+ (display (G_ "Usage: guix shell [OPTION] PACKAGES... [-- COMMAND...]+Build an environment that includes PACKAGES and execute COMMAND or an+interactive shell in that environment.\n"))+ (newline)++ ;; These two options differ from 'guix environment'.+ (display (G_ "+ -D, --development include the development inputs of the next package"))+ (display (G_ "+ -f, --file=FILE create environment for the package that the code within+ FILE evaluates to"))++ (show-environment-options-help)+ (newline)+ (show-build-options-help)+ (newline)+ (show-transformation-options-help)+ (newline)+ (display (G_ "+ -h, --help display this help and exit"))+ (display (G_ "+ -V, --version display version information and exit"))+ (newline)+ (show-bug-report-information))++(define (tag-package-arg opts arg)+ "Return a two-element list with the form (TAG ARG) that tags ARG with either+'ad-hoc' in OPTS has the 'ad-hoc?' key set to #t, or 'inputs' otherwise."+ (if (assoc-ref opts 'ad-hoc?)+ `(ad-hoc-package ,arg)+ `(package ,arg)))++(define (ensure-ad-hoc alist)+ (if (assq-ref alist 'ad-hoc?)+ alist+ `((ad-hoc? . #t) ,@alist)))++(define (wrapped-option opt)+ "Wrap OPT, a SRFI-37 option, such that its processor always adds the+'ad-hoc?' flag to the resulting alist."+ (option (option-names opt)+ (option-required-arg? opt)+ (option-optional-arg? opt)+ (compose ensure-ad-hoc (option-processor opt))))++(define %options+ ;; Specification of the command-line options.+ (let ((to-remove '("ad-hoc" "inherit" "load" "help" "version")))+ (append+ (list (option '(#\h "help") #f #f+ (lambda args+ (show-help)+ (exit 0)))+ (option '(#\V "version") #f #f+ (lambda args+ (show-version-and-exit "guix shell")))++ (option '(#\D "development") #f #f+ (lambda (opt name arg result)+ ;; Temporarily remove the 'ad-hoc?' flag from result.+ ;; The next option will put it back thanks to+ ;; 'wrapped-option'.+ (alist-delete 'ad-hoc? result)))++ ;; For consistency with 'guix package', support '-f' rather than+ ;; '-l' like 'guix environment' does.+ (option '(#\f "file") #t #f+ (lambda (opt name arg result)+ (alist-cons 'load (tag-package-arg result arg)+ result))))+ (filter-map (lambda (opt)+ (and (not (any (lambda (name)+ (member name to-remove))+ (option-names opt)))+ (wrapped-option opt)))+ %environment-options))))++(define %default-options+ `((ad-hoc? . #t) ;always true+ ,@%environment-default-options))++(define (parse-args args)+ "Parse the list of command line arguments ARGS."+ (define (handle-argument arg result)+ (alist-cons 'package (tag-package-arg result arg)+ (ensure-ad-hoc result)))++ ;; The '--' token is used to separate the command to run from the rest of+ ;; the operands.+ (let ((args command (break (cut string=? "--" <>) args)))+ (let ((opts (parse-command-line args %options (list %default-options)+ #:argument-handler handle-argument)))+ (match command+ (() opts)+ (("--") opts)+ (("--" command ...) (alist-cons 'exec command opts))))))++ +(define-command (guix-shell . args)+ (category development)+ (synopsis "spawn one-off software environments")++ (guix-environment* (parse-args args)))diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.inindex f5b76bf582..f8abeb2d38 100644--- a/po/guix/POTFILES.in+++ b/po/guix/POTFILES.in@@ -99,6 +99,7 @@ guix/derivations.scm guix/scripts/archive.scm guix/scripts/build.scm guix/scripts/environment.scm+guix/scripts/shell.scm guix/scripts/time-machine.scm guix/scripts/import/cpan.scm guix/scripts/import/crate.scmdiff --git a/tests/guix-shell.sh b/tests/guix-shell.shnew file mode 100644index 0000000000..f08637f7ff--- /dev/null+++ b/tests/guix-shell.sh@@ -0,0 +1,54 @@+# GNU Guix --- Functional package management for GNU+# Copyright © 2021 Ludovic Courtès <ludo@gnu.org>+#+# This file is part of GNU Guix.+#+# GNU Guix is free software; you can redistribute it and/or modify it+# under the terms of the GNU General Public License as published by+# the Free Software Foundation; either version 3 of the License, or (at+# your option) any later version.+#+# GNU Guix is distributed in the hope that it will be useful, but+# WITHOUT ANY WARRANTY; without even the implied warranty of+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+# GNU General Public License for more details.+#+# You should have received a copy of the GNU General Public License+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++#+# Test the 'guix shell' alias.+#++guix shell --version++tmpdir="t-guix-shell-$$"+trap 'rm -r "$tmpdir"' EXIT+mkdir "$tmpdir"++guix shell --bootstrap --pure guile-bootstrap -- guile --version++# '--ad-hoc' is a thing of the past.+! guix shell --ad-hoc guile-bootstrap++if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null+then+ # Compute the build environment for the initial GNU Make.+ guix shell --bootstrap --no-substitutes --search-paths --pure \+ -D -e '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/a"++ # Make sure bootstrap binaries are in the profile.+ profile=`grep "^export PATH" "$tmpdir/a" | sed -r 's|^.*="(.*)/bin"|\1|'`++ # Make sure the bootstrap binaries are all listed where they belong.+ grep -E "^export PATH=\"$profile/bin\"" "$tmpdir/a"+ grep -E "^export CPATH=\"$profile/include\"" "$tmpdir/a"+ grep -E "^export LIBRARY_PATH=\"$profile/lib\"" "$tmpdir/a"+ for dep in bootstrap-binaries-0 gcc-bootstrap-0 glibc-bootstrap-0+ do+ guix gc --references "$profile" | grep "$dep"+ done++ # 'make-boot0' itself must not be listed.+ ! guix gc --references "$profile" | grep make-boot0+fi-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 08/11] environment: Do not connect to the daemon when '--profile' is used.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-9-ludo@gnu.org
This further speeds up the 'guix environment -p PROFILE' case.
* guix/scripts/environment.scm (guix-environment*)[store-needed?]: Newvariable.[with-store/maybe]: New macro.Use it instead of 'with-store', and remove 'with-build-handler' form.--- guix/scripts/environment.scm | 169 +++++++++++++++++++---------------- 1 file changed, 93 insertions(+), 76 deletions(-)
Toggle diff (195 lines)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 32f376fdd2..e23d52df39 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -691,6 +691,26 @@ (define (guix-environment* opts) (mappings (pick-all opts 'file-system-mapping)) (white-list (pick-all opts 'inherit-regexp))) + (define store-needed?+ ;; Whether connecting to the daemon is needed.+ (or container? (not profile)))++ (define-syntax-rule (with-store/maybe store exp ...)+ ;; Evaluate EXP... with STORE bound to a connection, unless+ ;; STORE-NEEDED? is false, in which case STORE is bound to #f.+ (let ((proc (lambda (store) exp ...)))+ (if store-needed?+ (with-store s+ (set-build-options-from-command-line s opts)+ (with-build-handler (build-notifier #:use-substitutes?+ (assoc-ref opts 'substitutes?)+ #:verbosity+ (assoc-ref opts 'verbosity)+ #:dry-run?+ (assoc-ref opts 'dry-run?))+ (proc s)))+ (proc #f))))+ (when container? (assert-container-features)) (when (and (not container?) link-prof?)@@ -701,88 +721,85 @@ (define (guix-environment* opts) (leave (G_ "--no-cwd cannot be used without --container~%"))) - (with-store store- (with-build-handler (build-notifier #:use-substitutes?- (assoc-ref opts 'substitutes?)- #:verbosity- (assoc-ref opts 'verbosity)- #:dry-run?- (assoc-ref opts 'dry-run?))- (with-status-verbosity (assoc-ref opts 'verbosity)- (define manifest-from-opts- (options/resolve-packages store opts))+ (with-store/maybe store+ (with-status-verbosity (assoc-ref opts 'verbosity)+ (define manifest-from-opts+ (options/resolve-packages store opts)) - (define manifest- (if profile- (profile-manifest profile)- manifest-from-opts))+ (define manifest+ (if profile+ (profile-manifest profile)+ manifest-from-opts)) - (when (and profile- (> (length (manifest-entries manifest-from-opts)) 0))- (leave (G_ "'--profile' cannot be used with package options~%")))+ (when (and profile+ (> (length (manifest-entries manifest-from-opts)) 0))+ (leave (G_ "'--profile' cannot be used with package options~%"))) - (when (null? (manifest-entries manifest))- (warning (G_ "no packages specified; creating an empty environment~%")))+ (when (null? (manifest-entries manifest))+ (warning (G_ "no packages specified; creating an empty environment~%"))) - (set-build-options-from-command-line store opts)+ ;; Use the bootstrap Guile when requested.+ (parameterize ((%graft? (assoc-ref opts 'graft?))+ (%guile-for-build+ (and store-needed?+ (package-derivation+ store+ (if bootstrap?+ %bootstrap-guile+ (default-guile))))))+ (run-with-store store+ ;; Containers need a Bourne shell at /bin/sh.+ (mlet* %store-monad ((bash (environment-bash container?+ bootstrap?+ system))+ (prof-drv (if profile+ (return #f)+ (manifest->derivation+ manifest system bootstrap?)))+ (profile -> (if profile+ (readlink* profile)+ (derivation->output-path prof-drv)))+ (gc-root -> (assoc-ref opts 'gc-root))) - ;; Use the bootstrap Guile when requested.- (parameterize ((%graft? (assoc-ref opts 'graft?))- (%guile-for-build- (and (or container? (not profile))- (package-derivation- store- (if bootstrap?- %bootstrap-guile- (default-guile))))))- (run-with-store store- ;; Containers need a Bourne shell at /bin/sh.- (mlet* %store-monad ((bash (environment-bash container?- bootstrap?- system))- (prof-drv (if profile- (return #f)- (manifest->derivation- manifest system bootstrap?)))- (profile -> (if profile- (readlink* profile)- (derivation->output-path prof-drv)))- (gc-root -> (assoc-ref opts 'gc-root)))-- ;; First build the inputs. This is necessary even for- ;; --search-paths. Additionally, we might need to build bash for- ;; a container.- (mbegin %store-monad+ ;; First build the inputs. This is necessary even for+ ;; --search-paths. Additionally, we might need to build bash for+ ;; a container.+ (mbegin %store-monad+ (mwhen store-needed? (built-derivations (append (if prof-drv (list prof-drv) '())- (if (derivation? bash) (list bash) '())))- (mwhen gc-root- (register-gc-root profile gc-root))+ (if (derivation? bash) (list bash) '()))))+ (mwhen gc-root+ (register-gc-root profile gc-root)) - (cond- ((assoc-ref opts 'search-paths)- (show-search-paths profile manifest #:pure? pure?)- (return #t))- (container?- (let ((bash-binary- (if bootstrap?- (derivation->output-path bash)- (string-append (derivation->output-path bash)- "/bin/sh"))))- (launch-environment/container #:command command- #:bash bash-binary- #:user user- #:user-mappings mappings- #:profile profile- #:manifest manifest- #:white-list white-list- #:link-profile? link-prof?- #:network? network?- #:map-cwd? (not no-cwd?))))+ (cond+ ((assoc-ref opts 'search-paths)+ (show-search-paths profile manifest #:pure? pure?)+ (return #t))+ (container?+ (let ((bash-binary+ (if bootstrap?+ (derivation->output-path bash)+ (string-append (derivation->output-path bash)+ "/bin/sh"))))+ (launch-environment/container #:command command+ #:bash bash-binary+ #:user user+ #:user-mappings mappings+ #:profile profile+ #:manifest manifest+ #:white-list white-list+ #:link-profile? link-prof?+ #:network? network?+ #:map-cwd? (not no-cwd?)))) - (else- (return- (exit/status- (launch-environment/fork command profile manifest- #:white-list white-list- #:pure? pure?)))))))))))))))+ (else+ (return+ (exit/status+ (launch-environment/fork command profile manifest+ #:white-list white-list+ #:pure? pure?))))))))))))))++;;; Local Variables:+;;; (put 'with-store/maybe 'scheme-indent-function 1)+;;; End:-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 09/11] environment: Autoload some modules.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-10-ludo@gnu.org
This further speeds up the 'guix environment -p PROFILE' case.
* guix/scripts/environment.scm: Autoload a bunch of modules.--- guix/scripts/environment.scm | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-)
Toggle diff (33 lines)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex e23d52df39..05a43659da 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -34,15 +34,18 @@ (define-module (guix scripts environment) #:use-module (guix scripts) #:use-module (guix scripts build) #:use-module (guix transformations)- #:use-module (gnu build linux-container)- #:use-module (gnu build accounts)- #:use-module ((guix build syscalls) #:select (set-network-interface-up))- #:use-module (gnu system linux-container)+ #:autoload (gnu build linux-container) (call-with-container %namespaces+ user-namespace-supported?+ unprivileged-user-namespace-supported?+ setgroups-supported?)+ #:autoload (gnu build accounts) (password-entry group-entry+ password-entry-name password-entry-directory+ write-passwd write-group)+ #:autoload (guix build syscalls) (set-network-interface-up) #:use-module (gnu system file-systems)- #:use-module (gnu packages)- #:use-module (gnu packages bash)- #:use-module ((gnu packages bootstrap)- #:select (bootstrap-executable %bootstrap-guile))+ #:autoload (gnu packages) (specification->package+output)+ #:autoload (gnu packages bash) (bash)+ #:autoload (gnu packages bootstrap) (bootstrap-executable %bootstrap-guile) #:use-module (ice-9 match) #:use-module (srfi srfi-1) #:use-module (srfi srfi-11)-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 10/11] cache: Gracefully handle non-existent cache.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-11-ludo@gnu.org
* guix/cache.scm (maybe-remove-expired-cache-entries): Ignore ENOENTwhen writing EXPIRY-FILE.--- guix/cache.scm | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
Toggle diff (22 lines)diff --git a/guix/cache.scm b/guix/cache.scmindex 0401a9d428..51009809bd 100644--- a/guix/cache.scm+++ b/guix/cache.scm@@ -101,7 +101,13 @@ (define last-expiry-date #:now now #:entry-expiration entry-expiration #:delete-entry delete-entry)- (call-with-output-file expiry-file- (cute write (time-second now) <>))))+ (catch 'system-error+ (lambda ()+ (call-with-output-file expiry-file+ (cute write (time-second now) <>)))+ (lambda args+ ;; ENOENT means CACHE does not exist.+ (unless (= ENOENT (system-error-errno args))+ (apply throw args)))))) ;;; cache.scm ends here-- 2.33.0
L
L
Ludovic Courtès wrote on 11 Oct 23:38 +0200
[PATCH v2 11/11] shell: Maintain a profile cache.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211011213809.17482-12-ludo@gnu.org
shell: Maintain a profile cache.
With this change, running "guix shell" (no arguments) is equivalent to:
guix environment -r ~/.cache/guix/profiles/some-root -l guix.scm
This is the cache miss. On cache hit, it's equivalent to:
guix environment -p ~/.cache/guix/profiles/some-root
... which can run in 0.1s.
* guix/scripts/shell.scm (options-with-caching): New procedure.(parse-args): Use it.(%profile-cache-directory): New variable.(profile-cache-key, profile-cached-gc-root): New procedures.(show-help, %options): Add '--rebuild-cache'.(guix-shell)[cache-entries, entry-expiration]: New procedures.Add call to 'maybe-remove-expired-cache-entries'.* doc/guix.texi (Invoking guix shell): Document '--rebuild-cache'.--- doc/guix.texi | 11 ++++ guix/scripts/shell.scm | 127 ++++++++++++++++++++++++++++++++++++++--- 2 files changed, 130 insertions(+), 8 deletions(-)
Toggle diff (199 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex b95025a39f..f3be6b5085 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -5768,6 +5768,17 @@ This is similar to the same-named option in @command{guix package} (@pxref{profile-manifest, @option{--manifest}}) and uses the same manifest files. +@item --rebuild-cache+When using @option{--manifest}, @option{--file}, or when invoked without+arguments, @command{guix shell} caches the environment so that+subsequent uses are instantaneous. The cache is invalidated anytime the+file is modified.++The @option{--rebuild-cache} forces the cached environment to be+refreshed even if the file has not changed. This is useful if the+@command{guix.scm} or @command{manifest.scm} has external dependencies,+or if its behavior depends, say, on environment variables.+ @item --pure Unset existing environment variables when building the new environment, except those specified with @option{--preserve} (see below). This has the effect ofdiff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmindex 45fd536145..4062e8155d 100644--- a/guix/scripts/shell.scm+++ b/guix/scripts/shell.scm@@ -31,7 +31,15 @@ (define-module (guix scripts shell) #:use-module (srfi srfi-71) #:use-module (ice-9 match) #:autoload (ice-9 rdelim) (read-line)- #:autoload (guix utils) (config-directory)+ #:autoload (guix base32) (bytevector->base32-string)+ #:autoload (rnrs bytevectors) (string->utf8)+ #:autoload (guix utils) (config-directory cache-directory)+ #:autoload (guix describe) (current-channels)+ #:autoload (guix channels) (channel-commit)+ #:autoload (gcrypt hash) (sha256)+ #:use-module ((guix build utils) #:select (mkdir-p))+ #:use-module (guix cache)+ #:use-module ((ice-9 ftw) #:select (scandir)) #:export (guix-shell)) (define (show-help)@@ -48,6 +56,8 @@ (define (show-help) FILE evaluates to")) (display (G_ " -q inhibit loading of 'guix.scm' and 'manifest.scm'"))+ (display (G_ "+ --rebuild-cache rebuild cached environment, if any")) (show-environment-options-help) (newline)@@ -109,7 +119,10 @@ (define %options result))) (option '(#\q) #f #f (lambda (opt name arg result)- (alist-cons 'explicit-loading? #t result))))+ (alist-cons 'explicit-loading? #t result)))+ (option '("rebuild-cache") #f #f+ (lambda (opt name arg result)+ (alist-cons 'rebuild-cache? #t result)))) (filter-map (lambda (opt) (and (not (any (lambda (name) (member name to-remove))@@ -132,11 +145,12 @@ (define (handle-argument arg result) (let ((args command (break (cut string=? "--" <>) args))) (let ((opts (parse-command-line args %options (list %default-options) #:argument-handler handle-argument)))- (auto-detect-manifest- (match command- (() opts)- (("--") opts)- (("--" command ...) (alist-cons 'exec command opts)))))))+ (options-with-caching+ (auto-detect-manifest+ (match command+ (() opts)+ (("--") opts)+ (("--" command ...) (alist-cons 'exec command opts)))))))) (define (find-file-in-parent-directories candidates) "Find one of CANDIDATES in the current directory or one of its ancestors."@@ -187,6 +201,53 @@ (define (authorized-shell-directory? directory) line)))))))))) (const #f))) +(define (options-with-caching opts)+ "If OPTS contains exactly one 'load' or one 'manifest' key, automatically+add a 'profile' key (when a profile for that file is already in cache) or a+'gc-root' key (to add the profile to cache)."+ (define (single-file-for-caching opts)+ (let loop ((opts opts)+ (file #f))+ (match opts+ (() file)+ ((('package . _) . _) #f)+ ((('load . ('package candidate)) . rest)+ (and (not file) (loop rest candidate)))+ ((('manifest . candidate) . rest)+ (and (not file) (loop rest candidate)))+ ((('expression . _) . _) #f)+ ((_ . rest) (loop rest file)))))++ ;; Check whether there's a single 'load' or 'manifest' option. When that is+ ;; the case, arrange to automatically cache the resulting profile.+ (match (single-file-for-caching opts)+ (#f opts)+ (file+ (let* ((root (profile-cached-gc-root file))+ (stat (and root (false-if-exception (lstat root)))))+ (if (and (not (assoc-ref opts 'rebuild-cache?))+ stat+ (<= (stat:mtime ((@ (guile) stat) file))+ (stat:mtime stat)))+ (let ((now (current-time)))+ ;; Update the atime on ROOT to reflect usage.+ (utime root+ now (stat:mtime stat) 0 (stat:mtimensec stat)+ AT_SYMLINK_NOFOLLOW)+ (alist-cons 'profile root+ (remove (match-lambda+ (('load . _) #t)+ (('manifest . _) #t)+ (_ #f))+ opts))) ;load right away+ (if (and root (not (assq-ref opts 'gc-root)))+ (begin+ (if stat+ (delete-file root)+ (mkdir-p (dirname root)))+ (alist-cons 'gc-root root opts))+ opts))))))+ (define (auto-detect-manifest opts) "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or \"manifest.scm\" file from the current directory or one of its ancestors.@@ -236,9 +297,59 @@ (define disallow-implicit-load? (authorized-directory-file))) opts)))))) + +;;;+;;; Profile cache.+;;;++(define %profile-cache-directory+ ;; Directory where profiles created by 'guix shell' alone (without extra+ ;; options) are cached.+ (make-parameter (string-append (cache-directory #:ensure? #f)+ "/profiles")))++(define (profile-cache-key file)+ "Return the cache key for the profile corresponding to FILE, a 'guix.scm' or+'manifest.scm' file, or #f if we lack channel information."+ (match (current-channels)+ (() #f)+ (((= channel-commit commits) ...)+ (let ((stat (stat file)))+ (bytevector->base32-string+ ;; Since FILE is not canonicalized, only include the device/inode+ ;; numbers. XXX: In some rare cases involving Btrfs and NFS, this can+ ;; be insufficient: <https://lwn.net/Articles/866582/>.+ (sha256 (string->utf8+ (string-append (string-join commits) ":"+ (number->string (stat:dev stat)) ":"+ (number->string (stat:ino stat))))))))))++(define (profile-cached-gc-root file)+ "Return the cached GC root for FILE, a 'guix.scm' or 'manifest.scm' file, or+#f if we lack information to cache it."+ (match (profile-cache-key file)+ (#f #f)+ (key (string-append (%profile-cache-directory) "/" key))))+ (define-command (guix-shell . args) (category development) (synopsis "spawn one-off software environments") - (guix-environment* (parse-args args)))+ (define (cache-entries directory)+ (filter-map (match-lambda+ ((or "." "..") #f)+ (file (string-append directory "/" file)))+ (or (scandir directory) '())))++ (define* (entry-expiration file)+ ;; Return the time at which FILE, a cached profile, is considered expired.+ (match (false-if-exception (lstat file))+ (#f 0) ;FILE may have been deleted in the meantime+ (st (+ (stat:atime st) (* 60 60 24 7)))))++ (let ((result (guix-environment* (parse-args args))))+ (maybe-remove-expired-cache-entries (%profile-cache-directory)+ cache-entries+ #:entry-expiration entry-expiration)+ result))-- 2.33.0
Z
Z
zimoun wrote on 12 Oct 08:39 +0200
Re: [bug#50960] [PATCH v2 01/11] packages: Add 'package-development-inputs'.
(name . Ludovic Courtès)(address . ludo@gnu.org)
86czoaoigo.fsf@gmail.com
Hi Ludo,
On Mon, 11 Oct 2021 at 23:37, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (6 lines)> * guix/packages.scm (package-development-inputs): New procedure.> * guix/scripts/environment.scm (package-environment-inputs): Use it.> * tests/packages.scm ("package-development-inputs")> ("package-development-inputs, cross-compilation"): New tests.> * doc/guix.texi (package Reference): Document it.
LGTM, except…
Toggle quote (3 lines)> guix/packages.scm | 10 +++++++++> guix/scripts/environment.scm | 2 +-
[...]
Toggle quote (9 lines)> +(define* (package-development-inputs package> + #:optional (system (%current-system))> + #:key target)> + "Return the list of inputs required by PACKAGE for development purposes on> +SYSTEM. When TARGET is true, return the inputs needed to cross-compile> +PACKAGE from SYSTEM to TRIPLET, where TRIPLET is a triplet such as> +\"aarch64-linux-gnu\"."> + (bag-transitive-inputs (package->bag package system target)))
[...]
Toggle quote (5 lines)> ;; Remove non-package inputs such as origin records.> (filter-map input->manifest-entry> - (bag-transitive-inputs (package->bag package))))> + (package-development-inputs package system)))
Why ’system’ is used here? Why is not simply?
+ (package-development-inputs package)))
because ’system’ should be not defined, or I do not know where IIUC.

Cheers,simon
Z
Z
zimoun wrote on 12 Oct 08:43 +0200
Re: [bug#50960] [PATCH v2 02/11] profiles: Add 'package->development-manifest'.
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
868ryyoi8q.fsf@gmail.com
Hi Ludo,
On Mon, 11 Oct 2021 at 23:38, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (9 lines)> From: Ludovic Courtès <ludovic.courtes@inria.fr>>> * guix/profiles.scm (package->development-manifest): New procedure.> * guix/scripts/environment.scm (input->manifest-entry)> (package-environment-inputs): Remove.> * guix/scripts/environment.scm (options/resolve-packages): Use> 'package->development-manifest' instead of 'package-environment-inputs'.> * tests/profiles.scm ("package->development-manifest"): New test.
LGTM!
(These patch 1 and 2 could be applied independently of the new “guixshell” command. :-))

Cheers,simon
P
P
pelzflorian (Florian Pelz) wrote on 12 Oct 10:53 +0200
Re: [bug#50960] [PATCH v2 00/11] 'guix shell' strikes again
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
20211012085346.v56ck7fzsfuscto6@pelzflorian.localdomain
On Mon, Oct 11, 2021 at 11:37:58PM +0200, Ludovic Courtès wrote:
Toggle quote (4 lines)> Hello Guix!> > Here’s a v2 with the following changes:
Nice!
Toggle quote (4 lines)> • Auto-loading happens if and only if the containing directory> is listed in ~/.config/guix/shell-authorized-directories, as> suggested by Florian.
Toggle quote (3 lines)> […]> How does that sound? :-)
Much better, thank you!
Regards,Florian
P
P
pelzflorian (Florian Pelz) wrote on 12 Oct 10:57 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
20211012085756.nfhgz55lqhgt6bdq@pelzflorian.localdomain
On Tue, Oct 12, 2021 at 10:53:55AM +0200, pelzflorian (Florian Pelz) wrote:
Toggle quote (8 lines)> On Mon, Oct 11, 2021 at 11:37:58PM +0200, Ludovic Courtès wrote:> > • Auto-loading happens if and only if the containing directory> > is listed in ~/.config/guix/shell-authorized-directories, as> > suggested by Florian.> > Liliana Marie Prikler suggested it> <https://lists.gnu.org/r/guix-patches/2021-10/msg00380.html>.
Or was it Nicoló Balzarotti?
https://lists.gnu.org/r/guix-patches/2021-10/msg00196.html
Enough confusion.
Regards,Florian
L
L
Ludovic Courtès wrote on 12 Oct 11:27 +0200
Re: [bug#50960] [PATCH v2 02/11] profiles: Add 'package->development-manifest'.
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 50960@debbugs.gnu.org)
878ryyy4n7.fsf@gnu.org
Hi!
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (3 lines)> (These patch 1 and 2 could be applied independently of the new “guix> shell” command. :-))
Yes, definitely; actually most of the patches are independent of ‘guixshell’.
Ludo’.
L
L
Ludovic Courtès wrote on 12 Oct 11:54 +0200
Re: [bug#50960] [PATCH v2 01/11] packages: Add 'package-development-inputs'.
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 50960@debbugs.gnu.org)
87v922va9b.fsf@gnu.org
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (22 lines)>> +(define* (package-development-inputs package>> + #:optional (system (%current-system))>> + #:key target)>> + "Return the list of inputs required by PACKAGE for development purposes on>> +SYSTEM. When TARGET is true, return the inputs needed to cross-compile>> +PACKAGE from SYSTEM to TRIPLET, where TRIPLET is a triplet such as>> +\"aarch64-linux-gnu\".">> + (bag-transitive-inputs (package->bag package system target)))>> [...]>>> ;; Remove non-package inputs such as origin records.>> (filter-map input->manifest-entry>> - (bag-transitive-inputs (package->bag package))))>> + (package-development-inputs package system)))>> Why ’system’ is used here? Why is not simply?>> + (package-development-inputs package)))>> because ’system’ should be not defined, or I do not know where IIUC.
It’s an optional parameter above.
Ludo’.
L
L
Ludovic Courtès wrote on 12 Oct 11:55 +0200
Re: [bug#50960] [PATCH v2 00/11] 'guix shell' strikes again
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 50960@debbugs.gnu.org)
87r1cqva7g.fsf@gnu.org
"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:
Toggle quote (13 lines)> On Tue, Oct 12, 2021 at 10:53:55AM +0200, pelzflorian (Florian Pelz) wrote:>> On Mon, Oct 11, 2021 at 11:37:58PM +0200, Ludovic Courtès wrote:>> > • Auto-loading happens if and only if the containing directory>> > is listed in ~/.config/guix/shell-authorized-directories, as>> > suggested by Florian.>> >> Liliana Marie Prikler suggested it>> <https://lists.gnu.org/r/guix-patches/2021-10/msg00380.html>.>> Or was it Nicoló Balzarotti?>> https://lists.gnu.org/r/guix-patches/2021-10/msg00196.html
My bad, sorry for misattributing it.
Anyhow: big thanks to all of you for providing feedback! :-)
Ludo’.
Z
Z
zimoun wrote on 12 Oct 13:52 +0200
Re: [bug#50960] [PATCH v2 01/11] packages: Add 'package-development-inputs'.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50960@debbugs.gnu.org)
86y26ylasy.fsf@gmail.com
On Tue, 12 Oct 2021 at 11:54, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (26 lines)> zimoun <zimon.toutoune@gmail.com> skribis:>>>> +(define* (package-development-inputs package>>> + #:optional (system (%current-system))>>> + #:key target)>>> + "Return the list of inputs required by PACKAGE for development purposes on>>> +SYSTEM. When TARGET is true, return the inputs needed to cross-compile>>> +PACKAGE from SYSTEM to TRIPLET, where TRIPLET is a triplet such as>>> +\"aarch64-linux-gnu\".">>> + (bag-transitive-inputs (package->bag package system target)))>>>> [...]>>>>> ;; Remove non-package inputs such as origin records.>>> (filter-map input->manifest-entry>>> - (bag-transitive-inputs (package->bag package))))>>> + (package-development-inputs package system)))>>>> Why ’system’ is used here? Why is not simply?>>>> + (package-development-inputs package)))>>>> because ’system’ should be not defined, or I do not know where IIUC.>> It’s an optional parameter above.
Sorry if I miss, to me, ’system’ is not bounded when calling:
Toggle snippet (6 lines) ;; Remove non-package inputs such as origin records. (filter-map input->manifest-entry- (bag-transitive-inputs (package->bag package))))+ (package-development-inputs package system)))
and it seems bounded to something:
;;; (#<procedure system (#:optional _)>)”
and I do not know from where this definition comes.
Maybe I misread what optional means. From my understanding, it means:
(package-development-inputs foo)
will use ’(%current-system)’ as default. And it also means that:
(package-development-inputs foo system)
where ’system’ is bounded to something and from my understanding thissomething is not necessary ’(%current-system)’.

Again, sorry if I misread something.
Cheers,simon
P
P
pelzflorian (Florian Pelz) wrote on 13 Oct 18:51 +0200
Re: [bug#50960] [PATCH v2 03/11] Add 'guix shell'.
(name . Ludovic Courtès)(address . ludo@gnu.org)
20211013165104.nlzlz7npsxcpbsjn@pelzflorian.localdomain
Hello Ludo!
When I pull, I can’t use guix shell, perhaps because you did not addit to guix/self.scm?
I like that the new documentation does not target only developers andhackers.
On Mon, Oct 11, 2021 at 11:38:01PM +0200, Ludovic Courtès wrote:
Toggle quote (9 lines)> […]> +The general syntax is:> +> +@example> +guix shell [@var{options}] [@var{package}@dots{}]> +@end example> +> +The following example creates a environment containing Python and NumPy,
*an* environment

Toggle quote (2 lines)> +building and downloading any missing package, and runs the
building *or* downloading

Toggle quote (6 lines)> […]> +By default, the shell session or command runs in an @emph{augmented}> +environment, where the new packages are added to search path environment> +variables such as @code{PATH}. You can, instead, choose to create an> +@emph{isolated} environment containing nothing but the packages you
Hmmph. I had seen the words “isolated environment” being used todescribe containers in "(guix)Managing Software the Guix Way".
Therefore I also had used the German term for “isolated” to explaincontainers in other parts of the manual. Probably this was notnecessary; if you apply guix shell with this term, I will change alloccurrences of “isoliert” in the German PO files.
But then again, maybe it would be better if you described pureenvironments not as isolated. I also see a mix-up further below:

Toggle quote (13 lines)> +This @option{--container} option can also prove useful if you wish to> +run a security-sensitive application, such as a web browser, in an> +isolated environment. For example, the command below launches> +Ungoogled-Chromium in an isolated environment, this time sharing network> +access with the host and preserving its @code{DISPLAY} environment> +variable, but without even sharing the current directory:> +> +@example> +guix shell --container --network --no-cwd ungoogled-chromium \> + --preserve='^DISPLAY$' -- chromium> +@end example> […]
Back to typos:
Toggle quote (5 lines)> +@node Invoking guix environment> +@section Invoking @command{guix environment}> +> +The purpose of @command{guix environment} is to assists in creating
is to *assist*
So much for the docs. I have not looked further.
Regards,Florian
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 01/10] packages: Add 'package-development-inputs'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211018195219.13898-2-ludo@gnu.org
* guix/packages.scm (package-development-inputs): New procedure.* guix/scripts/environment.scm (package-environment-inputs): Use it.* tests/packages.scm ("package-development-inputs")("package-development-inputs, cross-compilation"): New tests.* doc/guix.texi (package Reference): Document it.--- doc/guix.texi | 41 ++++++++++++++++++++++++++++++++++++ guix/packages.scm | 10 +++++++++ guix/scripts/environment.scm | 2 +- tests/packages.scm | 14 ++++++++++++ 4 files changed, 66 insertions(+), 1 deletion(-)
Toggle diff (120 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex a49abc0554..deb4eac4ac 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -6848,6 +6848,47 @@ cross-compiling: It is an error to refer to @code{this-package} outside a package definition. @end deffn +@cindex development inputs, of a package+@cindex implicit inputs, of a package+Sometimes you will want to obtain the list of inputs needed to+@emph{develop} a package---all the inputs that are visible when the+package is compiled. This is what the @code{package-development-inputs}+procedure returns.++@deffn {Scheme Procedure} package-development-inputs @var{package} @+ [@var{system}] [#:target #f]+Return the list of inputs required by @var{package} for development+purposes on @var{system}. When @var{target} is true, return the inputs+needed to cross-compile @var{package} from @var{system} to+@var{triplet}, where @var{triplet} is a triplet such as+@code{"aarch64-linux-gnu"}.++Note that the result includes both explicit inputs and implicit+inputs---inputs automatically added by the build system (@pxref{Build+Systems}). Let us take the @code{hello} package to illustrate that:++@lisp+(use-modules (gnu packages base) (guix packages))++hello+@result{} #<package hello@@2.10 gnu/packages/base.scm:79 7f585d4f6790>++(package-direct-inputs hello)+@result{} ()++(package-development-inputs hello)+@result{} (("source" @dots{}) ("tar" #<package tar@@1.32 @dots{}>) @dots{})+@end lisp++In this example, @code{package-direct-inputs} returns the empty list,+because @code{hello} has zero explicit dependencies. Conversely,+@code{package-development-inputs} includes inputs implicitly added by+@code{gnu-build-system} that are required to build @code{hello}: tar,+gzip, GCC, libc, Bash, and more. To visualize it, @command{guix graph+hello} would show you explicit inputs, whereas @command{guix graph -t+bag hello} would include implicit inputs (@pxref{Invoking guix graph}).+@end deffn+ Because packages are regular Scheme objects that capture a complete dependency graph and associated build procedures, it is often useful to write procedures that take a package and return a modified versiondiff --git a/guix/packages.scm b/guix/packages.scmindex 8c3a0b0b7b..43e0130793 100644--- a/guix/packages.scm+++ b/guix/packages.scm@@ -153,6 +153,7 @@ (define-module (guix packages) bag-transitive-host-inputs bag-transitive-build-inputs bag-transitive-target-inputs+ package-development-inputs package-closure default-guile@@ -1070,6 +1071,15 @@ (define (bag-transitive-target-inputs bag) (%current-system (bag-system bag))) (transitive-inputs (bag-target-inputs bag)))) +(define* (package-development-inputs package+ #:optional (system (%current-system))+ #:key target)+ "Return the list of inputs required by PACKAGE for development purposes on+SYSTEM. When TARGET is true, return the inputs needed to cross-compile+PACKAGE from SYSTEM to TRIPLET, where TRIPLET is a triplet such as+\"aarch64-linux-gnu\"."+ (bag-transitive-inputs (package->bag package system target)))+ (define* (package-closure packages #:key (system (%current-system))) "Return the closure of PACKAGES on SYSTEM--i.e., PACKAGES and the list of packages they depend on, recursively."diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 6958bd6238..418f11c37e 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -82,7 +82,7 @@ (define (package-environment-inputs package) packages for PACKAGE." ;; Remove non-package inputs such as origin records. (filter-map input->manifest-entry- (bag-transitive-inputs (package->bag package))))+ (package-development-inputs package))) (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]diff --git a/tests/packages.scm b/tests/packages.scmindex 3756877270..266b5aeb7a 100644--- a/tests/packages.scm+++ b/tests/packages.scm@@ -353,6 +353,20 @@ (define read-at (package-transitive-supported-systems d) (package-transitive-supported-systems e)))) +(test-assert "package-development-inputs"+ ;; Note: Due to propagated inputs, 'package-development-inputs' returns a+ ;; couple more inputs, such as 'linux-libre-headers'.+ (lset<= equal?+ `(("source" ,(package-source hello)) ,@(standard-packages))+ (package-development-inputs hello)))++(test-assert "package-development-inputs, cross-compilation"+ (lset<= equal?+ `(("source" ,(package-source hello))+ ,@(standard-cross-packages "mips64el-linux-gnu" 'host)+ ,@(standard-cross-packages "mips64el-linux-gnu" 'target))+ (package-development-inputs hello #:target "mips64el-linux-gnu")))+ (test-assert "package-closure" (let-syntax ((dummy-package/no-implicit (syntax-rules ()-- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 00/10] Adding 'guix shell': last call!
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211018195219.13898-1-ludo@gnu.org
Hi!
This is v3 of the ‘guix shell’ patch set. Changes since v2:
• Merged as a single commit the one that adds support for implicit ‘guix.scm’/‘manifest.scm’ and then one that adds authorization checks;
• Fixed ‘package-development-inputs’ bug/typo reported by zimoun;
• Fixed doc typos reported by Florian.
If there are no objections, I’ll go ahead with this patch seriesin a few days. I’ll also add a news entry, and I guess a blog postintroducing it would be welcome.
WDYT?
Ludo’.
Ludovic Courtès (10): packages: Add 'package-development-inputs'. profiles: Add 'package->development-manifest'. Add 'guix shell'. shell: By default load the local 'guix.scm' or 'manifest.scm' file. environment: Add tests for '--profile'. environment: Skip derivation computation when '--profile' is used. environment: Do not connect to the daemon when '--profile' is used. environment: Autoload some modules. cache: Gracefully handle non-existent cache. shell: Maintain a profile cache.
Makefile.am | 2 + doc/contributing.texi | 8 +- doc/guix.texi | 443 ++++++++++++++++++++++++++-- guix/cache.scm | 10 +- guix/packages.scm | 10 + guix/profiles.scm | 19 ++ guix/scripts/environment.scm | 260 ++++++++-------- guix/scripts/shell.scm | 355 ++++++++++++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-environment-container.sh | 8 + tests/guix-environment.sh | 7 + tests/guix-shell.sh | 100 +++++++ tests/packages.scm | 14 + tests/profiles.scm | 7 + 14 files changed, 1098 insertions(+), 146 deletions(-) create mode 100644 guix/scripts/shell.scm create mode 100644 tests/guix-shell.sh

base-commit: 9cda21cf20a5c9bdf97e3a6d6c8f901fc3e4307d-- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 05/10] environment: Add tests for '--profile'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211018195219.13898-6-ludo@gnu.org
This is a followup to a643deac2de81755a1843a3b41dd53857678bebc.
* tests/guix-environment-container.sh, tests/guix-environment.sh: Addtests for '--profile'.--- tests/guix-environment-container.sh | 8 ++++++++ tests/guix-environment.sh | 7 +++++++ 2 files changed, 15 insertions(+)
Toggle diff (39 lines)diff --git a/tests/guix-environment-container.sh b/tests/guix-environment-container.shindex f2d15c8d0c..2e238c501d 100644--- a/tests/guix-environment-container.sh+++ b/tests/guix-environment-container.sh@@ -44,6 +44,14 @@ else test $? = 42 fi +# Try '--root' and '--profile'.+root="$tmpdir/root"+guix environment -C --ad-hoc --bootstrap guile-bootstrap -r "$root" -- guile --version+guix environment -C -p "$root" --bootstrap -- guile --version+path1=$(guix environment -C -p "$root" --bootstrap -- guile -c '(display (getenv "PATH"))')+path2=$(guix environment -C --ad-hoc --bootstrap guile-bootstrap -- guile -c '(display (getenv "PATH"))')+test "$path1" = "$path2"+ # Make sure "localhost" resolves. guix environment --container --ad-hoc --bootstrap guile-bootstrap \ -- guile -c '(exit (pair? (getaddrinfo "localhost" "80")))'diff --git a/tests/guix-environment.sh b/tests/guix-environment.shindex afadcbe195..f4fc2e39ed 100644--- a/tests/guix-environment.sh+++ b/tests/guix-environment.sh@@ -119,6 +119,13 @@ test `readlink "$gcroot"` = "$expected" guix environment --bootstrap -r "$gcroot" --ad-hoc guile-bootstrap \ -- guile -c 1 test `readlink "$gcroot"` = "$expected"++# Make sure '-p' works as expected.+test $(guix environment -p "$gcroot" -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT') = "$expected"+paths1="$(guix environment -p "$gcroot" --search-paths)"+paths2="$(guix environment --bootstrap --ad-hoc guile-bootstrap --search-paths)"+test "$paths1" = "$paths2"+ rm "$gcroot" # Try '-r' with a relative file name.-- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 06/10] environment: Skip derivation computation when '--profile' is used.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211018195219.13898-7-ludo@gnu.org
* guix/scripts/environment.scm (guix-environment*): Bypass calls to'package-derivation' and to 'manifest->derivation' when PROFILE istrue.--- guix/scripts/environment.scm | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-)
Toggle diff (48 lines)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 77956fc018..32f376fdd2 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -729,18 +729,21 @@ (define manifest ;; Use the bootstrap Guile when requested. (parameterize ((%graft? (assoc-ref opts 'graft?)) (%guile-for-build- (package-derivation- store- (if bootstrap?- %bootstrap-guile- (default-guile)))))+ (and (or container? (not profile))+ (package-derivation+ store+ (if bootstrap?+ %bootstrap-guile+ (default-guile)))))) (run-with-store store ;; Containers need a Bourne shell at /bin/sh. (mlet* %store-monad ((bash (environment-bash container? bootstrap? system))- (prof-drv (manifest->derivation- manifest system bootstrap?))+ (prof-drv (if profile+ (return #f)+ (manifest->derivation+ manifest system bootstrap?))) (profile -> (if profile (readlink* profile) (derivation->output-path prof-drv)))@@ -750,9 +753,9 @@ (define manifest ;; --search-paths. Additionally, we might need to build bash for ;; a container. (mbegin %store-monad- (built-derivations (if (derivation? bash)- (list prof-drv bash)- (list prof-drv)))+ (built-derivations (append+ (if prof-drv (list prof-drv) '())+ (if (derivation? bash) (list bash) '()))) (mwhen gc-root (register-gc-root profile gc-root)) -- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 07/10] environment: Do not connect to the daemon when '--profile' is used.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211018195219.13898-8-ludo@gnu.org
This further speeds up the 'guix environment -p PROFILE' case.
* guix/scripts/environment.scm (guix-environment*)[store-needed?]: Newvariable.[with-store/maybe]: New macro.Use it instead of 'with-store', and remove 'with-build-handler' form.--- guix/scripts/environment.scm | 169 +++++++++++++++++++---------------- 1 file changed, 93 insertions(+), 76 deletions(-)
Toggle diff (195 lines)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 32f376fdd2..e23d52df39 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -691,6 +691,26 @@ (define (guix-environment* opts) (mappings (pick-all opts 'file-system-mapping)) (white-list (pick-all opts 'inherit-regexp))) + (define store-needed?+ ;; Whether connecting to the daemon is needed.+ (or container? (not profile)))++ (define-syntax-rule (with-store/maybe store exp ...)+ ;; Evaluate EXP... with STORE bound to a connection, unless+ ;; STORE-NEEDED? is false, in which case STORE is bound to #f.+ (let ((proc (lambda (store) exp ...)))+ (if store-needed?+ (with-store s+ (set-build-options-from-command-line s opts)+ (with-build-handler (build-notifier #:use-substitutes?+ (assoc-ref opts 'substitutes?)+ #:verbosity+ (assoc-ref opts 'verbosity)+ #:dry-run?+ (assoc-ref opts 'dry-run?))+ (proc s)))+ (proc #f))))+ (when container? (assert-container-features)) (when (and (not container?) link-prof?)@@ -701,88 +721,85 @@ (define (guix-environment* opts) (leave (G_ "--no-cwd cannot be used without --container~%"))) - (with-store store- (with-build-handler (build-notifier #:use-substitutes?- (assoc-ref opts 'substitutes?)- #:verbosity- (assoc-ref opts 'verbosity)- #:dry-run?- (assoc-ref opts 'dry-run?))- (with-status-verbosity (assoc-ref opts 'verbosity)- (define manifest-from-opts- (options/resolve-packages store opts))+ (with-store/maybe store+ (with-status-verbosity (assoc-ref opts 'verbosity)+ (define manifest-from-opts+ (options/resolve-packages store opts)) - (define manifest- (if profile- (profile-manifest profile)- manifest-from-opts))+ (define manifest+ (if profile+ (profile-manifest profile)+ manifest-from-opts)) - (when (and profile- (> (length (manifest-entries manifest-from-opts)) 0))- (leave (G_ "'--profile' cannot be used with package options~%")))+ (when (and profile+ (> (length (manifest-entries manifest-from-opts)) 0))+ (leave (G_ "'--profile' cannot be used with package options~%"))) - (when (null? (manifest-entries manifest))- (warning (G_ "no packages specified; creating an empty environment~%")))+ (when (null? (manifest-entries manifest))+ (warning (G_ "no packages specified; creating an empty environment~%"))) - (set-build-options-from-command-line store opts)+ ;; Use the bootstrap Guile when requested.+ (parameterize ((%graft? (assoc-ref opts 'graft?))+ (%guile-for-build+ (and store-needed?+ (package-derivation+ store+ (if bootstrap?+ %bootstrap-guile+ (default-guile))))))+ (run-with-store store+ ;; Containers need a Bourne shell at /bin/sh.+ (mlet* %store-monad ((bash (environment-bash container?+ bootstrap?+ system))+ (prof-drv (if profile+ (return #f)+ (manifest->derivation+ manifest system bootstrap?)))+ (profile -> (if profile+ (readlink* profile)+ (derivation->output-path prof-drv)))+ (gc-root -> (assoc-ref opts 'gc-root))) - ;; Use the bootstrap Guile when requested.- (parameterize ((%graft? (assoc-ref opts 'graft?))- (%guile-for-build- (and (or container? (not profile))- (package-derivation- store- (if bootstrap?- %bootstrap-guile- (default-guile))))))- (run-with-store store- ;; Containers need a Bourne shell at /bin/sh.- (mlet* %store-monad ((bash (environment-bash container?- bootstrap?- system))- (prof-drv (if profile- (return #f)- (manifest->derivation- manifest system bootstrap?)))- (profile -> (if profile- (readlink* profile)- (derivation->output-path prof-drv)))- (gc-root -> (assoc-ref opts 'gc-root)))-- ;; First build the inputs. This is necessary even for- ;; --search-paths. Additionally, we might need to build bash for- ;; a container.- (mbegin %store-monad+ ;; First build the inputs. This is necessary even for+ ;; --search-paths. Additionally, we might need to build bash for+ ;; a container.+ (mbegin %store-monad+ (mwhen store-needed? (built-derivations (append (if prof-drv (list prof-drv) '())- (if (derivation? bash) (list bash) '())))- (mwhen gc-root- (register-gc-root profile gc-root))+ (if (derivation? bash) (list bash) '()))))+ (mwhen gc-root+ (register-gc-root profile gc-root)) - (cond- ((assoc-ref opts 'search-paths)- (show-search-paths profile manifest #:pure? pure?)- (return #t))- (container?- (let ((bash-binary- (if bootstrap?- (derivation->output-path bash)- (string-append (derivation->output-path bash)- "/bin/sh"))))- (launch-environment/container #:command command- #:bash bash-binary- #:user user- #:user-mappings mappings- #:profile profile- #:manifest manifest- #:white-list white-list- #:link-profile? link-prof?- #:network? network?- #:map-cwd? (not no-cwd?))))+ (cond+ ((assoc-ref opts 'search-paths)+ (show-search-paths profile manifest #:pure? pure?)+ (return #t))+ (container?+ (let ((bash-binary+ (if bootstrap?+ (derivation->output-path bash)+ (string-append (derivation->output-path bash)+ "/bin/sh"))))+ (launch-environment/container #:command command+ #:bash bash-binary+ #:user user+ #:user-mappings mappings+ #:profile profile+ #:manifest manifest+ #:white-list white-list+ #:link-profile? link-prof?+ #:network? network?+ #:map-cwd? (not no-cwd?)))) - (else- (return- (exit/status- (launch-environment/fork command profile manifest- #:white-list white-list- #:pure? pure?)))))))))))))))+ (else+ (return+ (exit/status+ (launch-environment/fork command profile manifest+ #:white-list white-list+ #:pure? pure?))))))))))))))++;;; Local Variables:+;;; (put 'with-store/maybe 'scheme-indent-function 1)+;;; End:-- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 08/10] environment: Autoload some modules.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211018195219.13898-9-ludo@gnu.org
This further speeds up the 'guix environment -p PROFILE' case.
* guix/scripts/environment.scm: Autoload a bunch of modules.--- guix/scripts/environment.scm | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-)
Toggle diff (33 lines)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex e23d52df39..05a43659da 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -34,15 +34,18 @@ (define-module (guix scripts environment) #:use-module (guix scripts) #:use-module (guix scripts build) #:use-module (guix transformations)- #:use-module (gnu build linux-container)- #:use-module (gnu build accounts)- #:use-module ((guix build syscalls) #:select (set-network-interface-up))- #:use-module (gnu system linux-container)+ #:autoload (gnu build linux-container) (call-with-container %namespaces+ user-namespace-supported?+ unprivileged-user-namespace-supported?+ setgroups-supported?)+ #:autoload (gnu build accounts) (password-entry group-entry+ password-entry-name password-entry-directory+ write-passwd write-group)+ #:autoload (guix build syscalls) (set-network-interface-up) #:use-module (gnu system file-systems)- #:use-module (gnu packages)- #:use-module (gnu packages bash)- #:use-module ((gnu packages bootstrap)- #:select (bootstrap-executable %bootstrap-guile))+ #:autoload (gnu packages) (specification->package+output)+ #:autoload (gnu packages bash) (bash)+ #:autoload (gnu packages bootstrap) (bootstrap-executable %bootstrap-guile) #:use-module (ice-9 match) #:use-module (srfi srfi-1) #:use-module (srfi srfi-11)-- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 09/10] cache: Gracefully handle non-existent cache.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211018195219.13898-10-ludo@gnu.org
* guix/cache.scm (maybe-remove-expired-cache-entries): Ignore ENOENTwhen writing EXPIRY-FILE.--- guix/cache.scm | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
Toggle diff (22 lines)diff --git a/guix/cache.scm b/guix/cache.scmindex 0401a9d428..51009809bd 100644--- a/guix/cache.scm+++ b/guix/cache.scm@@ -101,7 +101,13 @@ (define last-expiry-date #:now now #:entry-expiration entry-expiration #:delete-entry delete-entry)- (call-with-output-file expiry-file- (cute write (time-second now) <>))))+ (catch 'system-error+ (lambda ()+ (call-with-output-file expiry-file+ (cute write (time-second now) <>)))+ (lambda args+ ;; ENOENT means CACHE does not exist.+ (unless (= ENOENT (system-error-errno args))+ (apply throw args)))))) ;;; cache.scm ends here-- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 02/10] profiles: Add 'package->development-manifest'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20211018195219.13898-3-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>
* guix/profiles.scm (package->development-manifest): New procedure.* guix/scripts/environment.scm (input->manifest-entry)(package-environment-inputs): Remove.* guix/scripts/environment.scm (options/resolve-packages): Use'package->development-manifest' instead of 'package-environment-inputs'.* tests/profiles.scm ("package->development-manifest"): New test.--- doc/guix.texi | 11 +++++++++++ guix/profiles.scm | 19 +++++++++++++++++++ guix/scripts/environment.scm | 27 +++++---------------------- tests/profiles.scm | 7 +++++++ 4 files changed, 42 insertions(+), 22 deletions(-)
Toggle diff (134 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex deb4eac4ac..c1045b331b 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -3341,6 +3341,17 @@ objects, like this: '("emacs" "guile@@2.2" "guile@@2.2:debug")) @end lisp +@findex package->development-manifest+You might also want to create a manifest for all the dependencies of a+package, rather than the package itself:++@lisp+(package->development-manifest (specification->package "emacs"))+@end lisp++The example above gives you all the software required to develop Emacs,+similar to what @command{guix environment emacs} provides.+ @xref{export-manifest, @option{--export-manifest}}, to learn how to obtain a manifest file from an existing profile. diff --git a/guix/profiles.scm b/guix/profiles.scmindex 2486f91d09..9f30349c69 100644--- a/guix/profiles.scm+++ b/guix/profiles.scm@@ -124,6 +124,7 @@ (define-module (guix profiles) profile-manifest package->manifest-entry+ package->development-manifest packages->manifest ca-certificate-bundle %default-profile-hooks@@ -400,6 +401,24 @@ (define* (package->manifest-entry package #:optional (output "out") (properties properties)))) entry)) +(define* (package->development-manifest package+ #:optional+ (system (%current-system))+ #:key target)+ "Return a manifest for the \"development inputs\" of PACKAGE for SYSTEM,+optionally when cross-compiling to TARGET. Development inputs include both+explicit and implicit inputs of PACKAGE."+ (manifest+ (filter-map (match-lambda+ ((label (? package? package))+ (package->manifest-entry package))+ ((label (? package? package) output)+ (package->manifest-entry package output))+ ;; TODO: Support <inferior-package>.+ (_+ #f))+ (package-development-inputs package system #:target target))))+ (define (packages->manifest packages) "Return a list of manifest entries, one for each item listed in PACKAGES. Elements of PACKAGES can be either package objects or package/string tuplesdiff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 418f11c37e..54f48a7482 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -66,24 +66,6 @@ (define* (show-search-paths profile manifest #:key pure?) (newline))) (profile-search-paths profile manifest))) -(define (input->manifest-entry input)- "Return a manifest entry for INPUT, or #f if INPUT does not correspond to a-package."- (match input- ((_ (? package? package))- (package->manifest-entry package))- ((_ (? package? package) output)- (package->manifest-entry package output))- (_- #f)))--(define (package-environment-inputs package)- "Return a list of manifest entries corresponding to the transitive input-packages for PACKAGE."- ;; Remove non-package inputs such as origin records.- (filter-map input->manifest-entry- (package-development-inputs package)))- (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] Build an environment that includes the dependencies of PACKAGE and execute@@ -297,11 +279,11 @@ (define (packages->outputs packages mode) ((? package? package) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package))- (package-environment-inputs package)))+ (manifest-entries (package->development-manifest package)))) (((? package? package) (? string? output)) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package output))- (package-environment-inputs package)))+ (manifest-entries (package->development-manifest package)))) ((lst ...) (append-map (cut packages->outputs <> mode) lst)))) @@ -313,8 +295,9 @@ (define (packages->outputs packages mode) (specification->package+output spec))) (list (package->manifest-entry* package output)))) (('package 'package (? string? spec))- (package-environment-inputs- (transform (specification->package+output spec))))+ (manifest-entries+ (package->development-manifest+ (transform (specification->package+output spec))))) (('expression mode str) ;; Add all the outputs of the package STR evaluates to. (packages->outputs (read/eval str) mode))diff --git a/tests/profiles.scm b/tests/profiles.scmindex 06a0387221..cac5b73347 100644--- a/tests/profiles.scm+++ b/tests/profiles.scm@@ -265,6 +265,13 @@ (define transform1 (manifest-transaction-removal-candidate? guile-2.0.9 t) (null? install) (null? downgrade) (null? upgrade))))) +(test-assert "package->development-manifest"+ (let ((manifest (package->development-manifest packages:hello)))+ (every (lambda (name)+ (manifest-installed? manifest+ (manifest-pattern (name name))))+ '("gcc" "binutils" "glibc" "coreutils" "grep" "sed"))))+ (test-assertm "profile-derivation" (mlet* %store-monad ((entry -> (package->manifest-entry %bootstrap-guile))-- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 04/10] shell: By default load the local 'guix.scm' or 'manifest.scm' file.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211018195219.13898-5-ludo@gnu.org
* guix/scripts/shell.scm (parse-args): Add call to 'auto-detect-manifest'.(authorized-directory-file, authorized-shell-directory?)(find-file-in-parent-directories, auto-detect-manifest): New procedures.* tests/guix-shell.sh: Add test.* doc/guix.texi (Invoking guix shell): Document it.--- doc/guix.texi | 14 +++++ guix/scripts/shell.scm | 119 +++++++++++++++++++++++++++++++++++++++-- tests/guix-shell.sh | 50 ++++++++++++++++- 3 files changed, 176 insertions(+), 7 deletions(-)
Toggle diff (260 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 63612728ed..7c8f0c1f9b 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -5621,6 +5621,20 @@ before @command{guix shell} was invoked. The next garbage collection (@pxref{Invoking guix gc}) may clean up packages that were installed in the environment and that are no longer used outside of it. +As an added convenience, when running from a directory that contains a+@file{guix.scm} or a @file{manifest.scm} file, possibly in a parent+directory, @command{guix shell} automatically loads the file---provided+the directory is listed in+@file{~/.config/guix/shell-authorized-directories}, and only for+interactive use:++@example+guix shell+@end example++This provides an easy way to define, share, and enter development+environments.+ By default, the shell session or command runs in an @emph{augmented} environment, where the new packages are added to search path environment variables such as @code{PATH}. You can, instead, choose to create andiff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmindex 190dd8837d..45fd536145 100644--- a/guix/scripts/shell.scm+++ b/guix/scripts/shell.scm@@ -18,15 +18,20 @@ (define-module (guix scripts shell) #:use-module (guix ui)+ #:use-module ((guix diagnostics) #:select (location)) #:use-module (guix scripts environment) #:autoload (guix scripts build) (show-build-options-help) #:autoload (guix transformations) (show-transformation-options-help) #:use-module (guix scripts)+ #:use-module (guix packages)+ #:use-module (guix profiles) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) #:use-module (srfi srfi-37) #:use-module (srfi srfi-71) #:use-module (ice-9 match)+ #:autoload (ice-9 rdelim) (read-line)+ #:autoload (guix utils) (config-directory) #:export (guix-shell)) (define (show-help)@@ -41,6 +46,8 @@ (define (show-help) (display (G_ " -f, --file=FILE create environment for the package that the code within FILE evaluates to"))+ (display (G_ "+ -q inhibit loading of 'guix.scm' and 'manifest.scm'")) (show-environment-options-help) (newline)@@ -99,7 +106,10 @@ (define %options (option '(#\f "file") #t #f (lambda (opt name arg result) (alist-cons 'load (tag-package-arg result arg)- result))))+ result)))+ (option '(#\q) #f #f+ (lambda (opt name arg result)+ (alist-cons 'explicit-loading? #t result)))) (filter-map (lambda (opt) (and (not (any (lambda (name) (member name to-remove))@@ -122,10 +132,109 @@ (define (handle-argument arg result) (let ((args command (break (cut string=? "--" <>) args))) (let ((opts (parse-command-line args %options (list %default-options) #:argument-handler handle-argument)))- (match command- (() opts)- (("--") opts)- (("--" command ...) (alist-cons 'exec command opts))))))+ (auto-detect-manifest+ (match command+ (() opts)+ (("--") opts)+ (("--" command ...) (alist-cons 'exec command opts)))))))++(define (find-file-in-parent-directories candidates)+ "Find one of CANDIDATES in the current directory or one of its ancestors."+ (define start (getcwd))+ (define device (stat:dev (stat start)))++ (let loop ((directory start))+ (let ((stat (stat directory)))+ (and (= (stat:uid stat) (getuid))+ (= (stat:dev stat) device)+ (or (any (lambda (candidate)+ (let ((candidate (string-append directory "/" candidate)))+ (and (file-exists? candidate) candidate)))+ candidates)+ (and (not (string=? directory "/"))+ (loop (dirname directory)))))))) ;lexical ".." resolution++(define (authorized-directory-file)+ "Return the name of the file listing directories for which 'guix shell' may+automatically load 'guix.scm' or 'manifest.scm' files."+ (string-append (config-directory) "/shell-authorized-directories"))++(define (authorized-shell-directory? directory)+ "Return true if DIRECTORY is among the authorized directories for automatic+loading. The list of authorized directories is read from+'authorized-directory-file'; each line must be either: an absolute file name,+a hash-prefixed comment, or a blank line."+ (catch 'system-error+ (lambda ()+ (call-with-input-file (authorized-directory-file)+ (lambda (port)+ (let loop ()+ (match (read-line port)+ ((? eof-object?) #f)+ ((= string-trim line)+ (cond ((string-prefix? "#" line) ;comment+ (loop))+ ((string-prefix? "/" line) ;absolute file name+ (or (string=? line directory)+ (loop)))+ ((string-null? (string-trim-right line)) ;blank line+ (loop))+ (else ;bogus line+ (let ((loc (location (port-filename port)+ (port-line port)+ (port-column port))))+ (warning loc (G_ "ignoring invalid file name: '~a'~%")+ line))))))))))+ (const #f)))++(define (auto-detect-manifest opts)+ "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or+\"manifest.scm\" file from the current directory or one of its ancestors.+Return the modified OPTS."+ (define (options-contain-payload? opts)+ (match opts+ (() #f)+ ((('package . _) . _) #t)+ ((('load . _) . _) #t)+ ((('manifest . _) . _) #t)+ ((('expression . _) . _) #t)+ ((_ . rest) (options-contain-payload? rest))))++ (define interactive?+ (not (assoc-ref opts 'exec)))++ (define disallow-implicit-load?+ (assoc-ref opts 'explicit-loading?))++ (if (or (not interactive?)+ disallow-implicit-load?+ (options-contain-payload? opts))+ opts+ (match (find-file-in-parent-directories '("guix.scm" "manifest.scm"))+ (#f+ (warning (G_ "no packages specified; creating an empty environment~%"))+ opts)+ (file+ (if (authorized-shell-directory? (dirname file))+ (begin+ (info (G_ "loading environment from '~a'...~%") file)+ (match (basename file)+ ("guix.scm" (alist-cons 'load `(package ,file) opts))+ ("manifest.scm" (alist-cons 'manifest file opts))))+ (begin+ (warning (G_ "not loading '~a' because not authorized to do so~%")+ file)+ (display-hint (format #f (G_ "To allow automatic loading of+@file{~a} when running @command{guix shell}, you must explicitly authorize its+directory, like so:++@example+echo ~a >> ~a+@end example\n")+ file+ (dirname file)+ (authorized-directory-file)))+ opts)))))) (define-command (guix-shell . args)diff --git a/tests/guix-shell.sh b/tests/guix-shell.shindex f08637f7ff..95725cba2d 100644--- a/tests/guix-shell.sh+++ b/tests/guix-shell.sh@@ -22,15 +22,55 @@ guix shell --version +configdir="t-guix-shell-config-$$" tmpdir="t-guix-shell-$$"-trap 'rm -r "$tmpdir"' EXIT-mkdir "$tmpdir"+trap 'rm -r "$tmpdir" "$configdir"' EXIT+mkdir "$tmpdir" "$configdir" "$configdir/guix"++XDG_CONFIG_HOME="$(realpath $configdir)"+export XDG_CONFIG_HOME guix shell --bootstrap --pure guile-bootstrap -- guile --version # '--ad-hoc' is a thing of the past. ! guix shell --ad-hoc guile-bootstrap +# Ignoring unauthorized files.+cat > "$tmpdir/guix.scm" <<EOF+This is a broken guix.scm file.+EOF+(cd "$tmpdir"; SHELL="$(type -P true)" guix shell --bootstrap)++# Authorize the directory.+echo "$(realpath "$tmpdir")" > "$configdir/guix/shell-authorized-directories"++# Ignoring 'manifest.scm' and 'guix.scm' in non-interactive use.+(cd "$tmpdir"; guix shell --bootstrap -- true)+mv "$tmpdir/guix.scm" "$tmpdir/manifest.scm"+(cd "$tmpdir"; guix shell --bootstrap -- true)+rm "$tmpdir/manifest.scm"++# Honoring the local 'manifest.scm' file.+cat > "$tmpdir/manifest.scm" <<EOF+(specifications->manifest '("guile-bootstrap"))+EOF+cat > "$tmpdir/fake-shell.sh" <<EOF+#!$SHELL+# This fake shell allows us to test interactive use.+exec echo "\$GUIX_ENVIRONMENT"+EOF+chmod +x "$tmpdir/fake-shell.sh"+profile1="$(cd "$tmpdir"; SHELL="$(realpath fake-shell.sh)" guix shell --bootstrap)"+profile2="$(guix shell --bootstrap guile-bootstrap -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT')"+test -n "$profile1"+test "$profile1" = "$profile2"+rm "$tmpdir/manifest.scm"++# Do not read manifest when passed '-q'.+echo "Broken manifest." > "$tmpdir/manifest.scm"+(cd "$tmpdir"; SHELL="$(realpath fake-shell.sh)" guix shell --bootstrap -q)+rm "$tmpdir/manifest.scm"+ if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null then # Compute the build environment for the initial GNU Make.@@ -51,4 +91,10 @@ then # 'make-boot0' itself must not be listed. ! guix gc --references "$profile" | grep make-boot0++ # Honoring the local 'guix.scm' file.+ echo '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/guix.scm"+ (cd "$tmpdir"; guix shell --bootstrap --search-paths --pure > "b")+ cmp "$tmpdir/a" "$tmpdir/b"+ rm "$tmpdir/guix.scm" fi-- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 10/10] shell: Maintain a profile cache.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20211018195219.13898-11-ludo@gnu.org
shell: Maintain a profile cache.
With this change, running "guix shell" (no arguments) is equivalent to:
guix environment -r ~/.cache/guix/profiles/some-root -l guix.scm
This is the cache miss. On cache hit, it's equivalent to:
guix environment -p ~/.cache/guix/profiles/some-root
... which can run in 0.1s.
* guix/scripts/shell.scm (options-with-caching): New procedure.(parse-args): Use it.(%profile-cache-directory): New variable.(profile-cache-key, profile-cached-gc-root): New procedures.(show-help, %options): Add '--rebuild-cache'.(guix-shell)[cache-entries, entry-expiration]: New procedures.Add call to 'maybe-remove-expired-cache-entries'.* doc/guix.texi (Invoking guix shell): Document '--rebuild-cache'.--- doc/guix.texi | 11 ++++ guix/scripts/shell.scm | 127 ++++++++++++++++++++++++++++++++++++++--- 2 files changed, 130 insertions(+), 8 deletions(-)
Toggle diff (199 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 7c8f0c1f9b..dabd7fea1e 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -5769,6 +5769,17 @@ This is similar to the same-named option in @command{guix package} (@pxref{profile-manifest, @option{--manifest}}) and uses the same manifest files. +@item --rebuild-cache+When using @option{--manifest}, @option{--file}, or when invoked without+arguments, @command{guix shell} caches the environment so that+subsequent uses are instantaneous. The cache is invalidated anytime the+file is modified.++The @option{--rebuild-cache} forces the cached environment to be+refreshed even if the file has not changed. This is useful if the+@command{guix.scm} or @command{manifest.scm} has external dependencies,+or if its behavior depends, say, on environment variables.+ @item --pure Unset existing environment variables when building the new environment, except those specified with @option{--preserve} (see below). This has the effect ofdiff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmindex 45fd536145..4062e8155d 100644--- a/guix/scripts/shell.scm+++ b/guix/scripts/shell.scm@@ -31,7 +31,15 @@ (define-module (guix scripts shell) #:use-module (srfi srfi-71) #:use-module (ice-9 match) #:autoload (ice-9 rdelim) (read-line)- #:autoload (guix utils) (config-directory)+ #:autoload (guix base32) (bytevector->base32-string)+ #:autoload (rnrs bytevectors) (string->utf8)+ #:autoload (guix utils) (config-directory cache-directory)+ #:autoload (guix describe) (current-channels)+ #:autoload (guix channels) (channel-commit)+ #:autoload (gcrypt hash) (sha256)+ #:use-module ((guix build utils) #:select (mkdir-p))+ #:use-module (guix cache)+ #:use-module ((ice-9 ftw) #:select (scandir)) #:export (guix-shell)) (define (show-help)@@ -48,6 +56,8 @@ (define (show-help) FILE evaluates to")) (display (G_ " -q inhibit loading of 'guix.scm' and 'manifest.scm'"))+ (display (G_ "+ --rebuild-cache rebuild cached environment, if any")) (show-environment-options-help) (newline)@@ -109,7 +119,10 @@ (define %options result))) (option '(#\q) #f #f (lambda (opt name arg result)- (alist-cons 'explicit-loading? #t result))))+ (alist-cons 'explicit-loading? #t result)))+ (option '("rebuild-cache") #f #f+ (lambda (opt name arg result)+ (alist-cons 'rebuild-cache? #t result)))) (filter-map (lambda (opt) (and (not (any (lambda (name) (member name to-remove))@@ -132,11 +145,12 @@ (define (handle-argument arg result) (let ((args command (break (cut string=? "--" <>) args))) (let ((opts (parse-command-line args %options (list %default-options) #:argument-handler handle-argument)))- (auto-detect-manifest- (match command- (() opts)- (("--") opts)- (("--" command ...) (alist-cons 'exec command opts)))))))+ (options-with-caching+ (auto-detect-manifest+ (match command+ (() opts)+ (("--") opts)+ (("--" command ...) (alist-cons 'exec command opts)))))))) (define (find-file-in-parent-directories candidates) "Find one of CANDIDATES in the current directory or one of its ancestors."@@ -187,6 +201,53 @@ (define (authorized-shell-directory? directory) line)))))))))) (const #f))) +(define (options-with-caching opts)+ "If OPTS contains exactly one 'load' or one 'manifest' key, automatically+add a 'profile' key (when a profile for that file is already in cache) or a+'gc-root' key (to add the profile to cache)."+ (define (single-file-for-caching opts)+ (let loop ((opts opts)+ (file #f))+ (match opts+ (() file)+ ((('package . _) . _) #f)+ ((('load . ('package candidate)) . rest)+ (and (not file) (loop rest candidate)))+ ((('manifest . candidate) . rest)+ (and (not file) (loop rest candidate)))+ ((('expression . _) . _) #f)+ ((_ . rest) (loop rest file)))))++ ;; Check whether there's a single 'load' or 'manifest' option. When that is+ ;; the case, arrange to automatically cache the resulting profile.+ (match (single-file-for-caching opts)+ (#f opts)+ (file+ (let* ((root (profile-cached-gc-root file))+ (stat (and root (false-if-exception (lstat root)))))+ (if (and (not (assoc-ref opts 'rebuild-cache?))+ stat+ (<= (stat:mtime ((@ (guile) stat) file))+ (stat:mtime stat)))+ (let ((now (current-time)))+ ;; Update the atime on ROOT to reflect usage.+ (utime root+ now (stat:mtime stat) 0 (stat:mtimensec stat)+ AT_SYMLINK_NOFOLLOW)+ (alist-cons 'profile root+ (remove (match-lambda+ (('load . _) #t)+ (('manifest . _) #t)+ (_ #f))+ opts))) ;load right away+ (if (and root (not (assq-ref opts 'gc-root)))+ (begin+ (if stat+ (delete-file root)+ (mkdir-p (dirname root)))+ (alist-cons 'gc-root root opts))+ opts))))))+ (define (auto-detect-manifest opts) "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or \"manifest.scm\" file from the current directory or one of its ancestors.@@ -236,9 +297,59 @@ (define disallow-implicit-load? (authorized-directory-file))) opts)))))) + +;;;+;;; Profile cache.+;;;++(define %profile-cache-directory+ ;; Directory where profiles created by 'guix shell' alone (without extra+ ;; options) are cached.+ (make-parameter (string-append (cache-directory #:ensure? #f)+ "/profiles")))++(define (profile-cache-key file)+ "Return the cache key for the profile corresponding to FILE, a 'guix.scm' or+'manifest.scm' file, or #f if we lack channel information."+ (match (current-channels)+ (() #f)+ (((= channel-commit commits) ...)+ (let ((stat (stat file)))+ (bytevector->base32-string+ ;; Since FILE is not canonicalized, only include the device/inode+ ;; numbers. XXX: In some rare cases involving Btrfs and NFS, this can+ ;; be insufficient: <https://lwn.net/Articles/866582/>.+ (sha256 (string->utf8+ (string-append (string-join commits) ":"+ (number->string (stat:dev stat)) ":"+ (number->string (stat:ino stat))))))))))++(define (profile-cached-gc-root file)+ "Return the cached GC root for FILE, a 'guix.scm' or 'manifest.scm' file, or+#f if we lack information to cache it."+ (match (profile-cache-key file)+ (#f #f)+ (key (string-append (%profile-cache-directory) "/" key))))+ (define-command (guix-shell . args) (category development) (synopsis "spawn one-off software environments") - (guix-environment* (parse-args args)))+ (define (cache-entries directory)+ (filter-map (match-lambda+ ((or "." "..") #f)+ (file (string-append directory "/" file)))+ (or (scandir directory) '())))++ (define* (entry-expiration file)+ ;; Return the time at which FILE, a cached profile, is considered expired.+ (match (false-if-exception (lstat file))+ (#f 0) ;FILE may have been deleted in the meantime+ (st (+ (stat:atime st) (* 60 60 24 7)))))++ (let ((result (guix-environment* (parse-args args))))+ (maybe-remove-expired-cache-entries (%profile-cache-directory)+ cache-entries+ #:entry-expiration entry-expiration)+ result))-- 2.33.0
L
L
Ludovic Courtès wrote 4 days ago
[PATCH v3 03/10] Add 'guix shell'.
(address . 50960@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20211018195219.13898-4-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>
* guix/scripts/shell.scm, tests/guix-shell.sh: New files.* Makefile.am (MODULES): Add 'shell.scm'.(SH_TESTS): Add 'tests/guix-shell.sh'.* guix/scripts/environment.scm (show-environment-options-help): Newprocedure.(show-help): Use it.(guix-environment*): New procedure.(guix-environment): Use it.* po/guix/POTFILES.in: Add it.* doc/guix.texi (Features): Refer to "guix shell"(Invoking guix package): Likewise.(Development): Likewise.(Invoking guix shell): New node.(Invoking guix environment): Add deprecation warning.(Debugging Build Failures): Use 'guix shell' in examples.(Invoking guix container): Refer to 'guix shell'.(Invoking guix processes, Virtualization Services): Adjust examples touse 'guix shell'.* doc/contributing.texi (Building from Git): Refer to 'guix shell'.--- Makefile.am | 2 + doc/contributing.texi | 8 +- doc/guix.texi | 366 ++++++++++++++++++++++++++++++++--- guix/scripts/environment.scm | 52 +++-- guix/scripts/shell.scm | 135 +++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-shell.sh | 54 ++++++ 7 files changed, 575 insertions(+), 43 deletions(-) create mode 100644 guix/scripts/shell.scm create mode 100644 tests/guix-shell.sh
Toggle diff (826 lines)diff --git a/Makefile.am b/Makefile.amindex 41ec19eb89..239387c2f4 100644--- a/Makefile.am+++ b/Makefile.am@@ -316,6 +316,7 @@ MODULES = \ guix/scripts/import/stackage.scm \ guix/scripts/import/texlive.scm \ guix/scripts/environment.scm \+ guix/scripts/shell.scm \ guix/scripts/publish.scm \ guix/scripts/edit.scm \ guix/scripts/size.scm \@@ -552,6 +553,7 @@ SH_TESTS = \ tests/guix-authenticate.sh \ tests/guix-environment.sh \ tests/guix-environment-container.sh \+ tests/guix-shell.sh \ tests/guix-graph.sh \ tests/guix-describe.sh \ tests/guix-repl.sh \diff --git a/doc/contributing.texi b/doc/contributing.texiindex 76ab913b0d..db0f836157 100644--- a/doc/contributing.texi+++ b/doc/contributing.texi@@ -73,10 +73,10 @@ all the dependencies and appropriate environment variables are set up to hack on Guix: @example-guix environment guix --pure+guix shell -D guix --pure @end example -@xref{Invoking guix environment}, for more information on that command.+@xref{Invoking guix shell}, for more information on that command. If you are unable to use Guix when building Guix from a checkout, the following are the required packages in addition to those mentioned in the@@ -92,10 +92,10 @@ installation instructions (@pxref{Requirements}). @end itemize On Guix, extra dependencies can be added by instead running @command{guix-environment} with @option{--ad-hoc}:+shell}: @example-guix environment guix --pure --ad-hoc help2man git strace+guix shell -D guix help2man git strace --pure @end example Run @command{./bootstrap} to generate the build system infrastructurediff --git a/doc/guix.texi b/doc/guix.texiindex c1045b331b..63612728ed 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -119,6 +119,7 @@ Documentation License''. @dircategory Software development @direntry+* guix shell: (guix)Invoking guix shell. Creating software environments. * guix environment: (guix)Invoking guix environment. Building development environments with Guix. * guix build: (guix)Invoking guix build. Building packages. * guix pack: (guix)Invoking guix pack. Creating binary bundles.@@ -262,6 +263,7 @@ Channels Development +* Invoking guix shell:: Spawning one-off software environments. * Invoking guix environment:: Setting up development environments. * Invoking guix pack:: Creating software bundles. * The GCC toolchain:: Working with languages supported by GCC.@@ -3067,10 +3069,10 @@ substitutes: they can force a local build and @emph{challenge} providers (@pxref{Invoking guix challenge}). Control over the build environment is a feature that is also useful for-developers. The @command{guix environment} command allows developers of+developers. The @command{guix shell} command allows developers of a package to quickly set up the right development environment for their package, without having to manually install the dependencies of the-package into their profile (@pxref{Invoking guix environment}).+package into their profile (@pxref{Invoking guix shell}). @cindex replication, of software environments @cindex provenance tracking, of software artifacts@@ -3234,7 +3236,7 @@ As an example, @var{file} might contain a definition like this Developers may find it useful to include such a @file{guix.scm} file in the root of their project source tree that can be used to test development snapshots and create reproducible development environments-(@pxref{Invoking guix environment}).+(@pxref{Invoking guix shell}). The @var{file} may also contain a JSON representation of one or more package definitions. Running @code{guix package -f} on@@ -5559,31 +5561,352 @@ If you are a software developer, Guix provides tools that you should find helpful---independently of the language you're developing in. This is what this chapter is about. -The @command{guix environment} command provides a convenient way to set up-@dfn{development environments} containing all the dependencies and tools-necessary to work on the software package of your choice. The @command{guix+The @command{guix shell} command provides a convenient way to set up+one-off software environments, be it for development purposes or to run+a command without installing it in your profile. The @command{guix pack} command allows you to create @dfn{application bundles} that can be easily distributed to users who do not run Guix. @menu+* Invoking guix shell:: Spawning one-off software environments. * Invoking guix environment:: Setting up development environments. * Invoking guix pack:: Creating software bundles. * The GCC toolchain:: Working with languages supported by GCC. * Invoking guix git authenticate:: Authenticating Git repositories. @end menu -@node Invoking guix environment-@section Invoking @command{guix environment}+@node Invoking guix shell+@section Invoking @command{guix shell} @cindex reproducible build environments @cindex development environments @cindex @command{guix environment} @cindex environment, package build environment-The purpose of @command{guix environment} is to assist hackers in-creating reproducible development environments without polluting their-package profile. The @command{guix environment} tool takes one or more-packages, builds all of their inputs, and creates a shell-environment to use them.+The purpose of @command{guix shell} is to make it easy to create one-off+software environments, without changing one's profile. It is typically+used to create development environments; it is also a convenient way to+run applications without ``polluting'' your profile.++@quotation Note+The @command{guix shell} command was recently introduced to supersede+@command{guix environment} (@pxref{Invoking guix environment}). If you+are familiar with @command{guix environment}, you will notice that it is+similar but also---we hope!---more convenient.+@end quotation++The general syntax is:++@example+guix shell [@var{options}] [@var{package}@dots{}]+@end example++The following example creates an environment containing Python and NumPy,+building or downloading any missing package, and runs the+@command{python3} command in that environment:++@example+guix shell python python-numpy -- python3+@end example++Development environments can be created as in the example below, which+spawns an interactive shell containing all the dependencies and+environment variables needed to work on Inkscape:++@example+guix shell --development inkscape+@end example++Exiting the shell places the user back in the original environment+before @command{guix shell} was invoked. The next garbage collection+(@pxref{Invoking guix gc}) may clean up packages that were installed in+the environment and that are no longer used outside of it.++By default, the shell session or command runs in an @emph{augmented}+environment, where the new packages are added to search path environment+variables such as @code{PATH}. You can, instead, choose to create an+@emph{isolated} environment containing nothing but the packages you+asked for. Passing the @option{--pure} option clears environment+variable definitions found in the parent environment@footnote{Users+sometimes wrongfully augment environment variables such as @env{PATH} in+their @file{~/.bashrc} file. As a consequence, when @command{guix+environment} launches it, Bash may read @file{~/.bashrc}, thereby+introducing ``impurities'' in these environment variables. It is an+error to define such environment variables in @file{.bashrc}; instead,+they should be defined in @file{.bash_profile}, which is sourced only by+log-in shells. @xref{Bash Startup Files,,, bash, The GNU Bash Reference+Manual}, for details on Bash start-up files.}; passing+@option{--container} goes one step further by spawning a @dfn{container}+isolated from the rest of the system:++@example+guix shell --container emacs gcc-toolchain+@end example++The command above spawns an interactive shell in a container when+nothing but @code{emacs}, @code{gcc-toolchain}, and their dependencies+is available. The container lacks network access and shares no files+other than the current working directory with the surrounding+environment. This is useful to prevent access to system-wide resources+such as @file{/usr/bin} on foreign distros.++This @option{--container} option can also prove useful if you wish to+run a security-sensitive application, such as a web browser, in an+isolated environment. For example, the command below launches+Ungoogled-Chromium in an isolated environment, this time sharing network+access with the host and preserving its @code{DISPLAY} environment+variable, but without even sharing the current directory:++@example+guix shell --container --network --no-cwd ungoogled-chromium \+ --preserve='^DISPLAY$' -- chromium+@end example++@vindex GUIX_ENVIRONMENT+@command{guix shell} defines the @env{GUIX_ENVIRONMENT}+variable in the shell it spawns; its value is the file name of the+profile of this environment. This allows users to, say, define a+specific prompt for development environments in their @file{.bashrc}+(@pxref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}):++@example+if [ -n "$GUIX_ENVIRONMENT" ]+then+ export PS1="\u@@\h \w [dev]\$ "+fi+@end example++@noindent+...@: or to browse the profile:++@example+$ ls "$GUIX_ENVIRONMENT/bin"+@end example++The available options are summarized below.++@table @code+@item --development+@itemx -D+Cause @command{guix shell} to include in the environment the+dependencies of the following package rather than the package itself.+This can be combined with other packages. For instance, the command+below starts an interactive shell containing the build-time dependencies+of GNU@tie{}Guile, plus Autoconf, Automake, and Libtool:++@example+guix shell -D guile autoconf automake libtool+@end example++@item --expression=@var{expr}+@itemx -e @var{expr}+Create an environment for the package or list of packages that+@var{expr} evaluates to.++For example, running:++@example+guix shell -D -e '(@@ (gnu packages maths) petsc-openmpi)'+@end example++starts a shell with the environment for this specific variant of the+PETSc package.++Running:++@example+guix shell -e '(@@ (gnu) %base-packages)'+@end example++starts a shell with all the base system packages available.++The above commands only use the default output of the given packages.+To select other outputs, two element tuples can be specified:++@example+guix shell -e '(list (@@ (gnu packages bash) bash) "include")'+@end example++@item --file=@var{file}+@itemx -f @var{file}+Create an environment containing the package or list of packages that+the code within @var{file} evaluates to.++As an example, @var{file} might contain a definition like this+(@pxref{Defining Packages}):++@lisp+@verbatiminclude environment-gdb.scm+@end lisp++With the file above, you can enter a development environment for GDB by+running:++@example+guix shell -D -f gdb-devel.scm+@end example++@item --manifest=@var{file}+@itemx -m @var{file}+Create an environment for the packages contained in the manifest object+returned by the Scheme code in @var{file}. This option can be repeated+several times, in which case the manifests are concatenated.++This is similar to the same-named option in @command{guix package}+(@pxref{profile-manifest, @option{--manifest}}) and uses the same+manifest files.++@item --pure+Unset existing environment variables when building the new environment, except+those specified with @option{--preserve} (see below). This has the effect of+creating an environment in which search paths only contain package inputs.++@item --preserve=@var{regexp}+@itemx -E @var{regexp}+When used alongside @option{--pure}, preserve the environment variables+matching @var{regexp}---in other words, put them on a ``white list'' of+environment variables that must be preserved. This option can be repeated+several times.++@example+guix shell --pure --preserve=^SLURM openmpi @dots{} \+ -- mpirun @dots{}+@end example++This example runs @command{mpirun} in a context where the only environment+variables defined are @env{PATH}, environment variables whose name starts+with @samp{SLURM}, as well as the usual ``precious'' variables (@env{HOME},+@env{USER}, etc.).++@item --search-paths+Display the environment variable definitions that make up the+environment.++@item --system=@var{system}+@itemx -s @var{system}+Attempt to build for @var{system}---e.g., @code{i686-linux}.++@item --container+@itemx -C+@cindex container+Run @var{command} within an isolated container. The current working+directory outside the container is mapped inside the container.+Additionally, unless overridden with @option{--user}, a dummy home+directory is created that matches the current user's home directory, and+@file{/etc/passwd} is configured accordingly.++The spawned process runs as the current user outside the container. Inside+the container, it has the same UID and GID as the current user, unless+@option{--user} is passed (see below).++@item --network+@itemx -N+For containers, share the network namespace with the host system.+Containers created without this flag only have access to the loopback+device.++@item --link-profile+@itemx -P+For containers, link the environment profile to @file{~/.guix-profile}+within the container and set @code{GUIX_ENVIRONMENT} to that.+This is equivalent to making @file{~/.guix-profile} a symlink to the+actual profile within the container.+Linking will fail and abort the environment if the directory already+exists, which will certainly be the case if @command{guix shell}+was invoked in the user's home directory.++Certain packages are configured to look in @file{~/.guix-profile} for+configuration files and data;@footnote{For example, the+@code{fontconfig} package inspects @file{~/.guix-profile/share/fonts}+for additional fonts.} @option{--link-profile} allows these programs to+behave as expected within the environment.++@item --user=@var{user}+@itemx -u @var{user}+For containers, use the username @var{user} in place of the current+user. The generated @file{/etc/passwd} entry within the container will+contain the name @var{user}, the home directory will be+@file{/home/@var{user}}, and no user GECOS data will be copied. Furthermore,+the UID and GID inside the container are 1000. @var{user}+need not exist on the system.++Additionally, any shared or exposed path (see @option{--share} and+@option{--expose} respectively) whose target is within the current user's+home directory will be remapped relative to @file{/home/USER}; this+includes the automatic mapping of the current working directory.++@example+# will expose paths as /home/foo/wd, /home/foo/test, and /home/foo/target+cd $HOME/wd+guix shell --container --user=foo \+ --expose=$HOME/test \+ --expose=/tmp/target=$HOME/target+@end example++While this will limit the leaking of user identity through home paths+and each of the user fields, this is only one useful component of a+broader privacy/anonymity solution---not one in and of itself.++@item --no-cwd+For containers, the default behavior is to share the current working+directory with the isolated container and immediately change to that+directory within the container. If this is undesirable,+@option{--no-cwd} will cause the current working directory to @emph{not}+be automatically shared and will change to the user's home directory+within the container instead. See also @option{--user}.++@item --expose=@var{source}[=@var{target}]+@itemx --share=@var{source}[=@var{target}]+For containers, @option{--expose} (resp. @option{--share}) exposes the+file system @var{source} from the host system as the read-only+(resp. writable) file system @var{target} within the container. If+@var{target} is not specified, @var{source} is used as the target mount+point in the container.++The example below spawns a Guile REPL in a container in which the user's+home directory is accessible read-only via the @file{/exchange}+directory:++@example+guix shell --container --expose=$HOME=/exchange guile -- guile+@end example++@item --root=@var{file}+@itemx -r @var{file}+@cindex persistent environment+@cindex garbage collector root, for environments+Make @var{file} a symlink to the profile for this environment, and+register it as a garbage collector root.++This is useful if you want to protect your environment from garbage+collection, to make it ``persistent''.++When this option is omitted, the environment is protected from garbage+collection only for the duration of the @command{guix shell}+session. This means that next time you recreate the same environment,+you could have to rebuild or re-download packages. @xref{Invoking guix+gc}, for more on GC roots.+@end table++@command{guix shell} also supports all of the common build options that+@command{guix build} supports (@pxref{Common Build Options}) as well as+package transformation options (@pxref{Package Transformation Options}).++@node Invoking guix environment+@section Invoking @command{guix environment}++The purpose of @command{guix environment} is to assist in creating+development environments.++@quotation Deprecation warning+The @command{guix environment} command is deprecated in favor of+@command{guix shell}, which performs similar functions but is more+convenient to use. @xref{Invoking guix shell}.++Being deprecated, @command{guix environment} is slated for eventual+removal, but the Guix project is committed to keeping it until May 1st,+2023. Please get in touch with us at @email{guix-devel@@gnu.org} if you+would like to discuss it.+@end quotation The general syntax is: @@ -11099,14 +11422,14 @@ a container similar to the one the build daemon creates: $ guix build -K foo @dots{} $ cd /tmp/guix-build-foo.drv-0-$ guix environment --no-grafts -C foo --ad-hoc strace gdb+$ guix shell --no-grafts -C foo strace gdb [env]# source ./environment-variables [env]# cd foo-1.2 @end example Here, @command{guix environment -C} creates a container and spawns a new-shell in it (@pxref{Invoking guix environment}). The @command{--ad-hoc-strace gdb} part adds the @command{strace} and @command{gdb} commands to+shell in it (@pxref{Invoking guix shell}). The @command{strace gdb}+part adds the @command{strace} and @command{gdb} commands to the container, which you may find handy while debugging. The @option{--no-grafts} option makes sure we get the exact same environment, with ungrafted packages (@pxref{Security Updates}, for more@@ -11120,7 +11443,7 @@ remove @file{/bin/sh}: @end example (Don't worry, this is harmless: this is all happening in the throw-away-container created by @command{guix environment}.)+container created by @command{guix shell}.) The @command{strace} command is probably not in the search path, but we can run:@@ -13316,8 +13639,8 @@ is subject to radical change in the future. The purpose of @command{guix container} is to manipulate processes running within an isolated environment, commonly known as a-``container'', typically created by the @command{guix environment}-(@pxref{Invoking guix environment}) and @command{guix system container}+``container'', typically created by the @command{guix shell}+(@pxref{Invoking guix shell}) and @command{guix system container} (@pxref{Invoking guix system}) commands. The general syntax is:@@ -13503,7 +13826,7 @@ listed.}. Here's an example of the information it returns: $ sudo guix processes SessionPID: 19002 ClientPID: 19090-ClientCommand: guix environment --ad-hoc python+ClientCommand: guix shell python SessionPID: 19402 ClientPID: 19367@@ -29695,8 +30018,7 @@ When the service is running, you can view its console by connecting to it with a VNC client, for example with: @example-guix environment --ad-hoc tigervnc-client -- \- vncviewer localhost:5900+guix shell tigervnc-client -- vncviewer localhost:5900 @end example The default configuration (see @code{hurd-vm-configuration} below)diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scmindex 54f48a7482..77956fc018 100644--- a/guix/scripts/environment.scm+++ b/guix/scripts/environment.scm@@ -50,7 +50,11 @@ (define-module (guix scripts environment) #:use-module (srfi srfi-37) #:use-module (srfi srfi-98) #:export (assert-container-features- guix-environment))+ guix-environment+ guix-environment*+ show-environment-options-help+ (%options . %environment-options)+ (%default-options . %environment-default-options))) (define %default-shell (or (getenv "SHELL") "/bin/sh"))@@ -66,23 +70,16 @@ (define* (show-search-paths profile manifest #:key pure?) (newline))) (profile-search-paths profile manifest))) -(define (show-help)- (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]-Build an environment that includes the dependencies of PACKAGE and execute-COMMAND or an interactive shell in that environment.\n"))+(define (show-environment-options-help)+ "Print help about options shared between 'guix environment' and 'guix+shell'." (display (G_ " -e, --expression=EXPR create environment for the package that EXPR evaluates to")) (display (G_ "- -l, --load=FILE create environment for the package that the code within- FILE evaluates to"))- (display (G_ " -m, --manifest=FILE create environment with the manifest from FILE")) (display (G_ " -p, --profile=PATH create environment from profile at PATH"))- (display (G_ "- --ad-hoc include all specified packages in the environment instead- of only their inputs")) (display (G_ " --pure unset existing environment variables")) (display (G_ "@@ -118,7 +115,24 @@ (define (show-help) (display (G_ " -v, --verbosity=LEVEL use the given verbosity LEVEL")) (display (G_ "- --bootstrap use bootstrap binaries to build the environment"))+ --bootstrap use bootstrap binaries to build the environment")))++(define (show-help)+ (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]+Build an environment that includes the dependencies of PACKAGE and execute+COMMAND or an interactive shell in that environment.\n"))+ (warning (G_ "This command is deprecated in favor of 'guix shell'.\n"))+ (newline)++ ;; These two options are left out in 'guix shell'.+ (display (G_ "+ -l, --load=FILE create environment for the package that the code within+ FILE evaluates to"))+ (display (G_ "+ --ad-hoc include all specified packages in the environment instead+ of only their inputs"))++ (show-environment-options-help) (newline) (show-build-options-help) (newline)@@ -649,11 +663,15 @@ (define (register-gc-root target root) (define-command (guix-environment . args) (category development)- (synopsis "spawn one-off software environments")+ (synopsis "spawn one-off software environments (deprecated)") + (guix-environment* (parse-args args)))++(define (guix-environment* opts)+ "Run the 'guix environment' command on OPTS, an alist resulting for+command-line option processing with 'parse-command-line'." (with-error-handling- (let* ((opts (parse-args args))- (pure? (assoc-ref opts 'pure))+ (let* ((pure? (assoc-ref opts 'pure)) (container? (assoc-ref opts 'container?)) (link-prof? (assoc-ref opts 'link-profile?)) (network? (assoc-ref opts 'network?))@@ -724,8 +742,8 @@ (define manifest (prof-drv (manifest->derivation manifest system bootstrap?)) (profile -> (if profile- (readlink* profile)- (derivation->output-path prof-drv)))+ (readlink* profile)+ (derivation->output-path prof-drv))) (gc-root -> (assoc-ref opts 'gc-root))) ;; First build the inputs. This is necessary even fordiff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scmnew file mode 100644index 0000000000..190dd8837d--- /dev/null+++ b/guix/scripts/shell.scm@@ -0,0 +1,135 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 Ludovic Courtès <ludo@gnu.org>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (guix scripts shell)+ #:use-module (guix ui)+ #:use-module (guix scripts environment)+ #:autoload (guix scripts build) (show-build-options-help)+ #:autoload (guix transformations) (show-transformation-options-help)+ #:use-module (guix scripts)+ #:use-module (srfi srfi-1)+ #:use-module (srfi srfi-26)+ #:use-module (srfi srfi-37)+ #:use-module (srfi srfi-71)+ #:use-module (ice-9 match)+ #:export (guix-shell))++(define (show-help)+ (display (G_ "Usage: guix shell [OPTION] PACKAGES... [-- COMMAND...]+Build an environment that includes PACKAGES and execute COMMAND or an+interactive shell in that environment.\n"))+ (newline)++ ;; These two options differ from 'guix environment'.+ (display (G_ "+ -D, --development include the development inputs of the next package"))+ (display (G_ "+ -f, --file=FILE create environment for the package that the code within+ FILE evaluates to"))++ (show-environment-options-help)+ (newline)+ (show-build-options-help)+ (newline)+ (show-transformation-options-help)+ (newline)+ (display (G_ "+ -h, --help display this help and exit"))+ (display (G_ "+ -V, --version display version information and exit"))+ (newline)+ (show-bug-report-information))++(define (tag-package-arg opts arg)+ "Return a two-element list with the form (TAG ARG) that tags ARG with either+'ad-hoc' in OPTS has the 'ad-hoc?' key set to #t, or 'inputs' otherwise."+ (if (assoc-ref opts 'ad-hoc?)+ `(ad-hoc-package ,arg)+ `(package ,arg)))++(define (ensure-ad-hoc alist)+ (if (assq-ref alist 'ad-hoc?)+ alist+ `((ad-hoc? . #t) ,@alist)))++(define (wrapped-option opt)+ "Wrap OPT, a SRFI-37 option, such that its processor always adds the+'ad-hoc?' flag to the resulting alist."+ (option (option-names opt)+ (option-required-arg? opt)+ (option-optional-arg? opt)+ (compose ensure-ad-hoc (option-processor opt))))++(define %options+ ;; Specification of the command-line options.+ (let ((to-remove '("ad-hoc" "inherit" "load" "help" "version")))+ (append+ (list (option '(#\h "help") #f #f+ (lambda args+ (show-help)+ (exit 0)))+ (option '(#\V "version") #f #f+ (lambda args+ (show-version-and-exit "guix shell")))++ (option '(#\D "development") #f #f+ (lambda (opt name arg result)+ ;; Temporarily remove the 'ad-hoc?' flag from result.+ ;; The next option will put it back thanks to+ ;; 'wrapped-option'.+ (alist-delete 'ad-hoc? result)))++ ;; For consistency with 'guix package', support '-f' rather than+ ;; '-l' like 'guix environment' does.+ (option '(#\f "file") #t #f+ (lambda (opt name arg result)+ (alist-cons 'load (tag-package-arg result arg)+ result))))+ (filter-map (lambda (opt)+ (and (not (any (lambda (name)+ (member name to-remove))+ (option-names opt)))+ (wrapped-option opt)))+ %environment-options))))++(define %default-options+ `((ad-hoc? . #t) ;always true+ ,@%environment-default-options))++(define (parse-args args)+ "Parse the list of command line arguments ARGS."+ (define (handle-argument arg result)+ (alist-cons 'package (tag-package-arg result arg)+ (ensure-ad-hoc result)))++ ;; The '--' token is used to separate the command to run from the rest of+ ;; the operands.+ (let ((args command (break (cut string=? "--" <>) args)))+ (let ((opts (parse-command-line args %options (list %default-options)+ #:argument-handler handle-argument)))+ (match command+ (() opts)+ (("--") opts)+ (("--" command ...) (alist-cons 'exec command opts))))))++ +(define-command (guix-shell . args)+ (category development)+ (synopsis "spawn one-off software environments")++ (guix-environment* (parse-args args)))diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.inindex 5e77b3c230..ee0adbcf3c 100644--- a/po/guix/POTFILES.in+++ b/po/guix/POTFILES.in@@ -140,5 +140,6 @@ guix/scripts/offload.scm guix/scripts/perform-download.scm guix/scripts/refresh.scm guix/scripts/repl.scm+guix/scripts/shell.scm guix/scripts/system/reconfigure.scm nix/nix-daemon/guix-daemon.ccdiff --git a/tests/guix-shell.sh b/tests/guix-shell.shnew file mode 100644index 0000000000..f08637f7ff--- /dev/null+++ b/tests/guix-shell.sh@@ -0,0 +1,54 @@+# GNU Guix --- Functional package management for GNU+# Copyright © 2021 Ludovic Courtès <ludo@gnu.org>+#+# This file is part of GNU Guix.+#+# GNU Guix is free software; you can redistribute it and/or modify it+# under the terms of the GNU General Public License as published by+# the Free Software Foundation; either version 3 of the License, or (at+# your option) any later version.+#+# GNU Guix is distributed in the hope that it will be useful, but+# WITHOUT ANY WARRANTY; without even the implied warranty of+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+# GNU General Public License for more details.+#+# You should have received a copy of the GNU General Public License+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++#+# Test the 'guix shell' alias.+#++guix shell --version++tmpdir="t-guix-shell-$$"+trap 'rm -r "$tmpdir"' EXIT+mkdir "$tmpdir"++guix shell --bootstrap --pure guile-bootstrap -- guile --version++# '--ad-hoc' is a thing of the past.+! guix shell --ad-hoc guile-bootstrap++if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null+then+ # Compute the build environment for the initial GNU Make.+ guix shell --bootstrap --no-substitutes --search-paths --pure \+ -D -e '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/a"++ # Make sure bootstrap binaries are in the profile.+ profile=`grep "^export PATH" "$tmpdir/a" | sed -r 's|^.*="(.*)/bin"|\1|'`++ # Make sure the bootstrap binaries are all listed where they belong.+ grep -E "^export PATH=\"$profile/bin\"" "$tmpdir/a"+ grep -E "^export CPATH=\"$profile/include\"" "$tmpdir/a"+ grep -E "^export LIBRARY_PATH=\"$profile/lib\"" "$tmpdir/a"+ for dep in bootstrap-binaries-0 gcc-bootstrap-0 glibc-bootstrap-0+ do+ guix gc --references "$profile" | grep "$dep"+ done++ # 'make-boot0' itself must not be listed.+ ! guix gc --references "$profile" | grep make-boot0+fi-- 2.33.0
Z
Z
zimoun wrote 3 days ago
Re: [bug#50960] [PATCH v3 00/10] Adding 'guix shell': last call!
(name . Ludovic Courtès)(address . ludo@gnu.org)
864k9dflq3.fsf@gmail.com
Hi,
On Mon, 18 Oct 2021 at 21:52, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (4 lines)> Ludovic Courtès (10):> packages: Add 'package-development-inputs'.> profiles: Add 'package->development-manifest'.
LGTM. I have not checked the others. :-)
Toggle quote (10 lines)> Add 'guix shell'.> shell: By default load the local 'guix.scm' or 'manifest.scm' file.> environment: Add tests for '--profile'.> environment: Skip derivation computation when '--profile' is used.> environment: Do not connect to the daemon when '--profile' is used.> environment: Autoload some modules.> cache: Gracefully handle non-existent cache.> shell: Maintain a profile cache.

Cheers,simon
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 50960@debbugs.gnu.org