[PATCH] ui: Include channels in load path before searching for commands.

  • Open
  • quality assurance status badge
Details
2 participants
  • Carlo Zancanaro
  • Ludovic Courtès
Owner
unassigned
Submitted by
Carlo Zancanaro
Severity
normal
Merged with
C
C
Carlo Zancanaro wrote on 18 Nov 2024 23:26
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
eb0ba012b1cafc05b8438eb99a9c736237031242.1731968797.git.carlo@zancanaro.id.au
* guix/ui.scm (extension-directories): Add channel directories to load path,
compiled load path, and returned list of directories.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Change-Id: I3063cbf7164a065b2c6c2a5c6473df79ce8cbe9b
---

This patch allows extensions in channels to be found by the Guix
CLI. Most notably, this means that a script can define commands by
defining an appropriate (guix scripts X) module, with a guix-X
function.

This is a slight modification to a diff that Ludovic sent through IRC.

I've tested that it works when I use "guix time-machine". Guix is able
to find a command that I define in a channel.

I haven't specifically tested that it finds commands when run through
"guix pull", but I have tested that this change does not break "guix
pull".

guix/ui.scm | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)

Toggle diff (44 lines)
diff --git a/guix/ui.scm b/guix/ui.scm
index 447550635c..a702b6f3f8 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -19,6 +19,7 @@
;;; Copyright © 2018 Steve Sprang <scs@stevesprang.com>
;;; Copyright © 2022 Taiju HIGASHI <higashi@taiju.info>
;;; Copyright © 2022 Liliana Marie Prikler <liliana.prikler@gmail.com>
+;;; Copyright © 2024 Carlo Zancanaro <carlo@zancanaro.id.au>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -73,6 +74,7 @@ (define-module (guix ui) ;import in user interfaces only
#:use-module (srfi srfi-31)
#:use-module (srfi srfi-34)
#:use-module (srfi srfi-35)
+ #:use-module (srfi srfi-71)
#:autoload (ice-9 ftw) (scandir)
#:use-module (ice-9 match)
#:use-module (ice-9 format)
@@ -2192,9 +2194,16 @@ (define* (command-files #:optional directory)
(define (extension-directories)
"Return the list of directories containing Guix extensions."
- (filter file-exists?
- (parse-path
- (getenv "GUIX_EXTENSIONS_PATH"))))
+ (define package-path-entries
+ ;; We need to resolve this lazily, because even using an #:autoload is too
+ ;; much and breaks compilation during "guix pull".
+ (module-ref (resolve-module `(guix describe))
+ (symbol-append 'package-path-entries)))
+ (let ((scm-path go-path (package-path-entries)))
+ (set! %load-path (append %load-path scm-path))
+ (set! %load-compiled-path (append %load-compiled-path go-path))
+ (filter file-exists?
+ (parse-path (getenv "GUIX_EXTENSIONS_PATH") scm-path))))
(define (commands)
"Return the list of commands, alphabetically sorted."

base-commit: 23cbbe6860782c5d4a0ba599ea1cda0642e91661
--
2.46.0
L
L
Ludovic Courtès wrote on 21 Nov 2024 13:14
(name . Carlo Zancanaro)(address . carlo@zancanaro.id.au)
87jzcwss1j.fsf@gnu.org
Hello,

Carlo Zancanaro <carlo@zancanaro.id.au> skribis:

Toggle quote (6 lines)
> * guix/ui.scm (extension-directories): Add channel directories to load path,
> compiled load path, and returned list of directories.
>
> Co-authored-by: Ludovic Courtès <ludo@gnu.org>
> Change-Id: I3063cbf7164a065b2c6c2a5c6473df79ce8cbe9b

Nice, thanks for looking into it!

Toggle quote (16 lines)
> (define (extension-directories)
> "Return the list of directories containing Guix extensions."
> - (filter file-exists?
> - (parse-path
> - (getenv "GUIX_EXTENSIONS_PATH"))))
> + (define package-path-entries
> + ;; We need to resolve this lazily, because even using an #:autoload is too
> + ;; much and breaks compilation during "guix pull".
> + (module-ref (resolve-module `(guix describe))
> + (symbol-append 'package-path-entries)))
> + (let ((scm-path go-path (package-path-entries)))
> + (set! %load-path (append %load-path scm-path))
> + (set! %load-compiled-path (append %load-compiled-path go-path))
> + (filter file-exists?
> + (parse-path (getenv "GUIX_EXTENSIONS_PATH") scm-path))))

I think you could do:

#:autoload (guix describe) (package-path-entries)

instead of doing the trick above.

Two things:

1. The ‘%load-path’ and ‘%load-compiled-path’ modifications can now be
removed from (gnu packages), after checking that GUIX_PACKAGE_PATH
still takes precedence.

2. The performance impact of this change must be tested, in particular
the startup time of ‘guix’ commands since they’ll now be loading
(guix channels) & co. unconditionally.

One way to do that might be:

PROFILE=$(guix time-machine -q --url=/path/to/checkout)
guix shell time -- time $PROFILE/bin/guix build --help
strace -c $PROFILE/bin/guix build --help
Could you give it a try?

Ludo’.
C
C
Carlo Zancanaro wrote on 22 Nov 2024 00:25
(name . Ludovic Courtès)(address . ludo@gnu.org)
87iksgxj8k.fsf@zancanaro.id.au
On Thu, Nov 21 2024, Ludovic Courtès wrote:
Toggle quote (6 lines)
> I think you could do:
>
> #:autoload (guix describe) (package-path-entries)
>
> instead of doing the trick above.

I tried that, and it breaks "guix pull" (as mentioned in the comment in
the code). I don't have the specific error easily accessible, but I
think it was "no code for module: (git)". I assume it was an issue with
how build-program in (build-self) puts things together, but I wasn't
able to investigate further than that.

Toggle quote (4 lines)
> 1. The ‘%load-path’ and ‘%load-compiled-path’ modifications can now be
> removed from (gnu packages), after checking that GUIX_PACKAGE_PATH
> still takes precedence.

With this change, (extension-directories) does not include the paths in
GUIX_PACKAGE_PATH. Perhaps it should, but I wasn't sure given the
distinction between GUIX_PACKAGE_PATH and GUIX_EXTENSIONS_PATH.

Do we want to run commands from GUIX_PACKAGE_PATH? Or load packages from
GUIX_EXTENSIONS_PATH? At the moment I think the code says "no" to both.

My other concern with removing the code in (gnu packages) is that it may
break code that is using Guix as a library. That is, something like
this:

Toggle snippet (3 lines)
guix shell guix guile -- guile -c '(use-modules (gnu packages)) (pk (find-packages-by-name "guile"))'

might not add the right things to the load path.

That said, the current patch will add duplicate entries to the load path
when using the Guix CLI, which also isn't good. Perhaps this needs to be
factored out into a single place where we can keep track of whether
we've already added to the load paths.

Toggle quote (12 lines)
> 2. The performance impact of this change must be tested, in particular
> the startup time of ‘guix’ commands since they’ll now be loading
> (guix channels) & co. unconditionally.
>
> One way to do that might be:
>
> PROFILE=$(guix time-machine -q --url=/path/to/checkout)
> guix shell time -- time $PROFILE/bin/guix build --help
> strace -c $PROFILE/bin/guix build --help
>
> Could you give it a try?

Sure!

With the change in my patch, I'm seeing:

strace: number of calls and errors is roughly the same both with and
without my patch.

time: all the numbers look pretty similar with and without my patch.

I haven't done any statistical analysis of the various values recorded
by strace and time, but the numbers don't vary *too* much between runs,
so I'm fairly confident this isn't having too much of an effect.

I also tested leaving off the "-q" in the time machine call (so my other
channels were also included), but this didn't make any meaningful
difference to the numbers when running "guix build --help".

Carlo
L
L
Ludovic Courtès wrote on 26 Dec 2024 22:33
control message for bug #74633
(address . control@debbugs.gnu.org)
87msgiyvs2.fsf@gnu.org
merge 74633 74425
quit
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 74425@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 74425
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch