[PATCH 00/11] Improve startup time and memory footprint for short-lived commands

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Simon Tournier
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 1 Apr 22:22 +0200
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
cover.1712002698.git.ludo@gnu.org
Hello!

This is a pretty boring series adding #:autoload in strategic places.
The goal is to avoid loading tons of modules and shared objects when
running ‘guix describe’ or ‘guix shell’ on a cache hit.

There’s a tiny bit of reshuffling to make that easier:

• Bits moved from (guix channels) to (guix git);

• ‘call-with-temporary-output-file’ no longer re-exported by
(guix utils).

I measured the impact with things like:

strace -e openat -o /tmp/log.strace guix describe
grep 'openat.*\.go' </tmp/log.strace | wc -l

and also checking specifically whether things like Guile-Git,
Guile-Gcrypt, and Guile-GnuTLS were being loaded.

Feedback welcome!

Ludo’.

Ludovic Courtès (11):
channels: Use SRFI-71 instead of SRFI-11.
git: Add ‘repository-info’ and use it in (guix channels).
channels: Move ‘commit-short-id’ to (guix git).
git: Add ‘tag->commit’ and use it in (guix channels).
channels: Autoload (git …) modules.
guix system: Autoload some more.
utils: Don’t re-export ‘call-with-temporary-output-file’.
guix: Delay loading of (gnutls).
ui: Delay use of (guix build syscalls).
Autoload (guix build syscalls).
Autoload (gcrypt hash).

guix/channels.scm | 56 +++++++++++++++++----------------------
guix/derivations.scm | 6 ++---
guix/discovery.scm | 4 +--
guix/download.scm | 4 +--
guix/git.scm | 52 +++++++++++++++++++++++++++---------
guix/import/hackage.scm | 4 +--
guix/import/hexpm.scm | 7 ++---
guix/import/opam.scm | 7 ++---
guix/import/pypi.scm | 5 ++--
guix/nar.scm | 4 +--
guix/packages.scm | 5 ++--
guix/scripts.scm | 7 +++--
guix/scripts/build.scm | 4 +--
guix/scripts/describe.scm | 4 +--
guix/scripts/system.scm | 12 ++++++---
guix/store.scm | 15 ++++++-----
guix/ui.scm | 22 +++------------
guix/utils.scm | 17 ++++++------
tests/cpio.scm | 6 ++---
tests/egg.scm | 5 ++--
tests/opam.scm | 5 ++--
tests/publish.scm | 5 ++--
tests/store-database.scm | 6 ++---
tests/utils.scm | 3 ++-
24 files changed, 143 insertions(+), 122 deletions(-)


base-commit: 16c713083516e60b5ae30b3a8b49d5bf8d4cadc3
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 01/11] channels: Use SRFI-71 instead of SRFI-11.
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
6f4f3bb9ebafe98416d378e9438c9453fae613d2.1712002698.git.ludo@gnu.org
* guix/channels.scm (latest-channel-instance): Use SRFI-71.

Change-Id: I73531c98b3034e228006ed91518cc7bfedc784fd
---
guix/channels.scm | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)

Toggle diff (36 lines)
diff --git a/guix/channels.scm b/guix/channels.scm
index 66f3122f79..10f0e3800f 100644
--- a/guix/channels.scm
+++ b/guix/channels.scm
@@ -43,10 +43,10 @@ (define-module (guix channels)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
#:use-module (srfi srfi-9)
- #:use-module (srfi srfi-11)
#:use-module (srfi srfi-26)
#:use-module (srfi srfi-34)
#:use-module (srfi srfi-35)
+ #:use-module (srfi srfi-71)
#:autoload (guix describe) (current-channels) ;XXX: circular dep
#:autoload (guix self) (whole-package make-config.scm)
#:autoload (guix inferior) (gexp->derivation-in-inferior) ;FIXME: circular dep
@@ -411,12 +411,11 @@ (define* (latest-channel-instance store channel
(and (string=? (basename file) ".git")
(eq? 'directory (stat:type stat))))
- (let-values (((channel)
- (ensure-default-introduction channel))
- ((checkout commit relation)
- (update-cached-checkout (channel-url channel)
- #:ref (channel-reference channel)
- #:starting-commit starting-commit)))
+ (let ((channel (ensure-default-introduction channel))
+ (checkout commit relation
+ (update-cached-checkout (channel-url channel)
+ #:ref (channel-reference channel)
+ #:starting-commit starting-commit)))
(when relation
(validate-pull channel starting-commit commit relation))
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 02/11] git: Add ‘repository-info ’ and use it in (guix channels).
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
0b963eb469dfa61649ca3a75f6745f3ca0a041a9.1712002698.git.ludo@gnu.org
* guix/git.scm (repository-info): New procedure.
* guix/channels.scm (repository->guix-channel): Use it instead of local
code.

Change-Id: I74c758c73a22e16031571ca4271cc9cab0492f6e
---
guix/channels.scm | 20 ++++++++------------
guix/git.scm | 19 ++++++++++++++++++-
2 files changed, 26 insertions(+), 13 deletions(-)

Toggle diff (83 lines)
diff --git a/guix/channels.scm b/guix/channels.scm
index 10f0e3800f..f26ccbc3ae 100644
--- a/guix/channels.scm
+++ b/guix/channels.scm
@@ -24,6 +24,7 @@ (define-module (guix channels)
#:autoload (guix git) (update-cached-checkout
url+commit->name
commit-difference
+ repository-info
with-repository)
#:autoload (guix git-authenticate) (authenticate-repository)
#:autoload (guix openpgp) (openpgp-public-key-fingerprint
@@ -207,18 +208,13 @@ (define* (repository->guix-channel directory
channel that uses that repository and the commit HEAD currently points to; use
INTRODUCTION as the channel's introduction. Return #f if no Git repository
could be found at DIRECTORY or one of its ancestors."
- (catch 'git-error
- (lambda ()
- (with-repository (repository-discover directory) repository
- (let* ((head (repository-head repository))
- (commit (oid->string (reference-target head))))
- (channel
- (inherit %default-guix-channel)
- (url (repository-working-directory repository))
- (commit commit)
- (branch (reference-shorthand head))
- (introduction introduction)))))
- (const #f)))
+ (let ((directory commit branch (repository-info directory)))
+ (channel
+ (inherit %default-guix-channel)
+ (url directory)
+ (commit commit)
+ (branch branch)
+ (introduction introduction))))
(define-record-type <channel-instance>
(channel-instance channel commit checkout)
diff --git a/guix/git.scm b/guix/git.scm
index cbcdb1904b..760b064a9c 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -1,6 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2017, 2020 Mathieu Othacehe <m.othacehe@gmail.com>
-;;; Copyright © 2018-2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2018-2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2021 Kyle Meyer <kyle@kyleam.com>
;;; Copyright © 2021 Marius Bakke <marius@gnu.org>
;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
@@ -59,6 +59,7 @@ (define-module (guix git)
with-repository
with-git-error-handling
false-if-git-not-found
+ repository-info
update-cached-checkout
url+commit->name
latest-repository-commit
@@ -330,6 +331,22 @@ (define-syntax-rule (with-git-error-handling body ...)
(lambda (key err)
(report-git-error err))))
+(define (repository-info directory)
+ "Open the Git repository in DIRECTORY or one of its parent and return three
+values: the working directory of that repository, its checked out commit ID,
+and its checked out reference (such as a branch name). Return #f (three
+values) if DIRECTORY does not hold a readable Git repository."
+ (catch 'git-error
+ (lambda ()
+ (with-repository (repository-discover directory) repository
+ (let* ((head (repository-head repository))
+ (commit (oid->string (reference-target head))))
+ (values (repository-working-directory repository)
+ commit
+ (reference-shorthand head)))))
+ (lambda _
+ (values #f #f #f))))
+
(define* (update-submodules repository
#:key (log-port (current-error-port))
(fetch-options #f))
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 03/11] channels: Move ‘commit-short- id’ to (guix git).
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
40070393934abb8d6a785857dc3806217e64817b.1712002698.git.ludo@gnu.org
* guix/channels.scm (commit-short-id): Move to…
* guix/git.scm (commit-short-id): … here.

Change-Id: If4b34b1d82b1aa5068d157f26e57e8aecc967061
---
guix/channels.scm | 4 +---
guix/git.scm | 4 ++++
2 files changed, 5 insertions(+), 3 deletions(-)

Toggle diff (46 lines)
diff --git a/guix/channels.scm b/guix/channels.scm
index f26ccbc3ae..0b776ab211 100644
--- a/guix/channels.scm
+++ b/guix/channels.scm
@@ -25,6 +25,7 @@ (define-module (guix channels)
url+commit->name
commit-difference
repository-info
+ commit-short-id
with-repository)
#:autoload (guix git-authenticate) (authenticate-repository)
#:autoload (guix openpgp) (openpgp-public-key-fingerprint
@@ -339,9 +340,6 @@ (define (apply-patches checkout commit patches)
(apply-patch patch checkout))
(loop rest)))))
-(define commit-short-id
- (compose (cut string-take <> 7) oid->string commit-id))
-
(define* (authenticate-channel channel checkout commit
#:key (keyring-reference-prefix "origin/"))
"Authenticate the given COMMIT of CHANNEL, available at CHECKOUT, a
diff --git a/guix/git.scm b/guix/git.scm
index 760b064a9c..eab84ea798 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -67,6 +67,7 @@ (define-module (guix git)
commit-relation
commit-descendant?
commit-id?
+ commit-short-id
remote-refs
@@ -233,6 +234,9 @@ (define (commit-id? str)
(and (= (string-length str) 40)
(string-every char-set:hex-digit str)))
+(define commit-short-id
+ (compose (cut string-take <> 7) oid->string commit-id))
+
(define (resolve-reference repository ref)
"Resolve the branch, commit or tag specified by REF, and return the
corresponding Git object."
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 04/11] git: Add ‘tag->commit ’ and use it in (guix channels).
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
db8007e3c85705035d9f4f54753e36088f863b6b.1712002698.git.ludo@gnu.org
* guix/git.scm (tag->commit): New procedure, taken from…
(resolve-reference): … here. Use it in the ‘tag’ case.
* guix/channels.scm (resolve-channel-news-entry-tag): Use ‘tag->commit’
instead of custom code.

Change-Id: I46ea387345dc1b695ce0702991a52d0cde29e2f0
---
guix/channels.scm | 11 +++--------
guix/git.scm | 24 +++++++++++++++---------
2 files changed, 18 insertions(+), 17 deletions(-)

Toggle diff (80 lines)
diff --git a/guix/channels.scm b/guix/channels.scm
index 0b776ab211..70608561f9 100644
--- a/guix/channels.scm
+++ b/guix/channels.scm
@@ -26,6 +26,7 @@ (define-module (guix channels)
commit-difference
repository-info
commit-short-id
+ tag->commit
with-repository)
#:autoload (guix git-authenticate) (authenticate-repository)
#:autoload (guix openpgp) (openpgp-public-key-fingerprint
@@ -1148,14 +1149,8 @@ (define (resolve-channel-news-entry-tag repository entry)
cannot be found."
(if (channel-news-entry-commit entry)
entry
- (let* ((tag (channel-news-entry-tag entry))
- (reference (reference-lookup repository
- (string-append "refs/tags/" tag)))
- (target (reference-target reference))
- (oid (let ((obj (object-lookup repository target)))
- (if (= OBJ-TAG (object-type obj)) ;annotated tag?
- (tag-target-id (tag-lookup repository target))
- target))))
+ (let* ((tag (channel-news-entry-tag entry))
+ (oid (object-id (tag->commit repository tag))))
(channel-news-entry (oid->string oid) tag
(channel-news-entry-title entry)
(channel-news-entry-body entry)))))
diff --git a/guix/git.scm b/guix/git.scm
index eab84ea798..8e1d863976 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -68,6 +68,7 @@ (define-module (guix git)
commit-descendant?
commit-id?
commit-short-id
+ tag->commit
remote-refs
@@ -237,6 +238,19 @@ (define (commit-id? str)
(define commit-short-id
(compose (cut string-take <> 7) oid->string commit-id))
+(define (tag->commit repository tag)
+ "Resolve TAG in REPOSITORY and return the corresponding object, usually a
+commit."
+ (let* ((oid (reference-name->oid repository
+ (string-append "refs/tags/" tag)))
+ (obj (object-lookup repository oid)))
+ ;; OID may designate an "annotated tag" object or a "commit" object.
+ ;; Return the commit object in both cases.
+ (if (= OBJ-TAG (object-type obj))
+ (object-lookup repository
+ (tag-target-id (tag-lookup repository oid)))
+ obj)))
+
(define (resolve-reference repository ref)
"Resolve the branch, commit or tag specified by REF, and return the
corresponding Git object."
@@ -283,15 +297,7 @@ (define (resolve-reference repository ref)
;; There's no such tag, so it must be a commit ID.
(resolve `(commit . ,str)))))))
(('tag . tag)
- (let* ((oid (reference-name->oid repository
- (string-append "refs/tags/" tag)))
- (obj (object-lookup repository oid)))
- ;; OID may designate an "annotated tag" object or a "commit" object.
- ;; Return the commit object in both cases.
- (if (= OBJ-TAG (object-type obj))
- (object-lookup repository
- (tag-target-id (tag-lookup repository oid)))
- obj))))))
+ (tag->commit repository tag)))))
(define (switch-to-ref repository ref)
"Switch to REPOSITORY's branch, commit or tag specified by REF. Return the
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 05/11] channels: Autoload (git …) modules.
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
b8b31c5056e8e01990e7ce22d1c07aaced22ec76.1712002698.git.ludo@gnu.org
Autoloading Guile-Git is important in cases where (guix channels) is
used for little more than the <channel> definition. This is the case,
for example, of ‘guix describe’ or ‘guix shell’.

This reduces from 177 to 121 the number of .go files loaded when
running:

./pre-inst-env strace -e openat -o /tmp/log.strace \
guix describe -p /var/guix/profiles/per-user/$USER/current-guix
grep 'openat.*\.go.* = [0-9]' < /tmp/log.strace |wc -l

Likewise, it reduces the max RSS (as measured by ‘time -f %M guix
describe -p …’) from 54 to 37 MiB.

* guix/channels.scm: Autoload (git …) modules.

Change-Id: Ia58a99c865bf0f6fe461a1e71390d075e760f8d6
---
guix/channels.scm | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)

Toggle diff (21 lines)
diff --git a/guix/channels.scm b/guix/channels.scm
index 70608561f9..51024dcad4 100644
--- a/guix/channels.scm
+++ b/guix/channels.scm
@@ -20,7 +20,13 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix channels)
- #:use-module (git) ;TODO: autoload
+ #:autoload (git commit) (commit-lookup
+ commit-id)
+ #:autoload (git oid) (oid->string
+ string->oid)
+ #:autoload (git object) (object-id)
+ #:autoload (git errors) (GIT_ENOTFOUND)
+ #:autoload (git structs) (git-error-code)
#:autoload (guix git) (update-cached-checkout
url+commit->name
commit-difference
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 06/11] guix system: Autoload some more.
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
45625999e3252f94a596de1f75b70b5186a765cf.1712002698.git.ludo@gnu.org
* guix/scripts/system.scm: Autoload more modules.

Change-Id: I665857109bbfd1e3755135daacc01affcb3eb2eb
---
guix/scripts/system.scm | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)

Toggle diff (44 lines)
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index bf3d2f9044..6952d454e6 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2014-2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2014-2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2016 Alex Kost <alezost@gmail.com>
;;; Copyright © 2016, 2017, 2018 Chris Marusich <cmmarusich@gmail.com>
;;; Copyright © 2017, 2019 Mathieu Othacehe <m.othacehe@gmail.com>
@@ -37,7 +37,7 @@ (define-module (guix scripts system)
#:autoload (guix store database)
(sqlite-register store-database-file call-with-database)
#:autoload (guix build store-copy) (copy-store-item)
- #:use-module (guix describe)
+ #:autoload (guix describe) (current-profile)
#:use-module (guix gexp)
#:use-module (guix derivations)
#:use-module (guix diagnostics)
@@ -47,7 +47,10 @@ (define-module (guix scripts system)
#:use-module (guix records)
#:use-module (guix profiles)
#:use-module (guix scripts)
- #:use-module (guix channels)
+ #:autoload (guix channels) (channel-name
+ channel-url
+ channel-branch
+ channel-commit)
#:use-module (guix scripts build)
#:autoload (guix scripts package) (delete-generations
delete-matching-generations
@@ -57,7 +60,8 @@ (define-module (guix scripts system)
graph-backend-name lookup-backend)
#:use-module (guix scripts system reconfigure)
#:use-module (guix build utils)
- #:use-module (guix progress)
+ #:autoload (guix progress) (progress-reporter/bar
+ call-with-progress-reporter)
#:use-module ((guix docker) #:select (%docker-image-max-layers))
#:use-module (gnu build image)
#:use-module (gnu build install)
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 08/11] guix: Delay loading of (gnutls).
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
2c9f4d6f2b10b0975be8952a1ae0bc0fc0bbe0d6.1712002698.git.ludo@gnu.org
(web …) modules pull in (gnutls) indirectly. Arrange to load them
lazily, thereby reducing I/O and allocations when GnuTLS is not needed
such as when running ‘guix describe’ or ‘guix shell’ on a cache hit.

* guix/download.scm: Autoload (web uri).
* guix/scripts/describe.scm: Likewise.
* guix/store.scm: Likewise.
(%default-substitute-urls): Remove ‘resolve-interface’ call and use
https URLs unconditionally.

Change-Id: Ide470c556a14866e8740966d25821df487a79859
---
guix/download.scm | 2 +-
guix/scripts/describe.scm | 4 ++--
guix/store.scm | 13 ++++++++-----
3 files changed, 11 insertions(+), 8 deletions(-)

Toggle diff (71 lines)
diff --git a/guix/download.scm b/guix/download.scm
index 192c47f113..b251e1f6c0 100644
--- a/guix/download.scm
+++ b/guix/download.scm
@@ -32,7 +32,7 @@ (define-module (guix download)
#:use-module (guix monads)
#:use-module (guix gexp)
#:autoload (guix build utils) (call-with-temporary-output-file)
- #:use-module (web uri)
+ #:autoload (web uri) (string->uri uri-scheme uri-path)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (%download-methods
diff --git a/guix/scripts/describe.scm b/guix/scripts/describe.scm
index 449ab4b252..70ae84e9f6 100644
--- a/guix/scripts/describe.scm
+++ b/guix/scripts/describe.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2018, 2019, 2020, 2021, 2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2018, 2019, 2020, 2021, 2023, 2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2018 Oleg Pykhalov <go.wigust@gmail.com>
;;; Copyright © 2020 Ekaitz Zarraga <ekaitz@elenq.tech>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
@@ -37,7 +37,7 @@ (define-module (guix scripts describe)
#:use-module (ice-9 match)
#:use-module (ice-9 format)
#:autoload (ice-9 pretty-print) (pretty-print)
- #:use-module (web uri)
+ #:autoload (web uri) (string->uri uri-host)
#:export (display-profile-content
channel-commit-hyperlink
diff --git a/guix/store.scm b/guix/store.scm
index 97c4f32a5b..e808b43ba9 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012-2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2018 Jan Nieuwenhuizen <janneke@gnu.org>
;;; Copyright © 2019, 2020 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2020 Florian Pelz <pelzflorian@pelzflorian.de>
@@ -49,7 +49,12 @@ (define-module (guix store)
#:use-module (ice-9 popen)
#:autoload (ice-9 threads) (current-processor-count)
#:use-module (ice-9 format)
- #:use-module (web uri)
+ #:autoload (web uri) (uri?
+ string->uri
+ uri-scheme
+ uri-host
+ uri-port
+ uri-path)
#:export (%daemon-socket-uri
%gc-roots-directory
%default-substitute-urls
@@ -764,9 +769,7 @@ (define %default-substitute-urls
;; Default list of substituters. This is *not* the list baked in
;; 'guix-daemon', but it is used by 'guix-service-type' and and a couple of
;; clients ('guix build --log-file' uses it.)
- (map (if (false-if-exception (resolve-interface '(gnutls)))
- (cut string-append "https://" <>)
- (cut string-append "http://" <>))
+ (map (cut string-append "https://" <>)
'("ci.guix.gnu.org"
"bordeaux.guix.gnu.org")))
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 07/11] utils: Don’t re-export ‘call-with-temporary-output-file’.
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
0347e5d07c684f680ae2e983b3b44830cbcd74a5.1712002698.git.ludo@gnu.org
* guix/utils.scm: Remove re-export of ‘call-with-temporary-output-file’.
Autoload a number of modules.
* guix/download.scm, guix/import/hackage.scm,
guix/import/hexpm.scm, guix/import/opam.scm,
guix/import/pypi.scm, tests/cpio.scm, tests/egg.scm,
tests/opam.scm, tests/publish.scm, tests/store-database.scm,
tests/utils.scm: Adjust imports accordingly.

Change-Id: I3f5e94631397996a30be2ea4ff8b50a3371e8ee7
---
guix/download.scm | 2 +-
guix/import/hackage.scm | 4 ++--
guix/import/hexpm.scm | 7 ++++---
guix/import/opam.scm | 7 ++++---
guix/import/pypi.scm | 5 +++--
guix/utils.scm | 17 ++++++++---------
tests/cpio.scm | 6 +++---
tests/egg.scm | 5 +++--
tests/opam.scm | 5 +++--
tests/publish.scm | 5 +++--
tests/store-database.scm | 6 +++---
tests/utils.scm | 3 ++-
12 files changed, 39 insertions(+), 33 deletions(-)

Toggle diff (271 lines)
diff --git a/guix/download.scm b/guix/download.scm
index 3dfe143e9f..192c47f113 100644
--- a/guix/download.scm
+++ b/guix/download.scm
@@ -31,7 +31,7 @@ (define-module (guix download)
#:autoload (guix build download) (url-fetch)
#:use-module (guix monads)
#:use-module (guix gexp)
- #:use-module (guix utils)
+ #:autoload (guix build utils) (call-with-temporary-output-file)
#:use-module (web uri)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
diff --git a/guix/import/hackage.scm b/guix/import/hackage.scm
index bbaee73a06..79a51d3300 100644
--- a/guix/import/hackage.scm
+++ b/guix/import/hackage.scm
@@ -8,7 +8,7 @@
;;; Copyright © 2021 Sarah Morgensen <iskarian@mgsn.dev>
;;; Copyright © 2019 Simon Tournier <zimon.toutoune@gmail.com>
;;; Copyright © 2022 Hartmut Goebel <h.goebel@crazy-compilers.com>
-;;; Copyright © 2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2023-2024 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -47,7 +47,7 @@ (define-module (guix import hackage)
#:use-module (guix upstream)
#:use-module (guix packages)
#:autoload (guix build-system haskell) (hackage-uri)
- #:use-module ((guix utils) #:select (call-with-temporary-output-file))
+ #:autoload (guix build utils) (call-with-temporary-output-file)
#:export (%hackage-url
hackage->guix-package
hackage-recursive-import
diff --git a/guix/import/hexpm.scm b/guix/import/hexpm.scm
index 628a44ff24..71a54ba973 100644
--- a/guix/import/hexpm.scm
+++ b/guix/import/hexpm.scm
@@ -1,7 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015 Cyril Roelandt <tipecaml@gmail.com>
;;; Copyright © 2016 David Craven <david@craven.ch>
-;;; Copyright © 2017, 2019-2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2017, 2019-2021, 2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2019 Martin Becze <mjbecze@riseup.net>
;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;; Copyright © 2020-2022 Hartmut Goebel <h.goebel@crazy-compilers.com>
@@ -28,10 +28,11 @@ (define-module (guix import hexpm)
#:use-module ((guix build utils)
#:select ((package-name->name+version
. hyphen-package-name->name+version)
- dump-port))
+ dump-port
+ call-with-temporary-output-file))
#:use-module (guix packages)
#:use-module (guix upstream)
- #:use-module (guix utils)
+ #:autoload (guix utils) (version>? file-sans-extension)
#:use-module (ice-9 match)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
diff --git a/guix/import/opam.scm b/guix/import/opam.scm
index 86e82cde59..a7f8092507 100644
--- a/guix/import/opam.scm
+++ b/guix/import/opam.scm
@@ -31,7 +31,9 @@ (define-module (guix import opam)
#:use-module ((srfi srfi-26) #:select (cut))
#:use-module (srfi srfi-34)
#:use-module ((web uri) #:select (string->uri uri->string))
- #:use-module ((guix build utils) #:select (dump-port find-files mkdir-p))
+ #:use-module ((guix build utils)
+ #:select (dump-port find-files mkdir-p
+ call-with-temporary-output-file))
#:use-module (guix build-system)
#:use-module (guix i18n)
#:use-module (guix diagnostics)
@@ -39,8 +41,7 @@ (define-module (guix import opam)
#:use-module (guix packages)
#:use-module (guix upstream)
#:use-module ((guix utils) #:select (cache-directory
- version>?
- call-with-temporary-output-file))
+ version>?))
#:use-module ((guix import utils) #:select (beautify-description
guix-hash-url
recursive-import
diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 1a3070fb36..6719fde330 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -1,7 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2014 David Thompson <davet@gnu.org>
;;; Copyright © 2015 Cyril Roelandt <tipecaml@gmail.com>
-;;; Copyright © 2015-2017, 2019-2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2015-2017, 2019-2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2018, 2023 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com>
@@ -52,7 +52,8 @@ (define-module (guix import pypi)
#:select ((package-name->name+version
. hyphen-package-name->name+version)
find-files
- invoke))
+ invoke
+ call-with-temporary-output-file))
#:use-module (guix import utils)
#:use-module (guix import json)
#:use-module (json)
diff --git a/guix/utils.scm b/guix/utils.scm
index 29ad09d9f7..d8ce6ed886 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012-2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2022, 2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2013, 2014, 2015 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2014 Eric Bavier <bavier@member.fsf.org>
;;; Copyright © 2014 Ian Denhardt <ian@zenhack.net>
@@ -47,11 +47,12 @@ (define-module (guix utils)
#:use-module (rnrs io ports) ;need 'port-position' etc.
#:use-module ((rnrs bytevectors) #:select (bytevector-u8-set!))
#:use-module (guix memoization)
- #:use-module ((guix build utils)
- #:select (dump-port mkdir-p delete-file-recursively
- call-with-temporary-output-file %xz-parallel-args))
- #:use-module ((guix build syscalls) #:select (mkdtemp! fdatasync))
- #:use-module ((guix combinators) #:select (fold2))
+ #:autoload (guix build utils) (dump-port
+ mkdir-p
+ delete-file-recursively
+ %xz-parallel-args)
+ #:autoload (guix build syscalls) (mkdtemp! fdatasync)
+ #:autoload (guix combinators) (fold2)
#:use-module (guix diagnostics) ;<location>, &error-location, etc.
#:use-module (ice-9 format)
#:use-module ((ice-9 iconv) #:prefix iconv:)
@@ -76,9 +77,7 @@ (define-module (guix utils)
&fix-hint
fix-hint?
- condition-fix-hint
-
- call-with-temporary-output-file)
+ condition-fix-hint)
#:export (strip-keyword-arguments
default-keyword-arguments
substitute-keyword-arguments
diff --git a/tests/cpio.scm b/tests/cpio.scm
index 832101d1bb..35a704822b 100644
--- a/tests/cpio.scm
+++ b/tests/cpio.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2015, 2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2015, 2022, 2024 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -19,8 +19,8 @@
(define-module (test-cpio)
#:use-module (guix cpio)
#:use-module (guix tests)
- #:use-module ((guix build utils) #:select (which))
- #:use-module ((guix utils) #:select (call-with-temporary-output-file))
+ #:use-module ((guix build utils)
+ #:select (which call-with-temporary-output-file))
#:use-module (ice-9 match)
#:use-module (ice-9 popen)
#:use-module (rnrs io ports)
diff --git a/tests/egg.scm b/tests/egg.scm
index a7d3378dd7..c74f954683 100644
--- a/tests/egg.scm
+++ b/tests/egg.scm
@@ -24,8 +24,9 @@ (define-module (test-eggs)
#:use-module (gcrypt hash)
#:use-module (guix tests)
#:use-module ((guix build syscalls) #:select (mkdtemp!))
- #:use-module ((guix build utils) #:select (delete-file-recursively mkdir-p which))
- #:use-module ((guix utils) #:select (call-with-temporary-output-file))
+ #:use-module ((guix build utils)
+ #:select (delete-file-recursively mkdir-p which
+ call-with-temporary-output-file))
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-64)
#:use-module (web uri)
diff --git a/tests/opam.scm b/tests/opam.scm
index 832fea1d9b..f444ef302e 100644
--- a/tests/opam.scm
+++ b/tests/opam.scm
@@ -24,8 +24,9 @@ (define-module (test-opam)
#:use-module (gcrypt hash)
#:use-module (guix tests)
#:use-module ((guix build syscalls) #:select (mkdtemp!))
- #:use-module ((guix build utils) #:select (delete-file-recursively mkdir-p which))
- #:use-module ((guix utils) #:select (call-with-temporary-output-file))
+ #:use-module ((guix build utils)
+ #:select (delete-file-recursively mkdir-p which
+ call-with-temporary-output-file))
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-64)
#:use-module (web uri)
diff --git a/tests/publish.scm b/tests/publish.scm
index efb5698bed..d5ec3c954f 100644
--- a/tests/publish.scm
+++ b/tests/publish.scm
@@ -1,7 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2015 David Thompson <davet@gnu.org>
;;; Copyright © 2020 by Amar M. Singh <nly@disroot.org>
-;;; Copyright © 2016-2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2016-2022, 2024 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -25,7 +25,8 @@ (define-module (test-publish)
#:use-module (guix scripts publish)
#:use-module (guix tests)
#:use-module (guix config)
- #:use-module (guix utils)
+ #:use-module ((guix utils) #:select (call-with-temporary-directory))
+ #:use-module ((guix build utils) #:select (call-with-temporary-output-file))
#:use-module (gcrypt hash)
#:use-module (guix store)
#:use-module (guix derivations)
diff --git a/tests/store-database.scm b/tests/store-database.scm
index d8f3ce8070..0f44d33191 100644
--- a/tests/store-database.scm
+++ b/tests/store-database.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2017, 2018, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2017-2018, 2020-2021, 2024 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -21,9 +21,9 @@ (define-module (test-store-database)
#:use-module (guix store)
#:use-module (guix store database)
#:use-module (guix build store-copy)
- #:use-module ((guix utils) #:select (call-with-temporary-output-file))
#:use-module ((guix build utils)
- #:select (mkdir-p delete-file-recursively))
+ #:select (mkdir-p delete-file-recursively
+ call-with-temporary-output-file))
#:use-module (srfi srfi-26)
#:use-module (srfi srfi-64))
diff --git a/tests/utils.scm b/tests/utils.scm
index 52f3b58ede..462e43e2b1 100644
--- a/tests/utils.scm
+++ b/tests/utils.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2021, 2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2014 Eric Bavier <bavier@member.fsf.org>
;;; Copyright © 2016 Mathieu Lirzin <mthl@gnu.org>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
@@ -25,6 +25,7 @@
(define-module (test-utils)
#:use-module ((guix config) #:select (%gzip))
#:use-module (guix utils)
+ #:use-module ((guix build utils) #:select (call-with-temporary-output-file))
#:use-module ((guix store) #:select (%store-prefix store-path-package-name))
#:use-module ((guix search-paths) #:select (string-tokenize*))
#:use-module (srfi srfi-1)
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 09/11] ui: Delay use of (guix build syscalls).
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
52b532346c5eb27375681127542bf3c8152723db.1712002698.git.ludo@gnu.org
This ensures (guix build syscalls) is loaded only when needed.

* guix/ui.scm (%text-width): Unconditionally alias ‘*line-width*’.
Remove initialization.
<top level>: Remove code for Guile < 2.2.7.
(package->recutils): Change default #:width to (terminal-columns).

Change-Id: I990a1b5b0f20a6243e47e314d1d3d4f8298b7151
---
guix/ui.scm | 22 ++++------------------
1 file changed, 4 insertions(+), 18 deletions(-)

Toggle diff (42 lines)
diff --git a/guix/ui.scm b/guix/ui.scm
index 34ff210930..d82fa533cc 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -1473,23 +1473,9 @@ (define* (fill-paragraph str width #:optional (column 0))
;;;
(define %text-width
- ;; '*line-width*' was introduced in Guile 2.2.7/3.0.1. On older versions of
- ;; Guile, monkey-patch 'wrap*' below.
- (if (defined? '*line-width*)
- (let ((parameter (fluid->parameter *line-width*)))
- (parameter (terminal-columns))
- parameter)
- (make-parameter (terminal-columns))))
-
-(unless (defined? '*line-width*) ;Guile < 2.2.7
- (set! (@@ (texinfo plain-text) wrap*)
- ;; XXX: Monkey patch this private procedure to let 'package->recutils'
- ;; parameterize the fill of description field correctly.
- (lambda strings
- (let ((indent (fluid-ref (@@ (texinfo plain-text) *indent*))))
- (fill-string (string-concatenate strings)
- #:line-width (%text-width) #:initial-indent indent
- #:subsequent-indent indent)))))
+ ;; '*line-width*' was introduced in Guile 2.2.7/3.0.1. Keep this alias for
+ ;; backward-compatibility and for convenience.
+ (fluid->parameter *line-width*))
(define (texi->plain-text str)
"Return a plain-text representation of texinfo fragment STR."
@@ -1535,7 +1521,7 @@ (define (string->recutils str)
'()
str)))
-(define* (package->recutils p port #:optional (width (%text-width))
+(define* (package->recutils p port #:optional (width (terminal-columns))
#:key
(hyperlinks? (supports-hyperlinks? port))
(extra-fields '())
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 10/11] Autoload (guix build syscalls).
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
8d433a8fb3e0b79dd42f526005acadd54335c019.1712002698.git.ludo@gnu.org
* guix/discovery.scm, guix/git.scm, guix/nar.scm,
guix/scripts.scm, guix/scripts/build.scm: Autoload (guix build syscalls).
* guix/packages.scm: Autoload (guix build utils).

Change-Id: Ia7703b5f46e55fbfadff63b13c35bfe097ce2220
---
guix/discovery.scm | 4 ++--
guix/git.scm | 3 +--
guix/nar.scm | 4 ++--
guix/packages.scm | 5 ++---
guix/scripts.scm | 7 +++++--
guix/scripts/build.scm | 4 ++--
6 files changed, 14 insertions(+), 13 deletions(-)

Toggle diff (120 lines)
diff --git a/guix/discovery.scm b/guix/discovery.scm
index 0edc7fd1ae..2febfcdcb7 100644
--- a/guix/discovery.scm
+++ b/guix/discovery.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2019, 2024 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -20,7 +20,7 @@ (define-module (guix discovery)
#:use-module (guix i18n)
#:use-module (guix modules)
#:use-module (guix combinators)
- #:use-module (guix build syscalls)
+ #:autoload (guix build syscalls) (scandir*)
#:use-module (srfi srfi-1)
#:use-module (ice-9 match)
#:use-module (ice-9 vlist)
diff --git a/guix/git.scm b/guix/git.scm
index 8e1d863976..b22c8ac02a 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -33,8 +33,7 @@ (define-module (guix git)
#:use-module (guix store)
#:use-module (guix utils)
#:use-module (guix records)
- #:use-module ((guix build syscalls)
- #:select (terminal-string-width))
+ #:autoload (guix build syscalls) (terminal-string-width)
#:use-module (guix gexp)
#:autoload (guix git-download)
(git-reference-url git-reference-commit git-reference-recursive?)
diff --git a/guix/nar.scm b/guix/nar.scm
index a817b56007..a50c191f9d 100644
--- a/guix/nar.scm
+++ b/guix/nar.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2016, 2018-2020, 2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2014 Mark H Weaver <mhw@netris.org>
;;;
;;; This file is part of GNU Guix.
@@ -19,7 +19,7 @@
(define-module (guix nar)
#:use-module (guix serialization)
- #:use-module (guix build syscalls)
+ #:autoload (guix build syscalls) (lock-file unlock-file)
#:use-module ((guix build utils)
#:select (delete-file-recursively with-directory-excursion))
diff --git a/guix/packages.scm b/guix/packages.scm
index 930b1a3b0e..bd72b284b1 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012-2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2014, 2015, 2017, 2018, 2019 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2015 Eric Bavier <bavier@member.fsf.org>
;;; Copyright © 2016 Alex Kost <alezost@gmail.com>
@@ -27,8 +27,7 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix packages)
- #:use-module ((guix build utils) #:select (compressor tarball?
- strip-store-file-name))
+ #:autoload (guix build utils) (compressor tarball? strip-store-file-name)
#:use-module (guix utils)
#:use-module (guix records)
#:use-module (guix store)
diff --git a/guix/scripts.scm b/guix/scripts.scm
index 5d11ce7fe9..c4849816ea 100644
--- a/guix/scripts.scm
+++ b/guix/scripts.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2013, 2014, 2015, 2017, 2018, 2019, 2020, 2021, 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2013-2015, 2017-2021, 2021, 2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2014 Deck Pickard <deck.r.pickard@gmail.com>
;;; Copyright © 2015, 2016 Alex Kost <alezost@gmail.com>
;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
@@ -29,7 +29,10 @@ (define-module (guix scripts)
#:use-module (guix packages)
#:use-module (guix derivations)
#:autoload (guix describe) (current-profile-date)
- #:use-module (guix build syscalls)
+ #:autoload (guix build syscalls) (statfs
+ file-system-block-size
+ file-system-blocks-available
+ file-system-block-count)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-19)
#:use-module (srfi srfi-37)
diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm
index 05f022a92e..da4859eeaa 100644
--- a/guix/scripts/build.scm
+++ b/guix/scripts/build.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012-2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2013 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2020 Marius Bakke <mbakke@fastmail.com>
;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
@@ -45,7 +45,7 @@ (define-module (guix scripts build)
#:use-module (guix platform)
#:use-module ((guix status) #:select (with-status-verbosity))
#:use-module ((guix progress) #:select (current-terminal-columns))
- #:use-module ((guix build syscalls) #:select (terminal-columns))
+ #:autoload (guix build syscalls) (terminal-columns)
#:use-module (guix transformations)
#:export (log-url
--
2.41.0
L
L
Ludovic Courtès wrote on 1 Apr 22:25 +0200
[PATCH 11/11] Autoload (gcrypt hash).
(address . 70132@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
fd678dbe1ee7c5d97def65992f3607a117d3fdc0.1712002698.git.ludo@gnu.org
* guix/derivations.scm: Autoload (guix utils) and (gcrypt hash).
* guix/git.scm, guix/store.scm: Autoload (gcrypt hash).

Change-Id: I6145231d41c61f2d8c36e28f29e91074910bdd15
---
guix/derivations.scm | 6 +++---
guix/git.scm | 2 +-
guix/store.scm | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)

Toggle diff (57 lines)
diff --git a/guix/derivations.scm b/guix/derivations.scm
index 9fec7f4f0b..a91c1ae984 100644
--- a/guix/derivations.scm
+++ b/guix/derivations.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012-2021, 2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2021, 2023-2024 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2016, 2017 Mathieu Lirzin <mthl@gnu.org>
;;;
;;; This file is part of GNU Guix.
@@ -32,7 +32,7 @@ (define-module (guix derivations)
#:use-module (ice-9 rdelim)
#:use-module (ice-9 vlist)
#:use-module (guix store)
- #:use-module (guix utils)
+ #:autoload (guix utils) (%current-system string-replace-substring)
#:use-module (guix base16)
#:use-module (guix memoization)
#:use-module (guix combinators)
@@ -40,7 +40,7 @@ (define-module (guix derivations)
#:use-module (guix diagnostics)
#:use-module (guix i18n)
#:use-module (guix monads)
- #:use-module (gcrypt hash)
+ #:autoload (gcrypt hash) (sha256)
#:use-module (guix sets)
#:export (<derivation>
derivation?
diff --git a/guix/git.scm b/guix/git.scm
index b22c8ac02a..d75a301f98 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -27,7 +27,7 @@ (define-module (guix git)
#:use-module (guix i18n)
#:use-module (guix base32)
#:use-module (guix cache)
- #:use-module (gcrypt hash)
+ #:autoload (gcrypt hash) (sha256)
#:use-module ((guix build utils)
#:select (mkdir-p delete-file-recursively invoke/quiet))
#:use-module (guix store)
diff --git a/guix/store.scm b/guix/store.scm
index e808b43ba9..7ff6bf7eec 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -29,7 +29,7 @@ (define-module (guix store)
#:use-module (guix records)
#:use-module (guix base16)
#:use-module (guix base32)
- #:use-module (gcrypt hash)
+ #:autoload (gcrypt hash) (sha256)
#:use-module (guix profiling)
#:autoload (guix build syscalls) (terminal-columns)
#:autoload (guix build utils) (dump-port)
--
2.41.0
L
L
Ludovic Courtès wrote on 15 Apr 23:43 +0200
Re: [bug#70132] [PATCH 00/11] Improve startup time and memory footprint for short-lived commands
(address . 70132-done@debbugs.gnu.org)
87zftumhe8.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (12 lines)
> channels: Use SRFI-71 instead of SRFI-11.
> git: Add ‘repository-info’ and use it in (guix channels).
> channels: Move ‘commit-short-id’ to (guix git).
> git: Add ‘tag->commit’ and use it in (guix channels).
> channels: Autoload (git …) modules.
> guix system: Autoload some more.
> utils: Don’t re-export ‘call-with-temporary-output-file’.
> guix: Delay loading of (gnutls).
> ui: Delay use of (guix build syscalls).
> Autoload (guix build syscalls).
> Autoload (gcrypt hash).

Rebased and pushed as 8a74bb8030f2433155f00332475fc21191ef2952.

Ludo’.
Closed
S
S
Simon Tournier wrote on 16 Apr 00:41 +0200
Re: [bug#70132] [PATCH 01/11] channels: Use SRFI-71 instead of SRFI-11.
87v84iw8nj.fsf@gmail.com
Hi Ludo,

Sorry for being late and out of my curiosity; improving my Guile-fu. ;-)

On lun., 01 avril 2024 at 22:25, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (1 lines)
> - #:use-module (srfi srfi-11)
[...]
Toggle quote (14 lines)
> + #:use-module (srfi srfi-71)

> - (let-values (((channel)
> - (ensure-default-introduction channel))
> - ((checkout commit relation)
> - (update-cached-checkout (channel-url channel)
> - #:ref (channel-reference channel)
> - #:starting-commit starting-commit)))
> + (let ((channel (ensure-default-introduction channel))
> + (checkout commit relation
> + (update-cached-checkout (channel-url channel)
> + #:ref (channel-reference channel)
> + #:starting-commit starting-commit)))

Is it only aesthetic? Or does it change some performance?

Cheers,
simon
S
S
Simon Tournier wrote on 16 Apr 00:45 +0200
Re: [bug#70132] [PATCH 05/11] channels: Autoload (git …) modules.
87r0f6w8hr.fsf@gmail.com
Hi,

On lun., 01 avril 2024 at 22:25, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (41 lines)
> Autoloading Guile-Git is important in cases where (guix channels) is
> used for little more than the <channel> definition. This is the case,
> for example, of ‘guix describe’ or ‘guix shell’.
>
> This reduces from 177 to 121 the number of .go files loaded when
> running:
>
> ./pre-inst-env strace -e openat -o /tmp/log.strace \
> guix describe -p /var/guix/profiles/per-user/$USER/current-guix
> grep 'openat.*\.go.* = [0-9]' < /tmp/log.strace |wc -l
>
> Likewise, it reduces the max RSS (as measured by ‘time -f %M guix
> describe -p …’) from 54 to 37 MiB.
>
> * guix/channels.scm: Autoload (git …) modules.
>
> Change-Id: Ia58a99c865bf0f6fe461a1e71390d075e760f8d6
> ---
> guix/channels.scm | 8 +++++++-
> 1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/guix/channels.scm b/guix/channels.scm
> index 70608561f9..51024dcad4 100644
> --- a/guix/channels.scm
> +++ b/guix/channels.scm
> @@ -20,7 +20,13 @@
> ;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
>
> (define-module (guix channels)
> - #:use-module (git) ;TODO: autoload
> + #:autoload (git commit) (commit-lookup
> + commit-id)
> + #:autoload (git oid) (oid->string
> + string->oid)
> + #:autoload (git object) (object-id)
> + #:autoload (git errors) (GIT_ENOTFOUND)
> + #:autoload (git structs) (git-error-code)
> #:autoload (guix git) (update-cached-checkout
> url+commit->name
> commit-difference

Oh! Awesome!! Thanks for the tricks.

Cheers,
simon
S
S
Simon Tournier wrote on 16 Apr 00:57 +0200
Not receiving Emails from Debbugs (was Re: bug#70132: closed ...)
(name . GNU bug Tracking System)(address . help-debbugs@gnu.org)
87pluqw7xl.fsf_-_@gmail.com
Hi Ludo, all,

Cool! Thanks for the patch set.


Unrelated to this patch set, I am noticing that I do not receive any
email from Debbugs.

For instance, I am CC to the initial submission:

Toggle snippet (11 lines)
From: Ludovic Courtès <ludo@gnu.org>
Subject: [bug#70132] [PATCH 00/11] Improve startup time and memory footprint for short-lived commands
To: 70132@debbugs.gnu.org
Cc: Ludovic Courtès <ludo@gnu.org>, Christopher Baines <guix@cbaines.net>,
Josselin Poiret <dev@jpoiret.xyz>, Ludovic Courtès <ludo@gnu.org>, Mathieu
Othacehe <othacehe@gnu.org>, Ricardo Wurmus <rekado@elephly.net>, Simon
Tournier <zimon.toutoune@gmail.com>, Tobias Geerinckx-Rice <me@tobias.gr>
Date: Mon, 1 Apr 2024 22:22:53 +0200 (2 weeks, 2 hours, 12 minutes ago)
Resent-From: Ludovic Courtès <ludo@gnu.org>

But I have never received this email. And yes, I have checked my spam
folder. ;-) I do not find any trace of it inside my inbox.

I only receive the one closing – sent to 70132-done@debbugs.gnu.org –
where I was specifically CC by you; not X-Debbugs-CC. Similarly for
others.

Similarly I have never received an email back from Debbugs once I sent
to guix-patches then opening 70276.

Hum, weird isn’t it? What could be wrong?

Cheers,
simon




On lun., 15 avril 2024 at 21:44, help-debbugs@gnu.org (GNU bug Tracking System) wrote:
Toggle quote (120 lines)
> Your bug report
>
> #70132: [PATCH 00/11] Improve startup time and memory footprint for
> short-lived commands
>
> which was filed against the guix-patches package, has been closed.
>
> The explanation is attached below, along with your original report.
> If you require more details, please reply to 70132@debbugs.gnu.org.
>
> --
> 70132: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70132
> GNU Bug Tracking System
> Contact help-debbugs@gnu.org with problems
>
> From: Ludovic Courtès <ludo@gnu.org>
> Subject: Re: [bug#70132] [PATCH 00/11] Improve startup time and memory
> footprint for short-lived commands
> To: 70132-done@debbugs.gnu.org
> Cc: Josselin Poiret <dev@jpoiret.xyz>,
> Simon Tournier <zimon.toutoune@gmail.com>, Mathieu Othacehe
> <othacehe@gnu.org>,
> Tobias Geerinckx-Rice <me@tobias.gr>, Ricardo Wurmus <rekado@elephly.net>,
> Christopher Baines <guix@cbaines.net>
> Date: Mon, 15 Apr 2024 23:43:11 +0200 (50 minutes, 47 seconds ago)
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> channels: Use SRFI-71 instead of SRFI-11.
>> git: Add ‘repository-info’ and use it in (guix channels).
>> channels: Move ‘commit-short-id’ to (guix git).
>> git: Add ‘tag->commit’ and use it in (guix channels).
>> channels: Autoload (git …) modules.
>> guix system: Autoload some more.
>> utils: Don’t re-export ‘call-with-temporary-output-file’.
>> guix: Delay loading of (gnutls).
>> ui: Delay use of (guix build syscalls).
>> Autoload (guix build syscalls).
>> Autoload (gcrypt hash).
>
> Rebased and pushed as 8a74bb8030f2433155f00332475fc21191ef2952.
>
> Ludo’.
>
> ----------
>
> From: Ludovic Courtès <ludo@gnu.org>
> Subject: [PATCH 00/11] Improve startup time and memory footprint for
> short-lived commands
> To: guix-patches@gnu.org
> Cc: Ludovic Courtès <ludo@gnu.org>
> Date: Mon, 1 Apr 2024 22:22:53 +0200
> Date: Mon, 1 Apr 2024 22:22:53 +0200 (2 weeks, 2 hours, 11 minutes ago)
>
> Hello!
>
> This is a pretty boring series adding #:autoload in strategic places.
> The goal is to avoid loading tons of modules and shared objects when
> running ‘guix describe’ or ‘guix shell’ on a cache hit.
>
> There’s a tiny bit of reshuffling to make that easier:
>
> • Bits moved from (guix channels) to (guix git);
>
> • ‘call-with-temporary-output-file’ no longer re-exported by
> (guix utils).
>
> I measured the impact with things like:
>
> strace -e openat -o /tmp/log.strace guix describe
> grep 'openat.*\.go' </tmp/log.strace | wc -l
>
> and also checking specifically whether things like Guile-Git,
> Guile-Gcrypt, and Guile-GnuTLS were being loaded.
>
> Feedback welcome!
>
> Ludo’.
>
> Ludovic Courtès (11):
> channels: Use SRFI-71 instead of SRFI-11.
> git: Add ‘repository-info’ and use it in (guix channels).
> channels: Move ‘commit-short-id’ to (guix git).
> git: Add ‘tag->commit’ and use it in (guix channels).
> channels: Autoload (git …) modules.
> guix system: Autoload some more.
> utils: Don’t re-export ‘call-with-temporary-output-file’.
> guix: Delay loading of (gnutls).
> ui: Delay use of (guix build syscalls).
> Autoload (guix build syscalls).
> Autoload (gcrypt hash).
>
> guix/channels.scm | 56 +++++++++++++++++----------------------
> guix/derivations.scm | 6 ++---
> guix/discovery.scm | 4 +--
> guix/download.scm | 4 +--
> guix/git.scm | 52 +++++++++++++++++++++++++++---------
> guix/import/hackage.scm | 4 +--
> guix/import/hexpm.scm | 7 ++---
> guix/import/opam.scm | 7 ++---
> guix/import/pypi.scm | 5 ++--
> guix/nar.scm | 4 +--
> guix/packages.scm | 5 ++--
> guix/scripts.scm | 7 +++--
> guix/scripts/build.scm | 4 +--
> guix/scripts/describe.scm | 4 +--
> guix/scripts/system.scm | 12 ++++++---
> guix/store.scm | 15 ++++++-----
> guix/ui.scm | 22 +++------------
> guix/utils.scm | 17 ++++++------
> tests/cpio.scm | 6 ++---
> tests/egg.scm | 5 ++--
> tests/opam.scm | 5 ++--
> tests/publish.scm | 5 ++--
> tests/store-database.scm | 6 ++---
> tests/utils.scm | 3 ++-
> 24 files changed, 143 insertions(+), 122 deletions(-)
>
>
> base-commit: 16c713083516e60b5ae30b3a8b49d5bf8d4cadc3
L
L
Ludovic Courtès wrote on 16 Apr 18:42 +0200
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87ttk1mf8b.fsf@gnu.org
Hi!

Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (18 lines)
> Unrelated to this patch set, I am noticing that I do not receive any
> email from Debbugs.
>
> For instance, I am CC to the initial submission:
>
> From: Ludovic Courtès <ludo@gnu.org>
> Subject: [bug#70132] [PATCH 00/11] Improve startup time and memory footprint for short-lived commands
> To: 70132@debbugs.gnu.org
> Cc: Ludovic Courtès <ludo@gnu.org>, Christopher Baines <guix@cbaines.net>,
> Josselin Poiret <dev@jpoiret.xyz>, Ludovic Courtès <ludo@gnu.org>, Mathieu
> Othacehe <othacehe@gnu.org>, Ricardo Wurmus <rekado@elephly.net>, Simon
> Tournier <zimon.toutoune@gmail.com>, Tobias Geerinckx-Rice <me@tobias.gr>
> Date: Mon, 1 Apr 2024 22:22:53 +0200 (2 weeks, 2 hours, 12 minutes ago)
> Resent-From: Ludovic Courtès <ludo@gnu.org>
>
> But I have never received this email. And yes, I have checked my spam
> folder. ;-) I do not find any trace of it inside my inbox.

Could it be a gmail thing? I heard some of the maintainers were not
receiving messages sent to guix-maintainers@gnu.org due to complicated
gmail-related issues.

(I’m Cc’ing two addresses of yours here, just to see.)

Ludo’.
?