[PATCH 0/4] Tests for 'guix refresh'

  • Done
  • quality assurance status badge
Details
2 participants
  • Hartmut Goebel
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 3 Jan 2023 17:44
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103164439.16477-1-ludo@gnu.org
Hi!

Following the discussion at
I thought it’s about time to have tests for the ‘guix refresh’ CLI
(so far we had tests for importers, for (guix gnu-maintenance), and
for (guix upstream), but that did not cover the logic in ‘guix refresh’
itself as we’ve seen).

The strategy is to define a “pseudo updater”: the test defines an
environment variable that specifies which package updates the updater
should simulate, and then we go on using ‘guix refresh’.

Thoughts?

Ludo’.

Ludovic Courtès (4):
gnu-maintenance: Factorize 'false-if-networking-failure'.
gnu-maintenance: 'gnu' and 'gnu-ftp' predicates catch networking
errors.
import: stackage: Catch networking errors in predicate.
refresh: Add CLI tests.

Makefile.am | 4 +-
guix/gnu-maintenance.scm | 31 ++++-------
guix/import/stackage.scm | 15 ++---
guix/import/test.scm | 88 +++++++++++++++++++++++++++++
guix/import/utils.scm | 24 +++++++-
tests/guix-refresh.sh | 117 +++++++++++++++++++++++++++++++++++++++
6 files changed, 251 insertions(+), 28 deletions(-)
create mode 100644 guix/import/test.scm
create mode 100644 tests/guix-refresh.sh


base-commit: 473692b812b4ab4267d9bddad0fb27787d2112ff
--
2.38.1
L
L
Ludovic Courtès wrote on 3 Jan 2023 18:03
[PATCH 2/4] gnu-maintenance: 'gnu' and 'gnu-ftp' predicates catch networking errors.
(address . 60520@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103170319.16637-2-ludo@gnu.org
Previously, in a networking-less environment such as 'guix shell -C -D
guix', 'guix refresh --list-updaters' would crash due to a
'gettaddrinfo-error' exception in these predicates.

* guix/gnu-maintenance.scm (%gnu-updater)[pred]: Wrap in
'false-if-networking-error'.
(%gnu-ftp-updater)[pred]: Likewise.
---
guix/gnu-maintenance.scm | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)

Toggle diff (28 lines)
diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm
index 0418c297f2..0aa70243b5 100644
--- a/guix/gnu-maintenance.scm
+++ b/guix/gnu-maintenance.scm
@@ -883,7 +883,8 @@ (define %gnu-updater
(upstream-updater
(name 'gnu)
(description "Updater for GNU packages")
- (pred gnu-hosted?)
+ (pred (lambda (package)
+ (false-if-networking-error (gnu-hosted? package))))
(import import-gnu-release)))
(define %gnu-ftp-updater
@@ -893,8 +894,9 @@ (define %gnu-ftp-updater
(name 'gnu-ftp)
(description "Updater for GNU packages only available via FTP")
(pred (lambda (package)
- (and (not (gnu-hosted? package))
- (pure-gnu-package? package))))
+ (false-if-networking-error
+ (and (not (gnu-hosted? package))
+ (pure-gnu-package? package)))))
(import import-release*)))
(define %savannah-updater
--
2.38.1
L
L
Ludovic Courtès wrote on 3 Jan 2023 18:03
[PATCH 3/4] import: stackage: Catch networking errors in predicate.
(address . 60520@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103170319.16637-3-ludo@gnu.org
* guix/import/stackage.scm (stackage-lts-package?): Wrap body in
'false-if-networking-error'.
---
guix/import/stackage.scm | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)

Toggle diff (35 lines)
diff --git a/guix/import/stackage.scm b/guix/import/stackage.scm
index 70d3e271f4..c0284e48a4 100644
--- a/guix/import/stackage.scm
+++ b/guix/import/stackage.scm
@@ -3,7 +3,7 @@
;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;; Copyright © 2021 Xinglu Chem <public@yoctocell.xyz>
-;;; Copyright © 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2021, 2023, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2022 Hartmut Goebel <h.goebel@crazy-compilers.com>
;;;
;;; This file is part of GNU Guix.
@@ -170,12 +170,13 @@ (define latest-lts-release
(define (stackage-lts-package? package)
"Return whether PACKAGE is available on the default Stackage LTS release."
(and (hackage-package? package)
- (let ((packages (stackage-lts-packages
- (stackage-lts-info-fetch %default-lts-version)))
- (hackage-name (guix-package->hackage-name package)))
- (find (lambda (package)
- (string=? (stackage-package-name package) hackage-name))
- packages))))
+ (false-if-networking-error
+ (let ((packages (stackage-lts-packages
+ (stackage-lts-info-fetch %default-lts-version)))
+ (hackage-name (guix-package->hackage-name package)))
+ (find (lambda (package)
+ (string=? (stackage-package-name package) hackage-name))
+ packages)))))
(define %stackage-updater
(upstream-updater
--
2.38.1
L
L
Ludovic Courtès wrote on 3 Jan 2023 18:03
[PATCH 1/4] gnu-maintenance: Factorize 'false-if-networking-failure'.
(address . 60520@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103170319.16637-1-ludo@gnu.org
* guix/import/utils.scm (call-with-networking-exception-handler): New
procedure.
(false-if-networking-error): New macro.
* guix/gnu-maintenance.scm (import-html-updatable-release): Use it
instead of inline code.
---
guix/gnu-maintenance.scm | 23 +++++++----------------
guix/import/utils.scm | 24 +++++++++++++++++++++++-
2 files changed, 30 insertions(+), 17 deletions(-)

Toggle diff (99 lines)
diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm
index 8e60e52ea0..0418c297f2 100644
--- a/guix/gnu-maintenance.scm
+++ b/guix/gnu-maintenance.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2010-2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2010-2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2012, 2013 Nikita Karetnikov <nikita@karetnikov.org>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
@@ -43,6 +43,7 @@ (define-module (guix gnu-maintenance)
#:use-module (guix records)
#:use-module (guix upstream)
#:use-module (guix packages)
+ #:autoload (guix import utils) (false-if-networking-error)
#:autoload (zlib) (call-with-gzip-input-port)
#:autoload (htmlprag) (html->sxml) ;from Guile-Lib
#:export (gnu-package-name
@@ -871,21 +872,11 @@ (define* (import-html-updatable-release package #:key (version #f))
""
(dirname (uri-path uri))))
(package (package-upstream-name package)))
- (catch #t
- (lambda ()
- (guard (c ((http-get-error? c) #f))
- (import-html-release package
- #:version version
- #:base-url base
- #:directory directory)))
- (lambda (key . args)
- ;; Return false and move on upon connection failures and bogus HTTP
- ;; servers.
- (unless (memq key '(gnutls-error tls-certificate-error
- system-error getaddrinfo-error
- bad-header bad-header-component))
- (apply throw key args))
- #f))))
+ (false-if-networking-error
+ (import-html-release package
+ #:version version
+ #:base-url base
+ #:directory directory))))
(define %gnu-updater
;; This is for everything at ftp.gnu.org.
diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 41311cb86e..72795d2c61 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012, 2013, 2018, 2019, 2020, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
;;; Copyright © 2016 David Craven <david@craven.ch>
;;; Copyright © 2017, 2019, 2020, 2022 Ricardo Wurmus <rekado@elephly.net>
@@ -54,10 +54,12 @@ (define-module (guix import utils)
#:use-module (srfi srfi-9)
#:use-module (srfi srfi-11)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-34)
#:use-module (srfi srfi-71)
#:export (factorize-uri
flatten
+ false-if-networking-error
url-fetch
guix-hash-url
@@ -122,6 +124,26 @@ (define (flatten lst)
(cons elem memo)))
'() lst))
+(define (call-with-networking-exception-handler thunk)
+ "Invoke THUNK, returning #f if one of the usual networking exception is
+thrown."
+ (catch #t
+ (lambda ()
+ (guard (c ((http-get-error? c) #f))
+ (thunk)))
+ (lambda (key . args)
+ ;; Return false and move on upon connection failures and bogus HTTP
+ ;; servers.
+ (unless (memq key '(gnutls-error tls-certificate-error
+ system-error getaddrinfo-error
+ bad-header bad-header-component))
+ (apply throw key args))
+ #f)))
+
+(define-syntax-rule (false-if-networking-error exp)
+ "Evaluate EXP, returning #f if a networking-related exception is thrown."
+ (call-with-networking-exception-handler (lambda () exp)))
+
(define (url-fetch url file-name)
"Save the contents of URL to FILE-NAME. Return #f on failure."
(parameterize ((current-output-port (current-error-port)))
--
2.38.1
L
L
Ludovic Courtès wrote on 3 Jan 2023 18:03
[PATCH 4/4] refresh: Add CLI tests.
(address . 60520@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103170319.16637-4-ludo@gnu.org
* guix/import/test.scm, tests/guix-refresh.sh: New files.
* Makefile.am (MODULES, SH_TESTS): Add them.
---
Makefile.am | 4 +-
guix/import/test.scm | 88 +++++++++++++++++++++++++++++++
tests/guix-refresh.sh | 117 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 208 insertions(+), 1 deletion(-)
create mode 100644 guix/import/test.scm
create mode 100644 tests/guix-refresh.sh

Toggle diff (246 lines)
diff --git a/Makefile.am b/Makefile.am
index 8b026b6da6..d9d23eec88 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,5 +1,5 @@
# GNU Guix --- Functional package management for GNU
-# Copyright © 2012-2022 Ludovic Courtès <ludo@gnu.org>
+# Copyright © 2012-2023 Ludovic Courtès <ludo@gnu.org>
# Copyright © 2013 Andreas Enge <andreas@enge.fr>
# Copyright © 2015, 2017 Alex Kost <alezost@gmail.com>
# Copyright © 2016, 2018 Mathieu Lirzin <mthl@gnu.org>
@@ -289,6 +289,7 @@ MODULES = \
guix/import/print.scm \
guix/import/pypi.scm \
guix/import/stackage.scm \
+ guix/import/test.scm \
guix/import/texlive.scm \
guix/import/utils.scm \
guix/scripts.scm \
@@ -595,6 +596,7 @@ SH_TESTS = \
tests/guix-authenticate.sh \
tests/guix-environment.sh \
tests/guix-environment-container.sh \
+ tests/guix-refresh.sh \
tests/guix-shell.sh \
tests/guix-shell-export-manifest.sh \
tests/guix-graph.sh \
diff --git a/guix/import/test.scm b/guix/import/test.scm
new file mode 100644
index 0000000000..767dcd5b61
--- /dev/null
+++ b/guix/import/test.scm
@@ -0,0 +1,88 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2023 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import test)
+ #:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
+ #:use-module (web uri)
+ #:use-module (guix packages)
+ #:use-module (guix upstream)
+ #:use-module ((guix utils) #:select (version-prefix?))
+ #:use-module (ice-9 vlist)
+ #:use-module (ice-9 match)
+ #:export (%test-updater))
+
+;;; Commentary:
+;;;
+;;; This module defines a pseudo updater whose sole purpose is to allow
+;;; testing of the whole 'guix refresh' command.
+;;;
+;;; Code:
+
+(define test-target-version
+ ;; VHash that maps package names to version/URL tuples.
+ (make-parameter
+ (or (and=> (getenv "GUIX_TEST_UPDATER_TARGETS")
+ (lambda (str)
+ (alist->vhash (call-with-input-string str read))))
+ vlist-null)))
+
+(define (available-updates package)
+ "Return the list of available <upstream-source> records for PACKAGE."
+ (vhash-fold* (lambda (version+updates result)
+ (match version+updates
+ ((version (updates ...))
+ (if (version-prefix? version
+ (package-version package))
+ (append (map (match-lambda
+ ((version url)
+ (upstream-source
+ (package (package-name package))
+ (version version)
+ (urls (list url)))))
+ updates)
+ result)
+ result))))
+ '()
+ (package-name package)
+ (test-target-version)))
+
+(define (test-package? package)
+ "Return true if PACKAGE has pseudo updates available."
+ (and (not (vlist-null? (test-target-version))) ;cheap test
+ (pair? (available-updates package))))
+
+(define* (import-release package #:key (version #f))
+ "Return the <upstream-source> record denoting either the latest version of
+PACKAGE or VERSION."
+ (match (available-updates package)
+ (() #f)
+ ((sources ...)
+ (if version
+ (find (lambda (source)
+ (string=? (upstream-source-version source)
+ version))
+ sources)
+ (first sources)))))
+
+(define %test-updater
+ (upstream-updater
+ (name 'test)
+ (description "Pseudo updater for testing purposes.")
+ (pred test-package?)
+ (import import-release)))
diff --git a/tests/guix-refresh.sh b/tests/guix-refresh.sh
new file mode 100644
index 0000000000..de094a6c1d
--- /dev/null
+++ b/tests/guix-refresh.sh
@@ -0,0 +1,117 @@
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2023 Ludovic Courtès <ludo@gnu.org>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+#
+# Test the 'guix refresh' command-line utility.
+#
+
+guix refresh --version
+
+manifest="t-guix-refresh-manifest-$$.scm"
+module_dir="t-guix-refresh-modules-$$"
+trap 'rm -f "$manifest"; rm -rf "$module_dir"' EXIT
+
+# Tell the 'test' updater what to simulate.
+export GUIX_TEST_UPDATER_TARGETS
+idutils_version="$(guix package -A ^idutils$ | cut -f2)"
+GUIX_TEST_UPDATER_TARGETS='
+ (("guile" "3" (("12.5" "file:///dev/null")
+ ("1.6.4" "file:///dev/null")))
+ ("libreoffice" "" (("1.0" "file:///dev/null")))
+ ("idutils" "" (("'$idutils_version'" "file:///dev/null")))
+ ("the-test-package" "" (("5.5" "file://'$PWD/$module_dir'/source"))))'
+
+# No newer version available.
+! guix refresh -t test idutils
+case "$(guix refresh -t test idutils 2>&1)" in
+ *"$idutils_version"*"already the latest version"*) true;;
+ *) false;;
+esac
+! guix refresh -t test libreoffice
+case "$(guix refresh -t test libreoffice 2>&1)" in
+ *"greater than the latest known version"*"1.0"*) true;;
+ *) false;;
+esac
+
+# Various ways to specify packages.
+cat > "$manifest" <<EOF
+(specifications->manifest (list "guile@3.0"))
+EOF
+default_IFS="$IFS"
+IFS=X
+for spec in "guile" \
+ "guile@3.0" \
+ "-eX(@ (gnu packages guile) guile-3.0)" \
+ "-mX$manifest" \
+ "guileX-r" \
+ "-sXcore"
+do
+ guix refresh -t test $spec
+ case "$(guix refresh -t test $spec 2>&1)" in
+ *"would be upgraded"*"12.5"*)
+ true;;
+ *)
+ false;;
+ esac
+done
+IFS="$default_IFS"
+
+# Actually updating.
+mkdir "$module_dir"
+echo hello > "$module_dir/source"
+cat > "$module_dir/sample.scm"<<EOF
+(define-module (sample)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (gnu packages base))
+
+(define-public my-thing
+ (package
+ (inherit hello)
+ (name "the-test-package")
+ (version "4.3")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append "mirror://gnu/hello/hello-" version
+ ".tar.gz"))
+ (sha256
+ (base32
+ "086vqwk2wl8zfs47sq2xpjc9k066ilmb8z6dn0q6ymwjzlm196cd"))))))
+EOF
+guix refresh -t test -L "$module_dir" the-test-package
+guix refresh -t test -L "$module_dir" the-test-package -u
+grep 'version "5.5"' "$module_dir/sample.scm"
+grep "$(guix hash -H sha256 -f nix-base32 "$module_dir/source")" "$module_dir/sample.scm"
+
+# Specifying a target version.
+! guix refresh -t test guile=2.0.0
+case "$(guix refresh -t test guile=2.0.0 2>&1)" in
+ *"failed to find"*"2.0.0"*) true;;
+ *) false;;
+esac
+for spec in "guile=1.6.4" "guile@3=1.6.4"
+do
+ guix refresh -t test "$spec"
+ case "$(guix refresh -t test "$spec" 2>&1)" in
+ *"would be downgraded"*"1.6.4"*) true;;
+ *) false;;
+ esac
+done
+
+# Listing updaters. This should work whether or not networking is available.
+guix refresh --list-updaters
--
2.38.1
H
H
Hartmut Goebel wrote on 3 Jan 2023 19:51
Re: [bug#60520] [PATCH 0/4] Tests for 'guix refresh'
(name . Ricardo Wurmus)(address . rekado@elephly.net)
f7b927ef-e822-301b-1f47-0cbeca89e979@crazy-compilers.com
Am 03.01.23 um 17:44 schrieb Ludovic Courtès:
Toggle quote (2 lines)
> Thoughts?

Thanks for implementing this,  It was terribly missing.

Two mini remarks:

+IFS=X

What about using a symbol like underscore?

+ "guileX-r" \

For symetry I'd write "-r guile".

--
Regards
Hartmut Goebel

| Hartmut Goebel |h.goebel@crazy-compilers.com |
|www.crazy-compilers.com | compilers which you thought are impossible |
Attachment: file
L
L
Ludovic Courtès wrote on 8 Jan 2023 16:16
Re: bug#60520: [PATCH 0/4] Tests for 'guix refresh'
(name . Hartmut Goebel)(address . h.goebel@crazy-compilers.com)
875ydhukdm.fsf_-_@gnu.org
Hi,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

Toggle quote (10 lines)
> Two mini remarks:
>
> +IFS=X
>
> What about using a symbol like underscore?
>
> + "guileX-r" \
>
> For symetry I'd write "-r guile".

Good ideas. I made those changes and pushed:

04f247be81 * refresh: Add CLI tests.
2e9c0e1ff6 * import: stackage: Catch networking errors in predicate.
472dd29f37 * gnu-maintenance: 'gnu' and 'gnu-ftp' predicates catch networking errors.
f3edf29c67 * gnu-maintenance: Factorize 'false-if-networking-failure'.

Thanks,
Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 60520@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 60520
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch