[PATCH 0/4] Tests for 'guix refresh'

  • Done
  • quality assurance status badge
Details
2 participants
  • Hartmut Goebel
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 3 Jan 2023 17:44
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103164439.16477-1-ludo@gnu.org
Hi!

Following the discussion at
I thought it’s about time to have tests for the ‘guix refresh’ CLI
(so far we had tests for importers, for (guix gnu-maintenance), and
for (guix upstream), but that did not cover the logic in ‘guix refresh’
itself as we’ve seen).

The strategy is to define a “pseudo updater”: the test defines an
environment variable that specifies which package updates the updater
should simulate, and then we go on using ‘guix refresh’.

Thoughts?

Ludo’.

Ludovic Courtès (4):
gnu-maintenance: Factorize 'false-if-networking-failure'.
gnu-maintenance: 'gnu' and 'gnu-ftp' predicates catch networking
errors.
import: stackage: Catch networking errors in predicate.
refresh: Add CLI tests.

Makefile.am | 4 +-
guix/gnu-maintenance.scm | 31 ++++-------
guix/import/stackage.scm | 15 ++---
guix/import/test.scm | 88 +++++++++++++++++++++++++++++
guix/import/utils.scm | 24 +++++++-
tests/guix-refresh.sh | 117 +++++++++++++++++++++++++++++++++++++++
6 files changed, 251 insertions(+), 28 deletions(-)
create mode 100644 guix/import/test.scm
create mode 100644 tests/guix-refresh.sh


base-commit: 473692b812b4ab4267d9bddad0fb27787d2112ff
--
2.38.1
L
L
Ludovic Courtès wrote on 3 Jan 2023 18:03
[PATCH 2/4] gnu-maintenance: 'gnu' and 'gnu-ftp' predicates catch networking errors.
(address . 60520@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103170319.16637-2-ludo@gnu.org
Previously, in a networking-less environment such as 'guix shell -C -D
guix', 'guix refresh --list-updaters' would crash due to a
'gettaddrinfo-error' exception in these predicates.

* guix/gnu-maintenance.scm (%gnu-updater)[pred]: Wrap in
'false-if-networking-error'.
(%gnu-ftp-updater)[pred]: Likewise.
---
guix/gnu-maintenance.scm | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)

Toggle diff (28 lines)
diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm
index 0418c297f2..0aa70243b5 100644
--- a/guix/gnu-maintenance.scm
+++ b/guix/gnu-maintenance.scm
@@ -883,7 +883,8 @@ (define %gnu-updater
(upstream-updater
(name 'gnu)
(description "Updater for GNU packages")
- (pred gnu-hosted?)
+ (pred (lambda (package)
+ (false-if-networking-error (gnu-hosted? package))))
(import import-gnu-release)))
(define %gnu-ftp-updater
@@ -893,8 +894,9 @@ (define %gnu-ftp-updater
(name 'gnu-ftp)
(description "Updater for GNU packages only available via FTP")
(pred (lambda (package)
- (and (not (gnu-hosted? package))
- (pure-gnu-package? package))))
+ (false-if-networking-error
+ (and (not (gnu-hosted? package))
+ (pure-gnu-package? package)))))
(import import-release*)))
(define %savannah-updater
--
2.38.1
L
L
Ludovic Courtès wrote on 3 Jan 2023 18:03
[PATCH 3/4] import: stackage: Catch networking errors in predicate.
(address . 60520@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103170319.16637-3-ludo@gnu.org
* guix/import/stackage.scm (stackage-lts-package?): Wrap body in
'false-if-networking-error'.
---
guix/import/stackage.scm | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)

Toggle diff (35 lines)
diff --git a/guix/import/stackage.scm b/guix/import/stackage.scm
index 70d3e271f4..c0284e48a4 100644
--- a/guix/import/stackage.scm
+++ b/guix/import/stackage.scm
@@ -3,7 +3,7 @@
;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
;;; Copyright © 2021 Xinglu Chem <public@yoctocell.xyz>
-;;; Copyright © 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2021, 2023, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2022 Hartmut Goebel <h.goebel@crazy-compilers.com>
;;;
;;; This file is part of GNU Guix.
@@ -170,12 +170,13 @@ (define latest-lts-release
(define (stackage-lts-package? package)
"Return whether PACKAGE is available on the default Stackage LTS release."
(and (hackage-package? package)
- (let ((packages (stackage-lts-packages
- (stackage-lts-info-fetch %default-lts-version)))
- (hackage-name (guix-package->hackage-name package)))
- (find (lambda (package)
- (string=? (stackage-package-name package) hackage-name))
- packages))))
+ (false-if-networking-error
+ (let ((packages (stackage-lts-packages
+ (stackage-lts-info-fetch %default-lts-version)))
+ (hackage-name (guix-package->hackage-name package)))
+ (find (lambda (package)
+ (string=? (stackage-package-name package) hackage-name))
+ packages)))))
(define %stackage-updater
(upstream-updater
--
2.38.1
L
L
Ludovic Courtès wrote on 3 Jan 2023 18:03
[PATCH 1/4] gnu-maintenance: Factorize 'false-if-networking-failure'.
(address . 60520@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103170319.16637-1-ludo@gnu.org
* guix/import/utils.scm (call-with-networking-exception-handler): New
procedure.
(false-if-networking-error): New macro.
* guix/gnu-maintenance.scm (import-html-updatable-release): Use it
instead of inline code.
---
guix/gnu-maintenance.scm | 23 +++++++----------------
guix/import/utils.scm | 24 +++++++++++++++++++++++-
2 files changed, 30 insertions(+), 17 deletions(-)

Toggle diff (99 lines)
diff --git a/guix/gnu-maintenance.scm b/guix/gnu-maintenance.scm
index 8e60e52ea0..0418c297f2 100644
--- a/guix/gnu-maintenance.scm
+++ b/guix/gnu-maintenance.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2010-2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2010-2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2012, 2013 Nikita Karetnikov <nikita@karetnikov.org>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
@@ -43,6 +43,7 @@ (define-module (guix gnu-maintenance)
#:use-module (guix records)
#:use-module (guix upstream)
#:use-module (guix packages)
+ #:autoload (guix import utils) (false-if-networking-error)
#:autoload (zlib) (call-with-gzip-input-port)
#:autoload (htmlprag) (html->sxml) ;from Guile-Lib
#:export (gnu-package-name
@@ -871,21 +872,11 @@ (define* (import-html-updatable-release package #:key (version #f))
""
(dirname (uri-path uri))))
(package (package-upstream-name package)))
- (catch #t
- (lambda ()
- (guard (c ((http-get-error? c) #f))
- (import-html-release package
- #:version version
- #:base-url base
- #:directory directory)))
- (lambda (key . args)
- ;; Return false and move on upon connection failures and bogus HTTP
- ;; servers.
- (unless (memq key '(gnutls-error tls-certificate-error
- system-error getaddrinfo-error
- bad-header bad-header-component))
- (apply throw key args))
- #f))))
+ (false-if-networking-error
+ (import-html-release package
+ #:version version
+ #:base-url base
+ #:directory directory))))
(define %gnu-updater
;; This is for everything at ftp.gnu.org.
diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 41311cb86e..72795d2c61 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012, 2013, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012, 2013, 2018, 2019, 2020, 2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
;;; Copyright © 2016 David Craven <david@craven.ch>
;;; Copyright © 2017, 2019, 2020, 2022 Ricardo Wurmus <rekado@elephly.net>
@@ -54,10 +54,12 @@ (define-module (guix import utils)
#:use-module (srfi srfi-9)
#:use-module (srfi srfi-11)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-34)
#:use-module (srfi srfi-71)
#:export (factorize-uri
flatten
+ false-if-networking-error
url-fetch
guix-hash-url
@@ -122,6 +124,26 @@ (define (flatten lst)
(cons elem memo)))
'() lst))
+(define (call-with-networking-exception-handler thunk)
+ "Invoke THUNK, returning #f if one of the usual networking exception is
+thrown."
+ (catch #t
+ (lambda ()
+ (guard (c ((http-get-error? c) #f))
+ (thunk)))
+ (lambda (key . args)
+ ;; Return false and move on upon connection failures and bogus HTTP
+ ;; servers.
+ (unless (memq key '(gnutls-error tls-certificate-error
+ system-error getaddrinfo-error
+ bad-header bad-header-component))
+ (apply throw key args))
+ #f)))
+
+(define-syntax-rule (false-if-networking-error exp)
+ "Evaluate EXP, returning #f if a networking-related exception is thrown."
+ (call-with-networking-exception-handler (lambda () exp)))
+
(define (url-fetch url file-name)
"Save the contents of URL to FILE-NAME. Return #f on failure."
(parameterize ((current-output-port (current-error-port)))
--
2.38.1
L
L
Ludovic Courtès wrote on 3 Jan 2023 18:03
[PATCH 4/4] refresh: Add CLI tests.
(address . 60520@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20230103170319.16637-4-ludo@gnu.org
* guix/import/test.scm, tests/guix-refresh.sh: New files.
* Makefile.am (MODULES, SH_TESTS): Add them.
---
Makefile.am | 4 +-
guix/import/test.scm | 88 +++++++++++++++++++++++++++++++
tests/guix-refresh.sh | 117 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 208 insertions(+), 1 deletion(-)
create mode 100644 guix/import/test.scm
create mode 100644 tests/guix-refresh.sh

Toggle diff (246 lines)
diff --git a/Makefile.am b/Makefile.am
index 8b026b6da6..d9d23eec88 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,5 +1,5 @@
# GNU Guix --- Functional package management for GNU
-# Copyright © 2012-2022 Ludovic Courtès <ludo@gnu.org>
+# Copyright © 2012-2023 Ludovic Courtès <ludo@gnu.org>
# Copyright © 2013 Andreas Enge <andreas@enge.fr>
# Copyright © 2015, 2017 Alex Kost <alezost@gmail.com>
# Copyright © 2016, 2018 Mathieu Lirzin <mthl@gnu.org>
@@ -289,6 +289,7 @@ MODULES = \
guix/import/print.scm \
guix/import/pypi.scm \
guix/import/stackage.scm \
+ guix/import/test.scm \
guix/import/texlive.scm \
guix/import/utils.scm \
guix/scripts.scm \
@@ -595,6 +596,7 @@ SH_TESTS = \
tests/guix-authenticate.sh \
tests/guix-environment.sh \
tests/guix-environment-container.sh \
+ tests/guix-refresh.sh \
tests/guix-shell.sh \
tests/guix-shell-export-manifest.sh \
tests/guix-graph.sh \
diff --git a/guix/import/test.scm b/guix/import/test.scm
new file mode 100644
index 0000000000..767dcd5b61
--- /dev/null
+++ b/guix/import/test.scm
@@ -0,0 +1,88 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2023 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import test)
+ #:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
+ #:use-module (web uri)
+ #:use-module (guix packages)
+ #:use-module (guix upstream)
+ #:use-module ((guix utils) #:select (version-prefix?))
+ #:use-module (ice-9 vlist)
+ #:use-module (ice-9 match)
+ #:export (%test-updater))
+
+;;; Commentary:
+;;;
+;;; This module defines a pseudo updater whose sole purpose is to allow
+;;; testing of the whole 'guix refresh' command.
+;;;
+;;; Code:
+
+(define test-target-version
+ ;; VHash that maps package names to version/URL tuples.
+ (make-parameter
+ (or (and=> (getenv "GUIX_TEST_UPDATER_TARGETS")
+ (lambda (str)
+ (alist->vhash (call-with-input-string str read))))
+ vlist-null)))
+
+(define (available-updates package)
+ "Return the list of available <upstream-source> records for PACKAGE."
+ (vhash-fold* (lambda (version+updates result)
+ (match version+updates
+ ((version (updates ...))
+ (if (version-prefix? version
+ (package-version package))
+ (append (map (match-lambda
+ ((version url)
+ (upstream-source
+ (package (package-name package))
+ (version version)
+ (urls (list url)))))
+ updates)
+ result)
+ result))))
+ '()
+ (package-name package)
+ (test-target-version)))
+
+(define (test-package? package)
+ "Return true if PACKAGE has pseudo updates available."
+ (and (not (vlist-null? (test-target-version))) ;cheap test
+ (pair? (available-updates package))))
+
+(define* (import-release package #:key (version #f))
+ "Return the <upstream-source> record denoting either the latest version of
+PACKAGE or VERSION."
+ (match (available-updates package)
+ (() #f)
+ ((sources ...)
+ (if version
+ (find (lambda (source)
+ (string=? (upstream-source-version source)
+ version))
+ sources)
+ (first sources)))))
+
+(define %test-updater
+ (upstream-updater
+ (name 'test)
+ (description "Pseudo updater for testing purposes.")
+ (pred test-package?)
+ (import import-release)))
diff --git a/tests/guix-refresh.sh b/tests/guix-refresh.sh
new file mode 100644
index 0000000000..de094a6c1d
--- /dev/null
+++ b/tests/guix-refresh.sh
@@ -0,0 +1,117 @@
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2023 Ludovic Courtès <ludo@gnu.org>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+#
+# Test the 'guix refresh' command-line utility.
+#
+
+guix refresh --version
+
+manifest="t-guix-refresh-manifest-$$.scm"
+module_dir="t-guix-refresh-modules-$$"
+trap 'rm -f "$manifest"; rm -rf "$module_dir"' EXIT
+
+# Tell the 'test' updater what to simulate.
+export GUIX_TEST_UPDATER_TARGETS
+idutils_version="$(guix package -A ^idutils$ | cut -f2)"
+GUIX_TEST_UPDATER_TARGETS='
+ (("guile" "3" (("12.5" "file:///dev/null")
+ ("1.6.4" "file:///dev/null")))
+ ("libreoffice" "" (("1.0" "file:///dev/null")))
+ ("idutils" "" (("'$idutils_version'" "file:///dev/null")))
+ ("the-test-package" "" (("5.5" "file://'$PWD/$module_dir'/source"))))'
+
+# No newer version available.
+! guix refresh -t test idutils
+case "$(guix refresh -t test idutils 2>&1)" in
+ *"$idutils_version"*"already the latest version"*) true;;
+ *) false;;
+esac
+! guix refresh -t test libreoffice
+case "$(guix refresh -t test libreoffice 2>&1)" in
+ *"greater than the latest known version"*"1.0"*) true;;
+ *) false;;
+esac
+
+# Various ways to specify packages.
+cat > "$manifest" <<EOF
+(specifications->manifest (list "guile@3.0"))
+EOF
+default_IFS="$IFS"
+IFS=X
+for spec in "guile" \
+ "guile@3.0" \
+ "-eX(@ (gnu packages guile) guile-3.0)" \
+ "-mX$manifest" \
+ "guileX-r" \
+ "-sXcore"
+do
+ guix refresh -t test $spec
+ case "$(guix refresh -t test $spec 2>&1)" in
+ *"would be upgraded"*"12.5"*)
+ true;;
+ *)
+ false;;
+ esac
+done
+IFS="$default_IFS"
+
+# Actually updating.
+mkdir "$module_dir"
+echo hello > "$module_dir/source"
+cat > "$module_dir/sample.scm"<<EOF
+(define-module (sample)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (gnu packages base))
+
+(define-public my-thing
+ (package
+ (inherit hello)
+ (name "the-test-package")
+ (version "4.3")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append "mirror://gnu/hello/hello-" version
+ ".tar.gz"))
+ (sha256
+ (base32
+ "086vqwk2wl8zfs47sq2xpjc9k066ilmb8z6dn0q6ymwjzlm196cd"))))))
+EOF
+guix refresh -t test -L "$module_dir" the-test-package
+guix refresh -t test -L "$module_dir" the-test-package -u
+grep 'version "5.5"' "$module_dir/sample.scm"
+grep "$(guix hash -H sha256 -f nix-base32 "$module_dir/source")" "$module_dir/sample.scm"
+
+# Specifying a target version.
+! guix refresh -t test guile=2.0.0
+case "$(guix refresh -t test guile=2.0.0 2>&1)" in
+ *"failed to find"*"2.0.0"*) true;;
+ *) false;;
+esac
+for spec in "guile=1.6.4" "guile@3=1.6.4"
+do
+ guix refresh -t test "$spec"
+ case "$(guix refresh -t test "$spec" 2>&1)" in
+ *"would be downgraded"*"1.6.4"*) true;;
+ *) false;;
+ esac
+done
+
+# Listing updaters. This should work whether or not networking is available.
+guix refresh --list-updaters
--
2.38.1
H
H
Hartmut Goebel wrote on 3 Jan 2023 19:51
Re: [bug#60520] [PATCH 0/4] Tests for 'guix refresh'
(name . Ricardo Wurmus)(address . rekado@elephly.net)
f7b927ef-e822-301b-1f47-0cbeca89e979@crazy-compilers.com
Am 03.01.23 um 17:44 schrieb Ludovic Courtès:
Toggle quote (2 lines)
> Thoughts?

Thanks for implementing this,  It was terribly missing.

Two mini remarks:

+IFS=X

What about using a symbol like underscore?

+ "guileX-r" \

For symetry I'd write "-r guile".

--
Regards
Hartmut Goebel

| Hartmut Goebel |h.goebel@crazy-compilers.com |
|www.crazy-compilers.com | compilers which you thought are impossible |
Attachment: file
L
L
Ludovic Courtès wrote on 8 Jan 2023 16:16
Re: bug#60520: [PATCH 0/4] Tests for 'guix refresh'
(name . Hartmut Goebel)(address . h.goebel@crazy-compilers.com)
875ydhukdm.fsf_-_@gnu.org
Hi,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

Toggle quote (10 lines)
> Two mini remarks:
>
> +IFS=X
>
> What about using a symbol like underscore?
>
> + "guileX-r" \
>
> For symetry I'd write "-r guile".

Good ideas. I made those changes and pushed:

04f247be81 * refresh: Add CLI tests.
2e9c0e1ff6 * import: stackage: Catch networking errors in predicate.
472dd29f37 * gnu-maintenance: 'gnu' and 'gnu-ftp' predicates catch networking errors.
f3edf29c67 * gnu-maintenance: Factorize 'false-if-networking-failure'.

Thanks,
Ludo’.
Closed
?