[PATCH 0/6] Support #:tests? in guile-build-system

  • Open
  • quality assurance status badge
Details
3 participants
  • Efraim Flashner
  • Ludovic Courtès
  • Maxime Devos
Owner
unassigned
Submitted by
Maxime Devos
Severity
normal
M
M
Maxime Devos wrote on 7 Oct 2022 22:47
(name . guix-patches@gnu.org)(address . guix-patches@gnu.org)
e39412d9-edf3-25c7-1e86-5442ecc0a1fd@telenet.be
guile-build-system can't guess where the tests are located, so currently
it's opt-in, via #:test-script. After this patch series, it is
currently only used by guile-ffi-fftw, but other guile-build-system
packages that use SRFI-64 could potentially use it too (e.g.
guile-srfi-117, which is not packages or even released yet).
* First commit: package a test driver, for nicer output and for proper
exit status. I copied test-driver.scm to aux-files such that
test-driver.scm can be changed without worrying about rebuilds. OTOH,
it isn't modified that often, so that could be left out.
Also, copying test-driver.scm causes the following:
guix build: warning: failed to load '(gnu packages aux-files test-driver)':
no code for module (gnu packages aux-files test-driver)
hint: File
`/home/antipode/source-code/guix-master/gnu/packages/aux-files/test-driver.scm'
should probably start with:
(define-module (gnu packages aux-files test-driver))
Whatever code does this loading, should probably be modified to not load
things under aux-files.
Going by "git log", the upstream of this test driver is Guix itself.
* Second commit: add support for #:tests?, using the test driver.
* Third commit: use it in guile-ffi-fftw.
* Fourth and fifth commit: cleanup of guile-ffi-fftw.
* Sixth commit: adjust guile-ac-d-bus to new 'check' phase
Only the initial commit is required for adding support for tests.
I'll leave building dependencies to https://qa.guix.gnu.org/.
Greetings,
Maxime.
Attachment: OpenPGP_signature
M
M
Maxime Devos wrote on 7 Oct 2022 22:53
[PATCH 6/6] guile-ac-d-bus: Don't duplicate 'check' phase.
(address . 58365@debbugs.gnu.org)(name . Maxime Devos)(address . maximedevos@telenet.be)
20221007205352.1282-6-maximedevos@telenet.be
Now guile-build-system has a 'check' phase, the 'check' phase is duplicated.
It's harmless, but let's not.

* gnu/packages/guile-xyz.scm
(guile-ac-d-bus)[arguments]<#:phases>{check}: Instead of adding the phase,
replace it.
---
gnu/packages/guile-xyz.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (15 lines)
diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index 711e2eade9..0339bf8281 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -3911,7 +3911,7 @@ (define (guile-sls->sls file)
(lambda _
(with-directory-excursion "docs"
(invoke "makeinfo" "ac-d-bus"))))
- (add-after 'build-doc 'check
+ (replace 'check
(lambda* (#:key (tests? #t) #:allow-other-keys)
(when tests?
;; There is no locale for the ö character, which crashes
--
2.37.3
M
M
Maxime Devos wrote on 7 Oct 2022 22:53
[PATCH 3/6] guile-ffi-fftw: Respect #:tests?.
(address . 58365@debbugs.gnu.org)(name . Maxime Devos)(address . maximedevos@telenet.be)
20221007205352.1282-3-maximedevos@telenet.be
* gnu/packages/guile-xyz.scm
(guile-ffi-fftw)[arguments]<#:test-script>: Set it.
[arguments]{#:phases}{check}: Remove custom phase.
[native-inputs]: Add native-input required by uncustomised 'check' phase.
---
gnu/packages/guile-xyz.scm | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)

Toggle diff (28 lines)
diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index 56f0bb8acc..2ab5683d19 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -3682,6 +3682,7 @@ (define-public guile-ffi-fftw
(build-system guile-build-system)
(arguments
`(#:source-directory "mod"
+ #:test-script "../test/test-ffi-fftw.scm"
#:phases
(modify-phases %standard-phases
(add-after 'unpack 'prepare-build
@@ -3689,11 +3690,8 @@ (define-public guile-ffi-fftw
(substitute* "mod/ffi/fftw.scm"
(("\\(getenv \"GUILE_FFI_FFTW_LIBFFTW3_PATH\"\\)")
(format #f "\"~a/lib\"" (assoc-ref inputs "fftw"))))
- #t))
- (add-after 'build 'check
- (lambda _
- (invoke "guile" "-L" "mod"
- "-s" "test/test-ffi-fftw.scm"))))))
+ #t)))))
+ (native-inputs (list guile-test-driver))
(inputs
(list fftw guile-2.2))
(home-page "https://github.com/lloda/guile-ffi-fftw/")
--
2.37.3
M
M
Maxime Devos wrote on 7 Oct 2022 22:53
[PATCH 5/6] guile-ffi-fftw: Update to new Guile version.
(address . 58365@debbugs.gnu.org)(name . Maxime Devos)(address . maximedevos@telenet.be)
20221007205352.1282-5-maximedevos@telenet.be
It seems to build (including tests) fine and there are no dependents, so it
appears safe to update to a newer Guile version.

* gnu/packages/guile-xyz.scm
(guile-ffi-fftw)[inputs]{guile-2.2}: Replace by guile-3.0.
---
gnu/packages/guile-xyz.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (15 lines)
diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index a664e37968..711e2eade9 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -3693,7 +3693,7 @@ (define-public guile-ffi-fftw
(dirname
(search-input-file inputs "lib/libfftw3.so"))))))))))
(native-inputs (list guile-test-driver))
- (inputs (list fftw guile-2.2))
+ (inputs (list fftw guile-3.0))
(home-page "https://github.com/lloda/guile-ffi-fftw/")
(synopsis "Access FFTW through Guile's FFI")
(description
--
2.37.3
M
M
Maxime Devos wrote on 7 Oct 2022 22:53
[PATCH 2/6] build-system/guile: Run SRFI-64 tests.
(address . 58365@debbugs.gnu.org)(name . Maxime Devos)(address . maximedevos@telenet.be)
20221007205352.1282-2-maximedevos@telenet.be
The build system needs a hint for the location of the test suite(s),
but aside from that tests are now run automatically. I expect having
multiple test suites to be the exception, so I used 'test-script'
instead of 'test-scripts'.


* guix/build-system/guile.scm (guile-build): Pass on 'tests?', 'test-script'
and 'test-arguments'.
(guile-cross-build): Likewise.
* guix/build/guile-build-system.scm (guile-check): New phase.
(%standard-phases)[check]: Use it.
* doc/guix.texi (Build Systems)[guile-build-system]: Document new behaviour.
---
doc/guix.texi | 15 ++++++++---
guix/build-system/guile.scm | 13 +++++++++
guix/build/guile-build-system.scm | 44 +++++++++++++++++++++++++++++--
3 files changed, 67 insertions(+), 5 deletions(-)

Toggle diff (167 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 08dab8e5b1..858bcec96f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9022,13 +9022,22 @@ code and that are so lean that they don't even have a makefile, let alone a
@file{configure} script. It compiles Scheme code using @command{guild
compile} (@pxref{Compilation,,, guile, GNU Guile Reference Manual}) and
installs the @file{.scm} and @file{.go} files in the right place. It also
-installs documentation.
+installs documentation and runs SRFI-64 test suites.
+
+Test suites cannot be found automatically; the location of the test
+suite can be set with the @code{#:test-script} argument (as a string).
+If there are multiple test scripts, this argument can be set to a staged
+list of locations. Sometimes some tests need to be skipped, if so,
+@code{#:test-arguments} can be a staged list with extra arguments to
+pass to the test driver (see @code{test-driver.scm --help} for details).
This build system supports cross-compilation by using the
@option{--target} option of @samp{guild compile}.
-Packages built with @code{guile-build-system} must provide a Guile package in
-their @code{native-inputs} field.
+Packages built with @code{guile-build-system} must provide a Guile
+package in their @code{native-inputs} field. Additionally, if
+@code{#:test-script} is used, a test driver (usually
+@code{guile-test-driver}) must be provided.
@end defvr
@defvr {Scheme Variable} julia-build-system
diff --git a/guix/build-system/guile.scm b/guix/build-system/guile.scm
index 36a88e181a..75d62fe403 100644
--- a/guix/build-system/guile.scm
+++ b/guix/build-system/guile.scm
@@ -1,5 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2018-2019, 2021-2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -78,6 +79,9 @@ (define %compile-flags
(define* (guile-build name inputs
#:key source
+ (tests? #true)
+ (test-script #false)
+ (test-arguments ''())
(guile #f)
(phases '%standard-phases)
(outputs '("out"))
@@ -98,6 +102,9 @@ (define builder
(guile-build #:name #$name
#:source #+source
+ #:tests? #$tests?
+ #:test-script #$test-script
+ #:test-arguments #$test-arguments
#:source-directory #$source-directory
#:scheme-file-regexp #$scheme-file-regexp
#:not-compiled-file-regexp #$not-compiled-file-regexp
@@ -122,6 +129,9 @@ (define* (guile-cross-build name
build-inputs target-inputs host-inputs
(guile #f)
source
+ (tests? #false)
+ (test-script #false)
+ (test-arguments ''())
(outputs '("out"))
(search-paths '())
(native-search-paths '())
@@ -149,6 +159,9 @@ (define %outputs
#$(outputs->gexp outputs))
(guile-build #:source #+source
+ #:tests? #$tests?
+ #:test-script #$test-script
+ #:test-arguments #$test-arguments
#:system #$system
#:target #$target
#:outputs %outputs
diff --git a/guix/build/guile-build-system.scm b/guix/build/guile-build-system.scm
index 32a431d347..838cb13089 100644
--- a/guix/build/guile-build-system.scm
+++ b/guix/build/guile-build-system.scm
@@ -1,5 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2018, 2019 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2022 Maxime Devos <maximedevos@telenet.be>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -19,8 +20,10 @@
(define-module (guix build guile-build-system)
#:use-module ((guix build gnu-build-system) #:prefix gnu:)
#:use-module (guix build utils)
+ #:use-module (rnrs exceptions)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-34)
#:use-module (ice-9 match)
#:use-module (ice-9 popen)
#:use-module (ice-9 rdelim)
@@ -29,7 +32,8 @@ (define-module (guix build guile-build-system)
#:use-module (guix build utils)
#:export (target-guile-effective-version
%standard-phases
- guile-build))
+ guile-build
+ guile-check))
(define* (target-guile-effective-version #:optional guile)
"Return the effective version of GUILE or whichever 'guile' is in $PATH.
@@ -201,6 +205,42 @@ (define* (build #:key outputs inputs native-inputs
source-files))
#t))
+(define* (guile-check #:key tests? test-script (test-arguments '())
+ (source-directory ".")
+ native-inputs inputs
+ #:allow-other-keys)
+ (when tests?
+ ;; Let Guile find the source code of newly compiled modules,
+ ;; otherwise the modules won't be found even if Guile knows
+ ;; where the compiled code is.
+ (setenv "GUILE_LOAD_PATH"
+ (string-append source-directory
+ (match (getenv "GUILE_LOAD_PATH")
+ (#f "")
+ (path (string-append ":" path)))))
+ (for-each
+ (lambda (test-script)
+ (guard (c ((invoke-error? c)
+ (when (equal? (list (invoke-error-exit-status c)
+ (invoke-error-term-signal c)
+ (invoke-error-stop-signal c))
+ '(127 #false #false))
+ (display "hint: Make sure 'guile-test-driver' is in\
+ 'native-inputs'.\n"
+ (current-error-port)))
+ (raise-continuable c)))
+ (apply invoke "test-driver.scm"
+ (string-append "--test-name=" test-script) test-arguments)))
+ (match test-script
+ ;; Tests can be separated over multiple files.
+ ((? list? test-scripts) test-scripts)
+ ((? string? test-script) (list test-script))
+ (#false
+ (format (current-error-port)
+ "warning: location of test suite is unknown; not running\
+ tests~%")
+ '())))))
+
(define* (install-documentation #:key outputs
(documentation-file-regexp
%documentation-file-regexp)
@@ -222,7 +262,7 @@ (define %standard-phases
(replace 'build build)
(add-after 'build 'install-documentation
install-documentation)
- (delete 'check)
+ (replace 'check guile-check)
(delete 'strip)
(delete 'validate-runpath)
(delete 'install)))
--
2.37.3
M
M
Maxime Devos wrote on 7 Oct 2022 22:53
[PATCH 4/6] guile-ffi-fftw: Modernise style.
(address . 58365@debbugs.gnu.org)(name . Maxime Devos)(address . maximedevos@telenet.be)
20221007205352.1282-4-maximedevos@telenet.be
The 'object->string' change is more a personal thing.
"guix style" adds modify-phases on the same line as #:phases, causing the
80-columns limit to be exceeded, so it has been partially ignored.

* gnu/packages/guile-xyz.scm
(guile-ffi-fftw)[arguments]<#:test-script>: Set it.
[arguments]{#:phases}{check}: Remove custom phase.
[native-inputs]: Add native-input required by uncustomised 'check' phase.
---
gnu/packages/guile-xyz.scm | 27 ++++++++++++++-------------
1 file changed, 14 insertions(+), 13 deletions(-)

Toggle diff (43 lines)
diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm
index 2ab5683d19..a664e37968 100644
--- a/gnu/packages/guile-xyz.scm
+++ b/gnu/packages/guile-xyz.scm
@@ -3681,22 +3681,23 @@ (define-public guile-ffi-fftw
"08j40a5p6a8pgvhffmzb5rfdnrav2mksy3gfjkdqy93jfj1z5afg"))))
(build-system guile-build-system)
(arguments
- `(#:source-directory "mod"
- #:test-script "../test/test-ffi-fftw.scm"
- #:phases
- (modify-phases %standard-phases
- (add-after 'unpack 'prepare-build
- (lambda* (#:key inputs #:allow-other-keys)
- (substitute* "mod/ffi/fftw.scm"
- (("\\(getenv \"GUILE_FFI_FFTW_LIBFFTW3_PATH\"\\)")
- (format #f "\"~a/lib\"" (assoc-ref inputs "fftw"))))
- #t)))))
+ (list #:source-directory "mod"
+ #:test-script "../test/test-ffi-fftw.scm"
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'prepare-build
+ (lambda* (#:key inputs #:allow-other-keys)
+ (substitute* "mod/ffi/fftw.scm"
+ (("\\(getenv \"GUILE_FFI_FFTW_LIBFFTW3_PATH\"\\)")
+ (object->string
+ (dirname
+ (search-input-file inputs "lib/libfftw3.so"))))))))))
(native-inputs (list guile-test-driver))
- (inputs
- (list fftw guile-2.2))
+ (inputs (list fftw guile-2.2))
(home-page "https://github.com/lloda/guile-ffi-fftw/")
(synopsis "Access FFTW through Guile's FFI")
- (description "This is a minimal set of Guile FFI bindings for the FFTW
+ (description
+ "This is a minimal set of Guile FFI bindings for the FFTW
library's ‘guru interface’. It provides two functions: @code{fftw-dft! rank
sign in out} and @code{fftw-dft rank sign in}. These bindings being minimal,
there is no support for computing & reusing plans, or split r/i transforms, or
--
2.37.3
M
M
Maxime Devos wrote on 7 Oct 2022 22:53
[PATCH 1/6] gnu: Add guile-test-driver.
(address . 58365@debbugs.gnu.org)(name . Maxime Devos)(address . maximedevos@telenet.be)
20221007205352.1282-1-maximedevos@telenet.be
A copy is made of test-driver.scm to avoid potentially surprising rebuilds
when changes are made.

* gnu/packages/guile.scm (guile-test-driver): New variable.
* gnu/packages/aux-files/test-driver.scm: New file.
* Makefile.am (AUX_FILES): Register it.
* build-aux/test-driver.scm: Add a note.
---
Makefile.am | 3 +-
build-aux/test-driver.scm | 2 +
gnu/packages/aux-files/test-driver.scm | 284 +++++++++++++++++++++++++
gnu/packages/guile.scm | 39 +++-
4 files changed, 326 insertions(+), 2 deletions(-)
create mode 100755 gnu/packages/aux-files/test-driver.scm

Toggle diff (397 lines)
diff --git a/Makefile.am b/Makefile.am
index bfabf0bf2e..e1f1a4573e 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -427,7 +427,8 @@ AUX_FILES = \
gnu/packages/aux-files/python/sanity-check.py \
gnu/packages/aux-files/python/sitecustomize.py \
gnu/packages/aux-files/renpy/renpy.in \
- gnu/packages/aux-files/run-in-namespace.c
+ gnu/packages/aux-files/run-in-namespace.c \
+ gnu/packages/aux-files/test-driver.scm
# Templates, examples.
EXAMPLES = \
diff --git a/build-aux/test-driver.scm b/build-aux/test-driver.scm
index 1cdd4ff8f7..7ff8d45031 100755
--- a/build-aux/test-driver.scm
+++ b/build-aux/test-driver.scm
@@ -2,6 +2,8 @@
exec guile --no-auto-compile -e main -s "$0" "$@"
!#
;;;; test-driver.scm - Guile test driver for Automake testsuite harness
+;;;; When update this code, consider updating
+;;;; gnu/packages/aux-files/test-driver.scm as well.
(define script-version "2021-02-02.05") ;UTC
diff --git a/gnu/packages/aux-files/test-driver.scm b/gnu/packages/aux-files/test-driver.scm
new file mode 100755
index 0000000000..1cdd4ff8f7
--- /dev/null
+++ b/gnu/packages/aux-files/test-driver.scm
@@ -0,0 +1,284 @@
+#!/bin/sh
+exec guile --no-auto-compile -e main -s "$0" "$@"
+!#
+;;;; test-driver.scm - Guile test driver for Automake testsuite harness
+
+(define script-version "2021-02-02.05") ;UTC
+
+;;; Copyright © 2015, 2016 Mathieu Lirzin <mthl@gnu.org>
+;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;;
+;;; This program is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; This program is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+;;;; Commentary:
+;;;
+;;; This script provides a Guile test driver using the SRFI-64 Scheme API for
+;;; test suites. SRFI-64 is distributed with Guile since version 2.0.9.
+;;;
+;;;; Code:
+
+(use-modules (ice-9 format)
+ (ice-9 getopt-long)
+ (ice-9 pretty-print)
+ (ice-9 regex)
+ (srfi srfi-1)
+ (srfi srfi-19)
+ (srfi srfi-26)
+ (srfi srfi-64))
+
+(define (show-help)
+ (display "Usage:
+ test-driver --test-name=NAME --log-file=PATH --trs-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--select=REGEXP] [--exclude=REGEXP] [--errors-only={yes|no}]
+ [--enable-hard-errors={yes|no}] [--brief={yes|no}}]
+ [--show-duration={yes|no}] [--]
+ TEST-SCRIPT [TEST-SCRIPT-ARGUMENTS]
+The '--test-name' option is mandatory. The '--select' and '--exclude' options
+allow selecting or excluding individual test cases via a regexp, respectively.
+The '--errors-only' option can be set to \"yes\" to limit the logged test case
+metadata to only those test cases that failed. When set to \"yes\", the
+'--brief' option disables printing the individual test case result to the
+console. When '--show-duration' is set to \"yes\", the time elapsed per test
+case is shown.\n"))
+
+(define %options
+ '((test-name (value #t))
+ (log-file (value #t))
+ (trs-file (value #t))
+ (select (value #t))
+ (exclude (value #t))
+ (errors-only (value #t))
+ (color-tests (value #t))
+ (expect-failure (value #t)) ;XXX: not implemented yet
+ (enable-hard-errors (value #t)) ;not implemented in SRFI-64
+ (brief (value #t))
+ (show-duration (value #t))
+ (help (single-char #\h) (value #f))
+ (version (single-char #\V) (value #f))))
+
+(define (option->boolean options key)
+ "Return #t if the value associated with KEY in OPTIONS is \"yes\"."
+ (and=> (option-ref options key #f) (cut string=? <> "yes")))
+
+(define* (test-display field value #:optional (port (current-output-port))
+ #:key pretty?)
+ "Display \"FIELD: VALUE\\n\" on PORT."
+ (if pretty?
+ (begin
+ (format port "~A:~%" field)
+ (pretty-print value port #:per-line-prefix "+ "))
+ (format port "~A: ~S~%" field value)))
+
+(define* (result->string symbol #:key colorize?)
+ "Return SYMBOL as an upper case string. Use colors when COLORIZE is #t."
+ (let ((result (string-upcase (symbol->string symbol))))
+ (if colorize?
+ (string-append (case symbol
+ ((pass) "") ;green
+ ((xfail) "") ;light green
+ ((skip) "") ;blue
+ ((fail xpass) "") ;red
+ ((error) "")) ;magenta
+ result
+ "") ;no color
+ result)))
+
+
+;;;
+;;; SRFI 64 custom test runner.
+;;;
+
+(define* (test-runner-gnu test-name #:key color? brief? errors-only?
+ show-duration?
+ (out-port (current-output-port))
+ (trs-port (%make-void-port "w"))
+ select exclude)
+ "Return an custom SRFI-64 test runner. TEST-NAME is a string specifying the
+file name of the current the test. COLOR? specifies whether to use colors.
+When BRIEF? is true, the individual test cases results are masked and only the
+summary is shown. ERRORS-ONLY? reduces the amount of test case metadata
+logged to only that of the failed test cases. OUT-PORT and TRS-PORT must be
+output ports. OUT-PORT defaults to the current output port, while TRS-PORT
+defaults to a void port, which means no TRS output is logged. SELECT and
+EXCLUDE may take a regular expression to select or exclude individual test
+cases based on their names."
+
+ (define test-cases-start-time (make-hash-table))
+
+ (define (test-on-test-begin-gnu runner)
+ ;; Procedure called at the start of an individual test case, before the
+ ;; test expression (and expected value) are evaluated.
+ (let ((test-case-name (test-runner-test-name runner))
+ (start-time (current-time time-monotonic)))
+ (hash-set! test-cases-start-time test-case-name start-time)))
+
+ (define (test-skipped? runner)
+ (eq? 'skip (test-result-kind runner)))
+
+ (define (test-failed? runner)
+ (not (or (test-passed? runner)
+ (test-skipped? runner))))
+
+ (define (test-on-test-end-gnu runner)
+ ;; Procedure called at the end of an individual test case, when the result
+ ;; of the test is available.
+ (let* ((results (test-result-alist runner))
+ (result? (cut assq <> results))
+ (result (cut assq-ref results <>))
+ (test-case-name (test-runner-test-name runner))
+ (start (hash-ref test-cases-start-time test-case-name))
+ (end (current-time time-monotonic))
+ (time-elapsed (time-difference end start))
+ (time-elapsed-seconds (+ (time-second time-elapsed)
+ (* 1e-9 (time-nanosecond time-elapsed)))))
+ (unless (or brief? (and errors-only? (test-skipped? runner)))
+ ;; Display the result of each test case on the console.
+ (format out-port "~a: ~a - ~a ~@[[~,3fs]~]~%"
+ (result->string (test-result-kind runner) #:colorize? color?)
+ test-name test-case-name
+ (and show-duration? time-elapsed-seconds)))
+
+ (unless (and errors-only? (not (test-failed? runner)))
+ (format #t "test-name: ~A~%" (result 'test-name))
+ (format #t "location: ~A~%"
+ (string-append (result 'source-file) ":"
+ (number->string (result 'source-line))))
+ (test-display "source" (result 'source-form) #:pretty? #t)
+ (when (result? 'expected-value)
+ (test-display "expected-value" (result 'expected-value)))
+ (when (result? 'expected-error)
+ (test-display "expected-error" (result 'expected-error) #:pretty? #t))
+ (when (result? 'actual-value)
+ (test-display "actual-value" (result 'actual-value)))
+ (when (result? 'actual-error)
+ (test-display "actual-error" (result 'actual-error) #:pretty? #t))
+ (format #t "result: ~a~%" (result->string (result 'result-kind)))
+ (newline))
+
+ (format trs-port ":test-result: ~A ~A [~,3fs]~%"
+ (result->string (test-result-kind runner))
+ (test-runner-test-name runner) time-elapsed-seconds)))
+
+ (define (test-on-group-end-gnu runner)
+ ;; Procedure called by a 'test-end', including at the end of a test-group.
+ (let ((fail (or (positive? (test-runner-fail-count runner))
+ (positive? (test-runner-xpass-count runner))))
+ (skip (or (positive? (test-runner-skip-count runner))
+ (positive? (test-runner-xfail-count runner)))))
+ ;; XXX: The global results need some refinements for XPASS.
+ (format trs-port ":global-test-result: ~A~%"
+ (if fail "FAIL" (if skip "SKIP" "PASS")))
+ (format trs-port ":recheck: ~A~%"
+ (if fail "yes" "no"))
+ (format trs-port ":copy-in-global-log: ~A~%"
+ (if (or fail skip) "yes" "no"))
+ (when brief?
+ ;; Display the global test group result on the console.
+ (format out-port "~A: ~A~%"
+ (result->string (if fail 'fail (if skip 'skip 'pass))
+ #:colorize? color?)
+ test-name))
+ #f))
+
+ (let ((runner (test-runner-null)))
+ (test-runner-on-test-begin! runner test-on-test-begin-gnu)
+ (test-runner-on-test-end! runner test-on-test-end-gnu)
+ (test-runner-on-group-end! runner test-on-group-end-gnu)
+ (test-runner-on-bad-end-name! runner test-on-bad-end-name-simple)
+ runner))
+
+
+;;;
+;;; SRFI 64 test specifiers.
+;;;
+(define (test-match-name* regexp)
+ "Return a test specifier that matches a test name against REGEXP."
+ (lambda (runner)
+ (string-match regexp (test-runner-test-name runner))))
+
+(define (test-match-name*/negated regexp)
+ "Return a negated test specifier version of test-match-name*."
+ (lambda (runner)
+ (not (string-match regexp (test-runner-test-name runner)))))
+
+;;; XXX: test-match-all is a syntax, which isn't convenient to use with a list
+;;; of test specifiers computed at run time. Copy this SRFI 64 internal
+;;; definition here, which is the procedural equivalent of 'test-match-all'.
+(define (%test-match-all . pred-list)
+ (lambda (runner)
+ (let ((result #t))
+ (let loop ((l pred-list))
+ (if (null? l)
+ result
+ (begin
+ (if (not ((car l) runner))
+ (set! result #f))
+ (loop (cdr l))))))))
+
+
+;;;
+;;; Entry point.
+;;;
+
+(define (main . args)
+ (let* ((opts (getopt-long (command-line) %options))
+ (option (cut option-ref opts <> <>)))
+ (cond
+ ((option 'help #f) (show-help))
+ ((option 'version #f) (format #t "test-driver.scm ~A~%" script-version))
+ (else
+ (let* ((log (and=> (option 'log-file #f) (cut open-file <> "w0")))
+ (trs (and=> (option 'trs-file #f) (cut open-file <> "wl")))
+ (out (duplicate-port (current-output-port) "wl"))
+ (test-name (option 'test-name #f))
+ (select (option 'select #f))
+ (exclude (option 'exclude #f))
+ (test-specifiers (filter-map
+ identity
+ (list (and=> select test-match-name*)
+ (and=> exclude test-match-name*/negated))))
+ (test-specifier (apply %test-match-all test-specifiers))
+ (color-tests (if (assoc 'color-tests opts)
+ (option->boolean opts 'color-tests)
+ #t)))
+ (when log
+ (redirect-port log (current-output-port))
+ (redirect-port log (current-warning-port))
+ (redirect-port log (current-error-port)))
+ (test-with-runner
+ (test-runner-gnu test-name
+ #:color? color-tests
+ #:brief? (option->boolean opts 'brief)
+ #:errors-only? (option->boolean opts 'errors-only)
+ #:show-duration? (option->boolean
+ opts 'show-duration)
+ #:out-port out #:trs-port trs)
+ (test-apply test-specifier
+ (lambda _
+ (load-from-path test-name))))
+ (and=> log close-port)
+ (and=> trs close-port)
+ (close-port out))))
+ (exit 0)))
+
+;;; Local Variables:
+;;; eval: (add-hook 'write-file-functions 'time-stamp)
+;;; time-stamp-start: "(define script-version \""
+;;; time-stamp-format: "%:y-%02m-%02d.%02H"
+;;; time-stamp-time-zone: "UTC"
+;;; time-stamp-end: "\") ;UTC"
+;;; End:
+
+;;;; test-driver.scm ends here.
diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm
index fcdf75051c..b847ee6be4 100644
--- a/gnu/packages/guile.scm
+++ b/gnu/packages/guile.scm
@@ -16,7 +16,7 @@
;;; Copyright © 2018 Eric Bavier <bavier@member.fsf.org>
;;; Copyright © 2019 Taylan Kammer <taylan.kammer@gmail.com>
;;; Copyright © 2020, 2021, 2022 Efraim Flashner <efraim@flashner.co.il>
-;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be>
+;;; Copyright © 2021, 2022 Maxime Devos <maximedevos@telenet.be>
;;; Copyright © 2021 Timothy Sample <samplet@ngyro.com>
;;;
;;; This file is part of GNU Guix.
@@ -60,9 +60,11 @@ (define-module (gnu packages guile)
#:use-module (gnu packages version-control)
#:use-module (guix packages)
#:use-module (guix download)
+ #:use-module (guix gexp)
#:use-module (guix git-download)
#:use-module (guix build-system gnu)
#:use-module (guix build-system guile)
+ #:use-module (guix build-system trivial)
#:use-module (guix deprecation)
#:use-module (guix utils))
@@ -961,4 +963,39 @@ (define-public guile-lzma
libraries, like Guile-zlib.")
(license license:gpl3+)))
+(define-public guile-test-driver
+ (package
+ (name "guile-test-driver")
+ ;; (define script-version ...) in the source code
+ (version "2021-01-02.05")
+ ;; 'search-auxiliary-file' could be used here, but that causes warnings.
+ (source (local-file "../../gnu/packages/aux-files/test-driver.scm"))
+ (build-system gnu-build-system)
+ (inputs (list guile-3.0))
+ (arguments
+ (list #:phases
+ #~(modify-phases %standard-phases
+ (delete 'configure)
+ (delete 'check)
+ (delete 'install) ; no point in separating build and install
+ (replace 'build
+ (lambda _
+ (define destination
+ (string-append #$output "/bin/test-driver.scm"))
+ (mkdir-p (dirname destination))
+ (copy-file #$source destination)
+ (substitute* destination
+ (("/bin/sh")
+ ;; Reference to Guile will be patched by patch-shebangs.
+ "/bin/guile \\")
+ (("^exec guile(.*)$") "--no-auto-compile -e main -s\n"))
+ (chmod destination #o500))))))
+ (home-page "https://www.gnu.org/software/guix")
+ (synopsis "Guile test driver for SRFI-64 test suites")
+ (description "This package, also known as @file{test-driver.scm}, provides
+a Guile test driver using the SRFI-64 Scheme API for test suites. Unlike the
+default test runner, its output is consistent with other test drivers used
+by Automake.")
+ (license license:gpl3+)))
+
;;; guile.scm ends here

base-commit: 31a56967e2869c916b7a5e8ee570e8e10f0210a5
prerequisite-patch-id: 2712efb97bf33985fd0658e4dd8e936dc08be5fe
prerequisite-patch-id: 9d2409b480a8bff0fef029b4b095922d4957e06f
prerequisite-patch-id: 51a32abca3efec1ba67ead59b8694c5ea3129ad3
prerequisite-patch-id: 9092927761a340c07a99f5f3ed314a6add04cdee
prerequisite-patch-id: d0af09fbd5ee0ef60bdee53b87d729e46c1db2ca
prerequisite-patch-id: 4fee177b2d8c9478c6a7b8ce1ca9072942f39863
prerequisite-patch-id: c2b101598fa5b6f93470ae41d51a983dcb931b04
prerequisite-patch-id: 8fe65a852a4463203ea6b92abb3968bd819475b1
prerequisite-patch-id: 7585c78056095ec991615cbbe877e06f713aada9
prerequisite-patch-id: a9fb1bf1718ad8de6fc26d97a7dc5baf41dc38bd
prerequisite-patch-id: 1e5c9ef57d1df286042f4ae3eb420394c8b3b045
--
2.37.3
L
L
Ludovic Courtès wrote on 18 Oct 2022 14:36
Re: bug#58365: [PATCH 0/6] Support #:tests? in guile-build-system
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 58365@debbugs.gnu.org)
87sfjlnw9r.fsf_-_@gnu.org
Hello,

Maxime Devos <maximedevos@telenet.be> skribis:

Toggle quote (8 lines)
> A copy is made of test-driver.scm to avoid potentially surprising rebuilds
> when changes are made.
>
> * gnu/packages/guile.scm (guile-test-driver): New variable.
> * gnu/packages/aux-files/test-driver.scm: New file.
> * Makefile.am (AUX_FILES): Register it.
> * build-aux/test-driver.scm: Add a note.

I very much like the idea of supporting #:tests? in
‘guile-build-system’, but I’m skeptical about this approach.

The reason I’m skeptical is because ‘test-driver.scm’ is written as an
Automake test driver (it follows the “protocol” defined by Automake) for
tests that use SRFI-64—that’s a specific kind of tool, and one approach
to writing test suites in Guile.

I guess all I’m saying is that I doubt this is widely applicable, which
the diff seems to confirm:

Toggle quote (2 lines)
> 4 files changed, 326 insertions(+), 2 deletions(-)

Perhaps at this stage there really isn’t much we can factorize, after
all. It would be nice to fix this, but that’s probably work to be done
upstream—for example, by adding a “guild test” command. We can even
beta-test such a command as an external project before including it in
Guile proper.

WDYT?

Thanks,
Ludo’.
M
M
Maxime Devos wrote on 18 Oct 2022 15:30
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 58365@debbugs.gnu.org)
d3cb4d96-e135-a8d5-8600-7d5f4ec7a7c5@telenet.be
On 18-10-2022 14:36, Ludovic Courtès wrote:
Toggle quote (17 lines)
> Hello,
>
> Maxime Devos <maximedevos@telenet.be> skribis:
>
>> A copy is made of test-driver.scm to avoid potentially surprising rebuilds
>> when changes are made.
>>
>> * gnu/packages/guile.scm (guile-test-driver): New variable.
>> * gnu/packages/aux-files/test-driver.scm: New file.
>> * Makefile.am (AUX_FILES): Register it.
>> * build-aux/test-driver.scm: Add a note.
>
> I very much like the idea of supporting #:tests? in
> ‘guile-build-system’, but I’m skeptical about this approach.
>
> The reason I’m skeptical is because ‘test-driver.scm’ is written as an
> Automake test driver (it follows the “protocol” defined by Automake)
It is perfectly usable outside Automake, as this patch series
demonstrates -- it does not depend on any Automake details.
Also, the alternative would be to do "guile -l the-test-suite.scm"
directly (without a test runner) -- while sometimes this works,
sometimes the SRFI-64 implementation really cares that there is actually
some test runner (I got a failure for some unpackaged Scheme library
when running it without test-driver.scm), and additionally IMO the
output of "guile -l the-test-suite.scm" isn't very nice, the
test-driver.scm has much more nice output and is more consistent with
other packages' test output.
(A third alternative would be to ignore the tests altogether, but
running tests is good and I have some patches for adding support for the
tests.)
Toggle quote (1 lines)
> for tests that use SRFI-64—
That's the standard Scheme API for test suites in Scheme -- widely
applicable. (There's also SRFI-78, but that's currently unsupported by
Guile, so it's currently not relevant for guile-build-system.)
Toggle quote (2 lines)
> that’s a specific kind of tool, and one approach
> to writing test suites in Guile.
It's the only approach to my knowledge, aside from 'let's just put a
bunch of (when wrong? (error ...)) in a script and do 'guile -l
the-script.scm' or reimplement our own thing, but that's not a nice
approach, I would rather make it possible to use the more-standard and
nicer (even though it's somewhat stateful) SRFI-64.
Also, I don't see what's wrong with 'specific kind of tools' -- should
we remove 'nml' then because it's super-specific to openttd? And even
if there exists another tool that would fit here, why would that matter?
I mean, the 'gcc' approach to compiling C wasn't rejected in Guix even
though it's just 'one' approach to compiling C -- for example, 'clang'
is another approach.
If someone has another approach to running SRFI-64 test suites in
guile-build-system, they can propose it and it can be discussed which
one would be better for guile-build-system.
Additionally, a nitpick -- test-driver.scm is an approach to running
(SRFI-64) test suites, not for writing them, for writing test suites,
SRFI-64 is an approach.
Toggle quote (4 lines)
> I guess all I’m saying is that I doubt this is widely applicable, which
> the diff seems to confirm:
>
>> 4 files changed, 326 insertions(+), 2 deletions(-)
I don't how you get from the diff that it isn't widely applicable -- it
is applicable to every package that has a SRFI-64 test suite and uses
guile-build-system. I just didn't check every guile-build-system thing
for whether it has a SRFI-64 test suite. And once more Scheme libraries
are packaged (especially Scheme libraries not written with Guile in
mind, where gnu-build-system is likely inapplicable), presumably we
would get more packages where this patch series can run tests.
Additionally, have you considered that one possible reason for lack of
SRFI-64 test suites in guile-build-system packages, is that previously
there was no good method to run those test suites, so (for someone using
Guix and Guile) writing them would be pointless?
Toggle quote (2 lines)
> Perhaps at this stage there really isn’t much we can factorize, after
> all. It would be nice to fix this,
I don't see any problem to fix or anything to factorize. What is this
'this' you are referring to?
Toggle quote (2 lines)
> but that’s probably work to be done
> upstream—for example, by adding a “guild test” command.
Going by "git log build-aux/test-driver.scm", Guix is the upstream, not
Guile, though I suppose it could eventually be moved to Guile if desired
by both.
Toggle quote (4 lines)
> We can even
> beta-test such a command as an external project before including it in
> Guile proper. >
> WDYT?
I think it already has seen plenty of testing -- in Guix, in
scheme-gnunet, I think I've also seen it used in the GnuTLS bindings,
likely by most other Guile programs that use gnu-build-system and have a
test suite as well.
I mean, it existed since 2015 according to "git log", that's 7 years of
testing, seems plenty to me, and it also has been used by various projects.
Greetings,
Maxime.
Attachment: OpenPGP_signature
L
L
Ludovic Courtès wrote on 18 Oct 2022 17:44
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 58365@debbugs.gnu.org)
878rldkufc.fsf@gnu.org
Hi Maxime,

That’s quite a wall of text that you sent. :-)

Let me try to clarify what I think can be a fruitful way forward.
Again, I agree with the goal you have here, I’m just proposing a
different strategy.

To me, the problem that we have is lack of a standard way to run tests
in Guile packages that don’t use Autotools. That, in turn, leads to
duplication in package definitions—whether it’s ‘check’ phases in Guix
or something similar in Debian and other distros.

Instead of addressing it downstream (in Guix), my suggestion would be to
address it upstream—that is, to promote one way to run SRFI-64 test
suites, for example with a new “guild test” command.

That “guild test” command would probably be very similar to
‘build-aux/test-driver.scm’. As you note, this script is battle-tested
and provides useful functionality. If we would turn it into a (scripts
test) module, that ‘guild’ would pick up, then we’d have a good start.
With proper documentation, programmers who use Guile would have an
incentive to use this method; packagers would benefit because the
default ‘check’ phase would boil down to invoking “guild test”.

I hope this clarifies my proposal. WDYT?

Thanks,
Ludo’.
M
M
Maxime Devos wrote on 20 Nov 2023 01:34
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 58365@debbugs.gnu.org)
8a38968e-e3bb-9444-8b98-756f0178fe42@telenet.be
Hi,
Op 18-10-2022 om 17:44 schreef Ludovic Courtès:
Toggle quote (3 lines)
> Hi Maxime,
>
> That’s quite a wall of text that you sent. :-)
Going by the definition at
is not intentionally disruptive (neither unintentionally disruptive), it
is not an attempt at overwhelmening, it is not irrelevant and AFAICT it
is not due to an unawareness of good practices.
The only ‘wall-of-text’ property it might have is its length.
So, you're apparently ignoring all what I wrote previously?
Toggle quote (8 lines)
> Let me try to clarify what I think can be a fruitful way forward.
> Again, I agree with the goal you have here, I’m just proposing a
> different strategy.
>
> To me, the problem that we have is lack of a standard way to run tests
> in Guile packages that don’t use Autotools. That, in turn, leads to
> duplication in package definitions—whether it’s ‘check’ phases in Guix
> or something similar in Debian and other distros.
Err, this is the same for Autotools Guile packages -- Autotools Guile
packages have to copy test-driver.scm too (or do (exit 1)-style tests,
but those aren't really the tests this patch series). You can drop the
qualifier ‘
Also, there is, in fact a standard way to run tests in Guile packages:
copy the test-driver.scm (and in case of Autotools, parts of the
Makefile.am from another Guile package).
While this does sound like _a_ problem, it is not a problem this patch
series is about and hence not _the_ problem (see later for details).
Toggle quote (3 lines)
> Instead of addressing it downstream (in Guix), my suggestion would be to
> address it upstream—that is, to promote one way to run SRFI-64 test
> suites, for example with a new “guild test” command.
But Guix is the upstream (I mentioned this in the first e-mail *) --
AFAICT, Guix has the latest version of test-driver.scm, and AFAICT Guix
is where test-driver.scm + parts of Makefile.am is copied from.
(*) and also right in the e-mail you were responding to:
Toggle quote (3 lines)
> Going by "git log build-aux/test-driver.scm", Guix is the upstream, not
> Guile, though I suppose it could eventually be moved to Guile if desired
> by both.
See, I can make claims too. The difference is that you multiply times
claimed things _without_ evidence whereas when I claimed the contrary
_with_ evidence.
... did you even read it? My ‘wall of text’ wasn't for show.
Toggle quote (9 lines)
> That “guild test” command would probably be very similar to
> ‘build-aux/test-driver.scm’. As you note, this script is battle-tested
> and provides useful functionality. If we would turn it into a (scripts
> test) module, that ‘guild’ would pick up, then we’d have a good start.
> With proper documentation, programmers who use Guile would have an
> incentive to use this method; packagers would benefit because the
> default ‘check’ phase would boil down to invoking “guild test”.
>
> I hope this clarifies my proposal. WDYT?
I think that is a clear proposal, I'm not interested in implementing it,
I consider it out-of-scope and something that could be done separately
from this patch series, and I won't do it, no matter that you are
presenting it as if it were natural for me to do so -- I'm not paid
enough for this and don't want your money.
Finding who should be the upstream of test-driver.scm or finding a new
home for test-driver.scm is not at all the point of this patch series --
the point is to support #:tests? in guile-build-system, and for that a
test driver is needed, and look, there is a test driver right here in
the Guix repo, let's package it so that it becomes available to the
build/test code of guile-build-system (and as a bonus, it makes
test-driver.scm more discoverable).
IOW, going back to your ‘I'm just proposing a different strategy’
comment -- you aren't proposing a different strategy for the patch
series, you are proposing something supplemental, and that ‘something
supplemental’ is something for guile-devel@, not guix@.
Best regards (except to Ludo (*)),
Maxime Devos
(*) because of the unconstructive and disrespectful PMs, and other reasons
Attachment: OpenPGP_signature
M
M
Maxime Devos wrote on 20 Nov 2023 01:41
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 58365@debbugs.gnu.org)
7f81995d-b3a1-fa59-5651-86f5768d5797@telenet.be
(Some context I forgot to include:)
Around the same time at the last previous message on technical matters
(https://issues.guix.gnu.org/58365#9),Ludo' sent a PM on some
non-technical matters, and I answered to the PM instead of 58365,
leaving the technical matters message unreplied. So, now, at last, a reply.
Attachment: OpenPGP_signature
E
E
Efraim Flashner wrote on 3 Dec 2023 10:17
(name . Maxime Devos)(address . maximedevos@telenet.be)
ZWxHoWE3jMK1GWGQ@3900XT
Attachment: file
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmVsR50ACgkQQarn3Mo9
g1HOdg//Q2/tJdMWSDlzw/xQujuqJ6EBEoI406Z/XvYNbIQxmMUwOPiJNFbyooKi
r5T068KUGpvRCsg61hW8JrkFaEhCGic33irwyC70SlPiDKodEDnYl6GMk/+4ggOg
PoS/Q1/MucnBWIT4cj1fPKqlbHq2Umtgl0YKjd8E94zJeYaQX6C3OBgW/CVn4eKs
TR3iI+jDCEXkwClGFDwysbZsLQfY1kNgFVFI3A7mumZLbc+zw5zEA+xC57uzT0CQ
Y4IFQU9saQLUCwjeIwFepBo6Ofz+8T9fXB30jPQ7Ako7nP6wAG0R8QL/SL6M058G
UnL+sj83eMjvMiOR30XN6Abuo7yEWgFnYV3G7kr0Y+R+Im1k/4AU0O9a4c6zfPop
PO5U0735gPX2nfT+C0DfCYcsiuBfzEIc9UAIxw3mXT+s6kGpPMJRuW6dqNBeecV5
QjzV5qANYNH1CoT4CfJhs4HQXo1ojMacTPRivlFkhnfcVMLy8v6WN19Gr7TK2vbG
Umfkzdq+7HShRGKAVqyTbTNogqThIslLikshJERafq68oOt8L4eqLaVXlEkP889Z
b3WxsxErd0Uqg7ELIAZYr8rcEumLoCqfFurjtQHW/3D5cSkaDUFxWsfwVR5vpXUi
lkgxVkXuuiDx2+Zk2Ekj4FTir4N22f9UK9aW5C509VIM/Htx9p4=
=bK7B
-----END PGP SIGNATURE-----


?