On 18-10-2022 14:36, Ludovic Courtès wrote: > Hello, > > Maxime Devos skribis: > >> A copy is made of test-driver.scm to avoid potentially surprising rebuilds >> when changes are made. >> >> * gnu/packages/guile.scm (guile-test-driver): New variable. >> * gnu/packages/aux-files/test-driver.scm: New file. >> * Makefile.am (AUX_FILES): Register it. >> * build-aux/test-driver.scm: Add a note. > > I very much like the idea of supporting #:tests? in > ‘guile-build-system’, but I’m skeptical about this approach. > > The reason I’m skeptical is because ‘test-driver.scm’ is written as an > Automake test driver (it follows the “protocol” defined by Automake) It is perfectly usable outside Automake, as this patch series demonstrates -- it does not depend on any Automake details. Also, the alternative would be to do "guile -l the-test-suite.scm" directly (without a test runner) -- while sometimes this works, sometimes the SRFI-64 implementation really cares that there is actually some test runner (I got a failure for some unpackaged Scheme library when running it without test-driver.scm), and additionally IMO the output of "guile -l the-test-suite.scm" isn't very nice, the test-driver.scm has much more nice output and is more consistent with other packages' test output. (A third alternative would be to ignore the tests altogether, but running tests is good and I have some patches for adding support for the tests.) > for tests that use SRFI-64— That's the standard Scheme API for test suites in Scheme -- widely applicable. (There's also SRFI-78, but that's currently unsupported by Guile, so it's currently not relevant for guile-build-system.) > that’s a specific kind of tool, and one approach > to writing test suites in Guile. It's the only approach to my knowledge, aside from 'let's just put a bunch of (when wrong? (error ...)) in a script and do 'guile -l the-script.scm' or reimplement our own thing, but that's not a nice approach, I would rather make it possible to use the more-standard and nicer (even though it's somewhat stateful) SRFI-64. Also, I don't see what's wrong with 'specific kind of tools' -- should we remove 'nml' then because it's super-specific to openttd? And even if there exists another tool that would fit here, why would that matter? I mean, the 'gcc' approach to compiling C wasn't rejected in Guix even though it's just 'one' approach to compiling C -- for example, 'clang' is another approach. If someone has another approach to running SRFI-64 test suites in guile-build-system, they can propose it and it can be discussed which one would be better for guile-build-system. Additionally, a nitpick -- test-driver.scm is an approach to running (SRFI-64) test suites, not for writing them, for writing test suites, SRFI-64 is an approach. > I guess all I’m saying is that I doubt this is widely applicable, which > the diff seems to confirm: > >> 4 files changed, 326 insertions(+), 2 deletions(-) I don't how you get from the diff that it isn't widely applicable -- it is applicable to every package that has a SRFI-64 test suite and uses guile-build-system. I just didn't check every guile-build-system thing for whether it has a SRFI-64 test suite. And once more Scheme libraries are packaged (especially Scheme libraries not written with Guile in mind, where gnu-build-system is likely inapplicable), presumably we would get more packages where this patch series can run tests. Additionally, have you considered that one possible reason for lack of SRFI-64 test suites in guile-build-system packages, is that previously there was no good method to run those test suites, so (for someone using Guix and Guile) writing them would be pointless? > Perhaps at this stage there really isn’t much we can factorize, after > all. It would be nice to fix this, I don't see any problem to fix or anything to factorize. What is this 'this' you are referring to? > but that’s probably work to be done > upstream—for example, by adding a “guild test” command. Going by "git log build-aux/test-driver.scm", Guix is the upstream, not Guile, though I suppose it could eventually be moved to Guile if desired by both. > We can even > beta-test such a command as an external project before including it in > Guile proper. > > WDYT? I think it already has seen plenty of testing -- in Guix, in scheme-gnunet, I think I've also seen it used in the GnuTLS bindings, likely by most other Guile programs that use gnu-build-system and have a test suite as well. I mean, it existed since 2015 according to "git log", that's 7 years of testing, seems plenty to me, and it also has been used by various projects. Greetings, Maxime.