Making package transformation options available to 'guix pull'?

Hi Guix!

TL;DR Shall we add a `--without-tests` option (or something similar) for `guix time-machine`?

Our release workflow in Bitcoin is planned to shift over to Guix for the next release, and as such we have a lot more testers who are running Guix as we speak on their machines! Exciting!

Our build workflow basically boils down to something like:

The fact that this is even possible (and allows reproducibility across time) speaks to the awesomeness of Guix! :-)

One of the main reasons for switching over to Guix is for its bootstrappbility and the ability to perform `--bootstrap --no-substitutes` builds. As mentioned in the gnutls ticket, performing these source-based builds can sometimes reveal problems with tests suites:

1. Spurious tests

2. “Expiring” tests (like the gnutls one)

3. Foreign distro problems (0zmcam9sdiag9s1qx3xvlf8a3jnm53x2-sed-4.8.drv fails to build on SELinux-based distros because of a test failure in testsuite/inplace-selinux)

I think that the package transformation options are the perfect “hack-around” solution for this, namely `--without-tests`. In fact, since those who experience these problems are building from source, it makes no difference that the store item would be change: they would need to build everything anyway!

However, in the case of our workflow above, there is no way to specify `--without-tests=` to `guix time-machine`, which insists on building the broken `gnutls` package and running its tests. I’ve tried isolating the command to just:

To make sure that it’s `guix time-machine` which is trying to build gnutls and not the time-machine’d `guix environment`.

I’m not too familiar with how package transformation options work internally, but I’m guessing all we need here is a `--without-tests` option (or something similar) for `guix time-machine`? Is there anything else that’s necessary to successfully skip tests for my workflow?

Cheers,

Carl Dong

contact@carldong.me

"I fight for the users"

On Fri, Feb 19, 2021 at 06:24:57PM -0500, Carl Dong wrote:

I hate to throw a wrench in your plans, but I think that using package

transformation options on the command-line militates against many of the

benefits you are seeking to take advantage of, in terms of an

introspectable and bootstrappable software system.

Normally when using Guix, our Git tree provides a convenient and

comprehensive view of the entire software system. It also provides an

entry point into the system for making changes.

When using these command-line package transformations, you diverge from

the Guix Git tree and enter uncharted and untracked territory. If you

install some software or build some system using these transformations,

there is no Git tree you can check out and begin exploring. For example,

there will be no package definition of GnuTLS-without-tests that can be

referred to and shared.

Hi,

Carl Dong <contact@carldong.me> skribis:

Yes, these flaky/uncontrolled problems were one of the motivations for

‘--without-tests’.

Package transformation options currently only apply to packages

specified on the command line of ‘guix install’, ‘guix build’, etc.

These options are not available to ‘guix system’, ‘guix pull’, and other

non-package-oriented commands.

Guix as returned by ‘guix pull’ and ‘guix time-machine’ depends on

GnuTLS. So here, we have a special case: to build Guix itself, you need

to build GnuTLS first, and there’s no way to pass ‘--without-tests’ at

that level.

I’m not sure how to change that because unfortunately, ‘guix pull’ and

‘time-machine’ don’t even know that Guix depends on GnuTLS; only the

target Guix knows about this. :-/

Thanks,

Ludo’.

Leo Famulari <leo@famulari.name> skribis:

I think that’s OK: you’re building a different GnuTLS derivation, true,

but how that derivation is computed is fully under control as the

transformation option appears on the command line or in your manifest.

It’s really equivalent to forking Guix and adding #:tests? #f, only more

flexible.

Ludo’.

Hi,

On Sat, 20 Feb 2021 at 14:57, Ludovic Courtès <ludo@gnu.org> wrote:

If «“guix pull” and ’time-machine’ don’t even know that Guix depends on

GnuTLS» and «to build Guix itself, you need to build GnuTLS first», how

does it work? How does «you» know that they needs to build GnuTLS

first?

Well, I do not know if it is viable, one hack should to use the package

Guix to collect this missing knowledge. Why is it not possible to build

all the package dependencies of Guix with the option --without-tests?

Cheers,

simon

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

‘guix pull’ builds Guix using the method implemented in (guix self).

The entry point there is ‘compiled-guix’, which returns a compiled Guix.

It’s not a package so the package API doesn’t apply.

HTH,

Ludo’.

Hi Ludo,

On Wed, 24 Feb 2021 at 12:11, Ludovic Courtès <ludo@gnu.org> wrote:

Thanks. It perfectly answer to my question. :-)

I am confused. The 'specification->package' returns a regular

package, right?. And then,

So here, a transformation could be applied. For example, build all

the dependencies without testing them; at the 'define gnutls' (&co)

step or in the map. Why not?

Well, just to be able to travel back in time if some expiration changed.

Cheers,

simon

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

Sure, but how do you get a user-specified transformation to this line?

Ludo’.

Hi,

On Wed, 24 Feb 2021 at 18:21, Ludovic Courtès <ludo@gnu.org> wrote:

I do not know (yet?). :-)

First, we are only speaking about “guix time-machine” and second, only

about corner cases which could break in the future, as expiration.

Right?

We are not talking about general transformations applied to self (even

if it could be great).

So we could imagine something like ’set-build-from-command-line’ or

’set-build-option’ to tweak self only for the inferior. I have not

traced (yet) on the stack of calls from “guix time-machine” to self via

the inferiors, but somehow it looks like passing arguments.

Well, be able to tweak how to build Guix via the time-machine appears to

me better than configuring a build machine ahead of time. It is like

predicting the future and instead we should try to have a flexible

present in order to fix the past once in the unpredictable future. :-)

Cheers,

simon

retitle 46650 Making package transformation options available to 'guix pull'?

quit