'--with-input', '--with-git-url' etc. cause unnecessary rebuilds

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
important
Merged with
L
L
Ludovic Courtès wrote on 7 Nov 2019 13:35
‘--with-input’ causes unintended rebuilds
(address . bug-Guix@gnu.org)
87eeyjsw3g.fsf@inria.fr
Hello,

Consider this example:

Toggle snippet (12 lines)
$ guix build glib -nd
/gnu/store/9zz9hvzaz06f40a4cbvhskb183x676w4-glib-2.60.6.drv
$ guix build glib --with-input=inkscape=libreoffice -nd
/gnu/store/15f9jkpakmsaz8i2a0gy4kir1zyk29vi-glib-2.60.6.drv
$ guix describe
Generacio 114 Nov 02 2019 11:32:51 (nuna)
guix ab1c063
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: ab1c063ab08e069fbe62919828fa634a2e222bbf

Since GLib does not depend on Inkscape, the ‘--with-input’ flag should
have no effect: we should get the same glib derivation. However, we’re
not.

If we diff the ‘glibc-2.60.6-guile-builder’ files of each derivation, we
see that the second one has a duplicate entry:

(define %build-inputs
`(…
("python" . "/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4")
("python" . "/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4")
…))

whereas the first one doesn’t have this duplicate entry. IOW, the two
derivations are functionally equivalent but are not bit-identical.

Indeed, evaluating:

(bag-transitive-inputs
(package->bag ((package-input-rewriting '()) glib)))

shows that we have two “python” packages there that are not ‘eq?’.

To be continued…

Ludo’.
L
L
Ludovic Courtès wrote on 8 Nov 2019 22:06
(address . 38100@debbugs.gnu.org)
87ftiydqnb.fsf@gnu.org
Hi,

Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

Toggle quote (7 lines)
> Indeed, evaluating:
>
> (bag-transitive-inputs
> (package->bag ((package-input-rewriting '()) glib)))
>
> shows that we have two “python” packages there that are not ‘eq?’.

The problem is that ‘glib’ depends on ‘python-libxml2’, which uses
‘python-build-system’ and thus has ‘python’ as an implicit input.

‘package-input-rewriting’ doesn’t touch implicit inputs so it leaves
that implicit ‘python’ untouched.

Since ‘transitive-inputs’ (used by ‘bag-transitive-inputs’) uses pointer
equality, we end up with two “python” packages that are not ‘eq?’ but
are functionally equivalent: the one produced by
‘package-input-rewriting’, and the implicit dependency of
‘python-libxml2’. QED.

(This is essentially the same as https://bugs.gnu.org/30155.)

I’m not sure how to address it.

Ludo’.
L
L
Ludovic Courtès wrote on 2 Jul 2020 00:14
control message for bug #38100
(address . control@debbugs.gnu.org)
878sg2yjbj.fsf@gnu.org
merge 38100 42156
quit
L
L
Ludovic Courtès wrote on 2 Jul 2020 00:15
(address . control@debbugs.gnu.org)
87zh8ix4o8.fsf@gnu.org
severity 38100 important
quit
L
L
Ludovic Courtès wrote on 3 Aug 2020 16:30
control message for bug #42156
(address . control@debbugs.gnu.org)
87tuxjssyb.fsf@gnu.org
retitle 42156 '--with-input', '--with-git-url' etc. cause unnecessary rebuilds
quit
L
L
Ludovic Courtès wrote on 27 Sep 2020 23:46
Re: bug#38100: ‘--with-input’ causes unintended rebuilds
(address . 38100-done@debbugs.gnu.org)
87sgb2dhap.fsf@inria.fr
Hey there!

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (23 lines)
> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> Indeed, evaluating:
>>
>> (bag-transitive-inputs
>> (package->bag ((package-input-rewriting '()) glib)))
>>
>> shows that we have two “python” packages there that are not ‘eq?’.
>
> The problem is that ‘glib’ depends on ‘python-libxml2’, which uses
> ‘python-build-system’ and thus has ‘python’ as an implicit input.
>
> ‘package-input-rewriting’ doesn’t touch implicit inputs so it leaves
> that implicit ‘python’ untouched.
>
> Since ‘transitive-inputs’ (used by ‘bag-transitive-inputs’) uses pointer
> equality, we end up with two “python” packages that are not ‘eq?’ but
> are functionally equivalent: the one produced by
> ‘package-input-rewriting’, and the implicit dependency of
> ‘python-libxml2’. QED.
>
> (This is essentially the same as <https://bugs.gnu.org/30155>.)

Good news, this is fixed by 2bf6f962b91123b0474c0f7123cd17efe7f09a66,
which introduces package rewriting including implicit inputs!

Before getting there, this issue did get on my nerves for a while. Here
are several ways to address this issue that I thought of:

1. Have ‘package-input-rewriting/spec’ traverse implicit inputs, at
least optionally. We wouldn’t end up with an
equivalent-but-not-eq? ‘python’ in the example above. It does
change the semantics though, and it may be nice to keep a “shallow”
replacement option. That’s what
2bf6f962b91123b0474c0f7123cd17efe7f09a66 does.

2. Do (delete-duplicates input-drvs) in ‘bag->derivation’. That seems
wise, but it’s unfortunately impossible on ‘master’ because of

3. ‘package-input-rewriting/spec’ preserves eq?-ness for packages not
transformed; in the example above, the transformation result would
be eq? to ‘glib’ because ‘--with-input=libreoffice=inkscape’ had no
effect. Tricky to implement efficiently, perhaps not worth it.

I think #2 might still be worth investigating, but it may have
undesirable implications too. #3 is hardly doable.

All in all, I’m glad that #1 addresses the issue, because it’s also
something we wanted anyway.

Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 38100@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 38100
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch