'--with-input', '--with-git-url' etc. cause unnecessary rebuilds

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
important
Merged with

Debbugs page

Ludovic Courtès wrote 5 years ago
‘--with-input’ causes unintended rebuilds
(address . bug-Guix@gnu.org)
87eeyjsw3g.fsf@inria.fr
Hello,

Consider this example:

Toggle snippet (12 lines)
$ guix build glib -nd
/gnu/store/9zz9hvzaz06f40a4cbvhskb183x676w4-glib-2.60.6.drv
$ guix build glib --with-input=inkscape=libreoffice -nd
/gnu/store/15f9jkpakmsaz8i2a0gy4kir1zyk29vi-glib-2.60.6.drv
$ guix describe
Generacio 114 Nov 02 2019 11:32:51 (nuna)
guix ab1c063
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: ab1c063ab08e069fbe62919828fa634a2e222bbf

Since GLib does not depend on Inkscape, the ‘--with-input’ flag should
have no effect: we should get the same glib derivation. However, we’re
not.

If we diff the ‘glibc-2.60.6-guile-builder’ files of each derivation, we
see that the second one has a duplicate entry:

(define %build-inputs
`(…
("python" . "/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4")
("python" . "/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4")
…))

whereas the first one doesn’t have this duplicate entry. IOW, the two
derivations are functionally equivalent but are not bit-identical.

Indeed, evaluating:

(bag-transitive-inputs
(package->bag ((package-input-rewriting '()) glib)))

shows that we have two “python” packages there that are not ‘eq?’.

To be continued…

Ludo’.
Ludovic Courtès wrote 5 years ago
(address . 38100@debbugs.gnu.org)
87ftiydqnb.fsf@gnu.org
Hi,

Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

Toggle quote (7 lines)
> Indeed, evaluating:
>
> (bag-transitive-inputs
> (package->bag ((package-input-rewriting '()) glib)))
>
> shows that we have two “python” packages there that are not ‘eq?’.

The problem is that ‘glib’ depends on ‘python-libxml2’, which uses
‘python-build-system’ and thus has ‘python’ as an implicit input.

‘package-input-rewriting’ doesn’t touch implicit inputs so it leaves
that implicit ‘python’ untouched.

Since ‘transitive-inputs’ (used by ‘bag-transitive-inputs’) uses pointer
equality, we end up with two “python” packages that are not ‘eq?’ but
are functionally equivalent: the one produced by
‘package-input-rewriting’, and the implicit dependency of
‘python-libxml2’. QED.

(This is essentially the same as https://bugs.gnu.org/30155.)

I’m not sure how to address it.

Ludo’.
Ludovic Courtès wrote 5 years ago
control message for bug #38100
(address . control@debbugs.gnu.org)
878sg2yjbj.fsf@gnu.org
merge 38100 42156
quit
Ludovic Courtès wrote 5 years ago
(address . control@debbugs.gnu.org)
87zh8ix4o8.fsf@gnu.org
severity 38100 important
quit
Ludovic Courtès wrote 5 years ago
control message for bug #42156
(address . control@debbugs.gnu.org)
87tuxjssyb.fsf@gnu.org
retitle 42156 '--with-input', '--with-git-url' etc. cause unnecessary rebuilds
quit
Ludovic Courtès wrote 4 years ago
Re: bug#38100: ‘--with-input’ causes unintended rebuilds
(address . 38100-done@debbugs.gnu.org)
87sgb2dhap.fsf@inria.fr
Hey there!

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (23 lines)
> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> Indeed, evaluating:
>>
>> (bag-transitive-inputs
>> (package->bag ((package-input-rewriting '()) glib)))
>>
>> shows that we have two “python” packages there that are not ‘eq?’.
>
> The problem is that ‘glib’ depends on ‘python-libxml2’, which uses
> ‘python-build-system’ and thus has ‘python’ as an implicit input.
>
> ‘package-input-rewriting’ doesn’t touch implicit inputs so it leaves
> that implicit ‘python’ untouched.
>
> Since ‘transitive-inputs’ (used by ‘bag-transitive-inputs’) uses pointer
> equality, we end up with two “python” packages that are not ‘eq?’ but
> are functionally equivalent: the one produced by
> ‘package-input-rewriting’, and the implicit dependency of
> ‘python-libxml2’. QED.
>
> (This is essentially the same as <https://bugs.gnu.org/30155>.)

Good news, this is fixed by 2bf6f962b91123b0474c0f7123cd17efe7f09a66,
which introduces package rewriting including implicit inputs!

Before getting there, this issue did get on my nerves for a while. Here
are several ways to address this issue that I thought of:

1. Have ‘package-input-rewriting/spec’ traverse implicit inputs, at
least optionally. We wouldn’t end up with an
equivalent-but-not-eq? ‘python’ in the example above. It does
change the semantics though, and it may be nice to keep a “shallow”
replacement option. That’s what
2bf6f962b91123b0474c0f7123cd17efe7f09a66 does.

2. Do (delete-duplicates input-drvs) in ‘bag->derivation’. That seems
wise, but it’s unfortunately impossible on ‘master’ because of

3. ‘package-input-rewriting/spec’ preserves eq?-ness for packages not
transformed; in the example above, the transformation result would
be eq? to ‘glib’ because ‘--with-input=libreoffice=inkscape’ had no
effect. Tricky to implement efficiently, perhaps not worth it.

I think #2 might still be worth investigating, but it may have
undesirable implications too. #3 is hardly doable.

All in all, I’m glad that #1 addresses the issue, because it’s also
something we wanted anyway.

Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 38100@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 38100
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help