package-mapping with #:deep? #t doesn't get all the implicit inputs

  • Open
  • quality assurance status badge
Details
5 participants
  • Ludovic Courtès
  • Maxim Cournoyer
  • Csepp
  • Ulf Herrman
  • Simon Tournier
Owner
unassigned
Submitted by
Ulf Herrman
Severity
important
U
U
Ulf Herrman wrote on 31 Aug 2023 22:14
(address . bug-guix@gnu.org)
87h6ofufy5.fsf@tilde.club
#:deep? #t currently works by interposing a dummy build system that
lowers the package to a bag using the original build system, then
applies the supplied transformation to all of the bag's inputs, then
returns a new bag with the new inputs.

The problem with this approach is that it doesn't affect the bag
arguments. This means that packages passed in as arguments may still
end up being used without being transformed. Worse still, packages
*not* passed in as arguments may end up being used *unless one is
explicitly passed in as an argument*, as is the case with
qt-build-system's #:qtbase argument.

In short, the current approach of having the build-system lower
procedure leave the arguments mostly unchanged and letting the
bag->derivation procedure (the "bag builder") fill in lots of defaults
means that there are implicit inputs that cannot be touched at the
package level without adding special logic for every single build system
that does something like this.

I propose that we have the build system lower procedure (that is, the
one that converts from package to bag) completely fill in the argument
list with all defaults, and we modify build-system-with-package-mapping
to transform all arguments that look like a package or a list of
packages (or maybe even a tree containing packages).

Many large-scale package transformations have their purpose defeated if
even a single package slips through and has to be built or downloaded
separately (e.g. "I wanted to use a more minimal version of P, and now I
have both the original version of P *and* a more minimal version of P
*and* a duplicate copy of a large portion of the package graph..."). In
fact, in some situations, this could cause exponential growth of the
number of packages, e.g. a transformation is applied to qtbase and its
dependencies, but not to the implicit version and its dependencies, so
now all the dependencies of qtbase are duplicated, and if this happens
again at a higher level with something that depends on a package with
qt-build-system, now there are four duplicate subgraphs, and so on.

What do you think?

- Ulf
-----BEGIN PGP SIGNATURE-----

iQHIBAEBCAAyFiEEn6BUn0yca1D9JsMa1lV76sJM9mgFAmTw9MkUHHN0cmluZXNz
QHRpbGRlLmNsdWIACgkQ1lV76sJM9mhnnAwAmtKewwrTyG+f8sVRzZwHlipENflt
rQZmeK7LmkD2I6F4IG8tay+ro/ylIX3dFs2FPUDkBqY1biewXXO3M7Po3/oIUY3b
II16LQcp+FtJ8S8GZTHZnAdFgVoqVBiot5zftDCZCOQxZj/nbqFvT9EjAuUn6aCd
eEod96rS+VeecDQEvHiCr/x5B9OcMkgulo4Y6uv9GtxeE1vBj1FxV6dGCj6epPWP
Zj5+ED6P04T9vxXCmrlXMqtZBEZn7oQOauADribIQF/KAaqjuo+vjVyUorCCjkf9
wT+ukY2vikCd9esz+JJ7VXdRBS4q8RQ6kXIKdNvcJx2LdDgxPtF8+Iz2xls/J5sb
OoJvzRsfS1CTR8ZoTi92Nx+5DKl2qF5cRGlM78Ckowf0rhEs7VNwKDcJSe8nyUfy
LA43PXlB9J1NpzU9o9jAM6mM3rdxjGLf9lL0ARF9T3+XTu+XvVF4lHSdrQjsknMH
oILI2U6DOHsdUbpWOMk9wWkk8NwzXDN2cNGd
=3EXK
-----END PGP SIGNATURE-----

C
C
Csepp wrote on 1 Sep 2023 20:08
(name . Ulf Herrman)(address . striness@tilde.club)
87zg25hikd.fsf@riseup.net
Ulf Herrman <striness@tilde.club> writes:

Toggle quote (44 lines)
> [[PGP Signed Part:Undecided]]
> #:deep? #t currently works by interposing a dummy build system that
> lowers the package to a bag using the original build system, then
> applies the supplied transformation to all of the bag's inputs, then
> returns a new bag with the new inputs.
>
> The problem with this approach is that it doesn't affect the bag
> arguments. This means that packages passed in as arguments may still
> end up being used without being transformed. Worse still, packages
> *not* passed in as arguments may end up being used *unless one is
> explicitly passed in as an argument*, as is the case with
> qt-build-system's #:qtbase argument.
>
> In short, the current approach of having the build-system lower
> procedure leave the arguments mostly unchanged and letting the
> bag->derivation procedure (the "bag builder") fill in lots of defaults
> means that there are implicit inputs that cannot be touched at the
> package level without adding special logic for every single build system
> that does something like this.
>
> I propose that we have the build system lower procedure (that is, the
> one that converts from package to bag) completely fill in the argument
> list with all defaults, and we modify build-system-with-package-mapping
> to transform all arguments that look like a package or a list of
> packages (or maybe even a tree containing packages).
>
> Many large-scale package transformations have their purpose defeated if
> even a single package slips through and has to be built or downloaded
> separately (e.g. "I wanted to use a more minimal version of P, and now I
> have both the original version of P *and* a more minimal version of P
> *and* a duplicate copy of a large portion of the package graph..."). In
> fact, in some situations, this could cause exponential growth of the
> number of packages, e.g. a transformation is applied to qtbase and its
> dependencies, but not to the implicit version and its dependencies, so
> now all the dependencies of qtbase are duplicated, and if this happens
> again at a higher level with something that depends on a package with
> qt-build-system, now there are four duplicate subgraphs, and so on.
>
> What do you think?
>
> - Ulf
>
> [[End of PGP Signed Part]]

Sounds like a good idea to me.
M
M
Maxim Cournoyer wrote on 3 Sep 2023 15:02
(name . Ulf Herrman)(address . striness@tilde.club)
875y4rwgt4.fsf@gmail.com
Hello Ulf,

Ulf Herrman <striness@tilde.club> writes:

Toggle quote (19 lines)
> #:deep? #t currently works by interposing a dummy build system that
> lowers the package to a bag using the original build system, then
> applies the supplied transformation to all of the bag's inputs, then
> returns a new bag with the new inputs.
>
> The problem with this approach is that it doesn't affect the bag
> arguments. This means that packages passed in as arguments may still
> end up being used without being transformed. Worse still, packages
> *not* passed in as arguments may end up being used *unless one is
> explicitly passed in as an argument*, as is the case with
> qt-build-system's #:qtbase argument.
>
> In short, the current approach of having the build-system lower
> procedure leave the arguments mostly unchanged and letting the
> bag->derivation procedure (the "bag builder") fill in lots of defaults
> means that there are implicit inputs that cannot be touched at the
> package level without adding special logic for every single build system
> that does something like this.

This is indeed sub-optimal!

Toggle quote (6 lines)
> I propose that we have the build system lower procedure (that is, the
> one that converts from package to bag) completely fill in the argument
> list with all defaults, and we modify build-system-with-package-mapping
> to transform all arguments that look like a package or a list of
> packages (or maybe even a tree containing packages).

[...]

Toggle quote (2 lines)
> What do you think?

Like Csepp, I like your proposition! I'm CC'ing the core team, in case
they have something to add to it, but otherwise, we only need a
volunteer to turn it into code :-).

--
Thanks,
Maxim
S
S
Simon Tournier wrote on 5 Sep 2023 16:57
868r9ku0qc.fsf@gmail.com
Hi,

On Thu, 31 Aug 2023 at 15:14, Ulf Herrman <striness@tilde.club> wrote:

Toggle quote (6 lines)
> I propose that we have the build system lower procedure (that is, the
> one that converts from package to bag) completely fill in the argument
> list with all defaults, and we modify build-system-with-package-mapping
> to transform all arguments that look like a package or a list of
> packages (or maybe even a tree containing packages).

On principle, this looks a good idea.

Do you already have an implementation? Because transformations lead to
headache and sometimes the implementation is hard, well evil, details
and all that. :-)

Cheers,
simon
U
U
Ulf Herrman wrote on 16 Sep 2023 11:45
[PATCH] Really get all the implicit inputs.
(address . 65665@debbugs.gnu.org)
87msxmqwng.fsf@tilde.club
This patch series causes package-mapping to recurse into package and bag
arguments when #:deep? #t is given. It also recurses into gexps and
gexp inputs in search of packages to devour. It also ensures that build
systems leave all of their implicit package arguments in the bag
arguments, ready to be found by package-mapping. It also fixes a couple
build system errors I came across while modifying 40+ build systems and
testing a deep package transformation.

- ulfvonbelow
From dcd9b6505fa46d139d40141cf29d7eef15c46481 Mon Sep 17 00:00:00 2001
Message-Id: <dcd9b6505fa46d139d40141cf29d7eef15c46481.1694806866.git.striness@tilde.club>
In-Reply-To: <73b2dfe98591073104fd069622ede2acab0c7655.1694806866.git.striness@tilde.club>
References: <73b2dfe98591073104fd069622ede2acab0c7655.1694806866.git.striness@tilde.club>
From: Ulf Herrman <striness@tilde.club>
Date: Fri, 15 Sep 2023 07:48:48 -0500
Subject: [PATCH 2/5] build-system: asdf: use substitute-keyword-arguments with
default.

A two-argument substitute-keyword-arguments clause won't be evaluated when
there isn't already a keyword in the argument list. Using a three-argument
clause allows the phases-transformer to be run in all cases.

* guix/build-system/asdf.scm (package-with-build-system): use %standard-phases
as default value of `phases', use gexps.
---
guix/build-system/asdf.scm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

Toggle diff (16 lines)
diff --git a/guix/build-system/asdf.scm b/guix/build-system/asdf.scm
index 2b17cee37b..cd041eed4f 100644
--- a/guix/build-system/asdf.scm
+++ b/guix/build-system/asdf.scm
@@ -218,7 +218,8 @@ (define* (package-with-build-system from-build-system to-build-system
(build-system to-build-system)
(arguments
(substitute-keyword-arguments base-arguments
- ((#:phases phases) (list phases-transformer phases))))
+ ((#:phases phases #~%standard-phases)
+ #~(#$phases-transformer #$phases))))
(inputs (new-inputs package-inputs))
(propagated-inputs (new-propagated-inputs))
(native-inputs (append (if target-is-source?
--
2.40.1
From 32a455ab2852c6ff015a2126953075c67a3371b9 Mon Sep 17 00:00:00 2001
Message-Id: <32a455ab2852c6ff015a2126953075c67a3371b9.1694806866.git.striness@tilde.club>
In-Reply-To: <73b2dfe98591073104fd069622ede2acab0c7655.1694806866.git.striness@tilde.club>
References: <73b2dfe98591073104fd069622ede2acab0c7655.1694806866.git.striness@tilde.club>
From: Ulf Herrman <striness@tilde.club>
Date: Fri, 15 Sep 2023 08:53:10 -0500
Subject: [PATCH 5/5] build-system: dub: add #:ld-gold-wrapper to
private-keywords.

This isn't an accepted keyword to dub-build, so it will currently cause an
error if it is specified, which rather defeats the purpose of having it as an
argument. Putting it in private-keywords will cause it to be stripped from
the final bag arguments.

* guix/build-system/dub.scm (lower): add #:ld-gold-wrapper to
private-keywords.
---
guix/build-system/dub.scm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

Toggle diff (16 lines)
diff --git a/guix/build-system/dub.scm b/guix/build-system/dub.scm
index 1b9f21b052..b325357589 100644
--- a/guix/build-system/dub.scm
+++ b/guix/build-system/dub.scm
@@ -112,7 +112,8 @@ (define* (lower name
"Return a bag for NAME."
(define private-keywords
- '(#:target #:ldc #:dub #:pkg-config #:inputs #:native-inputs #:outputs))
+ '(#:target #:ldc #:dub #:pkg-config #:ld-gold-wrapper
+ #:inputs #:native-inputs #:outputs))
(and (not target) ;; TODO: support cross-compilation
(bag
--
2.40.1
-----BEGIN PGP SIGNATURE-----

iQHIBAEBCAAyFiEEn6BUn0yca1D9JsMa1lV76sJM9mgFAmUFeTYUHHN0cmluZXNz
QHRpbGRlLmNsdWIACgkQ1lV76sJM9mimLgv/d4J6jnXH+6Ujbx1z0WbaCkv3/r6b
zF0t6mkNiSQ08tjHJZcg1HKq/hXnObe8CTOt+i+AS94/2DmQRBAz5M4GcerbNglY
sMaFyhgxzCW2e8TBsPGD+9TAwJrIvwmSBLUDDGmhEi7u40oIXjqcZUUAXjqBVpw9
I0WAP7b8k8C1Wf/Gr2BIR7Y88ahqwt3nKtLKir5OCD/1o7vY3bI3DHN5MIEtRyli
yMEPbhy274ToGPhU84EeirJI9+Ug7zynNy3lyKiw7XcCiVNPBurEiX4rUBbQXHy0
XnHp21lYQrvyzs0nbESN6JAQPHmLpM4V/TwM7g/eEZi+dgz42WQC6rNMZgOk6rsY
ba5jSfImiQnqOL4dIHHl6WMhHG9X1iJZmEXvq36U9UwoBmxwkhwHZSoqAA7sCdVi
k2S14IE9ptaTQ2MdA8v2nPP1cbgT80YZKHPLRwOILpPEeWxIOCXHGK/FEBw9ifJ1
pfkM7j0Q3tAFL4cnwwRHnTr6X+cBOWg0y/GJ
=Aisx
-----END PGP SIGNATURE-----

M
M
Maxim Cournoyer wrote on 6 Oct 2023 04:36
(name . Ulf Herrman)(address . striness@tilde.club)
87a5swlbnp.fsf@gmail.com
Hello Ulf!

Ulf Herrman <striness@tilde.club> writes:

Toggle quote (8 lines)
> This patch series causes package-mapping to recurse into package and bag
> arguments when #:deep? #t is given. It also recurses into gexps and
> gexp inputs in search of packages to devour. It also ensures that build
> systems leave all of their implicit package arguments in the bag
> arguments, ready to be found by package-mapping. It also fixes a couple
> build system errors I came across while modifying 40+ build systems and
> testing a deep package transformation.

Nice series!

I've reviewed it, and it makes sense to me. I'd like to apply it to the
core-updates branch. Core team, what say you?

--
Thanks,
Maxim
U
U
Ulf Herrman wrote on 6 Oct 2023 09:37
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
874jj45hhl.fsf@tilde.club
I've also been using this patch, which rebinds `this-package' within
package variants to refer instead to the variant rather than the
original. When I tried applying a deep transformation to icecat, it
failed to reach icecat-minimal or something like that. Or maybe it was
some input of icecat - might have been my local version of mesa that had
breakage. Anyway, I've since forgotten exactly what happened and why (I
didn't actually write the commit message until quite some time later),
but I do know that this patch fixed it.

I might try to rediscover what the problem was later, but thought it
would be good to make you aware of this so as to hopefully consolidate
world rebuilds.

- Ulf
-----BEGIN PGP SIGNATURE-----

iQHIBAEBCAAyFiEEn6BUn0yca1D9JsMa1lV76sJM9mgFAmUfuTkUHHN0cmluZXNz
QHRpbGRlLmNsdWIACgkQ1lV76sJM9mhIQwwAgwcoLrdBgZ7EswiivPR1YcVVJtU+
gXErgi4FySkAtERIemOi5OSpwrwgnFPUJiSggW1og75iJjXB7edvq7Evi55wSBqy
oqDgUUezKxiRiP885xGdfjF+DEBh6j+TJuzwMU4JXP2AKfar81KQv9LSiLtoG+EW
QA6IOxTYOU/onMd18tLr4Fz1vGcVQFcXL9gSvNcFFSb61ovM2OtCj90nbCmYT3E5
HVE4NpNANI5NizHbraAJ35pCEZPQJo+6z4H+juccAxdBoKzVxLtAgKKAxCIJeJZL
MUfOQyVT7vFwoDc3NI+Z2LG4tCTLXRk3NUhj8evlM2gl5zCE6TCKoP+rsDlbCCiZ
ksMRpdgV7M+/I7RmRQkN1d3PfggKtm9VO3ZxhqiYbSA3RQrYRkkzi0zTFt60nhWC
ZUlGYonEDjzLr5ETcg4/uCsW1GqsmrLO8IbM+0+Kys0UM8qlaJbgDrMmmVFoOd3w
W04VlnNhjVRM2KA1gn15jFNQNT/8t0veKPD0
=xHJK
-----END PGP SIGNATURE-----

U
U
Ulf Herrman wrote on 7 Oct 2023 05:47
(name . Ulf Herrman)(address . striness@tilde.club)
87zg0v3xhj.fsf@tilde.club
Ulf Herrman <striness@tilde.club> writes:

Toggle quote (8 lines)
> Anyway, I've since forgotten exactly what happened and why (I didn't
> actually write the commit message until quite some time later), but I
> do know that this patch fixed it.
>
> I might try to rediscover what the problem was later, but thought it
> would be good to make you aware of this so as to hopefully consolidate
> world rebuilds.

I've since done some more checking to recall what the problem actually
was, and it indeed manifested as a combination of an icecat and mesa
problem: the wrap-program phase of icecat-minimal uses
`this-package-input' to get the mesa to point LD_LIBRARY_PATH to.
However, it then stuffs the resulting package inside a <file-append>
object, which we don't currently recurse through, so it ends up
compiling with one mesa and using another at runtime, which somehow
causes a segmentation fault.

Having looked at it again, I'm not sure that rebinding `this-package' is
the best solution - it's certainly not a general solution, since any old
package could be shoved into a <file-append> object (or really any
declarative file-like object) and thereby be hidden from
transformations. My understanding is that packaging guidelines already
discourage directly substituting top-level package references,
preferring instead tools like `this-package-input' so as to work nicely
with transformations. If those guidelines are adhered to, the
aforementioned patch should fix issues of this nature. I'm not sure
what the best way of handling objects like <file-append> and such is,
but as long as package references go through `this-package', it should
only matter for implicit inputs, and I don't think any of them use
declarative file-like objects other than <package>.

After thinking about it some more, I think it would be good if we had a
way of testing to make sure that every package is "transformable": that
is, if you apply a deep transformation to it, and lower the result to a
derivation, at no point within the dynamic extent of that lowering is a
derivation from an untransformed package introduced. This would allow
for testing for transformability en masse, and could be added to 'guix
lint'.

- Ulf
-----BEGIN PGP SIGNATURE-----

iQHIBAEBCAAyFiEEn6BUn0yca1D9JsMa1lV76sJM9mgFAmUg1LkUHHN0cmluZXNz
QHRpbGRlLmNsdWIACgkQ1lV76sJM9mjWFwwAq+08T2FDh1u+Hb6DHC/7Pexfe4G2
l7geedz0yRFFtbZ8BD593yA+TLGwdsNE05F7D6gLmQdsz3OIHmTEHxk0yaO3SqAI
GFRaue5btZ2gmbeQiNy2CLwX8lDGqyjO40IrDNntQ4iG7WqKKp066pWGpVj0jLvR
JifEjneB937IJrqo3MwwLiBUGAEMAnBbR3S4p4qGRAIyzBhrW204iXdMaRLgUgXN
kbm7hlrOFHi9ixOoc4abO+esywzLya8qyfyFh/0lENuKszszOUGgWUnqfMyxma73
Nzf67nG1fT69X5c+nfUx1vRt5YiZCu5y949S8pFHABUvSqo8/tcK4bYVIu+umcj+
9lOerIYRshInWPCrxipKnmkt0azBxz7I4PA0tCgfCrrmd7yvWY+sWpd6+kGDoAne
drWIeGAsXK8+0bYp4ENyoGlywZw04dZSEAwwplqFfIRMtE5Pk6SwOJsR0pe+Eu7O
sU6doFqccPWZkyYSZola9q4uzG5tv/kmYP+m
=fgOf
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 12 Oct 2023 09:07
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87h6mwpbcv.fsf@gnu.org
Hi!

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (15 lines)
> Ulf Herrman <striness@tilde.club> writes:
>
>> This patch series causes package-mapping to recurse into package and bag
>> arguments when #:deep? #t is given. It also recurses into gexps and
>> gexp inputs in search of packages to devour. It also ensures that build
>> systems leave all of their implicit package arguments in the bag
>> arguments, ready to be found by package-mapping. It also fixes a couple
>> build system errors I came across while modifying 40+ build systems and
>> testing a deep package transformation.
>
> Nice series!
>
> I've reviewed it, and it makes sense to me. I'd like to apply it to the
> core-updates branch. Core team, what say you?

Sorry for the delay, I’ll provide feedback within a couple of days.

Ludo’.
L
L
Ludovic Courtès wrote on 12 Oct 2023 15:47
Re: bug#65665: package-mapping with #:deep? #t doesn't get all the implicit inputs
(name . Ulf Herrman)(address . striness@tilde.club)
87h6mwlzpn.fsf_-_@gnu.org
Hi,

Ulf Herrman <striness@tilde.club> skribis:

Toggle quote (4 lines)
> I've also been using this patch, which rebinds `this-package' within
> package variants to refer instead to the variant rather than the
> original.

Could you send it as a separate issue?

(I think I once reported a bug for this issue, but I can’t find it.)

Ludo’.
L
L
Ludovic Courtès wrote on 12 Oct 2023 15:53
(name . Ulf Herrman)(address . striness@tilde.club)(address . 65665@debbugs.gnu.org)
87cyxklzgg.fsf@gnu.org
Hi Ulf,

Ulf Herrman <striness@tilde.club> skribis:

Toggle quote (7 lines)
> In short, the current approach of having the build-system lower
> procedure leave the arguments mostly unchanged and letting the
> bag->derivation procedure (the "bag builder") fill in lots of defaults
> means that there are implicit inputs that cannot be touched at the
> package level without adding special logic for every single build system
> that does something like this.

Could you share an example of what is fixed by these changes?

Let’s take ‘python-itsdangerous’ as an example. It has zero inputs,
only implicit dependencies, yet I can do this:

Toggle snippet (17 lines)
$ guix build python-itsdangerous -n
substitute: updating substitutes from 'http://192.168.1.48:8123'... 100.0%
0.0 MB would be downloaded:
/gnu/store/yhvhrmqd5znjs7vsq8kc55nc3rkg4w6x-python-itsdangerous-2.0.1

$ guix build python-itsdangerous --with-input=python=python2 -n
The following derivations would be built:
/gnu/store/b1hji2qzdiwfg2wx11l1fyjrgiy0f50v-python-itsdangerous-2.0.1.drv
/gnu/store/5gibs6x75acc6j0g0rh8m66191l9wq12-python-wrapper-3.10.7.drv
0.1 MB would be downloaded:
/gnu/store/b2qv97jbih850zn35b2j84n2acj079cv-itsdangerous-2.0.1.tar.gz

$ guix gc --references /gnu/store/5gibs6x75acc6j0g0rh8m66191l9wq12-python-wrapper-3.10.7.drv |grep python
/gnu/store/n0snl506x5bbs5c2496blh79yil3pf44-python2-2.7.18.drv
/gnu/store/whwrah24q7syyiqra16sm9mjvdxld1pv-python-wrapper-3.10.7-builder

‘python’ above is an implicit dependency, yet it was suitably replaced
by ‘--with-input’.

In what cases does ‘package-mapping #:deep? #t’ miss implicit inputs?

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 12 Oct 2023 16:06
(name . Ulf Herrman)(address . striness@tilde.club)(address . 65665@debbugs.gnu.org)
871qe0lytk.fsf_-_@gnu.org
Ulf Herrman <striness@tilde.club> skribis:

Toggle quote (17 lines)
> -(define (build-system-with-package-mapping bs rewrite)
> +(define (build-system-with-package-mapping bs rewrite-input rewrite-argument)
> "Return a variant of BS, a build system, that rewrites a bag's inputs by
> passing them through REWRITE, a procedure that takes an input tuplet and
> returns a \"rewritten\" input tuplet."
> @@ -1442,9 +1442,10 @@ (define (build-system-with-package-mapping bs rewrite)
> (let ((lowered (apply lower args)))
> (bag
> (inherit lowered)
> - (build-inputs (map rewrite (bag-build-inputs lowered)))
> - (host-inputs (map rewrite (bag-host-inputs lowered)))
> - (target-inputs (map rewrite (bag-target-inputs lowered))))))
> + (build-inputs (map rewrite-input (bag-build-inputs lowered)))
> + (host-inputs (map rewrite-input (bag-host-inputs lowered)))
> + (target-inputs (map rewrite-input (bag-target-inputs lowered)))
> + (arguments (map rewrite-argument (bag-arguments lowered))))))

Aah, now I understand. :-)

It’s indeed the case that arguments can capture references to packages
that won’t be caught by ‘package-mapping’. For instance, if you write:

(package
(arguments (list … #~(whatever #$coreutils))))

… then ‘coreutils’ here cannot be replaced.

To address this, the recommendation is to always add dependencies to
input fields and to use self-references within arguments. The example
above should be written like this:

(package
(arguments
(list … #~(whatever #$(this-package-input "coreutils")))))

It’s just a recommendation and one can perfectly ignore it, and I
suppose that was the impetus for this patch.

This is one of the things discussed while designing this change:

(search for “self-referential records”)


My take was and still is that it’s an acceptable limitation. Packagers
need to follow the guideline above if they want proper support for
rewriting, ‘guix graph’, and other tools.

WDYT?

Thanks,
Ludo’.
S
S
Simon Tournier wrote on 12 Oct 2023 16:22
Re: bug#65665: [PATCH] Really get all the implicit inputs.
86cyxj3op8.fsf@gmail.com
Hi Maxim,

On Thu, 05 Oct 2023 at 22:36, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (3 lines)
> I've reviewed it, and it makes sense to me. I'd like to apply it to the
> core-updates branch. Core team, what say you?

Well, I am not sure to deeply understand some details to get all the
implications here, so I have nothing relevant to say. For instance,
from my understanding, the core change is:

Toggle snippet (20 lines)
+ (define (rewrite-argument arg)
+ (match arg
+ ((? package? p)
+ (replace p))
+ ((? gexp-input? gi)
+ (gexp-input (rewrite-argument (gexp-input-thing gi))
+ (gexp-input-output gi)
+ #:native? (gexp-input-native? gi)))
+ ((? gexp? gxp)
+ (make-gexp (map rewrite-argument (gexp-references gxp))
+ (gexp-self-modules gxp)
+ (gexp-self-extensions gxp)
+ (gexp-proc gxp)
+ (%gexp-location gxp)))
+ ((lst ...)
+ (map rewrite-argument lst))
+ (_
+ arg)))

Toggle snippet (14 lines)
+ (arguments
+ (match p
+ (($ <package> _ _ _ _ args-proc)
+ ;; If we let ARGS-PROC be passed its original package,
+ ;; we somehow end up in an infinite (or maybe just
+ ;; exponential? Never seen it end...) loop. Should
+ ;; probably figure out what's causing that at some
+ ;; point.
+ (let ((args (args-proc this-package)))
+ (if deep?
+ (map rewrite-argument args)
+ args)))))

and I do not feel enough skilled here for getting the implications.
Equally for this kind of changes:

Toggle snippet (7 lines)
- (arguments (strip-keyword-arguments private-keywords arguments)))))
+ (arguments
+ (substitute-keyword-arguments
+ (strip-keyword-arguments private-keywords arguments)
+ ((#:guile _ #t) guile))))))

Therefore, I trust other opinions or I need some time for diving and
filling some gaps.

Cheers,
simon
M
M
Maxim Cournoyer wrote on 12 Oct 2023 17:53
Re: bug#65665: package-mapping with #:deep? #t doesn't get all the implicit inputs
(name . Ludovic Courtès)(address . ludo@gnu.org)
87lec796rc.fsf@gmail.com
Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (13 lines)
> Hi Ulf,
>
> Ulf Herrman <striness@tilde.club> skribis:
>
>> In short, the current approach of having the build-system lower
>> procedure leave the arguments mostly unchanged and letting the
>> bag->derivation procedure (the "bag builder") fill in lots of defaults
>> means that there are implicit inputs that cannot be touched at the
>> package level without adding special logic for every single build system
>> that does something like this.
>
> Could you share an example of what is fixed by these changes?

Ulf had mentioned the problems they were facing in the original post of
this issue:

Toggle quote (7 lines)
> The problem with this approach is that it doesn't affect the bag
> arguments. This means that packages passed in as arguments may still
> end up being used without being transformed. Worse still, packages
> *not* passed in as arguments may end up being used *unless one is
> explicitly passed in as an argument*, as is the case with
> qt-build-system's #:qtbase argument.

Examples would be an Qt packages, or packages using #:meson
meson-variant.

Any build system accepting package as arguments are subject to this
problem, if I understand correctly.

I hope that helps,

--
Thanks,
Maxim
M
M
Maxim Cournoyer wrote on 12 Oct 2023 18:00
(name . Ludovic Courtès)(address . ludo@gnu.org)
87h6mv96fy.fsf@gmail.com
Hi,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (55 lines)
> Ulf Herrman <striness@tilde.club> skribis:
>
>> -(define (build-system-with-package-mapping bs rewrite)
>> +(define (build-system-with-package-mapping bs rewrite-input rewrite-argument)
>> "Return a variant of BS, a build system, that rewrites a bag's inputs by
>> passing them through REWRITE, a procedure that takes an input tuplet and
>> returns a \"rewritten\" input tuplet."
>> @@ -1442,9 +1442,10 @@ (define (build-system-with-package-mapping bs rewrite)
>> (let ((lowered (apply lower args)))
>> (bag
>> (inherit lowered)
>> - (build-inputs (map rewrite (bag-build-inputs lowered)))
>> - (host-inputs (map rewrite (bag-host-inputs lowered)))
>> - (target-inputs (map rewrite (bag-target-inputs lowered))))))
>> + (build-inputs (map rewrite-input (bag-build-inputs lowered)))
>> + (host-inputs (map rewrite-input (bag-host-inputs lowered)))
>> + (target-inputs (map rewrite-input (bag-target-inputs lowered)))
>> + (arguments (map rewrite-argument (bag-arguments lowered))))))
>
> Aah, now I understand. :-)
>
> It’s indeed the case that arguments can capture references to packages
> that won’t be caught by ‘package-mapping’. For instance, if you write:
>
> (package
> …
> (arguments (list … #~(whatever #$coreutils))))
>
> … then ‘coreutils’ here cannot be replaced.
>
> To address this, the recommendation is to always add dependencies to
> input fields and to use self-references within arguments. The example
> above should be written like this:
>
> (package
> …
> (arguments
> (list … #~(whatever #$(this-package-input "coreutils")))))
>
> It’s just a recommendation and one can perfectly ignore it, and I
> suppose that was the impetus for this patch.
>
> This is one of the things discussed while designing this change:
>
> https://guix.gnu.org/en/blog/2021/the-big-change/
> (search for “self-referential records”)
>
> https://issues.guix.gnu.org/49169
>
> My take was and still is that it’s an acceptable limitation. Packagers
> need to follow the guideline above if they want proper support for
> rewriting, ‘guix graph’, and other tools.
>
> WDYT?

But not all packages found in a bag come from inputs; they may be
provided as an argument to the build system (cmake, meson, qt, etc. all
allow for this).

--
Thanks,
Maxim
U
U
Ulf Herrman wrote on 13 Oct 2023 03:49
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
871qdzfg0q.fsf@tilde.club
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

Toggle quote (3 lines)
> Any build system accepting package as arguments are subject to this
> problem, if I understand correctly.

It's not quite everywhere a package is accepted as an argument: there
are many cases where a package is passed as a package argument but
doesn't end up in the arguments list of the corresponding bag. This is
usually the case because that argument is only overriding a default
package that is just added as an input to the bag, with no special
treatment aside from being an implicit input. For example, #:cmake in
cmake-build-system works this way.

This is however not the case for #:qtbase in qt-build-system, and, much
more prominently, for #:guile. Neither qtbase nor guile are added to
bag inputs implicitly, but they do end up in the final builder, even if
not specified.

Concretely, this looks like this:
---------------------------------
(use-modules (guix packages)
(guix profiles)
(gnu packages base))

(define guile-named-lyle
(package
(inherit (default-guile))
(name "lyle")))

;; contrived example that only replaces hello's immediate dependencies
(define hello-transformer
(package-mapping (lambda (p0)
(if (eq? p0 (default-guile))
guile-named-lyle
p0))
(lambda (p)
(not (eq? p hello)))
#:deep? #t))

(define hello-with-lyle
(hello-transformer hello))

(packages->manifest (list hello hello-with-lyle))

;; $ guix build --derivations --manifest=THISFILE
;; Expectation: build for hello-with-lyle is done with guile-named-lyle.
;; Reality: derivation for hello-with-lyle is the same as hello.
---------------------------------
(and I have confirmed that the above results in guile-named-lyle being
used with the proposed patches applied)

A similar problem manifests when one tries replacing (default-qtbase) in
a package using qt-build-system. Both it and guile are in bag arguments
and not inputs because it's not enough that they be included as inputs,
the builder must also know which one is considered "used by the build
system" if there are multiple. One could argue that this ambiguity also
exists with other build systems - "which cmake should be used, which gcc
should be used", etc - but they choose to resolve it with "eh, whatever
shows up first in PATH matching the required name is probably fine".
It's not unimaginable that there could be cases where explicitly
specifying which package should be used for a particular role would be
necessary, though.

- Ulf
-----BEGIN PGP SIGNATURE-----

iQHIBAEBCAAyFiEEn6BUn0yca1D9JsMa1lV76sJM9mgFAmUooikUHHN0cmluZXNz
QHRpbGRlLmNsdWIACgkQ1lV76sJM9mgZ3gv+N+aiM4gJM3swNUPoc5C7QWKvgkU8
V0svynQ/4gFjHqDmqCBNzttgG4Cnquv7wm6RT8/U+PyifBCNzSsdRUmh/aGaajQy
NLB4rIQ3Db8Secgg416bMULcCRyabgGyYH7Yx4LfBq5vI1WJ8vknJ6h/SeBhkqnP
fPHzwaAO8L53GmlidlVkvYir8uLzr5I/si6Odzl6ewD2uT5K3DkGNwgqK5fD2HOz
o/dIa355lBxia/mXtTcY/cXcnr5F8skLJvGOZWkWn8pAzBz++DkS8tZPzW9mMwiM
VG/KKGT5mbEB2LvGajUh4u8dhVLlMIrCzT4HbH49GgrRBNfOl/IzdybZtBITlKHR
6g5K8aOWcBxGeBeuGY6N6if22fVUYDczseFNI5GicV15YPA6FgTTjgKUCNg3mYmU
HuNzR5RDwvXYXEKLD6ICm1vOtbOwrAS6yVyiuZFtaBKBXoI+uazJxH/r/anCTjar
Nwb5xoLqjDsPSZzBfynl/xMF9nnHNYc+Y22o
=OGXK
-----END PGP SIGNATURE-----

U
U
Ulf Herrman wrote on 13 Oct 2023 05:11
(name . Ludovic Courtès)(address . ludo@gnu.org)
87wmvrdxnf.fsf@tilde.club
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (42 lines)
> Ulf Herrman <striness@tilde.club> skribis:
>
>> -(define (build-system-with-package-mapping bs rewrite)
>> +(define (build-system-with-package-mapping bs rewrite-input rewrite-argument)
>> "Return a variant of BS, a build system, that rewrites a bag's inputs by
>> passing them through REWRITE, a procedure that takes an input tuplet and
>> returns a \"rewritten\" input tuplet."
>> @@ -1442,9 +1442,10 @@ (define (build-system-with-package-mapping bs rewrite)
>> (let ((lowered (apply lower args)))
>> (bag
>> (inherit lowered)
>> - (build-inputs (map rewrite (bag-build-inputs lowered)))
>> - (host-inputs (map rewrite (bag-host-inputs lowered)))
>> - (target-inputs (map rewrite (bag-target-inputs lowered))))))
>> + (build-inputs (map rewrite-input (bag-build-inputs lowered)))
>> + (host-inputs (map rewrite-input (bag-host-inputs lowered)))
>> + (target-inputs (map rewrite-input (bag-target-inputs lowered)))
>> + (arguments (map rewrite-argument (bag-arguments lowered))))))
>
> Aah, now I understand. :-)
>
> It’s indeed the case that arguments can capture references to packages
> that won’t be caught by ‘package-mapping’. For instance, if you write:
>
> (package
> …
> (arguments (list … #~(whatever #$coreutils))))
>
> … then ‘coreutils’ here cannot be replaced.
>
> To address this, the recommendation is to always add dependencies to
> input fields and to use self-references within arguments. The example
> above should be written like this:
>
> (package
> …
> (arguments
> (list … #~(whatever #$(this-package-input "coreutils")))))
>
> It’s just a recommendation and one can perfectly ignore it, and I
> suppose that was the impetus for this patch.

That and a growing thirst for a nuclear option for package rewriting
brought about by trying to debug deep transformations while
simultaneously experimenting with rewriting a manifest of around 270
packages.

There are really several distinct issues at play here:
1. The case of #:qtbase and #:guile being invisible to package-mapping.
This is what I first noticed, and cannot be fixed without modifying
the build systems. This is what prompted looking for packages in
package and bag arguments, and recursing into lists therein (just in
case an argument took a list of packages).
2. The (perceived) case of packages hiding inside arguments. In
hindsight, this was probably actually (3) causing this, though it's
hard to tell because I discovered it last. I attempted to resolve
this by recursing through <gexp>s and <gexp-input>s.
3. `this-package' referring to the inherited package in thunked fields,
rather than the package inheriting them. This is what prompted the
use of package-{inputs,arguments,etc}-with-package.

(1) could be resolved in several ways. I'm partial to looking for
arguments whose values are packages and transforming them (when #:deep?
#t is specified), and adjusting the build systems to make that work
consistently, which is what I've done.

(2) is probably not an issue, though it occurs to me that the technique
of recursively searching through arguments looking for packages could be
used to implement a sort of automated "transformability" check, which
could help a lot when trying to debug transformations.

(3) is a major issue; the entire strategy of using `this-package-input'
to enable transformations breaks because of it. My fix works for me at
least, though it requires exposing additional low-level procedures and
transformation authors using them. I'll open another bug about it, as
requested.

Toggle quote (13 lines)
> This is one of the things discussed while designing this change:
>
> https://guix.gnu.org/en/blog/2021/the-big-change/
> (search for “self-referential records”)
>
> https://issues.guix.gnu.org/49169
>
> My take was and still is that it’s an acceptable limitation. Packagers
> need to follow the guideline above if they want proper support for
> rewriting, ‘guix graph’, and other tools.
>
> WDYT?

I think it's probably reasonable, though I would like it if there were
tooling in place to detect cases where said guidelines are not
followed, so as to aid in debugging and verifying that a transformation
worked as intended.

- Ulf
-----BEGIN PGP SIGNATURE-----

iQHIBAEBCAAyFiEEn6BUn0yca1D9JsMa1lV76sJM9mgFAmUotWQUHHN0cmluZXNz
QHRpbGRlLmNsdWIACgkQ1lV76sJM9mgcHQv/cLA17ks/VPFW1MKvakcWKjbhqOya
ZoJcYoUsCporAu8xONfRKpZ3sSWPwbL4I/WS57DClxkYO5Xf3f5pgH5xJo24CT3h
gkorqQikxsoAUhfuLeoG9HchGrA+SX0MJIEQkCv0Fg7YXc4imsHAH97EOPnwy3L3
dslCDKLFLTJMDhnFLHOksaeAFlQUqhDNHnu6v8AhGVcvS5Vqxq0KkJwWS4knggzX
jvC+tO/JVRyXrxvvUVTtCCyXGvv6r9VA0KAXbwRT2FbII7DkfQBA1/xA8tkKOmCi
QgC4VddTd2poemWeUKL9RPeWv7VhhrTXyxFKssu5SMyo0Y1apcbBCstk9Py+FvSP
YUWzjZwUy1eKFax7WIXpTIE8KpRWjkOX0Rnb9ES3f4T+lbQ13O9Mb0Kjg6HLxV8K
Q7mvQ8P9H5rXWSE7AbnGXB+UlO+Jmlkxc2XBzxc7Zv7QHCinSiG04efZdSlPPSW+
wfA+Q7UdrkjBqZ7lWoa8PXNgKSWtvlb3YP/j
=lzX7
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 14 Oct 2023 16:47
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87jzrpgt1h.fsf@gnu.org
Hello,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (38 lines)
>> It’s indeed the case that arguments can capture references to packages
>> that won’t be caught by ‘package-mapping’. For instance, if you write:
>>
>> (package
>> …
>> (arguments (list … #~(whatever #$coreutils))))
>>
>> … then ‘coreutils’ here cannot be replaced.
>>
>> To address this, the recommendation is to always add dependencies to
>> input fields and to use self-references within arguments. The example
>> above should be written like this:
>>
>> (package
>> …
>> (arguments
>> (list … #~(whatever #$(this-package-input "coreutils")))))
>>
>> It’s just a recommendation and one can perfectly ignore it, and I
>> suppose that was the impetus for this patch.
>>
>> This is one of the things discussed while designing this change:
>>
>> https://guix.gnu.org/en/blog/2021/the-big-change/
>> (search for “self-referential records”)
>>
>> https://issues.guix.gnu.org/49169
>>
>> My take was and still is that it’s an acceptable limitation. Packagers
>> need to follow the guideline above if they want proper support for
>> rewriting, ‘guix graph’, and other tools.
>>
>> WDYT?
>
> But not all packages found in a bag come from inputs; they may be
> provided as an argument to the build system (cmake, meson, qt, etc. all
> allow for this).

Yes, but these packages (#:cmake for ‘cmake-build-system’, #:python for
‘python-build-system’, etc.) become inputs of the bag, and are taken
into account, as in this example:

Toggle snippet (33 lines)
$ ./pre-inst-env guix show tinyxml2
name: tinyxml2
version: 8.0.0
outputs:
+ out: everything
systems: x86_64-linux i686-linux
dependencies:
location: gnu/packages/xml.scm:1295:2
homepage: http://www.grinninglizard.com/tinyxml2/
license: Zlib
synopsis: Small XML parser for C++
description: TinyXML2 is a small and simple XML parsing library for the C++ programming
+ language.

$ ./pre-inst-env guix build tinyxml2
The following graft will be made:
/gnu/store/qf8w8ch1mrxk6k34g2d6bisny5pq8gnv-tinyxml2-8.0.0.drv
applying 2 grafts for tinyxml2-8.0.0 ...
grafting '/gnu/store/ndzyiwhypsbgrbaz7rmz3649b6ghlfal-tinyxml2-8.0.0' -> '/gnu/store/2da5frl36gd51653bjjypqvwdk0bh25v-tinyxml2-8.0.0'...
successfully built /gnu/store/qf8w8ch1mrxk6k34g2d6bisny5pq8gnv-tinyxml2-8.0.0.drv
/gnu/store/2da5frl36gd51653bjjypqvwdk0bh25v-tinyxml2-8.0.0
$ ./pre-inst-env guix build tinyxml2 --with-input=cmake-minimal=cmake
The following derivation will be built:
/gnu/store/hx0hx1nrmxzdhy9yw1d0md2q78y4spd4-tinyxml2-8.0.0.drv
8.4 MB will be downloaded:
/gnu/store/qwy25ivr63087x9fdl9km0m44hn5bimg-cmake-3.25.1
/gnu/store/3vh9b2i93na8wwdrj6n0q2qbgd2fzxik-tinyxml2-8.0.0-checkout

[...]

/gnu/store/cmn551x8iksy44cdqypyq2129lm8ym60-tinyxml2-8.0.0

(In this case I had to apply the patch below because ‘cmake-minimal’ was
mistakenly marked as hidden which, as implemented by
eee95b5a879b7096dffd533f24107cf8926b621e, meant that it could not be
replaced.)

I gave a similar example with ‘python-build-system’ here:


My understanding is that understanding is that this patch series is
about addressing cases as I explained above, where ‘arguments’ refer to
packages directly instead of using self references. Do I get this
right, Ulf?

Thanks,
Ludo’.
Toggle diff (12 lines)
diff --git a/gnu/packages/cmake.scm b/gnu/packages/cmake.scm
index bc14286070..1592703f8d 100644
--- a/gnu/packages/cmake.scm
+++ b/gnu/packages/cmake.scm
@@ -263,6 +263,7 @@ (define-public cmake-minimal
(package
(inherit cmake-bootstrap)
(name "cmake-minimal")
+ (properties (alist-delete 'hidden? (package-properties cmake-bootstrap)))
(source (origin
(inherit (package-source cmake-bootstrap))
;; Purge CMakes bundled dependencies as they are no longer needed.
L
L
Ludovic Courtès wrote on 14 Oct 2023 17:18
(name . Ulf Herrman)(address . striness@tilde.club)(address . 65665@debbugs.gnu.org)
87y1g5fd1e.fsf@gnu.org
Hello,

Ulf Herrman <striness@tilde.club> skribis:

Toggle quote (5 lines)
> That and a growing thirst for a nuclear option for package rewriting
> brought about by trying to debug deep transformations while
> simultaneously experimenting with rewriting a manifest of around 270
> packages.

On that topic of a catch-all option for rewriting: there’s an unused
procedure called ‘map-derivation’, which performs rewriting at the level
of derivations. It’s harder to use, more expensive, but who knows,
perhaps we’ll find a motivating use case… (Allowing for graph rewriting
has been a major goal for me since the beginning.)

Toggle quote (7 lines)
> There are really several distinct issues at play here:
> 1. The case of #:qtbase and #:guile being invisible to package-mapping.
> This is what I first noticed, and cannot be fixed without modifying
> the build systems. This is what prompted looking for packages in
> package and bag arguments, and recursing into lists therein (just in
> case an argument took a list of packages).

How are #:qtbase and #:guile invisible to package mapping?

They appear in the bag inputs and thus are definitely visible to
‘package-mapping’, as a I showed with the CMake and Python examples in
this thread.

For good measure :-) here’s an example with #:qtbase:

Toggle snippet (26 lines)
$ ./pre-inst-env guix build qgit -n
substitute: updating substitutes from 'http://192.168.1.48:8123'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
0.4 MB would be downloaded:
/gnu/store/7b20q17yg90b62404chgbnwgvd6ry1qf-qgit-2.10
$ ./pre-inst-env guix build qgit -n --with-latest=qtbase
following redirection to `https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/'...
following redirection to `https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/6.6/'...
guix build: warning: cannot authenticate source of 'qtbase', version 6.6.0

Starting download of /tmp/guix-file.CTehnY
From https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/6.6/6.6.0/submodules/qtbase-everywhere-src-6.6.0.tar.xz...
…-src-6.6.0.tar.xz 46.1MiB 12.9MiB/s 00:04 ???????????????????? 100.0%
substitute: updating substitutes from 'http://192.168.1.48:8123'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://guix.bordeaux.inria.fr'... 100.0%
The following derivations would be built:
/gnu/store/paixxkdaakv55bffggxx4l9hiknl8i5r-qgit-2.10.drv
/gnu/store/f9fdjk1g1s1aqmlmi4clla2kqns7283v-qtbase-6.6.0.drv
0.4 MB would be downloaded:
/gnu/store/nl9dadzfmjm9wg7v3r31jkx773dl683x-module-import-compiled
/gnu/store/6zryxmypw0wygayc9pvhyxkx47w0lyci-gperf-3.1
/gnu/store/a57n7wy8mdi7l52pr4zg07132blgj5xp-qgit-2.10-checkout

Now, package transformation options are almost all implemented in terms
of ‘package-input-rewriting/spec’, and I have to admit that it’s much
easier to use than ‘package-mapping’. The latter is low-level and it
can be a source of headaches.

Toggle quote (5 lines)
> 2. The (perceived) case of packages hiding inside arguments. In
> hindsight, this was probably actually (3) causing this, though it's
> hard to tell because I discovered it last. I attempted to resolve
> this by recursing through <gexp>s and <gexp-input>s.

Yeah. I think we have to understand that “hiding” is to some extent
inevitable; <package> is more concise than <bag>, which is more concise
than <derivation>. There are extra “inputs” showing up when we go from
one abstraction to the one below.

Toggle quote (4 lines)
> 3. `this-package' referring to the inherited package in thunked fields,
> rather than the package inheriting them. This is what prompted the
> use of package-{inputs,arguments,etc}-with-package.

Ah yes, I agree; I reported it here:


I think we can discuss it separately though, in that bug report
probably.

Toggle quote (10 lines)
> (1) could be resolved in several ways. I'm partial to looking for
> arguments whose values are packages and transforming them (when #:deep?
> #t is specified), and adjusting the build systems to make that work
> consistently, which is what I've done.
>
> (2) is probably not an issue, though it occurs to me that the technique
> of recursively searching through arguments looking for packages could be
> used to implement a sort of automated "transformability" check, which
> could help a lot when trying to debug transformations.

OK.

Toggle quote (6 lines)
> (3) is a major issue; the entire strategy of using `this-package-input'
> to enable transformations breaks because of it. My fix works for me at
> least, though it requires exposing additional low-level procedures and
> transformation authors using them. I'll open another bug about it, as
> requested.

Yeah, understood.

Thanks,
Ludo’.
U
U
Ulf Herrman wrote on 15 Oct 2023 09:12
(name . Ludovic Courtès)(address . ludo@gnu.org)
87o7h0e4us.fsf@tilde.club
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (28 lines)
> Hello,
>
> Ulf Herrman <striness@tilde.club> skribis:
>
>> That and a growing thirst for a nuclear option for package rewriting
>> brought about by trying to debug deep transformations while
>> simultaneously experimenting with rewriting a manifest of around 270
>> packages.
>
> On that topic of a catch-all option for rewriting: there’s an unused
> procedure called ‘map-derivation’, which performs rewriting at the level
> of derivations. It’s harder to use, more expensive, but who knows,
> perhaps we’ll find a motivating use case… (Allowing for graph rewriting
> has been a major goal for me since the beginning.)
>
>> There are really several distinct issues at play here:
>> 1. The case of #:qtbase and #:guile being invisible to package-mapping.
>> This is what I first noticed, and cannot be fixed without modifying
>> the build systems. This is what prompted looking for packages in
>> package and bag arguments, and recursing into lists therein (just in
>> case an argument took a list of packages).
>
> How are #:qtbase and #:guile invisible to package mapping?
>
> They appear in the bag inputs and thus are definitely visible to
> ‘package-mapping’, as a I showed with the CMake and Python examples in
> this thread.

"Invisible to package-mapping" was perhaps not the clearest phrasing;
"not completely replaced by package-mapping" might be better. qtbase
does indeed go in the bag inputs, but replacing the bag inputs has no
effect on the bag arguments, and even if we replaced it in the bag
arguments as well, it might not even be in there to begin with, in which
case the bag-builder would introduce a default that is only visible at
the level of derivations.

Toggle quote (28 lines)
>
> For good measure :-) here’s an example with #:qtbase:
>
> $ ./pre-inst-env guix build qgit -n
> substitute: updating substitutes from 'http://192.168.1.48:8123'... 100.0%
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> 0.4 MB would be downloaded:
> /gnu/store/7b20q17yg90b62404chgbnwgvd6ry1qf-qgit-2.10
> $ ./pre-inst-env guix build qgit -n --with-latest=qtbase
> following redirection to `https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/'...
> following redirection to `https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/6.6/'...
> guix build: warning: cannot authenticate source of 'qtbase', version 6.6.0
>
> Starting download of /tmp/guix-file.CTehnY
> From https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/6.6/6.6.0/submodules/qtbase-everywhere-src-6.6.0.tar.xz...
> …-src-6.6.0.tar.xz 46.1MiB 12.9MiB/s 00:04 ???????????????????? 100.0%
> substitute: updating substitutes from 'http://192.168.1.48:8123'... 100.0%
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
> substitute: updating substitutes from 'https://guix.bordeaux.inria.fr'... 100.0%
> The following derivations would be built:
> /gnu/store/paixxkdaakv55bffggxx4l9hiknl8i5r-qgit-2.10.drv
> /gnu/store/f9fdjk1g1s1aqmlmi4clla2kqns7283v-qtbase-6.6.0.drv
> 0.4 MB would be downloaded:
> /gnu/store/nl9dadzfmjm9wg7v3r31jkx773dl683x-module-import-compiled
> /gnu/store/6zryxmypw0wygayc9pvhyxkx47w0lyci-gperf-3.1
> /gnu/store/a57n7wy8mdi7l52pr4zg07132blgj5xp-qgit-2.10-checkout

And if you do a quick 'cat
/gnu/store/paixxkdaakv55bffggxx4l9hiknl8i5r-qgit-2.10.drv'...

------------------------------
Derive
([("out","/gnu/store/z8qrhfmicylxy2mwvcvh9sizfhd3x4d3-qgit-2.10","","")]
,[("/gnu/store/0jk33gpzyicyppbnh458213007v0qjhs-mesa-23.1.4.drv",["out"])
,("/gnu/store/0njvvgjlb52abhqnmb4rx22sfkxm2h9c-gcc-11.3.0.drv",["out"])
,("/gnu/store/0p1sns81qbgr8ayiv02fv4rm5drcycd7-libxdamage-1.1.5.drv",["out"])
,("/gnu/store/14a2ban238fng3c8632lrfkmz54y7m2c-binutils-2.38.drv",["out"])
,("/gnu/store/1n39zcbr528b7rh9bf1pwfrm7mv8nr8m-bzip2-1.0.8.drv",["out"])
,("/gnu/store/2pv3mjjiwh37b0m3m1hijxifnchrw76i-libpciaccess-0.16.drv",["out"])
,("/gnu/store/3fy3bf7wysi1n1qz9jz8xzx11sgy8m6d-git-2.41.0.drv",["out"])
,("/gnu/store/3r0r8j76l0qvxasmb7rgn7lvpikjdyn1-libxdmcp-1.1.3.drv",["out"])
,("/gnu/store/4jfy1ca1d5772z15jcyk1v8wdwdcllbi-gzip-1.12.drv",["out"])
,("/gnu/store/5nvwagz2hphvlax2bnj93smr1rgrzr8l-libx11-1.8.1.drv",["out"])
,("/gnu/store/64vwaah2spd7q66hji6sm1j2fl6pd1rn-diffutils-3.8.drv",["out"])
,("/gnu/store/6k4xxkp725r09vkn7rz2gc50asjjhpkk-xorgproto-2022.2.drv",["out"])
,("/gnu/store/91b6yraa6qax7lq7riqg1ag6lql2gfzi-tar-1.34.drv",["out"])
,("/gnu/store/9kcv1x0lrf6fdck2j42zarxrvjzxxznv-coreutils-9.1.drv",["out"])
,("/gnu/store/9piaq0aaf202r1gq7crig1cr131kx8zn-file-5.44.drv",["out"])
,("/gnu/store/9y28bf3ywai2ybhr92c904s3cxsc8apx-libpthread-stubs-0.4.drv",["out"])
,("/gnu/store/ak17xsjb4zcw7sf0r0lxxiy4xmh57i2h-findutils-4.9.0.drv",["out"])
,("/gnu/store/bmc1hqlb207n1mnf9rs7wy8zh9hhf0br-qgit-2.10-checkout.drv",["out"])
,("/gnu/store/cmd13zzya808ca8siibj8ib3llpkrr3f-sed-4.8.drv",["out"])
,("/gnu/store/cr84vpnszpy3hkwsm8ijwpnkknxvx8k4-grep-3.8.drv",["out"])
,("/gnu/store/cx6ws98i67qf6ba9msz7n63x78ky812c-libxxf86vm-1.1.4.drv",["out"])
,("/gnu/store/f9fdjk1g1s1aqmlmi4clla2kqns7283v-qtbase-6.6.0.drv",["out"])
,("/gnu/store/fpprrvfiqw6g0mn5gb7ac87nbmrsq64l-make-4.3.drv",["out"])
,("/gnu/store/g2z0fvzkh5xl4f00aixchkwxrwbbljlr-libxext-1.3.4.drv",["out"])
,("/gnu/store/gb247cil5nlnx175dhqmgg67q7ng7n2h-which-2.21.drv",["out"])
,("/gnu/store/hb2y5axynnpwqpx1znjsn5azspc9a4lw-module-import-compiled.drv",["out"])
,("/gnu/store/hjhr64r5x3bhdw63zz3a3v09vfrlkhrh-qtbase-5.15.10.drv",["out"])
,("/gnu/store/hvqfl21wx397k58jpn5dpn2l5y5k1dl0-patch-2.7.6.drv",["out"])
,("/gnu/store/kn7kcpkkcvy6gmrc74mf4mq0290dsf94-glibc-2.35.drv",["out","static"])
,("/gnu/store/l3ns54f8vgqmycwi50p5cwbr6l466kj3-libdrm-2.4.114.drv",["out"])
,("/gnu/store/lfsv077ggq8w8cbw86wf1075zhb3bhrw-xz-5.2.8.drv",["out"])
,("/gnu/store/lssiz5ppxfybwmr638x3bcmn4b6ixk89-linux-libre-headers-5.15.49.drv",["out"])
,("/gnu/store/n4p8vzp14k1pkgxyfb5mak12rm1yiwnr-cmake-minimal-3.24.2.drv",["out"])
,("/gnu/store/nldpb7xl1slkiigvr9rax7qhcky5flnc-util-macros-1.19.3.drv",["out"])
,("/gnu/store/nvaxgm6jvjy6agmp89ivahz7z9vl3ldh-libxcb-1.15.drv",["out"])
,("/gnu/store/pdkcmkk5h3zq82ws9fps4bxnfmsmz0kx-libxshmfence-1.3.drv",["out"])
,("/gnu/store/s5nyfh0cd3z9wsk69c9blj5y850kvvcm-libvdpau-1.5.drv",["out"])
,("/gnu/store/swblndkq2c4rzyv3xfkmsa4cjf6abphf-ld-wrapper-0.drv",["out"])
,("/gnu/store/vkns1vij5hjamh7dpd74zs5203526747-libxfixes-6.0.0.drv",["out"])
,("/gnu/store/w2mnryfds1bd7wyyqmdi0kblz0dym1bx-glibc-utf8-locales-2.35.drv",["out"])
,("/gnu/store/wwmk0rrnb5q8f48fm7h6grzar5qmslgi-libxau-1.0.10.drv",["out"])
,("/gnu/store/xwckz4hbfydkfiiaa7bgslbh317gdkiq-bash-minimal-5.1.16.drv",["out"])
,("/gnu/store/z7fxw9jj7avcr1ng88pak3ds3kxbfy15-gawk-5.2.1.drv",["out"])
,("/gnu/store/zraigp7miin3vzr5dcbr4i9rvds0i07r-guile-3.0.9.drv",["out"])]
,["/gnu/store/2lg0ibv3vw01xc83advvrhvmc8yxbhh0-qgit-2.10-builder","/gnu/store/p173g6kxa69qaypf8dvnw1ismd8g4k8q-module-import"]
,"x86_64-linux","/gnu/store/g8p09w6r78hhkl2rv1747pcp9zbk6fxv-guile-3.0.9/bin/guile",["--no-auto-compile","-L","/gnu/store/p173g6kxa69qaypf8dvnw1ismd8g4k8q-module-import","-C","/gnu/store/nl9dadzfmjm9wg7v3r31jkx773dl683x-module-import-compiled","/gnu/store/2lg0ibv3vw01xc83advvrhvmc8yxbhh0-qgit-2.10-builder"]
,[("out","/gnu/store/z8qrhfmicylxy2mwvcvh9sizfhd3x4d3-qgit-2.10")])
---------------------------

you'll see we have both a
("/gnu/store/f9fdjk1g1s1aqmlmi4clla2kqns7283v-qtbase-6.6.0.drv",["out"])
and a
("/gnu/store/hjhr64r5x3bhdw63zz3a3v09vfrlkhrh-qtbase-5.15.10.drv",["out"])
in use.

(and if this were a transformation that applied to all packages, it
would be using two variants of the entire world beneath qtbase also)

Toggle quote (10 lines)
>> 2. The (perceived) case of packages hiding inside arguments. In
>> hindsight, this was probably actually (3) causing this, though it's
>> hard to tell because I discovered it last. I attempted to resolve
>> this by recursing through <gexp>s and <gexp-input>s.
>
> Yeah. I think we have to understand that “hiding” is to some extent
> inevitable; <package> is more concise than <bag>, which is more concise
> than <derivation>. There are extra “inputs” showing up when we go from
> one abstraction to the one below.

Ideally no new packages would be introduced when going from bags to
derivations - there is also information at the package level that is
missing at the derivation level, like properties and arguments, that
transformations should be able to access.
Toggle quote (11 lines)
>
>> 3. `this-package' referring to the inherited package in thunked fields,
>> rather than the package inheriting them. This is what prompted the
>> use of package-{inputs,arguments,etc}-with-package.
>
> Ah yes, I agree; I reported it here:
>
> https://issues.guix.gnu.org/50335
>
> I think we can discuss it separately though, in that bug report
> probably.
Ah, in the meantime I created issue #66510. I suppose I should merge
that. I think the syntactic option presented there seems promising -
extending record constructors so that (inherit parent) can be replaced
with (inherit parent (fieldname variable-like-macro-name) ...) to cause
uses of variable-like-macro-name within the constructor to refer to the
parent's value of fieldname, with any field thunk of the parent
evaluated while `this-package' is bound to the package being defined.

- Ulf
-----BEGIN PGP SIGNATURE-----

iQHIBAEBCAAyFiEEn6BUn0yca1D9JsMa1lV76sJM9mgFAmUrkOwUHHN0cmluZXNz
QHRpbGRlLmNsdWIACgkQ1lV76sJM9mgTWAv+M3SCZPO5+RvL+fOfa1hVFhHkbwM7
JURJ43VQadC/I1VlPCHfAb/gJ+8aXG+Up/fkEU5tDEg2HLxwSsGRR6Ser/iAjtIC
Z508AZ2Y85bdwMk+8NAJIIfrQEZb6pJJKXcH6Nls6/sVkZFgtmjxjW6aDlLC/mkg
ETZQwjfcF5e5ZsimzFkLBT+B558gQDpvAtjDD42jQ10NZJOJlc0mRpeR1I3YzbMk
QBy1VjdqiBFYJ1hO5VRZ3egNki6m0aTOYrPNR57ogyJ1HQHg55sPqY/ZcjKhNWAz
SJHs+jX0Xvx33G9AGnS5/c71e7TxI6YDN7ANZlQMJlkUm+BBxb7yJJ+YxDflXI/A
eZzCsrSRg0R+IS/QeNwmrP4f3KNcHkii8Njn7HdngZJ2SINUWa0OsXRVGN+vYwVD
Ql8dMuCsigPGaObe4BgszTJdh/D7IJ066HlXfuZNTYbjTRLk5k+g0p+slfooZjQr
VBakEtN4co4UQjokOM/JDzUqM+PcOehcUlZJ
=NZ9m
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 21 Oct 2023 16:31
(name . Ulf Herrman)(address . striness@tilde.club)(address . 65665@debbugs.gnu.org)
87wmvgoxn7.fsf@gnu.org
Hi,

Ulf Herrman <striness@tilde.club> skribis:

Toggle quote (23 lines)
>> $ ./pre-inst-env guix build qgit -n --with-latest=qtbase
>> following redirection to `https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/'...
>> following redirection to `https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/6.6/'...
>> guix build: warning: cannot authenticate source of 'qtbase', version 6.6.0
>>
>> Starting download of /tmp/guix-file.CTehnY
>> From https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/6.6/6.6.0/submodules/qtbase-everywhere-src-6.6.0.tar.xz...
>> …-src-6.6.0.tar.xz 46.1MiB 12.9MiB/s 00:04 ???????????????????? 100.0%
>> substitute: updating substitutes from 'http://192.168.1.48:8123'... 100.0%
>> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
>> substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
>> substitute: updating substitutes from 'https://guix.bordeaux.inria.fr'... 100.0%
>> The following derivations would be built:
>> /gnu/store/paixxkdaakv55bffggxx4l9hiknl8i5r-qgit-2.10.drv
>> /gnu/store/f9fdjk1g1s1aqmlmi4clla2kqns7283v-qtbase-6.6.0.drv
>> 0.4 MB would be downloaded:
>> /gnu/store/nl9dadzfmjm9wg7v3r31jkx773dl683x-module-import-compiled
>> /gnu/store/6zryxmypw0wygayc9pvhyxkx47w0lyci-gperf-3.1
>> /gnu/store/a57n7wy8mdi7l52pr4zg07132blgj5xp-qgit-2.10-checkout
>
> And if you do a quick 'cat
> /gnu/store/paixxkdaakv55bffggxx4l9hiknl8i5r-qgit-2.10.drv'...

[...]

Toggle quote (9 lines)
> you'll see we have both a
> ("/gnu/store/f9fdjk1g1s1aqmlmi4clla2kqns7283v-qtbase-6.6.0.drv",["out"])
> and a
> ("/gnu/store/hjhr64r5x3bhdw63zz3a3v09vfrlkhrh-qtbase-5.15.10.drv",["out"])
> in use.
>
> (and if this were a transformation that applied to all packages, it
> would be using two variants of the entire world beneath qtbase also)

D’oh! Now we have a bug to chew. (Sorry if this was obvious to you
from the start; it wasn’t to me!)

I don’t know, should we start by having a proper bug report for this and
study how this happen?

Again I’m sorry if I’m slow to understand, but I’d like to make sure we
have a good understanding of the problem before we start discussing
solutions.

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 21 Oct 2023 16:31
control message for bug #65665
(address . control@debbugs.gnu.org)
87v8b0oxms.fsf@gnu.org
severity 65665 important
quit
U
U
Ulf Herrman wrote on 22 Oct 2023 00:22
Re: bug#65665: package-mapping with #:deep? #t doesn't get all the implicit inputs
(name . Ludovic Courtès)(address . ludo@gnu.org)
8734y3r4z1.fsf@tilde.club
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (3 lines)
> I don’t know, should we start by having a proper bug report for this and
> study how this happen?

Does that mean opening a new issue, or what? The bug you've described
is the one this issue was initially opened for.

Toggle quote (4 lines)
> Again I’m sorry if I’m slow to understand, but I’d like to make sure we
> have a good understanding of the problem before we start discussing
> solutions.

Okay, I suppose I should have given a concrete example of the behavior.
The qgit example can fill that role:

$ cat $(./pre-inst-env guix build qgit -n --derivations --no-grafts --with-latest=qtbase)
<observe that an untransformed qtbase is present>

The way this happens is that qt-build-system's bag-builder
(e.g. qt-build, qt-cross-build)) introduces a reference to the qtbase
from its #:qtbase argument into the gexp it creates a derivation from,
completely independently of any inputs. qgit doesn't explicitly specify
#:qtbase, and qt-build-system's 'lower' procedure doesn't add one, so
the default value for that keyword argument, (default-qtbase), is used
in qt-build.

This default value can only be overridden by explicitly passing #:qtbase
as a package or bag argument. This requirement doesn't map well to a
generic package transformation interface at all - it requires that every
transformer (e.g. package-mapping) check if the package it's
transforming is using qt-build-system, and if so explicitly supply a
#:qtbase that is the result of transforming the original implicit or
explicit value, after somehow figuring out what that may be (currently
only doable by manually reading guix/build-system/qt.scm).

This behavior is also currently exhibited by all build systems' handling
of #:guile. Here's a concrete example of that, taken from another
mailing I sent to this issue (https://issues.guix.gnu.org/65665#15):

---------------------------------
(use-modules (guix packages)
(guix profiles)
(gnu packages base))

(define guile-named-lyle
(package
(inherit (default-guile))
(name "lyle")))

;; contrived example that only replaces hello's immediate dependencies
(define hello-transformer
(package-mapping (lambda (p0)
(if (eq? p0 (default-guile))
guile-named-lyle
p0))
(lambda (p)
(not (eq? p hello)))
#:deep? #t))

(define hello-with-lyle
(hello-transformer hello))

(packages->manifest (list hello hello-with-lyle))

;; $ guix build --derivations --manifest=THISFILE
;; Expectation: build for hello-with-lyle is done with guile-named-lyle.
;; Reality: derivation for hello-with-lyle is the same as hello.
---------------------------------

Hopefully that makes it clear why this is happening.

As for solutions, some options that come to mind:

1. guile and qtbase, instead of being passed to bag builders as a
separate argument, are passed as a bag input that is looked up by
magic input label, e.g. "guile-for-build", "qtbase-for-build", etc.
Seems fragile, but requires no changes to package-mapping, etc.

2. Modify the build systems so that these kinds of implicit arguments
that are packages are always present in bags, and the defaults for
those keyword arguments in e.g. qt-build are never used. This is the
approach I've taken with
requires that bag arguments that are packages are transformed in
addition to inputs.

I hope that counts as a proper bug report.

- Ulf
-----BEGIN PGP SIGNATURE-----

iQHIBAEBCAAyFiEEn6BUn0yca1D9JsMa1lV76sJM9mgFAmU0TxIUHHN0cmluZXNz
QHRpbGRlLmNsdWIACgkQ1lV76sJM9mhmMAwAgJPzlVRPk86Mtw7un5F/MjUXNlzX
oWPXFqUYNymnt3soKeCKhO70JOu/zRWwnve23I7Ghjbt/6fLhMPT6mT1yvZPJ6Og
sCqFhxQc3iSh7brDFIC91MEVUugcb8lY2jTOIUObeRfVRmzp78cgCzPS1n1Lo9/5
MUtcCHXoq5myYdO71nC9dF4XgAi2kTx5tBlFHcDpNlgmAHkDNyjAZ2riwxYU/yIt
wEkpUp1RZMnPnWuBBsZDpRcZadkIp5px/crPbuGYzeuTMU/VTRuaW59HIZb9Wccj
w7WToQxHRqV9kdPnZvFUGC1DiDuUI5opIunndKzNPc3ep4XGsUMXn3dUwEyVODYt
m5F5qcQWwKCS9Le7TbdHm3FBN/AL5tIXYThYJ+D0ZcGBTCyH7LGEOgTEAQqOxt7u
XXwPHHtCv2BhV8RYy3DooHBguHIxcuhSQQ+7ZnqZwuvJ86I7YyXsEjp4c08mKtwM
Ah0VyLKGrs6H7QIyyAByqp3CFFNhEmYX79bG
=z2HY
-----END PGP SIGNATURE-----

S
S
Simon Tournier wrote on 23 Oct 2023 15:53
87r0lljvhc.fsf@gmail.com
Hi,

On Sat, 21 Oct 2023 at 17:22, Ulf Herrman <striness@tilde.club> wrote:

Toggle quote (3 lines)
>> I don’t know, should we start by having a proper bug report for this and
>> study how this happen?

[...]

Toggle quote (9 lines)
>> Again I’m sorry if I’m slow to understand, but I’d like to make sure we
>> have a good understanding of the problem before we start discussing
>> solutions.
>
> Okay, I suppose I should have given a concrete example of the behavior.
> The qgit example can fill that role:
>
> $ cat $(./pre-inst-env guix build qgit -n --derivations --no-grafts --with-latest=qtbase)

If I might, Guix revision a25a492f2b, my understanding is,

$ ./pre-inst-env guix build qgit -n --with-latest=qtbase
/gnu/store/gx5d03as0k1w6jv0pssi6j69n8glf6w5-qgit-2.10.drv
/gnu/store/h02aizdjy4p10n4gmcy0y35x14lmjx3n-qtbase-6.6.0.drv

Then the builder of the derivation
/gnu/store/gx5d03as0k1w6jv0pssi6j69n8glf6w5-qgit-2.10.drv reads,

Toggle snippet (20 lines)
(begin
(use-modules
(guix build qt-build-system)
(guix build utils))
(qt-build #:source "/gnu/store/a57n7wy8mdi7l52pr4zg07132blgj5xp-qgit-2.10-checkout" #:system "x86_64-linux" #:outputs
(list
(cons "out"
((@
(guile)
getenv)
"out")))
#:inputs
(quote
(("source" . "/gnu/store/a57n7wy8mdi7l52pr4zg07132blgj5xp-qgit-2.10-checkout")
("cmake" . "/gnu/store/ygab8v4ci9iklaykapq52bfsshpvi8pw-cmake-minimal-3.24.2")
("qtbase" . "/gnu/store/khlz8afih21pd0szn5x1ck6bp6w320cz-qtbase-6.6.0")
[...]
#:qtbase "/gnu/store/h8z3xhpb8m1ih2k45935kqx0wya5w7vq-qtbase-5.15.10"

Therefore, the transformation does not rewrite #:qtbase. And note that
qtbase-5.15.10 is not listed as #:inputs.

Without the transformations, it reads,

Toggle snippet (20 lines)
(begin
(use-modules
(guix build qt-build-system)
(guix build utils))
(qt-build #:source "/gnu/store/a57n7wy8mdi7l52pr4zg07132blgj5xp-qgit-2.10-checkout" #:system "x86_64-linux" #:outputs
(list
(cons "out"
((@
(guile)
getenv)
"out")))
#:inputs
(quote
(("source" . "/gnu/store/a57n7wy8mdi7l52pr4zg07132blgj5xp-qgit-2.10-checkout")
("cmake" . "/gnu/store/ygab8v4ci9iklaykapq52bfsshpvi8pw-cmake-minimal-3.24.2")
("qtbase" . "/gnu/store/h8z3xhpb8m1ih2k45935kqx0wya5w7vq-qtbase-5.15.10")
[...]
#:qtbase "/gnu/store/h8z3xhpb8m1ih2k45935kqx0wya5w7vq-qtbase-5.15.10"

Cheers,
simon
?