[PATCH] build-system: copy: Fix cross-compilation.

  • Open
  • quality assurance status badge
Details
3 participants
  • David Elsing
  • dan
  • Ludovic Courtès
Owner
unassigned
Submitted by
dan
Severity
normal
D
D
dan wrote on 21 Apr 09:12 +0200
(address . guix-patches@gnu.org)
20a614ca2a6064bfa597cd11b7b87ceb926de866.1713682643.git.i@dan.games
* guix/build-system/copy.scm (lower): Change arguments passed to host-inputs
and build-inputs.

Change-Id: I2991854b48587ab00ccc03712304e2850727e6b7
---

This patch tries to fix a issue related to grafting when cross-compiling.
See https://issues.guix.gnu.org/66866for more discussion.

guix/build-system/copy.scm | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)

Toggle diff (37 lines)
diff --git a/guix/build-system/copy.scm b/guix/build-system/copy.scm
index d58931b33c..64bd61a53f 100644
--- a/guix/build-system/copy.scm
+++ b/guix/build-system/copy.scm
@@ -3,6 +3,7 @@
;;; Copyright © 2020 Pierre Neidhardt <mail@ambrevar.xyz>
;;; Copyright © 2021, 2022 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2023 Jonathan Brielmaier <jonathan.brielmaier@web.de>
+;;; Copyright © 2024 dan <i@dan.games>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -66,13 +67,13 @@ (define* (lower name
(bag
(name name)
(system system)
- (host-inputs `(,@(if source
- `(("source" ,source))
- '())
- ,@inputs
- ;; Keep the standard inputs of 'gnu-build-system'.
- ,@(standard-packages)))
- (build-inputs native-inputs)
+ (build-inputs `(,@(if source
+ `(("source" ,source))
+ '())
+ ,@inputs
+ ,@native-inputs
+ ;; Keep the standard inputs of 'gnu-build-system'.
+ ,@(standard-packages)))
(outputs outputs)
(build copy-build)
(arguments (strip-keyword-arguments private-keywords arguments))))

base-commit: 38b88d710ea13ba024aed0543bc2862772cdb645
--
2.41.0
L
L
Ludovic Courtès wrote on 1 May 23:39 +0200
Re: bug#66866: Grafting breaks cross-compilation
(name . dan)(address . i@dan.games)
8734r143fv.fsf_-_@gnu.org
Hi,

dan <i@dan.games> skribis:

Toggle quote (34 lines)
> I spent some time digging into the rabbit hole. After changing
> the lower function of the copy-build-system to look more like the
> lower function of the gnu-build-system, I'm able to cross compile
> alsa-lib without the --no-grafts flag. The changes I made are
> like:
>
> diff --git a/guix/build-system/copy.scm
> b/guix/build-system/copy.scm
> index d58931b33c..74304b4bfb 100644
> --- a/guix/build-system/copy.scm
> +++ b/guix/build-system/copy.scm
> @@ -66,13 +66,13 @@ (define* (lower name
> (bag
> (name name)
> (system system)
> - (host-inputs `(,@(if source
> + (build-inputs `(,@(if source
> `(("source" ,source))
> '())
> - ,@inputs
> + ,@native-inputs
> ;; Keep the standard inputs of
> 'gnu-build-system'.
> ,@(standard-packages)))
> - (build-inputs native-inputs)
> + (host-inputs inputs)
> (outputs outputs)
> (build copy-build)
> (arguments (strip-keyword-arguments private-keywords
> arguments))))
>
> Can we put everything inside build-inputs? From my understanding,
> copy-build-system shouldn't care about cross-compilation at all.

Intuitively, if ‘copy-build-system’ is about copying
architecture-independent files, then it should do the same thing whether
or not we are cross-compiling.

However, users can and do add phases whose result is
architecture-dependent. Small sample:

• ‘desec-certbot-hook’ captures a reference to curl, so it would get
the wrong one when cross-compiling if we assumed build-inputs =
host-inputs.

• ‘chez-scheme-for-racket-bootstrap-bootfiles’ builds stuff when
cross-compiling. Philip, could you explain the intent and what you
expect here?

So it would seem we can’t just assume everything is a native input like

Now, as David and you found out, the use of inputs in
build-system/copy.scm:lower is bogus. It seems that it can be fixed by
following the intended definition of build/host inputs, as David
suggested:
Toggle diff (23 lines)
diff --git a/guix/build-system/copy.scm b/guix/build-system/copy.scm
index d58931b33c2..cf0214320bf 100644
--- a/guix/build-system/copy.scm
+++ b/guix/build-system/copy.scm
@@ -66,13 +66,13 @@ (define* (lower name
(bag
(name name)
(system system)
- (host-inputs `(,@(if source
+ (build-inputs `(,@(if source
`(("source" ,source))
'())
- ,@inputs
- ;; Keep the standard inputs of 'gnu-build-system'.
- ,@(standard-packages)))
- (build-inputs native-inputs)
+ ,@native-inputs
+ ;; Keep the standard inputs of 'gnu-build-system'.
+ ,@(standard-packages)))
+ (host-inputs inputs)
(outputs outputs)
(build copy-build)
(arguments (strip-keyword-arguments private-keywords arguments))))
But wait! That’s all theoretical because the bag always has (target #f)
and ‘copy-build’ bundles build and host inputs together, as if doing a
native build.

So it seems like https://issues.guix.gnu.org/70492(putting everything
in ‘build-inputs’) is OK, after all.

But still, there seem to be some expectation that ‘copy-build-system’
can support cross-compilation for real, so maybe we should add a
‘copy-cross-build’ procedure in addition to the patch above.

Thoughts?

Ludo’.
D
D
David Elsing wrote on 12 May 17:39 +0200
86o79byr7v.fsf@posteo.net
Hello,

I think I finally understand why the problem only occurs with grafts if
a dependency uses copy-build-system. It is actually somewhat
complicated.

When a package is lowered into a derivation by 'package->derivation' in
guix/packages.scm, a list of potentially applicable grafts is created by
the 'bag-grafts' procedure.
This works by first traversing the bag for native packages (where
bag-build-inputs and bag-target-inputs are followed recursively; and
also bag-host-inputs if not cross building) and building the native
derivations of the grafted versions of the packages which have a
replacement (such as glibc at the time of writing). Building the grafted
package (in the 'input-graft' procedure) is done again with
'package->derivation', which eventually calls 'graft-derivation' in
guix/grafts.scm, which in turn calls 'non-self-references', where the
ungrafted package is actually built (ignoring the store monad
indirection), not just its derivation.
In the case of cross builds, the bag is also traversed for the target
dependencies, where the bag-host-inputs are followed recursively.
Analogously, this causes the ungrafted packages and their ungrafted
replacement to be cross built to compute the grafted derivations.

As dan found out, the 'lower' procedure in guix/build-system/copy.scm
incorrectly puts standard-packages into the host inputs. They contain
the glibc-final package in gnu/packages/commencement.scm (which inherits
the replacement of glibc). Because copy-build-system puts them into
bag-host-inputs, the package replacement of the glibc-final package is
then cross built without grafts when calculating its grafted derivation,
which it does not support. Note that the glibc-final package and its
package replacement are distinct from the glibc package and its
replacement in gnu/packages/base.scm. I confirmed this by running
Toggle snippet (3 lines)
,use (guix)
(with-store store (run-with-store store (package->cross-derivation (package-replacement (@@ (gnu packages commencement) glibc-final)) "i686-linux-gnu" #:graft? #f)))
in the REPL, which returns the same incorrect glibc derivation that is
attempted to be built as a dependency when running
`guix build alsa-lib --target=i686-linux-gnu`.

It is actually possible to compute the graft derivations only when
needed by a store reference (which does not change the derivation of the
grafted package). Then, this problem does not occur even without the fix
for the copy-build-system, as the invalid derivations are never actually
built. With substitutes or after running GC, this may even prevent
building a package with replacement when the requested package output
which (transitively) depends on it has no (transitive) store reference
to it. I made a patch for that here: https://issues.guix.gnu.org/70895

Regarding the changes to the copy-build-system:

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (4 lines)
> But still, there seem to be some expectation that ‘copy-build-system’
> can support cross-compilation for real, so maybe we should add a
> ‘copy-cross-build’ procedure in addition to the patch above.

In guix/build-system/copy.scm, it is described as an extension of
gnu-build-system and the manual says "It adds much of the
gnu-build-system packages to the set of inputs. Because of this, the
copy-build-system does not require all the boilerplate code often needed
for the trivial-build-system."
Therefore, I think it makes sense to add the copy-cross-build procedure
so that copy-build-system actually behaves like gnu-build-system.

Cheers,
David
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 70492@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 70492
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch