Guile not in native-inputs when it should

Hi Guix,

(In response to bug#47027, but opened as a new bug.)

On Wed, 2021-03-17 at 21:52 +0100, Ludovic Courtès wrote:

FWIW, I tried to write a linter to catch these kind of issues.

(If there's a "guile" input, then there usually should also be

a "guile" native-input.) Currently, it has too many false positives

for my taste. I most likely won't be working on it in the near future

though. (Preliminary patch attached)

(Output attached)

Some suspicious things:

(in this case "guile" does not have to be in native-inputs).

The "wrap-script" procedure from (guix build utils) uses the

"which" procedure to determine where guile is located ...

but this is incorrect when cross-compiling!

(It is possible to override the "guile" binary used with a

keyword argument).

(I assume inputs in "inputs" do not contribute to the $PATH

in a cross-compilation environment; only "native-inputs" should

contribute to $PATH)

idk if it is feasible or if there are complications, but

IMHO the inputs in "inputs" shouldn't contribute to $PATH

at all (not even when not cross-compilation), only inputs

in $PATH.

There seems to be plenty of low-hanging cross-compilation fruit here!

Greetings,

Maxime

I made a spelling error in the command:

./pre-inst-env guix lint -c "inputs-should-also-be-native"

I forgot to attach the output of "guix lint -c ..." (now attached).

On Wed, 2021-03-17 at 22:58 +0100, Maxime Devos wrote:

Demonstration (host system: x86-64-linux with a childhurd running, without qemu binfmt),

using the "bats" package (the "bats" package is choosen because it doesn't have many

dependencies and it uses wrap-script). ("bats" actually uses wrap-script correctly,

so first remove the following line

":" (assoc-ref %build-inputs "guile") "/bin"

from the package definition to simulate a misbehaving package)

./pre-inst-env guix build --system=i586-gnu --target=x86-64-linux bats

--> tcl fails to build with plenty of failing test cases

^ TODO submit a bug report, for now try without tests

./pre-inst-env guix build --system=i586-gnu --target=x86-64-linux bats --without-tests=tcl

^ TODO this hangs the childhurd (something about paging?)

./pre-inst-env guix build --target=aarch64-linux bats

(warning: this takes some time building the cross-compiler)

--> install.sh: line 15: /gnu/store/...-coreutils-8.32/bin/install: cannot execute binary file: Exec format error

After adding "coreutils" to the native-inputs:

./pre-inst-env guix build --target=aarch64-linux bats

(success! --> some /gnu/store/something path $STORE_ITEM)

Let's look at $STORE_ITEM/bin/bats:

(start snip)

#!#f --no-auto-compile

#!#; Guix wrapper

#\-(begin (let ((current (getenv "PATH"))) (setenv "PATH" (if current (string-append "/gnu/store/qrj2w7a8ms7rkyvqhnrv8wrvqnbwv9bm-bash-

5.0.16/bin:/gnu/store/n8awazyldv9hbzb7pjcw76hiifmvrpvd-coreutils-8.32/bin:/gnu/store/3xi5vprn92r0jcb03lk9ykind5pi789j-grep-3.4/bin:/path-

not-set" ":" current) "/gnu/store/qrj2w7a8ms7rkyvqhnrv8wrvqnbwv9bm-bash-5.0.16/bin:/gnu/store/n8awazyldv9hbzb7pjcw76hiifmvrpvd-coreutils-

8.32/bin:/gnu/store/3xi5vprn92r0jcb03lk9ykind5pi789j-grep-3.4/bin:/path-not-set"))))

#\-(let ((cl (command-line))) (apply execl "/gnu/store/qrj2w7a8ms7rkyvqhnrv8wrvqnbwv9bm-bash-5.0.16/bin/bash" (car cl) (cons (car cl)

(append (quote ("")) cl))))

#!/gnu/store/qrj2w7a8ms7rkyvqhnrv8wrvqnbwv9bm-bash-5.0.16/bin/bash

set -e

BATS_READLINK='true'

[...]

(end snip)

I was worried for a moment that the inputs in "inputs" would contribute to

$PATH even when cross-compiling, but this turns out not to be the case.

However, I believe "wrap-script" should raise some kind of exception

instead of trying to use "#f" as interpreter.

--

Btw., "wrap-program" also uses "which" (but for finding the shell),

but fixing that would entail a world-rebuild as "wrap-program" doesn't

have a keyword argument

This fixes some uses of wrap-script.

On Thu, 2021-03-18 at 15:01 +0100, Maxime Devos wrote:

There's a bug in the patch (a missing #:input argument),

so don't apply yet. Will be fixed in the next revision

(along with more cross-compilation fixes).

Hi Guix,

The first patch ‘gnu: Explicitely pass the guile binary to wrap-script.’

does what it say what it does. This is important for picking up the guile

binary for the architecture in --target instead of the guile from native-inputs.

(The unpatched & patched package definitions do not have a guile native-input,

so before this patch they wouldn't pick up a guile at all -- packages in inputs

do not contribute to the $PATH when cross-compiling.)

The second patch ‘gnu: Make guile packages cross-compilable when the fix is trivial.’

only touches guile libraries. It adds guile to the native-inputs when required,

sometimes it adds guile to inputs and sometimes it copies propagated-inputs & inputs

to native-inputs when cross-compiling. It also fixes some other cross-compilations

issues like autoconf being in inputs instead of native-inputs.

The second patch only touches gnu/packages/guile-xyz.scm; other packages are ignored.

Also ignored are: emacsy-minimal (there have been patches and/or bug reports lately)

guile-bash (retired project) and guile-studio (it is an Emacs package).

Suggested testing method:

(start shell script)

# BAD_PACKAGES: fails to compile

BAD_PACKAGES="guile2.0-commonmark guile3.0-ncurses-with-gpm guile-dbi guile2.2-pfds"

# OK_PACKAGES: compiles & cross-compiles

OK_PACKAGES="guildhall guile-daemon guile-dsv guile-filesystem guile2.0-filesystem guile2.2-filesystem guile-syntax-highlight guile2.2-

syntax-highlight guile-sjson guile-sparql guile-email guile-mastodon guile-parted guile2.2-parted guile-config guile-hall guile-wisp

guile-commonmark guile-stis-parser guile-persist guile-file-names guile-srfi-158 guile-semver jupyter-guile-kernel guile-ics srfi-64-

driver guile-websocket g-wrap guile-webutils guile-srfi-159 guile-torrent guile-irc guile-machine-code guile2.2-sjson guile2.2-dsv

guile2.2-email guile2.2-config guile2.2-hall guile2.2-ics guile2.2-wisp guile2.2-commonmark guile2.2-semver guile2.2-webutils guile-redis

guile2.2-redis guile2.0-redis guile-irregex guile2.0-irregex guile2.2-irregex mcron guile2.2-mcron guile-srfi-89 guile-srfi-145 guile-

srfi-180 guile-jpeg guile-hashing guile2.2-hashing guile-packrat guile-ac-d-bus guile-lens guile2.2-lens guile-rdf guile-jsonld guile-

struct-pack guile-laesare guile-mkdir-p guile-jwt guile-r6rs-protobuf guile-shapefile schmutz guile-cbor guile-8sync guile-squee guile2.2-

squee guile-colorized guile2.2-colorized guile-pfds guile-prometheus guile-aa-tree guile-simple-zmq guile2.2-simple-zmq guile-debbugs

guile-email-latest guile-miniadapton guile-lib guile2.0-lib guile2.2-lib guile-minikanren guile2.0-minikanren guile2.2-minikanren python-

on-guile"

# NOT_CROSS_COMPILABLE: self-describing (compiles natively)

NOT_CROSS_COMPILABLE="guile-cv guile-gi guile-ncurses g-golf guile-picture-language guile-sly guile-aspell guile-fibers guile-sodium

guile-reader guile-udev haunt guile2.2-haunt guile2.0-haunt guile2.2-ncurses guile-ncurses-with-gpm guile-xosd artanis guile-xapian

guile2.2-xapian guile-newt guile2.2-newt guile2.2-reader guile-avahi guile2.0-pg guile-libyaml guile-eris guile-ffi-fftw "

make

# replace aarch64 with architecture of choice and maybe adjust -M and -c

./pre-inst-env guix build $OK_PACKAGES $NOT_CROSS_COMPILABLE -M6 -c1 --fallback

./pre-inst-env guix build $OK_PACKAGES -M6 -c1 --target=aarch64-linux-gnu --fallback

make as-derivation

(end shell script)

Greetings,

Maxime

merge 47221 58420

thanks