Wrapped python programs get native-inputs in PYTHONPATH

  • Open
  • quality assurance status badge
Details
10 participants
  • Arun Isaac
  • Efraim Flashner
  • Ricardo Wurmus via web
  • Lars-Dominik Braun
  • Leo Famulari
  • Ludovic Courtès
  • Maxim Cournoyer
  • Marius Bakke
  • Nicolas Graves
  • Ricardo Wurmus
Owner
unassigned
Submitted by
Marius Bakke
Severity
normal
Blocked by
M
M
Marius Bakke wrote on 20 Dec 2016 00:28
(address . bug-guix@gnu.org)
87eg13birp.fsf@kirby.i-did-not-set--mail-host-address--so-tickle-me
The 'wrap' phase of 'python-build-system' adds native-inputs to
PYTHONPATH of the wrapped programs. This causes unnecessary runtime
dependencies and may create problems when cross-compiling.

Given that all python dependencies are now supposed to be propagated,
perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAlhYbToACgkQoqBt8qM6
VPrqCgf9ET1NPk/I7q7M1zPHlbqOOrYWkk2fS7RAH3Un/X33Vve1DzKWsj7OQ7LO
YdU8ME0f8wJZ+tILUMawNRt/lKs/CmHug+9aOcqSfN7O8jawFVXxEPIE4U5wCM72
pMwhbu5TwRUZACD7G1168U257uyHpir26rf3wypbzWeAJFr50LEfxbPeBXjsR81v
YPYUUeJ1tCR1DedtU0Ih4TK8eW1ipveYMyw1iT9W+OMFO+y8qZwy1NyRNSrHgOcM
Ur/rmMqjJc+Yhvf37effW6Jg8OODBsqZzocVVqtvetk6Uo7S3jKKFu+Ov+SKyvSJ
FKNV5SC+DFxHSbdG2i35yMit8bGMcA==
=Ed8O
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 20 Dec 2016 15:00
(name . Marius Bakke)(address . mbakke@fastmail.com)(address . 25235@debbugs.gnu.org)
87y3zahf8t.fsf@gnu.org
Marius Bakke <mbakke@fastmail.com> skribis:

Toggle quote (7 lines)
> The 'wrap' phase of 'python-build-system' adds native-inputs to
> PYTHONPATH of the wrapped programs. This causes unnecessary runtime
> dependencies and may create problems when cross-compiling.
>
> Given that all python dependencies are now supposed to be propagated,
> perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.

We cannot simply remove it: it is meant for packages that provide
standalone programs, such as ‘certbot’. For these, it automatically
wraps bin/* and we don’t even have to have propagated inputs, which is
pretty cool.

However, you are right that native inputs shouldn’t be there. Probably
we need to add some filtering in the ‘wrap’ phase?

Ludo’.
L
L
Leo Famulari wrote on 26 Dec 2016 19:26
(name . Ludovic Courtès)(address . ludo@gnu.org)
20161226182608.GA20609@jasmine
On Tue, Dec 20, 2016 at 03:00:50PM +0100, Ludovic Courtès wrote:
Toggle quote (17 lines)
> Marius Bakke <mbakke@fastmail.com> skribis:
>
> > The 'wrap' phase of 'python-build-system' adds native-inputs to
> > PYTHONPATH of the wrapped programs. This causes unnecessary runtime
> > dependencies and may create problems when cross-compiling.
> >
> > Given that all python dependencies are now supposed to be propagated,
> > perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.
>
> We cannot simply remove it: it is meant for packages that provide
> standalone programs, such as ‘certbot’. For these, it automatically
> wraps bin/* and we don’t even have to have propagated inputs, which is
> pretty cool.
>
> However, you are right that native inputs shouldn’t be there. Probably
> we need to add some filtering in the ‘wrap’ phase?

While looking at the beets package definition with fresh eyes, I realize
that it will surely break once this change is made. So it will make a
good test case for this improvement :)
A
A
Arun Isaac wrote on 29 Mar 2017 13:27
(address . 25235@debbugs.gnu.org)
716a63e7.AEQAIoR8aXkAAAAAAAAAAAOwyEEAAAACwQwAAAAAAAW9WABY25oY@mailjet.com
I'm working on fixing this bug. I have modified
guix/build/python-build-system.scm for the same. In particular, I have
added #:use-module (guix packages) because I need the functions
`package-name' and `package-transitive-target-inputs'. But, when I try
building any python-build-system package with something like
"./pre-inst-env guix build scons", I get a ("no code for module" (guix
packages)) error. What am I missing?

Full backtrace follows:

Backtrace:
In ice-9/eval.scm:
432: 19 [eval # ()]
In system/base/compile.scm:
152: 18 [compile-file "/gnu/store/38rxls2y8v8zjva7xq7zv99rv6c81bq3-module-import/guix/build/python-build-system.scm" ...]
43: 17 [call-once #<procedure ae84c0 at system/base/compile.scm:56:5 ()>]
In ice-9/boot-9.scm:
174: 16 [with-throw-handler #t ...]
In system/base/compile.scm:
59: 15 [#<procedure ae8480 at system/base/compile.scm:58:9 ()>]
155: 14 [#<procedure ae8500 at system/base/compile.scm:153:8 (port)> #<closed: file 0>]
218: 13 [read-and-compile #<input: /gnu/store/38rxls2y8v8zjva7xq7zv99rv6c81bq3-module-import/guix/build/python-build-system.scm 6> ...]
234: 12 [lp () #f #<module (#{ g152}#) ae7a20>]
182: 11 [lp (#<procedure compile-tree-il (x e opts)>) (define-module # # ...) ...]
In ice-9/boot-9.scm:
2404: 10 [save-module-excursion #<procedure c967e0 at language/scheme/compile-tree-il.scm:29:3 ()>]
In language/scheme/compile-tree-il.scm:
31: 9 [#<procedure c967e0 at language/scheme/compile-tree-il.scm:29:3 ()>]
In ice-9/psyntax.scm:
1106: 8 [expand-top-sequence ((define-module # # # ...)) () ((top)) ...]
989: 7 [scan ((define-module # # # ...)) () ((top)) ...]
279: 6 [scan ((#(syntax-object let # ...) (#) (# #) ...)) () ...]
In ice-9/eval.scm:
411: 5 [eval # ()]
In ice-9/boot-9.scm:
2954: 4 [define-module* (guix build python-build-system) #:filename ...]
2929: 3 [resolve-imports ((# # gnu:) (#) (#) (#) ...)]
2870: 2 [resolve-interface (guix packages) #:select ...]
In unknown file:
?: 1 [scm-error misc-error #f "~A ~S" ("no code for module" (guix packages)) #f]
In ice-9/boot-9.scm:
109: 0 [#<procedure ae8440 at ice-9/boot-9.scm:100:6 (thrown-k . args)> misc-error ...]

ice-9/boot-9.scm:109:20: In procedure #<procedure ae8440 at ice-9/boot-9.scm:100:6 (thrown-k . args)>:
ice-9/boot-9.scm:109:20: no code for module (guix packages)
builder for `/gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv' failed with exit code 1
@ build-failed /gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv - 1 builder for `/gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv' failed with exit code 1
cannot build derivation `/gnu/store/pm2r6cbcfij8d7nzdxkzqx1dq880mhg8-scons-2.3.4.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/pm2r6cbcfij8d7nzdxkzqx1dq880mhg8-scons-2.3.4.drv' failed
L
L
Ludovic Courtès wrote on 30 Mar 2017 17:30
(name . Arun Isaac)(address . arunisaac@systemreboot.net)(address . 25235@debbugs.gnu.org)
87o9wiwzn2.fsf@gnu.org
Hello,

Arun Isaac <arunisaac@systemreboot.net> skribis:

Toggle quote (10 lines)
> I'm working on fixing this bug. I have modified
> guix/build/python-build-system.scm for the same. In particular, I have
> added #:use-module (guix packages) because I need the functions
> `package-name' and `package-transitive-target-inputs'. But, when I try
> building any python-build-system package with something like
> "./pre-inst-env guix build scons", I get a ("no code for module" (guix
> packages)) error. What am I missing?
>
> Full backtrace follows:

[...]

Toggle quote (6 lines)
> 2954: 4 [define-module* (guix build python-build-system) #:filename ...]
> 2929: 3 [resolve-imports ((# # gnu:) (#) (#) (#) ...)]
> 2870: 2 [resolve-interface (guix packages) #:select ...]
> In unknown file:
> ?: 1 [scm-error misc-error #f "~A ~S" ("no code for module" (guix packages)) #f]

“Build-side” modules, which typically live in (guix build …), should not
depend on “host-side” modules such as (guix packages). That’s because
if we did that, we’d effectively end up importing all of Guix on the
build side, but then we’d also have to serialize data structures such as
packages to pass them from one side to the other. (I hope this makes
sense to you, but if it doesn’t maybe the intro of
shed some light.)

So in short, we cannot use ‘package-name’ and
‘package-transitive-target-inputs’ in this module.

(Time passes…)

I wasn’t sure how to fix this bug myself so I gave it a try and ended up
with the patch below, but I haven’t tested in detail. (You’ll notice
(guix build-system python) is hard to work with because it doesn’t use
gexps yet.)

How does it look?

Thanks,
Ludo’.
Attachment: file
A
A
Arun Isaac wrote on 30 Mar 2017 21:07
(address . 25235@debbugs.gnu.org)
e01723b7.AEMAIlnS11YAAAAAAAAAAAOwyEEAAAACwQwAAAAAAAW9WABY3Vef@mailjet.com
Toggle quote (9 lines)
> “Build-side” modules, which typically live in (guix build …), should not
> depend on “host-side” modules such as (guix packages). That’s because
> if we did that, we’d effectively end up importing all of Guix on the
> build side, but then we’d also have to serialize data structures such as
> packages to pass them from one side to the other. (I hope this makes
> sense to you, but if it doesn’t maybe the intro of
> <https://gnu.org/s/guix/manual/html_node/G_002dExpressions.html> can
> shed some light.)

It makes some sense, but I'll read up more.

Toggle quote (3 lines)
> So in short, we cannot use ‘package-name’ and
> ‘package-transitive-target-inputs’ in this module.

Ok.

Toggle quote (9 lines)
> (Time passes…)
>
> I wasn’t sure how to fix this bug myself so I gave it a try and ended up
> with the patch below, but I haven’t tested in detail. (You’ll notice
> (guix build-system python) is hard to work with because it doesn’t use
> gexps yet.)
>
> How does it look?

I'm travelling now. I'll get back on Monday and study this patch.
A
A
Arun Isaac wrote on 4 Apr 2017 19:02
(address . 25235@debbugs.gnu.org)
a18c806b.AEEAJFKBgTAAAAAAAAAAAAOwyEEAAAACwQwAAAAAAAW9WABY49Gt@mailjet.com
Toggle quote (2 lines)
> How does it look?

I tried your patch. But native-inputs are still there in the wrapped
PYTHONPATH. I printed out the results from various steps from the wrap
phase. They are as follows:

`build-time-inputs' is

("/gnu/store/5wbi6vsi12bmpzygwhys3c0cc6sjdw40-python-beautifulsoup4-4.5.3"
"/gnu/store/kc5gl1j85y9ispjsgbg5f5c6jdy1k32s-python-flask-0.11.1"
"/gnu/store/3ib43rpca6qzwfcav8s9ivm3dikpxq87-python-mock-1.0.1"
"/gnu/store/6q4hszis7z6x44ig2zlwjwlcjy3vvfmp-python-mpd2-0.5.5"
"/gnu/store/5wx6yffx6dppd2dr91qfi1jlpqlp9f5m-python-nose-1.3.7"
"/gnu/store/kqjfgfwzm7sq43fnng1w23mjyr2xrd75-python-pathlib-1.0.1"
"/gnu/store/b9yn8bfd43pvrp2wyinf6rf2qw113r5p-python-pyxdg-0.25"
"/gnu/store/g4vfl4wy81y14fjhlpmgqh0i01cg37mb-python-pylast-1.6.0"
"/gnu/store/nim5r75b1kzawln3a1bgag2f12vdmb8x-python-rarfile-2.8"
"/gnu/store/wbaaacm1m7bzbhi3z3scwbz2p4pf2ybg-python-responses-0.5.1")

`path' is

("/gnu/store/5wbi6vsi12bmpzygwhys3c0cc6sjdw40-python-beautifulsoup4-4.5.3/lib/python3.5/site-packages"
"/gnu/store/r3f04sda4gh5vbsa7zb5x0ygf326cp75-python-flask-0.11.1/lib/python3.5/site-packages"
"/gnu/store/3ib43rpca6qzwfcav8s9ivm3dikpxq87-python-mock-1.0.1/lib/python3.5/site-packages"
"/gnu/store/5mkb6im3dijkw4f5146mjz4c49qgcklq-python-mpd2-0.5.5/lib/python3.5/site-packages"
"/gnu/store/9z450rcsza3x448ky83fkd5636ck9aqd-python-nose-1.3.7/lib/python3.5/site-packages"
"/gnu/store/kqjfgfwzm7sq43fnng1w23mjyr2xrd75-python-pathlib-1.0.1/lib/python3.5/site-packages"
"/gnu/store/b9yn8bfd43pvrp2wyinf6rf2qw113r5p-python-pyxdg-0.25/lib/python3.5/site-packages"
"/gnu/store/g4vfl4wy81y14fjhlpmgqh0i01cg37mb-python-pylast-1.6.0/lib/python3.5/site-packages"
"/gnu/store/nim5r75b1kzawln3a1bgag2f12vdmb8x-python-rarfile-2.8/lib/python3.5/site-packages"
"/gnu/store/wbaaacm1m7bzbhi3z3scwbz2p4pf2ybg-python-responses-0.5.1/lib/python3.5/site-packages"
"/gnu/store/m6ir6mj7laxcwsjrz0lvxm7q04w0anhp-python-discogs-client-2.2.1/lib/python3.5/site-packages"
"/gnu/store/v3xiyw0cj08qmpn08jaiznd8ak3nvdga-python-jellyfish-0.5.6/lib/python3.5/site-packages"
"/gnu/store/pwxvvi24950x6rgvl4f46z89sbpa2b7a-python-munkres-1.0.8/lib/python3.5/site-packages"
"/gnu/store/nws3867b5q78ri461wr3rdwgqzasvv2a-python-musicbrainzngs-0.6/lib/python3.5/site-packages"
"/gnu/store/wsw2nr30nhz29dww1hfjb6lgqn2ckrgy-python-mutagen-1.36/lib/python3.5/site-packages"
"/gnu/store/n7fh268b1g1g87s4djvm672p2xmwkd5r-python-pyyaml-3.12/lib/python3.5/site-packages"
"/gnu/store/y3ncl6ia04vqdkhnhgzb3iabcsczj0zb-python-unidecode-0.04.20/lib/python3.5/site-packages"
"/gnu/store/alk9r3rir93pjmv8im20f8xrvv90219z-python-3.5.2/lib/python3.5/site-packages"
"/gnu/store/z2mmzglkykws7hc9k2sr40fidiv62cpa-python-werkzeug-0.11.15/lib/python3.5/site-packages"
"/gnu/store/5w60y9zyjskv179sf4pmq2kzpff887dv-python-click-6.7/lib/python3.5/site-packages"
"/gnu/store/f4qqgaql0f95alxrz9r52n3m5ahjrc1s-python-jinja2-2.8/lib/python3.5/site-packages"
"/gnu/store/n0l3szawn661n2rjk6nnbajzzc02pmfj-python-itsdangerous-0.24/lib/python3.5/site-packages"
"/gnu/store/rpnfm1v7vnwxl3wpbadz2bz25s2x1bbg-python-six-1.10.0/lib/python3.5/site-packages"
"/gnu/store/64ahrdmjmmm8id5327i5kqjj8ppvi2kl-python-cookies-2.2.1/lib/python3.5/site-packages"
"/gnu/store/2iyrxbr6k1qdcrhfpv3cz1nh1znwnga5-python-requests-2.13.0/lib/python3.5/site-packages"
"/gnu/store/0c28plwcf33lnibf27h8ijig467pphiw-python-oauthlib-1.0.3/lib/python3.5/site-packages"
"/gnu/store/m48bcrmrzrcb79p6dhaq0diwr3aacfq3-python-markupsafe-0.23/lib/python3.5/site-packages")

(remove build-time-dependency? path) is

("/gnu/store/r3f04sda4gh5vbsa7zb5x0ygf326cp75-python-flask-0.11.1/lib/python3.5/site-packages"
"/gnu/store/5mkb6im3dijkw4f5146mjz4c49qgcklq-python-mpd2-0.5.5/lib/python3.5/site-packages"
"/gnu/store/9z450rcsza3x448ky83fkd5636ck9aqd-python-nose-1.3.7/lib/python3.5/site-packages"
"/gnu/store/m6ir6mj7laxcwsjrz0lvxm7q04w0anhp-python-discogs-client-2.2.1/lib/python3.5/site-packages"
"/gnu/store/v3xiyw0cj08qmpn08jaiznd8ak3nvdga-python-jellyfish-0.5.6/lib/python3.5/site-packages"
"/gnu/store/pwxvvi24950x6rgvl4f46z89sbpa2b7a-python-munkres-1.0.8/lib/python3.5/site-packages"
"/gnu/store/nws3867b5q78ri461wr3rdwgqzasvv2a-python-musicbrainzngs-0.6/lib/python3.5/site-packages"
"/gnu/store/wsw2nr30nhz29dww1hfjb6lgqn2ckrgy-python-mutagen-1.36/lib/python3.5/site-packages"
"/gnu/store/n7fh268b1g1g87s4djvm672p2xmwkd5r-python-pyyaml-3.12/lib/python3.5/site-packages"
"/gnu/store/y3ncl6ia04vqdkhnhgzb3iabcsczj0zb-python-unidecode-0.04.20/lib/python3.5/site-packages"
"/gnu/store/alk9r3rir93pjmv8im20f8xrvv90219z-python-3.5.2/lib/python3.5/site-packages"
"/gnu/store/z2mmzglkykws7hc9k2sr40fidiv62cpa-python-werkzeug-0.11.15/lib/python3.5/site-packages"
"/gnu/store/5w60y9zyjskv179sf4pmq2kzpff887dv-python-click-6.7/lib/python3.5/site-packages"
"/gnu/store/f4qqgaql0f95alxrz9r52n3m5ahjrc1s-python-jinja2-2.8/lib/python3.5/site-packages"
"/gnu/store/n0l3szawn661n2rjk6nnbajzzc02pmfj-python-itsdangerous-0.24/lib/python3.5/site-packages"
"/gnu/store/rpnfm1v7vnwxl3wpbadz2bz25s2x1bbg-python-six-1.10.0/lib/python3.5/site-packages"
"/gnu/store/64ahrdmjmmm8id5327i5kqjj8ppvi2kl-python-cookies-2.2.1/lib/python3.5/site-packages"
"/gnu/store/2iyrxbr6k1qdcrhfpv3cz1nh1znwnga5-python-requests-2.13.0/lib/python3.5/site-packages"
"/gnu/store/0c28plwcf33lnibf27h8ijig467pphiw-python-oauthlib-1.0.3/lib/python3.5/site-packages"
"/gnu/store/m48bcrmrzrcb79p6dhaq0diwr3aacfq3-python-markupsafe-0.23/lib/python3.5/site-packages")

Somehow, the paths in `build-time-inputs' are not the same as those in
`path'. Notice the different hashes. Hence `remove' does not succeed in
filtering them out.

Toggle quote (3 lines)
> + ;; TODO: Something like this:
> + ;; (disallowed-references native-inputs)

What is this part for?
R
R
Ricardo Wurmus wrote on 7 Feb 2019 14:22
(name . Arun Isaac)(address . arunisaac@systemreboot.net)(address . 25235@debbugs.gnu.org)
87pns34tts.fsf@elephly.net
Arun Isaac <arunisaac@systemreboot.net> writes:

Toggle quote (5 lines)
>> + ;; TODO: Something like this:
>> + ;; (disallowed-references native-inputs)
>
> What is this part for?

This is to ensure that no references to any native input are retained in
the output. (Think of it as an assertion that is checked at the end of
the build.)

~~ Ricardo
R
R
Ricardo Wurmus via web wrote on 6 Apr 2020 16:49
(no subject)
(address . 25235@debbugs.gnu.org)
7f6e5ae41520.2159a21282e87de@guile.gnu.org
Have you been able to figure out why some of the items differ in the two lists?
A
A
Arun Isaac wrote on 9 Apr 2020 02:02
(name . Ricardo Wurmus)(address . rekado@elephly.net)(address . 25235@debbugs.gnu.org)
cu78sj5pluo.fsf@systemreboot.net
Toggle quote (2 lines)
> Have you been able to figure out why some of the items differ in the two lists?

Hi! Are you asking me? I haven't worked on this in a very long
time. But I'll give it another shot now and get back to you.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAl6OZiEACgkQLiXui2GA
K7OAPQf/cyL5HaPfxKPxtfU7jAuY+tvJwnd0M/LxUu4UYiCLKLD4Ved/qWBOd9uK
ylo6qgMs5cqj0BBqoCjxZQPLC+T7gHc3qnDeyoCx5y/LnVjRCU5yBaYimrNnQJ9B
SS0OVuo8vw8/sxq8VhHHpweZsHTG89MXdts3H5u6PqxbypbMZoLNbb30kenwRuEy
hpurZtOcvl9t1X7+GV7UoG8nHZlH1DMURpFGfuleMeTGWApFSEnAbGepJonCU9bs
vpJFzbSmmb+wNWfwawJfyfUmQGMtixAAsOEdrpo+sjWBCA+Nj7T7gpTG3vNOi1w0
ypW9PdfkI1UWb2T6p7bE9u0WKGBP6A==
=Lxvu
-----END PGP SIGNATURE-----

A
A
Arun Isaac wrote on 21 Apr 2020 04:15
Re: bug#25235: Wrapped python programs get native-inputs in PYTHONPATH
(name . Ricardo Wurmus)(address . rekado@elephly.net)(address . 25235@debbugs.gnu.org)
cu7imht4m9n.fsf@systemreboot.net
Hi,

Toggle quote (3 lines)
> Have you been able to figure out why some of the items differ in the
> two lists?

The two lists are the same if `guix build' is invoked with the
`--no-grafts' flag. I don't understand grafts very well and am not sure
how that can be fixed. Any ideas?

See different output below with and without the `--no-grafts' flag.

$ guix build python-mock

Toggle snippet (20 lines)
`build-time-inputs' is
(/gnu/store/dhs3v8rfzazq9cb37pqs7fdm5hd7zmf0-python-unittest2-1.1.0)

`path' is
(/gnu/store/v70qcgga1r2m6scnard8s61dqh859mn2-python-unittest2-1.1.0/lib/python3.7/site-packages
/gnu/store/w408dkc1lynq5kqi2cxkvikalp6w35qw-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/bwqidwykphzv10n454kxv3hfippq8qrx-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/k4a9433nghpjhs101cm9v8k64bmd8iqf-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/3nl9hk7f59vsxgr7xmnqgc15y9c3rwpg-python-linecache2-1.0.0/lib/python3.7/site-packages)

`(remove build-time-dependency? path)' is
(/gnu/store/v70qcgga1r2m6scnard8s61dqh859mn2-python-unittest2-1.1.0/lib/python3.7/site-packages
/gnu/store/w408dkc1lynq5kqi2cxkvikalp6w35qw-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/bwqidwykphzv10n454kxv3hfippq8qrx-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/k4a9433nghpjhs101cm9v8k64bmd8iqf-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/3nl9hk7f59vsxgr7xmnqgc15y9c3rwpg-python-linecache2-1.0.0/lib/python3.7/site-packages)

$ guix build --no-grafts python-mock

Toggle snippet (19 lines)
`build-time-inputs' is
(/gnu/store/m01qbb9sgymxh2xqwpnbv58h60yl4pz0-python-unittest2-1.1.0)

`path' is
(/gnu/store/m01qbb9sgymxh2xqwpnbv58h60yl4pz0-python-unittest2-1.1.0/lib/python3.7/site-packages
/gnu/store/9k5hyd9d7b3r4yzadd6phphf12l2mfc8-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/p8p65yd0pf95yn50iib0jw05p0n4mr9p-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/dvn1fz82ccjg7dnqqmkxhwp720wnmriw-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/vnbk5jswscv68qasfbhmh7wqlsibyism-python-linecache2-1.0.0/lib/python3.7/site-packages)

`(remove build-time-dependency? path)' is
(/gnu/store/9k5hyd9d7b3r4yzadd6phphf12l2mfc8-python-pbr-minimal-3.0.1/lib/python3.7/site-packages
/gnu/store/p8p65yd0pf95yn50iib0jw05p0n4mr9p-python-six-1.12.0/lib/python3.7/site-packages
/gnu/store/78w7y0lxar70j512iqw8x3nimzj10yga-python-3.7.4/lib/python3.7/site-packages
/gnu/store/dvn1fz82ccjg7dnqqmkxhwp720wnmriw-python-traceback2-1.4.0/lib/python3.7/site-packages
/gnu/store/vnbk5jswscv68qasfbhmh7wqlsibyism-python-linecache2-1.0.0/lib/python3.7/site-packages)

Regards,
Arun
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEf3MDQ/Lwnzx3v3nTLiXui2GAK7MFAl6eV0UACgkQLiXui2GA
K7OzgwgAh0Sc2MhdnPj5HMsuWQnCiib+zKMI1P2y7TfsKuM+m/YiqWzs9V+ushmf
K7qErupO44DrqpKfl70iDDfiWjkzRkBloq3um40+dRebKNlfMaBbwlDRvxTc1OAw
GeLRTn4GLXkL2gwd78AHXU+ALmupWxJRaNYZCZwDKqOry5Xcvjb6pc10REGQyTBh
28VpP7ezUWBIk9e98xZf/Ct/wNyNXMSABiGgersUr/vBEuQmQ654hg86VrFOAgsi
X+TM9xxZzsw4ZJ7BiutOlKC1TIlzl7DJHxqTwtWngPEB5U8Qu9E+Rnz2Iy4yVD5a
bSvSF+2UxftTp8Q5JtvdBRmrsyMtUA==
=E6gx
-----END PGP SIGNATURE-----

M
M
Maxim Cournoyer wrote on 16 Jan 2023 22:29
[PATCH 0/1] build-system/pyproject: Do not wrap native-inputs.
(address . 25235@debbugs.gnu.org)
20230116212920.9641-1-maxim.cournoyer@gmail.com
This fixes a longstanding issue. The fix is made trivial by the changes made
in #60847 adding cross-compilation support for the pyproject build system.
Additionally, it's based on #60868, which touches the wrap phase to use
wrap-script instead of wrap-program.


Maxim Cournoyer (1):
build-system/pyproject: Do not wrap native-inputs.

guix/build/pyproject-build-system.scm | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)


base-commit: 9da36aa1e207ff8a8fb2af8bc9442f9c935dc8db
--
2.38.1
M
M
Maxim Cournoyer wrote on 16 Jan 2023 22:29
[PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.
(address . 25235@debbugs.gnu.org)
20230116212920.9641-2-maxim.cournoyer@gmail.com

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

guix/build/pyproject-build-system.scm | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)

Toggle diff (36 lines)
diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..cd418f7ec9 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
"A phase that just wraps the 'add-installed-pythonpath' procedure."
(add-installed-pythonpath native-inputs outputs))
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
(define (list-of-files dir)
(find-files dir (lambda (file stat)
(and (eq? 'regular (stat:type stat))
@@ -464,9 +464,17 @@ (define bindirs
(define %guile (delay (search-input-file inputs "bin/guile")))
(define (guile) (force %guile))
- (let* ((var `("GUIX_PYTHONPATH" prefix
- ,(search-path-as-string->list
- (or (getenv "GUIX_PYTHONPATH") "")))))
+ ;; Use the same strategy to compute the native-input file names.
+ (define %native-input-dirs (delay (match native-inputs
+ (((_ . dir) ...)
+ dir))))
+ (define (native-input-dirs) (force %native-input-dirs))
+
+ (let ((var `("GUIX_PYTHONPATH" prefix
+ ,(remove (lambda (x)
+ (any (cut string-prefix? <> x) (native-input-dirs)))
+ (search-path-as-string->list
+ (or (getenv "GUIX_PYTHONPATH") ""))))))
(for-each (lambda (dir)
(let ((files (list-of-files dir)))
(for-each (cut wrap-script <> #:guile (guile) var)
--
2.38.1
L
L
Ludovic Courtès wrote on 17 Jan 2023 10:28
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
874jspjyrv.fsf@gnu.org
Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (5 lines)
>
> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
> Filter out native inputs from the values in GUIX_PYTHONPATH.

Yay, great to see this fixed!

Toggle quote (35 lines)
> guix/build/pyproject-build-system.scm | 16 ++++++++++++----
> 1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
> index a66c1fb34a..cd418f7ec9 100644
> --- a/guix/build/pyproject-build-system.scm
> +++ b/guix/build/pyproject-build-system.scm
> @@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
> "A phase that just wraps the 'add-installed-pythonpath' procedure."
> (add-installed-pythonpath native-inputs outputs))
>
> -(define* (wrap #:key inputs outputs #:allow-other-keys)
> +(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
> (define (list-of-files dir)
> (find-files dir (lambda (file stat)
> (and (eq? 'regular (stat:type stat))
> @@ -464,9 +464,17 @@ (define bindirs
> (define %guile (delay (search-input-file inputs "bin/guile")))
> (define (guile) (force %guile))
>
> - (let* ((var `("GUIX_PYTHONPATH" prefix
> - ,(search-path-as-string->list
> - (or (getenv "GUIX_PYTHONPATH") "")))))
> + ;; Use the same strategy to compute the native-input file names.
> + (define %native-input-dirs (delay (match native-inputs
> + (((_ . dir) ...)
> + dir))))
> + (define (native-input-dirs) (force %native-input-dirs))
> +
> + (let ((var `("GUIX_PYTHONPATH" prefix
> + ,(remove (lambda (x)
> + (any (cut string-prefix? <> x) (native-input-dirs)))
> + (search-path-as-string->list
> + (or (getenv "GUIX_PYTHONPATH") ""))))))

I think there’s no need to delay+thunk things since we’re using them
right away. How about:

(define native-input-directories
(match native-inputs
…))

(let ((var `("GUIX_PYTHONPATH" prefix
,(remove (lambda (x) … native-input-directories …) …))))
…)

?

(Same for ‘%guile’ above, but I don’t see it in ‘master’; maybe that’s
from another branch?)

Thanks,
Ludo’.
M
M
Maxim Cournoyer wrote on 17 Jan 2023 16:49
(name . Ludovic Courtès)(address . ludo@gnu.org)
877cxl2mc3.fsf@gmail.com
Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (59 lines)
> Hi,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Fixes <https://issues.guix.gnu.org/25235>.
>>
>> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
>> Filter out native inputs from the values in GUIX_PYTHONPATH.
>
> Yay, great to see this fixed!
>
>> guix/build/pyproject-build-system.scm | 16 ++++++++++++----
>> 1 file changed, 12 insertions(+), 4 deletions(-)
>>
>> diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
>> index a66c1fb34a..cd418f7ec9 100644
>> --- a/guix/build/pyproject-build-system.scm
>> +++ b/guix/build/pyproject-build-system.scm
>> @@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
>> "A phase that just wraps the 'add-installed-pythonpath' procedure."
>> (add-installed-pythonpath native-inputs outputs))
>>
>> -(define* (wrap #:key inputs outputs #:allow-other-keys)
>> +(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
>> (define (list-of-files dir)
>> (find-files dir (lambda (file stat)
>> (and (eq? 'regular (stat:type stat))
>> @@ -464,9 +464,17 @@ (define bindirs
>> (define %guile (delay (search-input-file inputs "bin/guile")))
>> (define (guile) (force %guile))
>>
>> - (let* ((var `("GUIX_PYTHONPATH" prefix
>> - ,(search-path-as-string->list
>> - (or (getenv "GUIX_PYTHONPATH") "")))))
>> + ;; Use the same strategy to compute the native-input file names.
>> + (define %native-input-dirs (delay (match native-inputs
>> + (((_ . dir) ...)
>> + dir))))
>> + (define (native-input-dirs) (force %native-input-dirs))
>> +
>> + (let ((var `("GUIX_PYTHONPATH" prefix
>> + ,(remove (lambda (x)
>> + (any (cut string-prefix? <> x) (native-input-dirs)))
>> + (search-path-as-string->list
>> + (or (getenv "GUIX_PYTHONPATH") ""))))))
>
> I think there’s no need to delay+thunk things since we’re using them
> right away. How about:
>
> (define native-input-directories
> (match native-inputs
> …))
>
> (let ((var `("GUIX_PYTHONPATH" prefix
> ,(remove (lambda (x) … native-input-directories …) …))))
> …)
>
> ?

It's a (small) optimization, that was intended to delay things until
they are really needed that was already used for %bash (now %guile) in
the wrap phase of (guix build python-build-system).

The idea is to delay the evaluation of native-input-directories or the
lookup for guile in the inputs until it's really necessary. For
example, the computations are avoided when there are no "bin" or "sbin"
sub-directories in the outputs.

Toggle quote (3 lines)
> (Same for ‘%guile’ above, but I don’t see it in ‘master’; maybe that’s
> from another branch?)

Yes, the change build on both #60847 (Enable cross-compilation for the
pyproject-build-system) and #60868 (Use wrap-script instead of
wrap-program), which should be applied first. %guile appears in #60868,
and changes that make the above easy (native-inputs always appearing as
native-inputs, even for native builds) are from #60847.

--
Thanks,
Maxim
M
M
Maxim Cournoyer wrote on 17 Jan 2023 16:49
control message for bug #25235
(address . control@debbugs.gnu.org)
875yd52mb7.fsf@gmail.com
block 25235 by 60847
quit
E
E
Efraim Flashner wrote on 18 Jan 2023 10:51
Re: bug#25235: [PATCH 1/1] build-system/pyproject: Do not wrap native-inputs.
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
Y8fBFLZkBH8OZCDO@3900XT
I'm still on my first cup of coffee...

If a python input is in inputs and in native-inputs would it be included
in the wrapper or not? At first glance I'd say no. Does the search path
do deduplication? If it doesn't then it'd work to only remove the first
instance matching from native-input-dirs.

On Mon, Jan 16, 2023 at 04:29:19PM -0500, Maxim Cournoyer wrote:
Toggle quote (51 lines)
>
> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
> Filter out native inputs from the values in GUIX_PYTHONPATH.
>
> ---
>
> guix/build/pyproject-build-system.scm | 16 ++++++++++++----
> 1 file changed, 12 insertions(+), 4 deletions(-)
>
> diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
> index a66c1fb34a..cd418f7ec9 100644
> --- a/guix/build/pyproject-build-system.scm
> +++ b/guix/build/pyproject-build-system.scm
> @@ -445,7 +445,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
> "A phase that just wraps the 'add-installed-pythonpath' procedure."
> (add-installed-pythonpath native-inputs outputs))
>
> -(define* (wrap #:key inputs outputs #:allow-other-keys)
> +(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
> (define (list-of-files dir)
> (find-files dir (lambda (file stat)
> (and (eq? 'regular (stat:type stat))
> @@ -464,9 +464,17 @@ (define bindirs
> (define %guile (delay (search-input-file inputs "bin/guile")))
> (define (guile) (force %guile))
>
> - (let* ((var `("GUIX_PYTHONPATH" prefix
> - ,(search-path-as-string->list
> - (or (getenv "GUIX_PYTHONPATH") "")))))
> + ;; Use the same strategy to compute the native-input file names.
> + (define %native-input-dirs (delay (match native-inputs
> + (((_ . dir) ...)
> + dir))))
> + (define (native-input-dirs) (force %native-input-dirs))
> +
> + (let ((var `("GUIX_PYTHONPATH" prefix
> + ,(remove (lambda (x)
> + (any (cut string-prefix? <> x) (native-input-dirs)))
> + (search-path-as-string->list
> + (or (getenv "GUIX_PYTHONPATH") ""))))))
> (for-each (lambda (dir)
> (let ((files (list-of-files dir)))
> (for-each (cut wrap-script <> #:guile (guile) var)
> --
> 2.38.1
>
>
>
>

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmPHwRQACgkQQarn3Mo9
g1HZ0xAAhDHP3k2Q7DRb2PwIRbL5Yf22B7MnuVHWl3+lYPmtenu0uj4e2/FuP08f
qMhrOKUzXeP8jmAgk7yNd8OutsUfjkDML4ZezdCh0lZpw5fUokIaIGeNUCEH54Wi
TNLqpzatCMtFaORf30np/aEholzhkYRXy6JciUiapeTd0A5AtBq9GmuDrviRpq6C
NlKgBM7e4tBA7kBiHtrkhO9qPjC+PV7hoFJ8CVzFc3piXPgz0tH1IkB3pCtyHEfn
FGJL/g2hooIcdZ5n2T3WLOSnhf/zavlHSzv3vodb8lR6H+GxO0HS9ouQICXk+L/s
oThlqwoZk5yDfD9sE8gGZtSx5R5qoIOUSc62YmB0SdQ/ntrn/yruLQovbmL4VhMa
qcS55HsX/R5egfXvMILf9NSTCla03KtuMamKx+3RXL5vjkDKLMHyqoAdnrnGpARk
TLCvoxL8lu7Fc2kkJp2J6DQBMtbCOhpRxg3UOiCBbPutcfVOF9YFujF07eKAAw+3
GHYJWZkPKFudc6a1YY4FapZYudXGW6BFS2PrFPQ0viebjD59/oot31q7tSwYG+eE
B1gERew5jypMiftc9ND+pnho4MnAA5hErAKsiWVKNwsnE5oVwlSVW56PpiwepcEw
0GRjndBFQ81cLJQgg0nqcRlYrtLjLZ+hUhgRUrmmrouOyBDbEhQ=
=+g65
-----END PGP SIGNATURE-----


M
M
Maxim Cournoyer wrote on 18 Jan 2023 14:22
(name . Efraim Flashner)(address . efraim@flashner.co.il)
875yd47z9v.fsf@gmail.com
Hi Efraim,

Efraim Flashner <efraim@flashner.co.il> writes:

Toggle quote (7 lines)
> I'm still on my first cup of coffee...
>
> If a python input is in inputs and in native-inputs would it be included
> in the wrapper or not? At first glance I'd say no. Does the search path
> do deduplication? If it doesn't then it'd work to only remove the first
> instance matching from native-input-dirs.

It wouldn't be included with the current simple logic, which simply
remove any native input from the items of GUIX_PYTHONPATH. Good catch!
I'll send a v2 with the refined check.

--
Thanks,
Maxim
M
M
Maxim Cournoyer wrote on 18 Jan 2023 14:59
[PATCH v2 0/1] build-system/pyproject: Do not wrap native-inputs.
(address . 25235@debbugs.gnu.org)
20230118135908.13162-1-maxim.cournoyer@gmail.com
This fixes a longstanding issue. The fix is made trivial by the changes made
in #60847 adding cross-compilation support for the pyproject build system.
Additionally, it's based on #60868, which touches the wrap phase to use
wrap-script instead of wrap-program.

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and
inputs
- Enclose wrap computations in an 'unless' form and streamline

Maxim Cournoyer (1):
build-system/pyproject: Do not wrap native-inputs.

guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
1 file changed, 24 insertions(+), 15 deletions(-)


base-commit: 9da36aa1e207ff8a8fb2af8bc9442f9c935dc8db
--
2.39.1
M
M
Maxim Cournoyer wrote on 18 Jan 2023 15:00
[PATCH v2 1/1] build-system/pyproject: Do not wrap native-inputs.
(address . 25235@debbugs.gnu.org)
20230118140036.13436-2-maxim.cournoyer@gmail.com

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
1 file changed, 24 insertions(+), 15 deletions(-)

Toggle diff (66 lines)
diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..9da86bfc54 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net>
;;; Copyright © 2022 Marius Bakke <marius@gnu.org>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -445,7 +446,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
"A phase that just wraps the 'add-installed-pythonpath' procedure."
(add-installed-pythonpath native-inputs outputs))
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
(define (list-of-files dir)
(find-files dir (lambda (file stat)
(and (eq? 'regular (stat:type stat))
@@ -458,20 +459,28 @@ (define bindirs
(string-append dir "/sbin"))))
outputs))
- ;; Do not require "guile" to be present in the package inputs
- ;; even when there is nothing to wrap.
- ;; Also, calculate (guile) only once to prevent some I/O.
- (define %guile (delay (search-input-file inputs "bin/guile")))
- (define (guile) (force %guile))
-
- (let* ((var `("GUIX_PYTHONPATH" prefix
- ,(search-path-as-string->list
- (or (getenv "GUIX_PYTHONPATH") "")))))
- (for-each (lambda (dir)
- (let ((files (list-of-files dir)))
- (for-each (cut wrap-script <> #:guile (guile) var)
- files)))
- bindirs)))
+ (unless (null? bindirs)
+ (let* ((guile (search-input-file inputs "bin/guile"))
+ (native-input-dirs (match native-inputs
+ (((_ . dir) ...)
+ dir)))
+ (input-dirs (match inputs
+ (((_ . dir) ...)
+ dir)))
+ (build-inputs (lset-difference string=? native-input-dirs
+ input-dirs))
+ ;; A build input is an input listed in native-inputs and NOT in
+ ;; inputs.
+ (build-input? (lambda (x)
+ (any (cut string-prefix? <> x) build-inputs)))
+ (var `("GUIX_PYTHONPATH" prefix
+ ,(remove build-input? (search-path-as-string->list
+ (or (getenv "GUIX_PYTHONPATH") ""))))))
+ (for-each (lambda (dir)
+ (let ((files (list-of-files dir)))
+ (for-each (cut wrap-script <> #:guile guile var)
+ files)))
+ bindirs))))
(define* (rename-pth-file #:key name native-inputs outputs #:allow-other-keys
#:rest args)
--
2.39.1
L
L
Ludovic Courtès wrote on 23 Jan 2023 11:27
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
874jsh4kbq.fsf@gnu.org
Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (5 lines)
>
> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
> Filter out native inputs from the values in GUIX_PYTHONPATH.

LGTM!

Toggle quote (6 lines)
> - ;; Do not require "guile" to be present in the package inputs
> - ;; even when there is nothing to wrap.
> - ;; Also, calculate (guile) only once to prevent some I/O.
> - (define %guile (delay (search-input-file inputs "bin/guile")))
> - (define (guile) (force %guile))

I still don’t see what branch this snippet is on, what am I missing?
:-)

Thanks,
Ludo’.
M
M
Maxim Cournoyer wrote on 23 Jan 2023 14:34
[PATCH v3 0/1] build-system/pyproject: Do not wrap native-inputs.
(address . 25235@debbugs.gnu.org)
20230123133434.1852-1-maxim.cournoyer@gmail.com
This fixes a longstanding issue. The fix is made trivial by the changes made
in #60847 adding cross-compilation support for the pyproject build system.
Additionally, it's based on #60868, which touches the wrap phase to use
wrap-script instead of wrap-program.

Changes in v3:
- Rebase

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

Maxim Cournoyer (1):
build-system/pyproject: Do not wrap native-inputs.

guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
1 file changed, 24 insertions(+), 15 deletions(-)


base-commit: 490e386e10ca2b7496c3da7ed85299ed3e9c2404
--
2.39.1
M
M
Maxim Cournoyer wrote on 23 Jan 2023 14:34
[PATCH v3 1/1] build-system/pyproject: Do not wrap native-inputs.
(address . 25235@debbugs.gnu.org)
20230123133434.1852-2-maxim.cournoyer@gmail.com

* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

Changes in v3:
- Rebase

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
1 file changed, 24 insertions(+), 15 deletions(-)

Toggle diff (66 lines)
diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..9da86bfc54 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net>
;;; Copyright © 2022 Marius Bakke <marius@gnu.org>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -445,7 +446,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
"A phase that just wraps the 'add-installed-pythonpath' procedure."
(add-installed-pythonpath native-inputs outputs))
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
(define (list-of-files dir)
(find-files dir (lambda (file stat)
(and (eq? 'regular (stat:type stat))
@@ -458,20 +459,28 @@ (define bindirs
(string-append dir "/sbin"))))
outputs))
- ;; Do not require "guile" to be present in the package inputs
- ;; even when there is nothing to wrap.
- ;; Also, calculate (guile) only once to prevent some I/O.
- (define %guile (delay (search-input-file inputs "bin/guile")))
- (define (guile) (force %guile))
-
- (let* ((var `("GUIX_PYTHONPATH" prefix
- ,(search-path-as-string->list
- (or (getenv "GUIX_PYTHONPATH") "")))))
- (for-each (lambda (dir)
- (let ((files (list-of-files dir)))
- (for-each (cut wrap-script <> #:guile (guile) var)
- files)))
- bindirs)))
+ (unless (null? bindirs)
+ (let* ((guile (search-input-file inputs "bin/guile"))
+ (native-input-dirs (match native-inputs
+ (((_ . dir) ...)
+ dir)))
+ (input-dirs (match inputs
+ (((_ . dir) ...)
+ dir)))
+ (build-inputs (lset-difference string=? native-input-dirs
+ input-dirs))
+ ;; A build input is an input listed in native-inputs and NOT in
+ ;; inputs.
+ (build-input? (lambda (x)
+ (any (cut string-prefix? <> x) build-inputs)))
+ (var `("GUIX_PYTHONPATH" prefix
+ ,(remove build-input? (search-path-as-string->list
+ (or (getenv "GUIX_PYTHONPATH") ""))))))
+ (for-each (lambda (dir)
+ (let ((files (list-of-files dir)))
+ (for-each (cut wrap-script <> #:guile guile var)
+ files)))
+ bindirs))))
(define* (rename-pth-file #:key name native-inputs outputs #:allow-other-keys
#:rest args)
--
2.39.1
M
M
Maxim Cournoyer wrote on 23 Jan 2023 14:38
Re: [PATCH v2 1/1] build-system/pyproject: Do not wrap native-inputs.
(name . Ludovic Courtès)(address . ludo@gnu.org)
87bkmpz7zw.fsf@gmail.com
Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (20 lines)
> Hi,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Fixes <https://issues.guix.gnu.org/25235>.
>>
>> * guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
>> Filter out native inputs from the values in GUIX_PYTHONPATH.
>
> LGTM!
>
>> - ;; Do not require "guile" to be present in the package inputs
>> - ;; even when there is nothing to wrap.
>> - ;; Also, calculate (guile) only once to prevent some I/O.
>> - (define %guile (delay (search-input-file inputs "bin/guile")))
>> - (define (guile) (force %guile))
>
> I still don’t see what branch this snippet is on, what am I missing?
> :-)

It's based on https://issues.guix.gnu.org/60868,itself based on
https://issues.guix.gnu.org/60847. I've tried to express this
relationship by marking the dependent changes blocked by the submissions
they are based on.

I've just rebased the 3 submissions and added you in CC; hopefully that
helps!

--
Thanks,
Maxim
N
N
Nicolas Graves wrote on 1 May 11:36 +0200
[Maxim Cournoyer] bug#25235: [PATCH v3 1/1] build-system/pyproject: Do not wrap native-inputs.
(address . 25235@debbugs.gnu.org)
874jbheuvh.fsf@ngraves.fr
Hi Lars, Ricardo,

I'm advancing with my patch series, which I can submit soon. I was
curious about why this 25235 patch isn't in python-team branch yet since
it's also a very welcome change to the pyproject-build-system.

Cheers,

Nicolas

-------------------- Start of forwarded message --------------------
Subject: bug#25235: [PATCH v3 1/1] build-system/pyproject: Do not wrap
native-inputs.
To: 25235@debbugs.gnu.org
Cc: arunisaac@systemreboot.net, Maxim Cournoyer <maxim.cournoyer@gmail.com>,
ludo@gnu.org, Lars-Dominik Braun <lars@6xq.net>, efraim@flashner.co.il,
rekado@elephly.net, jgart <jgart@dismail.de>, marius@gnu.org
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Mon, 23 Jan 2023 08:34:33 -0500


* guix/build/pyproject-build-system.scm (wrap) [native-inputs]: New argument.
Filter out native inputs from the values in GUIX_PYTHONPATH.

---

Changes in v3:
- Rebase

Changes in v2:
- Add missing copyright line
- Rework wrap phase to avoid removing inputs found in both native-inputs and inputs
- Enclose wrap computations in an 'unless' form and streamline

guix/build/pyproject-build-system.scm | 39 ++++++++++++++++-----------
1 file changed, 24 insertions(+), 15 deletions(-)

Toggle diff (75 lines)
diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm
index a66c1fb34a..9da86bfc54 100644
--- a/guix/build/pyproject-build-system.scm
+++ b/guix/build/pyproject-build-system.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2021 Lars-Dominik Braun <lars@6xq.net>
;;; Copyright © 2022 Marius Bakke <marius@gnu.org>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -445,7 +446,7 @@ (define* (add-install-to-pythonpath #:key native-inputs outputs
"A phase that just wraps the 'add-installed-pythonpath' procedure."
(add-installed-pythonpath native-inputs outputs))
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key native-inputs inputs outputs #:allow-other-keys)
(define (list-of-files dir)
(find-files dir (lambda (file stat)
(and (eq? 'regular (stat:type stat))
@@ -458,20 +459,28 @@ (define bindirs
(string-append dir "/sbin"))))
outputs))
- ;; Do not require "guile" to be present in the package inputs
- ;; even when there is nothing to wrap.
- ;; Also, calculate (guile) only once to prevent some I/O.
- (define %guile (delay (search-input-file inputs "bin/guile")))
- (define (guile) (force %guile))
-
- (let* ((var `("GUIX_PYTHONPATH" prefix
- ,(search-path-as-string->list
- (or (getenv "GUIX_PYTHONPATH") "")))))
- (for-each (lambda (dir)
- (let ((files (list-of-files dir)))
- (for-each (cut wrap-script <> #:guile (guile) var)
- files)))
- bindirs)))
+ (unless (null? bindirs)
+ (let* ((guile (search-input-file inputs "bin/guile"))
+ (native-input-dirs (match native-inputs
+ (((_ . dir) ...)
+ dir)))
+ (input-dirs (match inputs
+ (((_ . dir) ...)
+ dir)))
+ (build-inputs (lset-difference string=? native-input-dirs
+ input-dirs))
+ ;; A build input is an input listed in native-inputs and NOT in
+ ;; inputs.
+ (build-input? (lambda (x)
+ (any (cut string-prefix? <> x) build-inputs)))
+ (var `("GUIX_PYTHONPATH" prefix
+ ,(remove build-input? (search-path-as-string->list
+ (or (getenv "GUIX_PYTHONPATH") ""))))))
+ (for-each (lambda (dir)
+ (let ((files (list-of-files dir)))
+ (for-each (cut wrap-script <> #:guile guile var)
+ files)))
+ bindirs))))
(define* (rename-pth-file #:key name native-inputs outputs #:allow-other-keys
#:rest args)
--
2.39.1




-------------------- End of forwarded message --------------------

--
Best regards,
Nicolas Graves
L
L
Lars-Dominik Braun wrote on 4 May 10:05 +0200
(name . Nicolas Graves)(address . ngraves@ngraves.fr)
ZjXsP3e-jjt9Psf9@noor.fritz.box
Hi,

Toggle quote (4 lines)
> I'm advancing with my patch series, which I can submit soon. I was
> curious about why this 25235 patch isn't in python-team branch yet since
> it's also a very welcome change to the pyproject-build-system.

I believe this change is based on #60847, which Ludo objected to

Lars
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 25235@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 25235
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch