Wrapped python programs get native-inputs in PYTHONPATH

  • Open
  • quality assurance status badge
Details
4 participants
  • Arun Isaac
  • Leo Famulari
  • Ludovic Courtès
  • Marius Bakke
Owner
unassigned
Submitted by
Marius Bakke
Severity
normal
Blocked by
M
M
Marius Bakke wrote on 20 Dec 2016 00:28
(address . bug-guix@gnu.org)
87eg13birp.fsf@kirby.i-did-not-set--mail-host-address--so-tickle-me
The 'wrap' phase of 'python-build-system' adds native-inputs to
PYTHONPATH of the wrapped programs. This causes unnecessary runtime
dependencies and may create problems when cross-compiling.

Given that all python dependencies are now supposed to be propagated,
perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAlhYbToACgkQoqBt8qM6
VPrqCgf9ET1NPk/I7q7M1zPHlbqOOrYWkk2fS7RAH3Un/X33Vve1DzKWsj7OQ7LO
YdU8ME0f8wJZ+tILUMawNRt/lKs/CmHug+9aOcqSfN7O8jawFVXxEPIE4U5wCM72
pMwhbu5TwRUZACD7G1168U257uyHpir26rf3wypbzWeAJFr50LEfxbPeBXjsR81v
YPYUUeJ1tCR1DedtU0Ih4TK8eW1ipveYMyw1iT9W+OMFO+y8qZwy1NyRNSrHgOcM
Ur/rmMqjJc+Yhvf37effW6Jg8OODBsqZzocVVqtvetk6Uo7S3jKKFu+Ov+SKyvSJ
FKNV5SC+DFxHSbdG2i35yMit8bGMcA==
=Ed8O
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 20 Dec 2016 15:00
(name . Marius Bakke)(address . mbakke@fastmail.com)(address . 25235@debbugs.gnu.org)
87y3zahf8t.fsf@gnu.org
Marius Bakke <mbakke@fastmail.com> skribis:

Toggle quote (7 lines)
> The 'wrap' phase of 'python-build-system' adds native-inputs to
> PYTHONPATH of the wrapped programs. This causes unnecessary runtime
> dependencies and may create problems when cross-compiling.
>
> Given that all python dependencies are now supposed to be propagated,
> perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.

We cannot simply remove it: it is meant for packages that provide
standalone programs, such as ‘certbot’. For these, it automatically
wraps bin/* and we don’t even have to have propagated inputs, which is
pretty cool.

However, you are right that native inputs shouldn’t be there. Probably
we need to add some filtering in the ‘wrap’ phase?

Ludo’.
L
L
Leo Famulari wrote on 26 Dec 2016 19:26
(name . Ludovic Courtès)(address . ludo@gnu.org)
20161226182608.GA20609@jasmine
On Tue, Dec 20, 2016 at 03:00:50PM +0100, Ludovic Courtès wrote:
Toggle quote (17 lines)
> Marius Bakke <mbakke@fastmail.com> skribis:
>
> > The 'wrap' phase of 'python-build-system' adds native-inputs to
> > PYTHONPATH of the wrapped programs. This causes unnecessary runtime
> > dependencies and may create problems when cross-compiling.
> >
> > Given that all python dependencies are now supposed to be propagated,
> > perhaps we can simply remove the wrap phase? It only touches PYTHONPATH.
>
> We cannot simply remove it: it is meant for packages that provide
> standalone programs, such as ‘certbot’. For these, it automatically
> wraps bin/* and we don’t even have to have propagated inputs, which is
> pretty cool.
>
> However, you are right that native inputs shouldn’t be there. Probably
> we need to add some filtering in the ‘wrap’ phase?

While looking at the beets package definition with fresh eyes, I realize
that it will surely break once this change is made. So it will make a
good test case for this improvement :)
A
A
Arun Isaac wrote on 29 Mar 2017 13:27
(address . 25235@debbugs.gnu.org)
716a63e7.AEQAIoR8aXkAAAAAAAAAAAOwyEEAAAACwQwAAAAAAAW9WABY25oY@mailjet.com
I'm working on fixing this bug. I have modified
guix/build/python-build-system.scm for the same. In particular, I have
added #:use-module (guix packages) because I need the functions
`package-name' and `package-transitive-target-inputs'. But, when I try
building any python-build-system package with something like
"./pre-inst-env guix build scons", I get a ("no code for module" (guix
packages)) error. What am I missing?

Full backtrace follows:

Backtrace:
In ice-9/eval.scm:
432: 19 [eval # ()]
In system/base/compile.scm:
152: 18 [compile-file "/gnu/store/38rxls2y8v8zjva7xq7zv99rv6c81bq3-module-import/guix/build/python-build-system.scm" ...]
43: 17 [call-once #<procedure ae84c0 at system/base/compile.scm:56:5 ()>]
In ice-9/boot-9.scm:
174: 16 [with-throw-handler #t ...]
In system/base/compile.scm:
59: 15 [#<procedure ae8480 at system/base/compile.scm:58:9 ()>]
155: 14 [#<procedure ae8500 at system/base/compile.scm:153:8 (port)> #<closed: file 0>]
218: 13 [read-and-compile #<input: /gnu/store/38rxls2y8v8zjva7xq7zv99rv6c81bq3-module-import/guix/build/python-build-system.scm 6> ...]
234: 12 [lp () #f #<module (#{ g152}#) ae7a20>]
182: 11 [lp (#<procedure compile-tree-il (x e opts)>) (define-module # # ...) ...]
In ice-9/boot-9.scm:
2404: 10 [save-module-excursion #<procedure c967e0 at language/scheme/compile-tree-il.scm:29:3 ()>]
In language/scheme/compile-tree-il.scm:
31: 9 [#<procedure c967e0 at language/scheme/compile-tree-il.scm:29:3 ()>]
In ice-9/psyntax.scm:
1106: 8 [expand-top-sequence ((define-module # # # ...)) () ((top)) ...]
989: 7 [scan ((define-module # # # ...)) () ((top)) ...]
279: 6 [scan ((#(syntax-object let # ...) (#) (# #) ...)) () ...]
In ice-9/eval.scm:
411: 5 [eval # ()]
In ice-9/boot-9.scm:
2954: 4 [define-module* (guix build python-build-system) #:filename ...]
2929: 3 [resolve-imports ((# # gnu:) (#) (#) (#) ...)]
2870: 2 [resolve-interface (guix packages) #:select ...]
In unknown file:
?: 1 [scm-error misc-error #f "~A ~S" ("no code for module" (guix packages)) #f]
In ice-9/boot-9.scm:
109: 0 [#<procedure ae8440 at ice-9/boot-9.scm:100:6 (thrown-k . args)> misc-error ...]

ice-9/boot-9.scm:109:20: In procedure #<procedure ae8440 at ice-9/boot-9.scm:100:6 (thrown-k . args)>:
ice-9/boot-9.scm:109:20: no code for module (guix packages)
builder for `/gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv' failed with exit code 1
@ build-failed /gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv - 1 builder for `/gnu/store/lsrxzd1hdsgpb9zhrdv02dsxihpmxmww-module-import-compiled.drv' failed with exit code 1
cannot build derivation `/gnu/store/pm2r6cbcfij8d7nzdxkzqx1dq880mhg8-scons-2.3.4.drv': 1 dependencies couldn't be built
guix build: error: build failed: build of `/gnu/store/pm2r6cbcfij8d7nzdxkzqx1dq880mhg8-scons-2.3.4.drv' failed
L
L
Ludovic Courtès wrote on 30 Mar 2017 17:30
(name . Arun Isaac)(address . arunisaac@systemreboot.net)(address . 25235@debbugs.gnu.org)
87o9wiwzn2.fsf@gnu.org
Hello,

Arun Isaac <arunisaac@systemreboot.net> skribis:

Toggle quote (10 lines)
> I'm working on fixing this bug. I have modified
> guix/build/python-build-system.scm for the same. In particular, I have
> added #:use-module (guix packages) because I need the functions
> `package-name' and `package-transitive-target-inputs'. But, when I try
> building any python-build-system package with something like
> "./pre-inst-env guix build scons", I get a ("no code for module" (guix
> packages)) error. What am I missing?
>
> Full backtrace follows:

[...]

Toggle quote (6 lines)
> 2954: 4 [define-module* (guix build python-build-system) #:filename ...]
> 2929: 3 [resolve-imports ((# # gnu:) (#) (#) (#) ...)]
> 2870: 2 [resolve-interface (guix packages) #:select ...]
> In unknown file:
> ?: 1 [scm-error misc-error #f "~A ~S" ("no code for module" (guix packages)) #f]

“Build-side” modules, which typically live in (guix build …), should not
depend on “host-side” modules such as (guix packages). That’s because
if we did that, we’d effectively end up importing all of Guix on the
build side, but then we’d also have to serialize data structures such as
packages to pass them from one side to the other. (I hope this makes
sense to you, but if it doesn’t maybe the intro of
shed some light.)

So in short, we cannot use ‘package-name’ and
‘package-transitive-target-inputs’ in this module.

(Time passes…)

I wasn’t sure how to fix this bug myself so I gave it a try and ended up
with the patch below, but I haven’t tested in detail. (You’ll notice
(guix build-system python) is hard to work with because it doesn’t use
gexps yet.)

How does it look?

Thanks,
Ludo’.
Attachment: file
?