[PATCH core-updates] build-system/python: Add a #:python-output argument.

  • Done
  • quality assurance status badge
Details
2 participants
  • Jakub K?dzio?ka
  • Leo Famulari
Owner
unassigned
Submitted by
Jakub K?dzio?ka
Severity
normal
J
J
Jakub K?dzio?ka wrote on 6 Apr 2020 16:48
(address . guix-patches@gnu.org)(address . leo@famulari.name)
20200406144809.30928-1-kuba@kadziolka.net
This simplifies packages that ship Python bindings in a separate output.

* guix/build-system/python.scm (python-build): Add the argument, pass it
build-side.
* guix/build/python-build-system.scm (site-packages): Add an #:output
argument.
(add-installed-pythonpath): Likewise.
(install): Use the #:python-output argument.
(wrap): Likewise.
(rename-pth-file): Likewise. Use the site-packages procedure.
---

See #40267 for an example of what this change wants to accomplish.

guix/build-system/python.scm | 2 ++
guix/build/python-build-system.scm | 27 ++++++++++++++-------------
2 files changed, 16 insertions(+), 13 deletions(-)

Toggle diff (103 lines)
diff --git a/guix/build-system/python.scm b/guix/build-system/python.scm
index e39c06528e..34cc487c8c 100644
--- a/guix/build-system/python.scm
+++ b/guix/build-system/python.scm
@@ -170,6 +170,7 @@ pre-defined variants."
(phases '(@ (guix build python-build-system)
%standard-phases))
(outputs '("out"))
+ (python-output "out")
(search-paths '())
(system (%current-system))
(guile #f)
@@ -196,6 +197,7 @@ provides a 'setup.py' file as its build system."
#:use-setuptools? ,use-setuptools?
#:phases ,phases
#:outputs %outputs
+ #:python-output ,python-output
#:search-paths ',(map search-path-specification->sexp
search-paths)
#:inputs %build-inputs)))
diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index 09bd8465c8..3b19072264 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -154,28 +154,29 @@
(major+minor (take components 2)))
(string-join major+minor ".")))
-(define (site-packages inputs outputs)
+(define* (site-packages inputs outputs #:key (output "out"))
"Return the path of the current output's Python site-package."
- (let* ((out (assoc-ref outputs "out"))
+ (let* ((out (assoc-ref outputs output))
(python (assoc-ref inputs "python")))
(string-append out "/lib/python"
(python-version python)
"/site-packages/")))
-(define (add-installed-pythonpath inputs outputs)
+(define* (add-installed-pythonpath inputs outputs #:key (output "out"))
"Prepend the Python site-package of OUTPUT to PYTHONPATH. This is useful
when running checks after installing the package."
(let ((old-path (getenv "PYTHONPATH"))
- (add-path (site-packages inputs outputs)))
+ (add-path (site-packages inputs outputs #:output output)))
(setenv "PYTHONPATH"
(string-append add-path
(if old-path (string-append ":" old-path) "")))
#t))
(define* (install #:key outputs (configure-flags '()) use-setuptools?
+ (python-output "out")
#:allow-other-keys)
"Install a given Python package."
- (let* ((out (assoc-ref outputs "out"))
+ (let* ((out (assoc-ref outputs python-output))
(params (append (list (string-append "--prefix=" out))
(if use-setuptools?
;; distutils does not accept these flags
@@ -186,7 +187,9 @@ when running checks after installing the package."
(call-setuppy "install" params use-setuptools?)
#t))
-(define* (wrap #:key inputs outputs #:allow-other-keys)
+(define* (wrap #:key inputs outputs
+ (python-output "out")
+ #:allow-other-keys)
(define (list-of-files dir)
(find-files dir (lambda (file stat)
(and (eq? 'regular (stat:type stat))
@@ -199,7 +202,7 @@ when running checks after installing the package."
(string-append dir "/sbin"))))
outputs))
- (let* ((out (assoc-ref outputs "out"))
+ (let* ((out (assoc-ref outputs python-output))
(python (assoc-ref inputs "python"))
(var `("PYTHONPATH" prefix
,(cons (string-append out "/lib/python"
@@ -214,17 +217,15 @@ when running checks after installing the package."
bindirs)
#t))
-(define* (rename-pth-file #:key name inputs outputs #:allow-other-keys)
+(define* (rename-pth-file #:key name inputs outputs
+ (python-output "out")
+ #:allow-other-keys)
"Rename easy-install.pth to NAME.pth to avoid conflicts between packages
installed with setuptools."
;; Even if the "easy-install.pth" is not longer created, we kept this phase.
;; There still may be packages creating an "easy-install.pth" manually for
;; some good reason.
- (let* ((out (assoc-ref outputs "out"))
- (python (assoc-ref inputs "python"))
- (site-packages (string-append out "/lib/python"
- (python-version python)
- "/site-packages"))
+ (let* ((site-packages (site-packages inputs outputs #:output python-output))
(easy-install-pth (string-append site-packages "/easy-install.pth"))
(new-pth (string-append site-packages "/" name ".pth")))
(when (file-exists? easy-install-pth)
--
2.26.0
L
L
Leo Famulari wrote on 29 May 2020 03:32
(name . Jakub K?dzio?ka)(address . kuba@kadziolka.net)(address . 40469@debbugs.gnu.org)
20200529013240.GG5443@jasmine.lan
On Mon, Apr 06, 2020 at 04:48:09PM +0200, Jakub K?dzio?ka wrote:
Toggle quote (14 lines)
> This simplifies packages that ship Python bindings in a separate output.
>
> * guix/build-system/python.scm (python-build): Add the argument, pass it
> build-side.
> * guix/build/python-build-system.scm (site-packages): Add an #:output
> argument.
> (add-installed-pythonpath): Likewise.
> (install): Use the #:python-output argument.
> (wrap): Likewise.
> (rename-pth-file): Likewise. Use the site-packages procedure.
> ---
>
> See #40267 for an example of what this change wants to accomplish.

Too bad it didn't make it into the recent core-updates cycle.

If you are ready to add Unicorn, you could do that on master with a TODO
comment about using this feature once it is live.

And I think you can push this to the current core-updates along with a
commit adjusting Unicorn. If you cherry-picked just this commit to
master then it would probably not take *too* long to test. That's what I
would do to test it, anyways.

Thanks for working on this!
J
J
Jakub K?dzio?ka wrote on 21 Jun 2020 03:27
[PATCH core-updates v2 2/2] gnu: unicorn: Use python-build-system with grace.
(address . 40469@debbugs.gnu.org)
20200621012749.3742-2-kuba@kadziolka.net
* gnu/packages/emulators.scm (unicorn)[arguments]: Remove the
install-bindings-to-python-output phase. Adjust output names used in
other phases.
---
gnu/packages/emulators.scm | 19 ++++---------------
1 file changed, 4 insertions(+), 15 deletions(-)

Toggle diff (55 lines)
diff --git a/gnu/packages/emulators.scm b/gnu/packages/emulators.scm
index ab5b662915..0e83d7e82c 100644
--- a/gnu/packages/emulators.scm
+++ b/gnu/packages/emulators.scm
@@ -1700,17 +1700,6 @@ performance, features, and ease of use.")
(guix build utils))
#:phases
(modify-phases %standard-phases
- (add-after 'unpack 'install-bindings-to-python-output
- (lambda* (#:key outputs #:allow-other-keys)
- ;; python-build-system will build the bindings and install them to
- ;; the "out" output, so change the build-internal names of the
- ;; outputs.
- ;;
- ;; TODO: remove this once #40469 lands, through the core-updates
- ;; holding zone, on master.
- (set-car! (assoc "out" outputs) "lib")
- (set-car! (assoc "python" outputs) "out")
- #t))
(add-before 'build 'build-library
(lambda* (#:key inputs #:allow-other-keys)
(invoke "make"
@@ -1723,7 +1712,7 @@ performance, features, and ease of use.")
"UNICORN_STATIC=no"
(string-append
"PREFIX="
- (assoc-ref outputs "lib")))))
+ (assoc-ref outputs "out")))))
(add-before 'build 'prepare-bindings
(lambda* (#:key outputs #:allow-other-keys)
(chdir "bindings/python")
@@ -1736,7 +1725,7 @@ performance, features, and ease of use.")
(("_path_list = \\[.*")
(string-append
"_path_list = [\""
- (assoc-ref outputs "lib")
+ (assoc-ref outputs "out")
;; eat the rest of the list
"/lib\"] + 0*[")))
#t))
@@ -1757,10 +1746,10 @@ performance, features, and ease of use.")
(let* ((python-samples (find-files "." "sample_.*"))
(c-samples (find-files "../../samples" ".*\\.c"))
(python-docdir
- (string-append (assoc-ref outputs "out")
+ (string-append (assoc-ref outputs "python")
"/share/doc/unicorn/samples"))
(c-docdir
- (string-append (assoc-ref outputs "lib")
+ (string-append (assoc-ref outputs "out")
"/share/doc/unicorn/samples")))
(for-each (cut install-file <> c-docdir) c-samples)
(for-each (cut install-file <> python-docdir) python-samples)
--
2.26.2
J
J
Jakub K?dzio?ka wrote on 21 Jun 2020 03:27
[PATCH core-updates v2 1/2] build-system/python: Install to the python output if present.
(address . 40469@debbugs.gnu.org)
20200621012749.3742-1-kuba@kadziolka.net
* gnu/build/python-build-system.scm (python-output): New procedure.
(site-packages, install): Use python-output to find the output path.
(wrap, rename-pth-file): Use site-packages where appropriate.
* doc/guix.texi (Build Systems): Mention the new behavior.
---

Following a discussion on IRC, I am changing the approach a bit. Also,
should this go on core-updates or staging? I'm not sure how to account
for all the dependent packages, but just grepping for
python-build-system puts this in the staging range.

$ rg python-build-system gnu/packages | wc -l
1474

doc/guix.texi | 7 +++++++
guix/build/python-build-system.scm | 24 +++++++++++-------------
2 files changed, 18 insertions(+), 13 deletions(-)

Toggle diff (85 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 2268e159a2..7af98d3e00 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -6685,6 +6685,13 @@ By default guix calls @code{setup.py} under control of
@code{setuptools}, much like @command{pip} does. Some packages are not
compatible with setuptools (and pip), thus you can disable this by
setting the @code{#:use-setuptools?} parameter to @code{#f}.
+
+If a @code{"python"} output is available, the package is installed into it
+instead of the default @code{"out"} output. This is useful for packages that
+include a Python package as only a part of the software, and thus want to
+combine the phases of @code{python-build-system} with another build system.
+Python bindings are a common usecase.
+
@end defvr
@defvr {Scheme Variable} perl-build-system
diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index 09bd8465c8..62e7a7b305 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -6,6 +6,7 @@
;;; Copyright © 2016 Hartmut Goebel <h.goebel@crazy-compilers.com>
;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2018 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2020 Jakub K?dzio?ka <kuba@kadziolka.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -154,9 +155,14 @@
(major+minor (take components 2)))
(string-join major+minor ".")))
+(define (python-output outputs)
+ "Return the path of the python output, if there is one, or fall-back to out."
+ (or (assoc-ref outputs "python")
+ (assoc-ref outputs "out")))
+
(define (site-packages inputs outputs)
"Return the path of the current output's Python site-package."
- (let* ((out (assoc-ref outputs "out"))
+ (let* ((out (python-output outputs))
(python (assoc-ref inputs "python")))
(string-append out "/lib/python"
(python-version python)
@@ -175,7 +181,7 @@ when running checks after installing the package."
(define* (install #:key outputs (configure-flags '()) use-setuptools?
#:allow-other-keys)
"Install a given Python package."
- (let* ((out (assoc-ref outputs "out"))
+ (let* ((out (python-output outputs))
(params (append (list (string-append "--prefix=" out))
(if use-setuptools?
;; distutils does not accept these flags
@@ -199,12 +205,8 @@ when running checks after installing the package."
(string-append dir "/sbin"))))
outputs))
- (let* ((out (assoc-ref outputs "out"))
- (python (assoc-ref inputs "python"))
- (var `("PYTHONPATH" prefix
- ,(cons (string-append out "/lib/python"
- (python-version python)
- "/site-packages")
+ (let* ((var `("PYTHONPATH" prefix
+ ,(cons (site-packages inputs outputs)
(search-path-as-string->list
(or (getenv "PYTHONPATH") ""))))))
(for-each (lambda (dir)
@@ -220,11 +222,7 @@ installed with setuptools."
;; Even if the "easy-install.pth" is not longer created, we kept this phase.
;; There still may be packages creating an "easy-install.pth" manually for
;; some good reason.
- (let* ((out (assoc-ref outputs "out"))
- (python (assoc-ref inputs "python"))
- (site-packages (string-append out "/lib/python"
- (python-version python)
- "/site-packages"))
+ (let* ((site-packages (site-packages inputs outputs))
(easy-install-pth (string-append site-packages "/easy-install.pth"))
(new-pth (string-append site-packages "/" name ".pth")))
(when (file-exists? easy-install-pth)
--
2.26.2
J
J
Jakub K?dzio?ka wrote on 18 Jul 2020 18:16
Re: [PATCH core-updates] build-system/python: Add a #:python-output argument.
(address . 40469-done@debbugs.gnu.org)
20200718161654.p322j7d74lk3gl4g@gravity
Patchstack pushed in commits @ 352e6c8a750c1b995ad8a357caf65eeded7e7774,
closing.
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEE5Xa/ss9usT31cTO54xWnWEYTFWQFAl8TIHYACgkQ4xWnWEYT
FWRTBxAAjVYXFRZrSUFuYtVrwP2U4OlEeefYmR3YFmN95PbZW82Hp13KwEGCJdrU
eUiub6/beFHj30gUlFrEcBkLbu7FqB3I0s0E8DfvQqb/UT0rRnVVEKOYNSDuTJgu
NWYWI49Ojvm5B2esu2eMpwajUtOwUcHWFcLwRWi2wDb8oJ6iMA5If9CTqxh28YRa
xsxj+PYrPmPE9OCGxPVSBk4BFlGnUJYZYqPW9dumVZwr0ll0IL9MHsem4kb2rtI9
WmO+sirJJS4IWgaYvvHkHCWxfQXaxHz8xnGvv0FSWeVcnHJ2MmvY5Wwn0WdAR3hK
dhHelG9wRKL3/0T6fDzdDOQCN/scyPy9cM0qexNGEVOPSHYvqKy0kz8OAnzuJuQr
IawEryGmV2E9OKm0zc6plhnEveCbNWRj8zFkGpE0W7WK1LOIQ4FQyRDHDrgrLYgz
+mvzyTyzXdtGyqEMfj2GQE+PBSdatrqGK8ENF3blELZv13XOL8Pq1/epXaE/2XHW
he0BHcPUI8IUyVEiJqMsi62ydbnRCFTTolHMAWx62E2l1RfHBcBPRtAqAst9i+wP
5f0CfEZv/skAOPxS4s8tO87Av2liTwXiJQtgAIEbgpZsZjgTQcOsdgzFdCIufl5/
nYc6XhPJCE1GaKm2gMmE53QbX9/ysGY5fpJlf+uLBCnxqNIliLc=
=xphP
-----END PGP SIGNATURE-----


Closed
?