[PATCH] gnu: Add python-blis

OpenSubmitted by Artyom V. Poptsov.
Details
2 participants
  • Ludovic Courtès
  • Artyom V. Poptsov
Owner
unassigned
Severity
normal
A
A
Artyom V. Poptsov wrote on 22 May 00:17 +0200
(address . guix-patches@gnu.org)
87tu9ifs3m.fsf@gmail.com
Hello,

this patch adds "blis" Python module[1] under the name "python-blis".

- Artyom

References:
1. "Fast matrix-multiplication as a self-contained Python library"
From 3d22d0e5f7929ac1b782217ec207a5e462316c66 Mon Sep 17 00:00:00 2001
From: "Artyom V. Poptsov" <poptsov.artyom@gmail.com>
Date: Sun, 22 May 2022 01:14:26 +0300
Subject: [PATCH] gnu: Add python-blis

* gnu/packages/python-xyz.scm (python-blis): New variable.
---
gnu/packages/python-xyz.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (34 lines)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 3c4fa16d16..95b85b9931 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -31449,3 +31449,27 @@ written in C.")
     (synopsis "Cython bindings for MurmurHash2")
     (description "This package provides Cython bindings for MurmurHash2.")
     (license license:expat)))
+
+(define-public python-blis
+  (package
+    (name "python-blis")
+    (version "0.9.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "blis" version))
+              (sha256
+               (base32
+                "0adl7knj11rglg6l9nfqccl8kgs2v1c7qllv9c5kf5k796hir7k9"))))
+    (build-system python-build-system)
+    (native-inputs (list python-pytest python-cython))
+    (inputs (list python python-numpy python-hypothesis))
+    (arguments
+     ;; XXX: Tests are failing with the following error:
+     ;; AttributeError: module 'numpy.__config__' has no attribute 'blas_ilp64_opt_info'
+     (list #:tests? #f))
+    (home-page "https://github.com/explosion/cython-blis")
+    (synopsis "Fast matrix-multiplication Python library")
+    (description
+     "This package provides the @url{https://github.com/flame/blis,
+Blis linear algebra} routines as a self-contained Python C-extension.")
+    (license license:bsd-3)))
-- 
2.25.1
--
Artyom "avp" Poptsov <poptsov.artyom@gmail.com>
CADR Hackerspace co-founder: https://cadrspace.ru/
GPG: D0C2 EAC1 3310 822D 98DE B57C E9C5 A2D9 0898 A02F
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEE0MLqwTMQgi2Y3rV86cWi2QiYoC8FAmKJZP0ACgkQ6cWi2QiY
oC8VnQf+OTRCZWGDEXb7ujqZNsjogn05h+RRG16jDfZZq+B0MFxpga8ycN+4oEHH
VFZX9A9UbOJR276krwtTVGgVBnsVp6NyNCwR9b69G39ihMITiS6fA8i7RXNT+aie
4rS0kgZIP1x8nWq01qACk27QTGMPLIIhGFiAub0ZyeI/gpHLTeGpxCUHva+rV0cP
zp/SoukUjl6KGpq9GXSlFEoKoJnOjU/Q0DtaT5g2us4OHOo3GDVoF+ep0NmVRruO
guzQPau/230OMuY0l/TKxLtrtWvdVJiDjXijBKlZlk1QxlFs1VBK4UsfpVYckjLl
R3mLuUxNgNwT9fVgxgzq3PfQKfTL2Q==
=8DRQ
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 27 May 23:23 +0200
(name . Artyom V. Poptsov)(address . poptsov.artyom@gmail.com)(address . 55565@debbugs.gnu.org)
87pmjyd60l.fsf@gnu.org
Hi Artyom,

"Artyom V. Poptsov" <poptsov.artyom@gmail.com> skribis:

Toggle quote (7 lines)
> From 3d22d0e5f7929ac1b782217ec207a5e462316c66 Mon Sep 17 00:00:00 2001
> From: "Artyom V. Poptsov" <poptsov.artyom@gmail.com>
> Date: Sun, 22 May 2022 01:14:26 +0300
> Subject: [PATCH] gnu: Add python-blis
>
> * gnu/packages/python-xyz.scm (python-blis): New variable.

[...]

Toggle quote (7 lines)
> + (native-inputs (list python-pytest python-cython))
> + (inputs (list python python-numpy python-hypothesis))
> + (arguments
> + ;; XXX: Tests are failing with the following error:
> + ;; AttributeError: module 'numpy.__config__' has no attribute 'blas_ilp64_opt_info'
> + (list #:tests? #f))

Doesn’t that failure suggest something’s really wrong? Seems worth
investigating.

Toggle quote (6 lines)
> + (synopsis "Fast matrix-multiplication Python library")
> + (description
> + "This package provides the @url{https://github.com/flame/blis,
> +Blis linear algebra} routines as a self-contained Python C-extension.")

Blis isn’t listed in ‘inputs’, and apparently that’s because it’s
bundled.

Could you “unbundle” it? That is, add Blis as an input, make sure it
gets used during the build process, and remove the bundled copy.

Thanks in advance,
Ludo’.
A
A
Artyom V. Poptsov wrote on 11 Jun 12:44 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 55565@debbugs.gnu.org)
87pmjfxywd.fsf@gmail.com
Hello Ludovic,

thanks for the feedback.

I tried to fix the patch and got some promising results. Please find
the updated patch attached.

Basically I patched 'blis/benchmark.py' to use "blas_opt_info" instead
of "blas_ilp64_opt_info" and tests went fine. An issue with different
numpy versions I guess? I also added "blis" and "python-numpy-next" to
the "native-inputs".

- Artyom
From 3d088c0cbeee76c59dc140db16119a5e3f837b08 Mon Sep 17 00:00:00 2001
From: "Artyom V. Poptsov" <poptsov.artyom@gmail.com>
Date: Sun, 22 May 2022 01:14:26 +0300
Subject: [PATCH] gnu: Add python-blis

* gnu/packages/python-xyz.scm (python-blis): New variable.
---
gnu/packages/python-xyz.scm | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)

Toggle diff (46 lines)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 5f9ce4fdfe..934dfd2959 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -29435,3 +29435,39 @@ Information about these scales can be
 @url{https://en.wikipedia.org/wiki/List_of_musical_scales_and_modes, found on
 Wikipedia}.")
     (license license:expat)))
+
+(define-public python-blis
+  (package
+    (name "python-blis")
+    (version "0.9.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/explosion/cython-blis")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "1p4y4mf8046kv2syf2s0v6mf4s000anr1ws7w3fny852gpp6czk4"))))
+    (build-system python-build-system)
+    (native-inputs (list python-numpy-next
+                         python-pytest
+                         python-cython
+                         blis
+                         python-hypothesis))
+    (arguments
+     (list #:tests? #t
+           #:phases
+           #~(modify-phases %standard-phases
+               (add-after 'unpack 'patch
+                 (lambda _
+                   (substitute* "./blis/benchmark.py"
+                     (("blas_ilp64_opt_info")
+                      "blas_opt_info")))))))
+    (home-page "https://github.com/explosion/cython-blis")
+    (synopsis "Fast matrix-multiplication Python library")
+    (description
+     "This package provides the @url{https://github.com/flame/blis,
+Blis linear algebra} routines as a self-contained Python C-extension.")
+    (license license:bsd-3)))
-- 
2.25.1
--
Artyom "avp" Poptsov <poptsov.artyom@gmail.com>
CADR Hackerspace co-founder: https://cadrspace.ru/
GPG: D0C2 EAC1 3310 822D 98DE B57C E9C5 A2D9 0898 A02F
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEE0MLqwTMQgi2Y3rV86cWi2QiYoC8FAmKkchIACgkQ6cWi2QiY
oC/YjQgApAafjyBkTS+z64ZdKCt7nBzeN0fcYRHBmjFIJjAkzzIiUEMCxga5hRoq
OhAynKks3NjYp78Nisj0IH3KyZW6xxGov/e6FioZrPnWeEp/WUSik9aQvMWuPgJ0
PacIatbAH7hfLi7LXPwC0p03PE5KTWFjN6LjhQfuoPt+k+3lkybfpwL4rySGf1K5
Uvxz00KLnhF/orbTA6DkMEYD6e375bVGO7l7EygRmhiPnl92IQzZl41Qm80AlU4z
KL5nSGwn9QqvXKPYLOr9UxdJBazcIAU9413f58Ni7ijilrHpjqwKCD37ZIyvfGJz
m+DXj5XmPOQFeUI9khUgK8X2wCMTug==
=XKhy
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 13 Jun 12:20 +0200
(name . Artyom V. Poptsov)(address . poptsov.artyom@gmail.com)(address . 55565@debbugs.gnu.org)
87wndksw4d.fsf@gnu.org
Hi Artyom,

"Artyom V. Poptsov" <poptsov.artyom@gmail.com> skribis:

Toggle quote (5 lines)
> Basically I patched 'blis/benchmark.py' to use "blas_opt_info" instead
> of "blas_ilp64_opt_info" and tests went fine. An issue with different
> numpy versions I guess? I also added "blis" and "python-numpy-next" to
> the "native-inputs".

Unfortunately this is not sufficient: the source bundles a copy of BLIS
under ‘blis/_src’ and it starts by building it (which is why it takes so
long), whether or not BLIS is among the inputs.

Could you (1) add a snippet that removes ‘blis/_src’, and (2) see
whether/how ‘setup.py’ can be patched to not build BLIS? It might be
that commenting out the line that reads:

cmdclass={"build_ext": ExtensionBuilder},

would be enough, I don’t know.

Anyhow, given that BLIS is the kind of package that’s highly tuned on
our side for performance and reproducibility configuration, it’s
important to not have several copies around.

Toggle quote (6 lines)
> + (native-inputs (list python-numpy-next
> + python-pytest
> + python-cython
> + blis
> + python-hypothesis))

NumPy and BLIS should definitely be ‘inputs’, not ‘native-inputs’; not
sure about ‘hypothesis’.

Also, please include a short comment explaining why numpy-next is used
rather than numpy (like “version >= X.Y required”).

Could you send an updated patch?

Sorry that this is providing trickier than we’d like!

Thanks,
Ludo’.
?