[PATCH 00/31] Unbundle and update python-pytorch

  • Done
  • quality assurance status badge
Details
4 participants
  • David Elsing
  • Ludovic Courtès
  • Ludovic Courtès
  • Maxim Cournoyer
Owner
unassigned
Submitted by
David Elsing
Severity
normal
D
D
David Elsing wrote on 6 Mar 20:37 +0100
(address . guix-patches@gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306193725.17463-1-david.elsing@posteo.net
This patch series unbundles all dependencies of python-pytorch (updated to
version 2.2.1) and removes all* autogenerated files, completing the work by
Ludo' [1]. I also applied the unbundling to python-pytorch-for-r-torch
(version 2.0.1) and made sure that r-torch still builds.

Originally, I wanted to package PyTorch with ROCm, but decided that it would
be better to first update the package and complete the unbundling. This was
more work than I initially thought though, so this patch series is quite long.

Caffe2 is heavily integrated in the PyTorch code, so unbundling it does
not make sense.

For QNNPACK, the situation is strange: There is an old version [2] and an
internal fork in aten/src/ATen/native/quantized/cpu/qnnpack, which PyTorch
both depends on (as well as XNNPACK). The internal fork does not depend on
other parts of the code currently, so I unbundled it as well as a package
variant of qnnpack.

ideep [3] has a separate branch for PyTorch (the master branch does not
seem maintained), which only contains header files without build
instructions. Therefore, I made a simple package with the header files.

miniz also requires a package variant for PyTorch.

Several scripts for autogenerated files depend on the compiled PyTorch
library. However, it was possible to patch the scripts with empty values
and later regenerate the files with the initial PyTorch version.
Therefore, I introduced the 'codegen2, 'build2 and 'install2 build
phases running after the initial library is installed. In 'codegen2, the
installed library is used to regenerate the files properly, which leads
to a recompilation of only 12 files in the 'build2 phase.

The core tests can be run as well, but I disabled them as they take
a long time after an already long build.

I also wanted to try and build python-pytorch on i686-linux, as it doesn't
require emulation on an x86_64 CPU, but it depends on Rust through
python-cryptography, for which there are no substitutes available currently.

Cheers,
David

* with the exception of test/cpp/api/init_baseline.h and
test/cpp/api/optim_baseline.h, which are just lists of previously
calculated numbers; or any other generated file I might have missed.


David Elsing (31):
gnu: asmjit: Update to commit 3ca5c18.
gnu: Add python-typing-extensions-4.10.
gnu: Add python-optree.
gnu: Add python-pytest-flakefinder.
gnu: Add python-pytest-shard.
gnu: Add python-expecttest.
gnu: Add python-pytest-rerunfailures-13.
gnu: Add miniz.
gnu: Add miniz-for-pytorch.
gnu: Add libnop.
gnu: Remove flatbuffers-next-shared.
gnu: python-flatbuffers-next: Update to 23.5.26.
gnu: pthreadpool: Update to commit 178e3e0.
gnu: cpuinfo: Update to commit aa4b216.
gnu: clog: Add "-DUSE_SYSTEM_LIBS=ON" configure flag.
gnu: nnpack: Update to commit 70a77f4.
gnu: oneapi-dnnl: Update to 3.3.5.
gnu: Add tensorpipe.
gnu: Add fbgemm.
gnu: Add qnnpack.
gnu: Add foxi.
gnu: Add ideep-pytorch.
gnu: xnnpack: Update to commit 51a9875.
gnu: Remove xnnpack-for-torch2.
gnu: Add qnnpack-pytorch.
gnu: python-pytorch: Update to 2.2.1 and unbundle dependencies.
gnu: python-torchvision: Update to 0.17.1.
gnu: Add ideep-pytorch-for-r-torch.
gnu: Add oneapi-dnnl-for-r-torch.
gnu: Add qnnpack-pytorch-for-r-torch.
gnu: python-pytorch-for-r-torch: Adjust to new python-pytorch.

gnu/local.mk | 9 +-
gnu/packages/compression.scm | 49 +
gnu/packages/cpp.scm | 6 +-
gnu/packages/machine-learning.scm | 874 +++-
gnu/packages/parallel.scm | 30 +-
.../patches/clog-fix-shared-build.patch | 39 +-
.../patches/cpuinfo-system-libraries.patch | 50 -
.../patches/fbgemm-use-system-libraries.patch | 62 +
gnu/packages/patches/foxi-fix-build.patch | 55 +
gnu/packages/patches/miniz-for-pytorch.patch | 47 +
.../patches/python-optree-fix-32-bit.patch | 122 +
...ython-pytorch-1.9.0-system-libraries.patch | 139 -
.../patches/python-pytorch-fix-codegen.patch | 167 +
...thon-pytorch-for-r-torch-fix-codegen.patch | 167 +
...pytorch-for-r-torch-system-libraries.patch | 390 ++
.../patches/python-pytorch-runpath.patch | 19 +-
.../python-pytorch-system-libraries.patch | 324 +-
.../python-pytorch-without-kineto.patch | 60 +
.../python-pytorch2-system-libraries.patch | 156 -
.../xnnpack-for-torch2-system-libraries.patch | 2660 -----------
.../patches/xnnpack-remove-broken-tests.patch | 335 ++
.../patches/xnnpack-system-libraries.patch | 4133 ++++++++++-------
gnu/packages/python-build.scm | 12 +
gnu/packages/python-check.scm | 88 +
gnu/packages/python-xyz.scm | 29 +
gnu/packages/serialization.scm | 53 +-
26 files changed, 5111 insertions(+), 4964 deletions(-)
delete mode 100644 gnu/packages/patches/cpuinfo-system-libraries.patch
create mode 100644 gnu/packages/patches/fbgemm-use-system-libraries.patch
create mode 100644 gnu/packages/patches/foxi-fix-build.patch
create mode 100644 gnu/packages/patches/miniz-for-pytorch.patch
create mode 100644 gnu/packages/patches/python-optree-fix-32-bit.patch
delete mode 100644 gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch
create mode 100644 gnu/packages/patches/python-pytorch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch
create mode 100644 gnu/packages/patches/python-pytorch-without-kineto.patch
delete mode 100644 gnu/packages/patches/python-pytorch2-system-libraries.patch
delete mode 100644 gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch
create mode 100644 gnu/packages/patches/xnnpack-remove-broken-tests.patch

--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 01/31] gnu: asmjit: Update to commit 3ca5c18.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-1-david.elsing@posteo.net
* gnu/packages/cpp.scm (asmjit): Update to commit 3ca5c18.
---
gnu/packages/cpp.scm | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Toggle diff (26 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index 3b65ce9f4a..6f1df246cb 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -154,8 +154,8 @@ (define-public argagg
(license license:expat))))
(define-public asmjit
- (let ((commit "4ec760a3d1f69e32ba460ecd2513f29b8428700b")
- (revision "0"))
+ (let ((commit "3ca5c186bf8922e5fe3018432e93651fd2fa4053")
+ (revision "1"))
(package
(name "asmjit")
(version (git-version "0.0.0" revision commit))
@@ -168,7 +168,7 @@ (define-public asmjit
(commit commit)))
(file-name (git-file-name name version))
(sha256
- (base32 "0skgccbpamcbg1byawfq5n6jzxgj64hnc7jznvk35nkskaaz1nlb"))))
+ (base32 "10k1zc0w8m0vnh52id9qlm1sb99qmpvr6k0ha8ag2h223n0d591g"))))
(build-system cmake-build-system)
(arguments
(list #:configure-flags #~(list "-DASMJIT_TEST=TRUE")))
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 02/31] gnu: Add python-typing-extensions-4.10.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-2-david.elsing@posteo.net
* gnu/packages/python-build.scm (python-typing-extensions-4.10): New variable.
---
gnu/packages/python-build.scm | 12 ++++++++++++
1 file changed, 12 insertions(+)

Toggle diff (25 lines)
diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 2ea457cdba..21e7e422f9 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -252,6 +252,18 @@ (define-public python-typing-extensions
@end enumerate\n")
(license license:psfl)))
+(define-public python-typing-extensions-4.10
+ (package
+ (inherit python-typing-extensions)
+ (version "4.10.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "typing_extensions" version))
+ (sha256
+ (base32
+ "1jxkj4pni8pdyrn79sq441lsp40xzw363n0qvfc6zfcgkv4dgaxh"))))))
+
;;;
;;; Python builder packages.
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 05/31] gnu: Add python-pytest-shard.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-5-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-pytest-shard): New variable.
---
gnu/packages/python-check.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 36d4025b9d..2338f32801 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -209,6 +209,30 @@ (define-public python-pytest-flakefinder
times and detect flakyness.")
(license license:asl2.0)))
+(define-public python-pytest-shard
+ (let ((commit "64610a08dac6b0511b6d51cf895d0e1040d162ad")
+ (revision "0"))
+ (package
+ (name "python-pytest-shard")
+ (version (git-version "0.1.2" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/AdamGleave/pytest-shard")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1h31m68igz670bzl307hazjrfbr8pk14mxflllar18ydmlrnl677"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs (list python-pytest))
+ (home-page "https://github.com/AdamGleave/pytest-shard")
+ (synopsis "Pytest plugin for sharding tests")
+ (description "This package provides a Pytest extension for sharding
+tests at the granularity of individual test cases, which can be run in
+parallel and on multiple machines.")
+ (license license:expat))))
+
(define-public python-testfixtures
(package
(name "python-testfixtures")
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 03/31] gnu: Add python-optree.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-3-david.elsing@posteo.net
* gnu/packages/python-xyz.scm (python-optree): New variable.
---
.../patches/python-optree-fix-32-bit.patch | 122 ++++++++++++++++++
gnu/packages/python-xyz.scm | 29 +++++
2 files changed, 151 insertions(+)
create mode 100644 gnu/packages/patches/python-optree-fix-32-bit.patch

Toggle diff (170 lines)
diff --git a/gnu/packages/patches/python-optree-fix-32-bit.patch b/gnu/packages/patches/python-optree-fix-32-bit.patch
new file mode 100644
index 0000000000..6a32c39bd8
--- /dev/null
+++ b/gnu/packages/patches/python-optree-fix-32-bit.patch
@@ -0,0 +1,122 @@
+In include/utils.h, ssize_t is an alias for py::ssize_t, which is an alias for
+Py_ssize_t in Python, which is an alias for the system ssize_t.
+The latter is defined in glibc as int if __WORDSIZE == 32 and as long int if
+__WORDSIZE == 64. Therefore, we need to remove the explicit template
+specialization for int in the first case.
+
+diff --git a/include/utils.h b/include/utils.h
+index 950a02b..82a9591 100644
+--- a/include/utils.h
++++ b/include/utils.h
+@@ -141,10 +141,12 @@ template <>
+ inline py::handle GET_ITEM_HANDLE<py::tuple>(const py::handle& container, const size_t& item) {
+ return PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::handle GET_ITEM_HANDLE<py::tuple>(const py::handle& container, const int& item) {
+ return PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#endif
+ template <>
+ inline py::handle GET_ITEM_HANDLE<py::list>(const py::handle& container, const ssize_t& item) {
+ return PyList_GET_ITEM(container.ptr(), item);
+@@ -153,10 +155,12 @@ template <>
+ inline py::handle GET_ITEM_HANDLE<py::list>(const py::handle& container, const size_t& item) {
+ return PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::handle GET_ITEM_HANDLE<py::list>(const py::handle& container, const int& item) {
+ return PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#endif
+
+ template <typename Container, typename Item>
+ inline py::object GET_ITEM_BORROW(const py::handle& container, const Item& item) {
+@@ -171,11 +175,13 @@ inline py::object GET_ITEM_BORROW<py::tuple>(const py::handle& container, const
+ return py::reinterpret_borrow<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_BORROW<py::tuple>(const py::handle& container, const int& item) {
+ return py::reinterpret_borrow<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+ template <>
+ inline py::object GET_ITEM_BORROW<py::list>(const py::handle& container, const ssize_t& item) {
+ return py::reinterpret_borrow<py::object>(PyList_GET_ITEM(container.ptr(), item));
+@@ -185,11 +191,13 @@ inline py::object GET_ITEM_BORROW<py::list>(const py::handle& container, const s
+ return py::reinterpret_borrow<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_BORROW<py::list>(const py::handle& container, const int& item) {
+ return py::reinterpret_borrow<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+
+ template <typename Container, typename Item>
+ inline py::object GET_ITEM_STEAL(const py::handle& container, const Item& item) {
+@@ -204,11 +212,13 @@ inline py::object GET_ITEM_STEAL<py::tuple>(const py::handle& container, const s
+ return py::reinterpret_steal<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_STEAL<py::tuple>(const py::handle& container, const int& item) {
+ return py::reinterpret_steal<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+ template <>
+ inline py::object GET_ITEM_STEAL<py::list>(const py::handle& container, const ssize_t& item) {
+ return py::reinterpret_steal<py::object>(PyList_GET_ITEM(container.ptr(), item));
+@@ -218,11 +228,13 @@ inline py::object GET_ITEM_STEAL<py::list>(const py::handle& container, const si
+ return py::reinterpret_steal<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_STEAL<py::list>(const py::handle& container, const int& item) {
+ return py::reinterpret_steal<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+
+ template <typename Container, typename Item>
+ inline void SET_ITEM(const py::handle& container, const Item& item, const py::handle& value) {
+@@ -240,12 +252,14 @@ inline void SET_ITEM<py::tuple>(const py::handle& container,
+ const py::handle& value) {
+ PyTuple_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline void SET_ITEM<py::tuple>(const py::handle& container,
+ const int& item,
+ const py::handle& value) {
+ PyTuple_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#endif
+ template <>
+ inline void SET_ITEM<py::list>(const py::handle& container,
+ const ssize_t& item,
+@@ -258,12 +272,14 @@ inline void SET_ITEM<py::list>(const py::handle& container,
+ const py::handle& value) {
+ PyList_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline void SET_ITEM<py::list>(const py::handle& container,
+ const int& item,
+ const py::handle& value) {
+ PyList_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#endif
+
+ template <typename PyType>
+ inline void AssertExact(const py::handle& object) {
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index fc6e997b6c..e1e68ca343 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -11635,6 +11635,35 @@ (define-public python-treelib
"This package provides a Python implementation of a tree structure.")
(license license:asl2.0)))
+(define-public python-optree
+ (package
+ (name "python-optree")
+ (version "0.10.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/metaopt/optree")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1q3wljk7cyl5rsam02sfsj8zjrqx4c3x9vic8j6xx13p8czpsisg"))
+ (patches (search-patches "python-optree-fix-32-bit.patch"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs (list python-typing-extensions))
+ (native-inputs
+ (list python-pytest
+ python-pytest-cov
+ python-pytest-xdist
+ cmake
+ pybind11))
+ (home-page "https://github.com/metaopt/optree")
+ (synopsis "Optimized PyTree Utilities")
+ (description "This package contains operations on PyTrees (a tree made of
+container data structures in Python).")
+ (license license:asl2.0)))
+
(define-public python-jupyter-core
(package
(name "python-jupyter-core")
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 04/31] gnu: Add python-pytest-flakefinder.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-4-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-pytest-flakefinder): New variable.
---
gnu/packages/python-check.scm | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 51a38bbcbe..36d4025b9d 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -191,6 +191,24 @@ (define-public python-pytest-csv
it adds to the Pytest command line interface (CLI).")
(license license:gpl3+)))
+(define-public python-pytest-flakefinder
+ (package
+ (name "python-pytest-flakefinder")
+ (version "1.1.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "pytest-flakefinder" version))
+ (sha256
+ (base32 "03iy80xlkpgzjs2kxa9rrj8dbnp9awyhpcl3hy8fgf5x40cjlhg2"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs (list python-pytest))
+ (home-page "https://github.com/dropbox/pytest-flakefinder")
+ (synopsis "Pytest plugin for finding flaky tests")
+ (description "This package provides a Pytest plugin to run tests multiple
+times and detect flakyness.")
+ (license license:asl2.0)))
+
(define-public python-testfixtures
(package
(name "python-testfixtures")
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 06/31] gnu: Add python-expecttest.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-6-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-expecttest): New variable.
---
gnu/packages/python-check.scm | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 2338f32801..94694316ca 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -2077,6 +2077,41 @@ (define-public python-eradicate
valid Python syntax that are likely to be commented out code.")
(license license:expat)))
+(define-public python-expecttest
+ (let ((commit "683b09a352cc426851adc2e3a9f46e0ab25e4dee")
+ (revision "0"))
+ (package
+ (name "python-expecttest")
+ (version (git-version "0.2.1" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/ezyang/expecttest")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1djwxp9x1hczzxbimv1b1bmd083am88v27l82nmlkhvzyg2cmpvv"))))
+ (build-system pyproject-build-system)
+ (arguments
+ (list
+ #:phases
+ #~(modify-phases %standard-phases
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ ;; The test runs tests expected to fail, so the output is
+ ;; confusing
+ (invoke "python3" "test_expecttest.py")))))))
+ (native-inputs (list python-hypothesis poetry))
+ (home-page "https://github.com/ezyang/expecttest")
+ (synopsis "Python module for expect tests")
+ (description "@code{expecttest} is a Python module for expect tests, where
+the initial expected value of a test can be automatically set by running the
+test itself.")
+ (license license:expat))))
+
(define-public python-robber
(package
(name "python-robber")
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 07/31] gnu: Add python-pytest-rerunfailures-13.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-7-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-pytest-rerunfailures-13): New variable.
---
gnu/packages/python-check.scm | 11 +++++++++++
1 file changed, 11 insertions(+)

Toggle diff (24 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 94694316ca..67822c4c50 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -2463,6 +2463,17 @@ (define-public python-pytest-rerunfailures
eliminate flaky failures.")
(license license:mpl2.0)))
+(define-public python-pytest-rerunfailures-13
+ (package
+ (inherit python-pytest-rerunfailures)
+ (version "13.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "pytest-rerunfailures" version))
+ (sha256
+ (base32 "16cin0chv59w4rvnd6r0fisp0s8avmp07rwn9da6yixw43jdncp1"))))))
+
(define-public python-xunitparser
(package
(name "python-xunitparser")
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 08/31] gnu: Add miniz.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-8-david.elsing@posteo.net
* gnu/packages/compression.scm (miniz): New variable.
---
gnu/packages/compression.scm | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)

Toggle diff (38 lines)
diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 5de17b6b51..74075df0ee 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -2838,3 +2838,31 @@ (define-public unrar-free
libarchive. It does not rival the non-free @code{unrar} in terms of features,
but special care has been taken to ensure it meets most user's needs.")
(license license:gpl2+)))
+
+(define-public miniz
+ (package
+ (name "miniz")
+ (version "3.0.2")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/richgel999/miniz")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0672q35vjrpakmsr1gwj9k5fwv5ihzhahm19bq4y74wqpn91p7fw"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ #:configure-flags
+ ''("-DBUILD_SHARED_LIBS=ON")
+ ;; No tests
+ #:tests? #f))
+ (home-page "https://github.com/richgel999/miniz")
+ (synopsis "Independent implementation of zlib and Deflate compression")
+ (description "Miniz is a lossless data compression library that implements
+the zlib (RFC 1950) and Deflate (RFC 1951) compressed data format
+specification standards. It supports the most commonly used functions
+exported by the zlib library.")
+ (license license:expat)))
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 09/31] gnu: Add miniz-for-pytorch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-9-david.elsing@posteo.net
* gnu/packages/compression.scm (miniz-for-pytorch): New variable.
* gnu/packages/patches/miniz-for-pytorch.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
gnu/packages/compression.scm | 21 +++++++++
gnu/packages/patches/miniz-for-pytorch.patch | 47 ++++++++++++++++++++
3 files changed, 69 insertions(+)
create mode 100644 gnu/packages/patches/miniz-for-pytorch.patch

Toggle diff (96 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 83937db4c8..06b3238684 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1687,6 +1687,7 @@ dist_patch_DATA = \
%D%/packages/patches/mingw-w64-reproducible-gendef.patch \
%D%/packages/patches/minisat-friend-declaration.patch \
%D%/packages/patches/minisat-install.patch \
+ %D%/packages/patches/miniz-for-pytorch.patch \
%D%/packages/patches/mit-krb5-hurd.patch \
%D%/packages/patches/mixxx-link-qtscriptbytearray-qtscript.patch \
%D%/packages/patches/mixxx-system-googletest-benchmark.patch \
diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 74075df0ee..dcea1f9ec7 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -2866,3 +2866,24 @@ (define-public miniz
specification standards. It supports the most commonly used functions
exported by the zlib library.")
(license license:expat)))
+
+(define-public miniz-for-pytorch
+ (package
+ (inherit miniz)
+ (version "pytorch-2.2.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/richgel999/miniz")
+ (commit "2.2.0")))
+ (file-name (git-file-name (package-name miniz) version))
+ (sha256
+ (base32
+ "09j9ihigfsavgcmk8l36zmbjvdf1x1w7h2v4rkww1qk1sb43y5zf"))
+ (patches (search-patches "miniz-for-pytorch.patch"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments miniz)
+ ((#:configure-flags flags '())
+ ;; The changes break the examples.
+ `(cons "-DBUILD_EXAMPLES=OFF" ,flags))))
+ (properties '((hidden? . #t)))))
diff --git a/gnu/packages/patches/miniz-for-pytorch.patch b/gnu/packages/patches/miniz-for-pytorch.patch
new file mode 100644
index 0000000000..19fa705a31
--- /dev/null
+++ b/gnu/packages/patches/miniz-for-pytorch.patch
@@ -0,0 +1,47 @@
+Modifications from the internal fork of the python-pytorch package.
+
+diff --git a/miniz.h b/miniz.h
+index 42e1ea2..5869ab9 100644
+--- a/miniz.h
++++ b/miniz.h
+@@ -123,7 +123,7 @@
+ /* If MINIZ_NO_TIME is specified then the ZIP archive functions will not be able to get the current time, or */
+ /* get/set file times, and the C run-time funcs that get/set times won't be called. */
+ /* The current downside is the times written to your archives will be from 1979. */
+-/*#define MINIZ_NO_TIME */
++#define MINIZ_NO_TIME
+
+ /* Define MINIZ_NO_ARCHIVE_APIS to disable all ZIP archive API's. */
+ /*#define MINIZ_NO_ARCHIVE_APIS */
+@@ -135,7 +135,7 @@
+ /*#define MINIZ_NO_ZLIB_APIS */
+
+ /* Define MINIZ_NO_ZLIB_COMPATIBLE_NAME to disable zlib names, to prevent conflicts against stock zlib. */
+-/*#define MINIZ_NO_ZLIB_COMPATIBLE_NAMES */
++#define MINIZ_NO_ZLIB_COMPATIBLE_NAMES
+
+ /* Define MINIZ_NO_MALLOC to disable all calls to malloc, free, and realloc.
+ Note if MINIZ_NO_MALLOC is defined then the user must always provide custom user alloc/free/realloc
+@@ -172,8 +172,7 @@
+ #if !defined(MINIZ_USE_UNALIGNED_LOADS_AND_STORES)
+ #if MINIZ_X86_OR_X64_CPU
+ /* Set MINIZ_USE_UNALIGNED_LOADS_AND_STORES to 1 on CPU's that permit efficient integer loads and stores from unaligned addresses. */
+-#define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 1
+-#define MINIZ_UNALIGNED_USE_MEMCPY
++#define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 0
+ #else
+ #define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 0
+ #endif
+diff --git a/miniz_zip.h b/miniz_zip.h
+index 82502bd..3d92255 100644
+--- a/miniz_zip.h
++++ b/miniz_zip.h
+@@ -150,7 +150,7 @@ typedef enum {
+ MZ_ZIP_TOTAL_ERRORS
+ } mz_zip_error;
+
+-typedef struct
++typedef struct mz_zip_archive
+ {
+ mz_uint64 m_archive_size;
+ mz_uint64 m_central_directory_file_ofs;
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 10/31] gnu: Add libnop.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-10-david.elsing@posteo.net
* gnu/packages/serialization.scm (libnop): New variable.
---
gnu/packages/serialization.scm | 38 ++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 331fae80cf..93d12e714b 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -936,3 +936,41 @@ (define-public python-feather-format
(description "This package provides a Python wrapper library to the
Apache Arrow-based Feather binary columnar serialization data frame format.")
(license license:asl2.0)))
+
+(define-public libnop
+ (let ((commit "35e800d81f28c632956c5a592e3cbe8085ecd430")
+ (revision "0"))
+ (package
+ (name "libnop")
+ (version (git-version "0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/google/libnop")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0qqbaljq54qiq0dky9nj47igfcs065ry526jg9a0aafbfl9krpy2"))))
+ (build-system gnu-build-system)
+ (arguments
+ (list
+ #:test-target "test"
+ #:phases
+ #~(modify-phases %standard-phases
+ (delete 'configure)
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ (invoke "out/test"))))
+ (replace 'install
+ (lambda _
+ (copy-recursively
+ "include" (string-append #$output "/include")))))))
+ (native-inputs (list googletest))
+ (home-page "https://github.com/google/libnop")
+ (synopsis "C++ Native Object Protocols")
+ (description "@code{libnop} is a header-only library for serializing and
+deserializing C++ data types without external code generators or runtime
+support libraries.")
+ (license license:asl2.0))))
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 11/31] gnu: Remove flatbuffers-next-shared.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-11-david.elsing@posteo.net
* gnu/packages/serialization.scm (flatbuffers-next-shared): Delete variable.
* gnu/packages/machine-learning.scm (tensorflow-lite)[arguments]: Use
flatbuffers-next instead of flatbuffers-next-shared.
---
gnu/packages/machine-learning.scm | 4 ++--
gnu/packages/serialization.scm | 11 -----------
2 files changed, 2 insertions(+), 13 deletions(-)

Toggle diff (46 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 5c18a2e9d5..0a50049b6f 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -3337,7 +3337,7 @@ (define-public tensorflow-lite
(string-append "-DEigen3_DIR=" #$(this-package-input "eigen")
"/share/eigen3/cmake")
(string-append "-DFlatBuffers_DIR="
- #$(this-package-input "flatbuffers-shared")
+ #$(this-package-input "flatbuffers")
"/lib/cmake/flatbuffers")
(string-append "-DNEON_2_SSE_DIR=" #$(this-package-input "neon2sse")
"/lib/cmake/NEON_2_SSE")
@@ -3424,7 +3424,7 @@ (define-public tensorflow-lite
("cpuinfo" ,cpuinfo)
("eigen" ,eigen)
("fp16" ,fp16)
- ("flatbuffers-shared" ,flatbuffers-next-shared)
+ ("flatbuffers" ,flatbuffers-next)
("gemmlowp" ,gemmlowp)
("mesa-headers" ,mesa-headers)
("neon2sse" ,neon2sse)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 93d12e714b..8f1d3dab17 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -889,17 +889,6 @@ (define-public flatbuffers-next
(base32
"1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
-(define-public flatbuffers-next-shared
- (package
- (inherit flatbuffers-next)
- (name "flatbuffers-shared")
- (version "23.1.21")
- (arguments
- (substitute-keyword-arguments (package-arguments flatbuffers-next)
- ((#:configure-flags flags)
- ;; Compile with -fPIC, needed for shared lib.
- #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags))))))
-
(define-public python-flatbuffers
(package
(name "python-flatbuffers")
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 16/31] gnu: nnpack: Update to commit 70a77f4.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-16-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (nnpack): Update to commit 70a77f4.
[supported-systems]: New field.
---
gnu/packages/machine-learning.scm | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)

Toggle diff (35 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 0a50049b6f..846fb7a718 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -3974,8 +3974,8 @@ (define-public python-umap-learn
(define-public nnpack
(let ((version "0.0")
- (commit "c07e3a0400713d546e0dea2d5466dd22ea389c73")
- (revision "1"))
+ (commit "70a77f485e8b934224f3a79efd8edcd84cd377b8")
+ (revision "2"))
(package
(name "nnpack")
(version (git-version version revision commit))
@@ -3986,7 +3986,7 @@ (define-public nnpack
(file-name (git-file-name name version))
(sha256
(base32
- "0s0kk3a35w3yzf0q447p72350sbsh4qhg6vm3y2djbj4xpg7jc8v"))
+ "0c4pw926279s3rlx7mg4l4vhnfy6dh374n6w7zqhcn0bxpym1hv1"))
(patches (search-patches "nnpack-system-libraries.patch"))))
(build-system cmake-build-system)
;; XXX: The test suite runs but it's very expensive, and on x86_64 CPUs
@@ -4019,6 +4019,8 @@ (define-public nnpack
googletest))
(native-inputs
(list python python-peachpy python-six))
+ ;; Supported for Linux
+ (supported-systems '("x86_64-linux" "armhf-linux" "aarch64-linux"))
(license license:bsd-2))))
(define-public xnnpack
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 14/31] gnu: cpuinfo: Update to commit aa4b216.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-14-david.elsing@posteo.net
* gnu/packages/parallel.scm (cpuinfo): Update to commit aa4b216.
[arguments]: Set the "-DUSE_SYSTEM_LIBS=ON" configure flag instead of using a
patch.
* gnu/packages/patches/cpuinfo-system-libraries.patch: Remove file.
* gnu/local.mk (dist_patch_DATA): Remove it.
---
gnu/local.mk | 1 -
gnu/packages/parallel.scm | 11 ++--
.../patches/cpuinfo-system-libraries.patch | 50 -------------------
3 files changed, 6 insertions(+), 56 deletions(-)
delete mode 100644 gnu/packages/patches/cpuinfo-system-libraries.patch

Toggle diff (87 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 06b3238684..c290d91cbb 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1065,7 +1065,6 @@ dist_patch_DATA = \
%D%/packages/patches/cool-retro-term-wctype.patch \
%D%/packages/patches/coreutils-gnulib-tests.patch \
%D%/packages/patches/cppcheck-disable-char-signedness-test.patch \
- %D%/packages/patches/cpuinfo-system-libraries.patch \
%D%/packages/patches/cpulimit-with-glib-2.32.patch \
%D%/packages/patches/crawl-upgrade-saves.patch \
%D%/packages/patches/crc32c-unbundle-googletest.patch \
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index 4e8ca9af35..4dc0bee898 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -479,8 +479,8 @@ (define-public pthreadpool
(define-public cpuinfo
;; There's currently no tag on this repo.
(let ((version "0.0")
- (revision "2")
- (commit "53298db833c5c5a1598639e9b47cc1a602bbac26"))
+ (revision "3")
+ (commit "aa4b2163b99ac9534194520f70b93eeefb0b3b4e"))
(package
(name "cpuinfo")
(version (git-version version revision commit))
@@ -491,12 +491,13 @@ (define-public cpuinfo
(file-name (git-file-name name version))
(sha256
(base32
- "01kfgxya2w32dz9bd3qm3i2d6nffw0qfyql11rxl7d3g830brj5k"))
- (patches (search-patches "cpuinfo-system-libraries.patch"))))
+ "12x4krkyzxngf1l2ck33lnsp8pyzf6gyjj9mp9cnka9mw3h6617m"))))
(build-system cmake-build-system)
(arguments
(list
- #:configure-flags '(list "-DBUILD_SHARED_LIBS=ON")
+ #:configure-flags
+ '(list "-DBUILD_SHARED_LIBS=ON"
+ "-DUSE_SYSTEM_LIBS=ON")
#:phases
'(modify-phases %standard-phases
(add-after 'unpack 'skip-bad-test
diff --git a/gnu/packages/patches/cpuinfo-system-libraries.patch b/gnu/packages/patches/cpuinfo-system-libraries.patch
deleted file mode 100644
index e25446e9da..0000000000
--- a/gnu/packages/patches/cpuinfo-system-libraries.patch
+++ /dev/null
@@ -1,50 +0,0 @@
-This patch allows the build process to use the provided dependencies instead
-of adding their source as CMake sub-directories (in which case "make install"
-would install googletest's and googlebenchmark's libraries and headers).
-
-diff --git a/CMakeLists.txt b/CMakeLists.txt
-index 57abc26..761c612 100644
---- a/CMakeLists.txt
-+++ b/CMakeLists.txt
-@@ -93,7 +93,7 @@ SET(CONFU_DEPENDENCIES_BINARY_DIR ${CMAKE_BINARY_DIR}/deps
- CACHE PATH "Confu-style dependencies binary directory")
-
- IF(CPUINFO_BUILD_MOCK_TESTS OR CPUINFO_BUILD_UNIT_TESTS)
-- IF(CPUINFO_SUPPORTED_PLATFORM AND NOT DEFINED GOOGLETEST_SOURCE_DIR)
-+ IF(FALSE)
- MESSAGE(STATUS "Downloading Google Test to ${CONFU_DEPENDENCIES_SOURCE_DIR}/googletest (define GOOGLETEST_SOURCE_DIR to avoid it)")
- CONFIGURE_FILE(cmake/DownloadGoogleTest.cmake "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download/CMakeLists.txt")
- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
-@@ -105,7 +105,7 @@ IF(CPUINFO_BUILD_MOCK_TESTS OR CPUINFO_BUILD_UNIT_TESTS)
- ENDIF()
-
- IF(CPUINFO_BUILD_BENCHMARKS)
-- IF(CPUINFO_SUPPORTED_PLATFORM AND NOT DEFINED GOOGLEBENCHMARK_SOURCE_DIR)
-+ IF(FALSE)
- MESSAGE(STATUS "Downloading Google Benchmark to ${CONFU_DEPENDENCIES_SOURCE_DIR}/googlebenchmark (define GOOGLEBENCHMARK_SOURCE_DIR to avoid it)")
- CONFIGURE_FILE(cmake/DownloadGoogleBenchmark.cmake "${CONFU_DEPENDENCIES_BINARY_DIR}/googlebenchmark-download/CMakeLists.txt")
- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
-@@ -271,14 +271,6 @@ INSTALL(TARGETS cpuinfo
-
- # ---[ cpuinfo micro-benchmarks
- IF(CPUINFO_SUPPORTED_PLATFORM AND CPUINFO_BUILD_BENCHMARKS)
-- # ---[ Build google benchmark
-- IF(NOT TARGET benchmark)
-- SET(BENCHMARK_ENABLE_TESTING OFF CACHE BOOL "")
-- ADD_SUBDIRECTORY(
-- "${GOOGLEBENCHMARK_SOURCE_DIR}"
-- "${CONFU_DEPENDENCIES_BINARY_DIR}/googlebenchmark")
-- ENDIF()
--
- IF(CMAKE_SYSTEM_NAME MATCHES "^(Linux|Android)$")
- ADD_EXECUTABLE(get-current-bench bench/get-current.cc)
- TARGET_LINK_LIBRARIES(get-current-bench cpuinfo benchmark)
-@@ -289,7 +281,7 @@ IF(CPUINFO_SUPPORTED_PLATFORM AND CPUINFO_BUILD_BENCHMARKS)
- ENDIF()
-
- IF(CPUINFO_SUPPORTED_PLATFORM)
-- IF(CPUINFO_BUILD_MOCK_TESTS OR CPUINFO_BUILD_UNIT_TESTS)
-+ IF(FALSE)
- # ---[ Build google test
- IF(NOT TARGET gtest)
- IF(MSVC AND NOT CPUINFO_RUNTIME_TYPE STREQUAL "static")
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 13/31] gnu: pthreadpool: Update to commit 178e3e0.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-13-david.elsing@posteo.net
* gnu/packages/parallel.scm (pthreadpool): Update to commit 178e3e0.
---
gnu/packages/parallel.scm | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Toggle diff (27 lines)
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index ddf4ad016b..4e8ca9af35 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -449,9 +449,9 @@ (define-public python-slurm-magic
(define-public pthreadpool
;; This repository has only one tag, 0.1, which is older than what users
;; such as XNNPACK expect.
- (let ((commit "1787867f6183f056420e532eec640cba25efafea")
+ (let ((commit "178e3e0646cc671708bf78e77c273940130ac637")
(version "0.1")
- (revision "1"))
+ (revision "2"))
(package
(name "pthreadpool")
(version (git-version version revision commit))
@@ -462,7 +462,7 @@ (define-public pthreadpool
(file-name (git-file-name name version))
(sha256
(base32
- "02hdvxfn5krw8zivkgjx3b4rk9p02yr4mpdjlp75lsv6z1xf5yrx"))
+ "1s86lnq9bahacf5wxn7y14w70jh3g9lq1l7y16ijwhifd01nc2km"))
(patches (search-patches "pthreadpool-system-libraries.patch"))))
(build-system cmake-build-system)
(arguments '(#:configure-flags '("-DBUILD_SHARED_LIBS=ON")))
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 12/31] gnu: python-flatbuffers-next: Update to 23.5.26.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-12-david.elsing@posteo.net
* gnu/packages/serialization.scm (python-flatbuffers-next): Update to 23.5.26.
---
gnu/packages/serialization.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 8f1d3dab17..b5b9669762 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -878,7 +878,7 @@ (define-public flatbuffers
(define-public flatbuffers-next
(package
(inherit flatbuffers)
- (version "23.1.21")
+ (version "23.5.26")
(source (origin
(method git-fetch)
(uri (git-reference
@@ -887,7 +887,7 @@ (define-public flatbuffers-next
(file-name (git-file-name "flatbuffers" version))
(sha256
(base32
- "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
+ "0cd12dvkzqdafz46q4302mzgpzbz589zmmiga7bq07f2sqy4vrvv"))))))
(define-public python-flatbuffers
(package
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 22/31] gnu: Add ideep-pytorch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-22-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (ideep-pytorch): New variable.
---
gnu/packages/machine-learning.scm | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)

Toggle diff (46 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 5c21e4ed46..bfad33eadb 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -51,6 +51,7 @@ (define-module (gnu packages machine-learning)
#:use-module (guix download)
#:use-module (guix svn-download)
#:use-module (guix build-system cmake)
+ #:use-module (guix build-system copy)
#:use-module (guix build-system gnu)
#:use-module (guix build-system ocaml)
#:use-module (guix build-system pyproject)
@@ -4244,6 +4245,31 @@ (define-public foxi
contains facebook extensions and is used by PyTorch.")
(license license:expat))))
+(define-public ideep-pytorch
+ (package
+ (name "ideep-pytorch")
+ (version "3.3.2")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/intel/ideep")
+ (commit (string-append "pytorch-rls-v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0y6r938qryi3bnf15rp0fbilsfimdcgmvsa0ygwrn3zifw6386rb"))))
+ (build-system copy-build-system)
+ (arguments
+ (list
+ #:install-plan
+ ''(("include" "include"))))
+ (home-page "https://github.com/intel/ideep")
+ (synopsis "Ideep headers for interal use by PyTorch")
+ (description "This library is used internally as header-only library by
+PyTorch.")
+ (license license:expat)))
+
;; Please also update python-torchvision when updating this package.
(define-public python-pytorch
(package
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 15/31] gnu: clog: Add "-DUSE_SYSTEM_LIBS=ON" configure flag.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-15-david.elsing@posteo.net
* gnu/packages/parallel.scm (clog)[arguments]: Add "-DUSE_SYSTEM_LIBS=ON"
to #:configure-flags.
* gnu/packages/patches/clog-fix-shared-build.patch: Adjust patch.
---
gnu/packages/parallel.scm | 13 ++++---
.../patches/clog-fix-shared-build.patch | 39 ++-----------------
2 files changed, 12 insertions(+), 40 deletions(-)

Toggle diff (56 lines)
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index 4dc0bee898..7d53f3a769 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -523,11 +523,14 @@ (define-public clog
(inherit (package-source cpuinfo))
(patches (search-patches "clog-fix-shared-build.patch"))))
(arguments
- (list #:configure-flags #~(list "-DBUILD_SHARED_LIBS=ON")
- #:phases #~(modify-phases %standard-phases
- (add-after 'unpack 'chdir
- (lambda _
- (chdir "deps/clog"))))))
+ (list
+ #:configure-flags
+ ''("-DBUILD_SHARED_LIBS=ON"
+ "-DUSE_SYSTEM_LIBS=ON")
+ #:phases #~(modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _
+ (chdir "deps/clog"))))))
(native-inputs (list googletest))
(inputs '())
(synopsis "C-style logging library based on printf")
diff --git a/gnu/packages/patches/clog-fix-shared-build.patch b/gnu/packages/patches/clog-fix-shared-build.patch
index bf80544b90..eb305339f5 100644
--- a/gnu/packages/patches/clog-fix-shared-build.patch
+++ b/gnu/packages/patches/clog-fix-shared-build.patch
@@ -1,50 +1,19 @@
Author: Antero Mejr <antero@mailbox.org>
-Notes: Disabled function visibility hacks and googletest download. Enabled
-non-static builds.
+Notes: Disabled function visibility hacks. Enabled non-static builds.
diff --git a/deps/clog/CMakeLists.txt b/deps/clog/CMakeLists.txt
-index 083f519..b7b225a 100644
+index 6e50c41..db02e95 100644
--- a/deps/clog/CMakeLists.txt
+++ b/deps/clog/CMakeLists.txt
-@@ -38,20 +38,8 @@ SET(CONFU_DEPENDENCIES_SOURCE_DIR ${CMAKE_SOURCE_DIR}/deps
- SET(CONFU_DEPENDENCIES_BINARY_DIR ${CMAKE_BINARY_DIR}/deps
- CACHE PATH "Confu-style dependencies binary directory")
+@@ -55,7 +55,7 @@ IF(CLOG_BUILD_TESTS)
+ ENDIF()
--IF(CLOG_BUILD_TESTS)
-- IF(NOT DEFINED GOOGLETEST_SOURCE_DIR)
-- MESSAGE(STATUS "Downloading Google Test to ${CONFU_DEPENDENCIES_SOURCE_DIR}/googletest (define GOOGLETEST_SOURCE_DIR to avoid it)")
-- CONFIGURE_FILE(cmake/DownloadGoogleTest.cmake "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download/CMakeLists.txt")
-- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
-- WORKING_DIRECTORY "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download")
-- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" --build .
-- WORKING_DIRECTORY "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download")
-- SET(GOOGLETEST_SOURCE_DIR "${CONFU_DEPENDENCIES_SOURCE_DIR}/googletest" CACHE STRING "Google Test source directory")
-- ENDIF()
--ENDIF()
--
# ---[ clog library
-ADD_LIBRARY(clog STATIC src/clog.c)
+ADD_LIBRARY(clog src/clog.c)
SET_TARGET_PROPERTIES(clog PROPERTIES
C_STANDARD 99
C_EXTENSIONS NO)
-@@ -74,16 +62,6 @@ INSTALL(TARGETS clog
-
- # ---[ clog tests
- IF(CLOG_BUILD_TESTS)
-- # ---[ Build google test
-- IF(NOT TARGET gtest)
-- IF(MSVC AND NOT CLOG_RUNTIME_TYPE STREQUAL "static")
-- SET(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
-- ENDIF()
-- ADD_SUBDIRECTORY(
-- "${GOOGLETEST_SOURCE_DIR}"
-- "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest")
-- ENDIF()
--
- ADD_EXECUTABLE(clog-test test/clog.cc)
- SET_TARGET_PROPERTIES(clog-test PROPERTIES
- CXX_STANDARD 11
diff --git a/deps/clog/include/clog.h b/deps/clog/include/clog.h
index 4143761..aa9000f 100644
--- a/deps/clog/include/clog.h
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 17/31] gnu: oneapi-dnnl: Update to 3.3.5.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-17-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (oneapi-dnnl): Update to 3.3.5.
[supported-systems]: New field.
---
gnu/packages/machine-learning.scm | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 846fb7a718..99d7399c12 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -5375,7 +5375,7 @@ (define-public python-brian2tools
(define-public oneapi-dnnl
(package
(name "oneapi-dnnl")
- (version "3.1")
+ (version "3.3.5")
(source
(origin
(method git-fetch)
@@ -5384,7 +5384,7 @@ (define-public oneapi-dnnl
(commit (string-append "v" version))))
(file-name (git-file-name name version))
(sha256
- (base32 "1jgmb5kl0bf4a2zfn94zlb117672r9lvvkkmwl86ihlyr1mpr3d0"))))
+ (base32 "05ra5zziys2hvn29y6ysiqzsf4jr9bf2bci5sc3swvf3bs2y5ihf"))))
(build-system cmake-build-system)
(arguments (if (target-riscv64?)
(list #:configure-flags #~'("-DDNNL_CPU_RUNTIME=SEQ"))
@@ -5394,6 +5394,7 @@ (define-public oneapi-dnnl
(description
"OneAPI Deep Neural Network Library (oneDNN) is a cross-platform
performance library of basic building blocks for deep learning applications.")
+ (supported-systems %64bit-supported-systems)
(license license:asl2.0)))
(define-public python-gguf
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 23/31] gnu: xnnpack: Update to commit 51a9875.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-23-david.elsing@posteo.net
The tests which fail to build if building the shared library are removed by a
patch. Autogenerated files are removed by a snippet and generated in a
separate build phase.

* gnu/packages/machine-learning.scm (xnnpack): Update to commit 51a9875.
[source]: Add patch for tests. Add snippet to remove autogenerated files.
[arguments]: New modules and phase arguments.
[inputs]: Add clog and cpuinfo.
[native-inputs]: New field.
* gnu/packages/patches/xnnpack-remove-broken-tests.patch: Add file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/patches/xnnpack-system-libraries.patch: Update patch.
---
gnu/local.mk | 1 +
gnu/packages/machine-learning.scm | 106 +-
.../patches/xnnpack-remove-broken-tests.patch | 335 ++
.../patches/xnnpack-system-libraries.patch | 4133 ++++++++++-------
4 files changed, 2922 insertions(+), 1653 deletions(-)
create mode 100644 gnu/packages/patches/xnnpack-remove-broken-tests.patch

Toggle diff (351 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index c88c21ac60..7b87bf93e5 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -2198,6 +2198,7 @@ dist_patch_DATA = \
%D%/packages/patches/xfce4-settings-defaults.patch \
%D%/packages/patches/xgboost-use-system-dmlc-core.patch \
%D%/packages/patches/xmonad-dynamic-linking.patch \
+ %D%/packages/patches/xnnpack-remove-broken-tests.patch \
%D%/packages/patches/xnnpack-system-libraries.patch \
%D%/packages/patches/xnnpack-for-torch2-system-libraries.patch \
%D%/packages/patches/xplanet-1.3.1-cxx11-eof.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index bfad33eadb..35d96ac012 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4077,41 +4077,109 @@ (define-public qnnpack
(define-public xnnpack
;; There's currently no tag on this repo.
(let ((version "0.0")
- (commit "ae108ef49aa5623b896fc93d4298c49d1750d9ba")
- (revision "2"))
+ (commit "51a987591a6fc9f0fc0707077f53d763ac132cbf")
+ (revision "3"))
(package
(name "xnnpack")
(version (git-version version revision commit))
(home-page "https://github.com/google/XNNPACK") ;fork of QNNPACK
- (source (origin
- (method git-fetch)
- (uri (git-reference (url home-page) (commit commit)))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "0q68q2jxiiiblx45q4337k13ppgh5vqjwrwznchcnpb8hawjj3zl"))
- (patches (search-patches "xnnpack-system-libraries.patch"))))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference (url home-page) (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1rzby82xq8d0rl1d148yz88jh9cpsw5c8b2yw7yg39mi7qmr55rm"))
+ ;; Some tests fail to link as they use internal symbols, which are
+ ;; not included in the shared library.
+ ;; XXX: Additionally, these tests fail on i686 due to incorrect results:
+ ;; 171 - f32-vlrelu-test (Failed)
+ ;; 211 - qs8-gavgpool-minmax-fp32-test (Failed)
+ ;; 224 - qu8-avgpool-minmax-fp32-test (Failed)
+ ;; 228 - qu8-gavgpool-minmax-fp32-test (Failed)
+ ;; 263 - x32-packx-test (Failed)
+ (patches (search-patches "xnnpack-remove-broken-tests.patch"))
+ (modules '((guix build utils)
+ (ice-9 ftw)
+ (ice-9 textual-ports)
+ (srfi srfi-26)))
+ (snippet
+ '(begin
+ ;; Remove autogenerated files
+ (for-each
+ (lambda (dir)
+ (let ((gendir (string-append "src/" dir "/gen")))
+ (when (file-exists? gendir)
+ (delete-file-recursively gendir)
+ ;; Needed for the scripts generating the files
+ (mkdir gendir))))
+ (scandir "src" (negate (cut member <> '("." "..")))))
+ (delete-file-recursively "google3")
+ (delete-file "cmake/microkernels.cmake")
+ ;; Additional autogenerated files which contain the string "Auto-generated file"
+ (for-each
+ (lambda (dir)
+ (for-each
+ (lambda (name)
+ (let ((path (string-append dir "/" name)))
+ (when (call-with-input-file path
+ (lambda (port)
+ (string-contains
+ (get-string-all port)
+ "Auto-generated file")))
+ (delete-file path))))
+ (scandir dir (negate (cut member <> '("." ".."))))))
+ '("test" "bench" "eval" "models" "src/enums" "src/xnnpack"))))))
(build-system cmake-build-system)
(arguments
- '(#:configure-flags '("-DXNNPACK_USE_SYSTEM_LIBS=YES"
+ (list
+ #:build-type "Release" ;; Debugging symbols require a lot of disk space
+ #:configure-flags ''("-DXNNPACK_USE_SYSTEM_LIBS=YES"
"-DBUILD_SHARED_LIBS=ON"
+ "-DCMAKE_POSITION_INDEPENDENT_CODE=ON"
"-DXNNPACK_LIBRARY_TYPE=shared"
- "-DXNNPACK_BUILD_TESTS=FALSE" ;FIXME: see below
"-DXNNPACK_BUILD_BENCHMARKS=FALSE")
-
- ;; FIXME: Building tests leads to a CMake error:
- ;;
- ;; ADD_LIBRARY cannot create target "all_microkernels" because
- ;; another target with the same name already exists.
- #:tests? #f))
+ #:modules '((ice-9 ftw)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'fix-cmake
+ (lambda _
+ (substitute* "CMakeLists.txt"
+ (("TARGET_INCLUDE_DIRECTORIES\\((pthreadpool|cpuinfo).*") "")
+ ((".*IF\\(NOT TARGET gtest\\).*")
+ "IF(FALSE)\n")
+ (("SET\\(CMAKE_CXX_STANDARD 11\\)")
+ "SET(CMAKE_CXX_STANDARD 14)")
+ (("AMD64") "x86_64"))))
+ (add-after 'patch-source-shebangs 'generate-files
+ (lambda _
+ (for-each
+ (lambda (name)
+ (when (and (string-prefix? "generate" name)
+ (string-suffix? ".sh" name)
+ (not (equal? "generate-amalgamation.sh" name)))
+ (display (string-append name "\n"))
+ (invoke "bash" (string-append "scripts/" name))))
+ (scandir "scripts"))
+ ;; These need to run after the above scripts
+ (display "Remaining files\n")
+ (invoke "python3" "tools/update-microkernels.py")
+ (substitute* "tools/amalgamate-microkernels.py"
+ (("BUILD") "BUILD.bazel"))
+ (invoke "bash" "scripts/generate-amalgamation.sh"))))))
(inputs
- (list cpuinfo
+ (list clog
+ cpuinfo
pthreadpool
googletest
googlebenchmark
fxdiv
fp16
psimd))
+ (native-inputs (list python-pyyaml python-wrapper))
(synopsis "Optimized floating-point neural network inference operators")
(description
"XNNPACK is a highly optimized library of floating-point neural network
diff --git a/gnu/packages/patches/xnnpack-remove-broken-tests.patch b/gnu/packages/patches/xnnpack-remove-broken-tests.patch
new file mode 100644
index 0000000000..529dd2590c
--- /dev/null
+++ b/gnu/packages/patches/xnnpack-remove-broken-tests.patch
@@ -0,0 +1,335 @@
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 95df99eae..ab57d5efa 100755
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -978,16 +978,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(add-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
+ ADD_TEST(NAME add-nd-eager-test COMMAND add-nd-eager-test)
+
+- ADD_EXECUTABLE(argmax-pooling-nhwc-test test/argmax-pooling-nhwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(argmax-pooling-nhwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(argmax-pooling-nhwc-test PRIVATE XNNPACK gtest gtest_main)
+- ADD_TEST(NAME argmax-pooling-nhwc-test COMMAND argmax-pooling-nhwc-test)
+-
+- ADD_EXECUTABLE(average-pooling-nhwc-test test/average-pooling-nhwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(average-pooling-nhwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(average-pooling-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME average-pooling-nhwc-test COMMAND average-pooling-nhwc-test)
+-
+ ADD_EXECUTABLE(bankers-rounding-nc-test test/bankers-rounding-nc.cc)
+ TARGET_INCLUDE_DIRECTORIES(bankers-rounding-nc-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(bankers-rounding-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+@@ -1066,11 +1056,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(copy-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
+ ADD_TEST(NAME copy-nc-eager-test COMMAND copy-nc-eager-test)
+
+- ADD_EXECUTABLE(deconvolution-nhwc-test test/deconvolution-nhwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(deconvolution-nhwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(deconvolution-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME deconvolution-nhwc-test COMMAND deconvolution-nhwc-test)
+-
+ ADD_EXECUTABLE(depth-to-space-nchw2nhwc-test test/depth-to-space-nchw2nhwc.cc)
+ TARGET_INCLUDE_DIRECTORIES(depth-to-space-nchw2nhwc-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(depth-to-space-nchw2nhwc-test PRIVATE XNNPACK gtest gtest_main)
+@@ -1116,11 +1101,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(floor-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
+ ADD_TEST(NAME floor-nc-eager-test COMMAND floor-nc-eager-test)
+
+- ADD_EXECUTABLE(global-average-pooling-nwc-test test/global-average-pooling-nwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(global-average-pooling-nwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(global-average-pooling-nwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME global-average-pooling-nwc-test COMMAND global-average-pooling-nwc-test)
+-
+ ADD_EXECUTABLE(global-average-pooling-ncw-test test/global-average-pooling-ncw.cc)
+ TARGET_INCLUDE_DIRECTORIES(global-average-pooling-ncw-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(global-average-pooling-ncw-test PRIVATE XNNPACK fp16 gtest gtest_main)
+@@ -1146,11 +1126,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(leaky-relu-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
+ ADD_TEST(NAME leaky-relu-nc-eager-test COMMAND leaky-relu-nc-eager-test)
+
+- ADD_EXECUTABLE(max-pooling-nhwc-test test/max-pooling-nhwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(max-pooling-nhwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(max-pooling-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME max-pooling-nhwc-test COMMAND max-pooling-nhwc-test)
+-
+ ADD_EXECUTABLE(maximum-nd-test test/maximum-nd.cc)
+ TARGET_INCLUDE_DIRECTORIES(maximum-nd-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(maximum-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
+@@ -1191,11 +1166,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(negate-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
+ ADD_TEST(NAME negate-nc-eager-test COMMAND negate-nc-eager-test)
+
+- ADD_EXECUTABLE(prelu-nc-test test/prelu-nc.cc)
+- TARGET_INCLUDE_DIRECTORIES(prelu-nc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(prelu-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME prelu-nc-test COMMAND prelu-nc-test)
+-
+ ADD_EXECUTABLE(resize-bilinear-nhwc-test test/resize-bilinear-nhwc.cc)
+ TARGET_INCLUDE_DIRECTORIES(resize-bilinear-nhwc-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(resize-bilinear-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+@@ -1312,16 +1282,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(fusion-test PRIVATE XNNPACK gtest gtest_main)
+ ADD_TEST(NAME fusion-test COMMAND fusion-test)
+
+- ADD_EXECUTABLE(memory-planner-test test/memory-planner-test.cc)
+- TARGET_INCLUDE_DIRECTORIES(memory-planner-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(memory-planner-test PRIVATE XNNPACK gtest gtest_main)
+- ADD_TEST(NAME memory-planner-test COMMAND memory-planner-test)
+-
+- ADD_EXECUTABLE(subgraph-fp16-test test/subgraph-fp16.cc)
+- TARGET_INCLUDE_DIRECTORIES(subgraph-fp16-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(subgraph-fp16-test PRIVATE XNNPACK gtest gtest_main)
+- ADD_TEST(NAME subgraph-fp16-test COMMAND subgraph-fp16-test)
+-
+ ADD_EXECUTABLE(subgraph-nchw-test test/subgraph-nchw.cc)
+ TARGET_INCLUDE_DIRECTORIES(subgraph-nchw-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(subgraph-nchw-test PRIVATE XNNPACK gtest gtest_main)
+@@ -1347,226 +1307,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_INCLUDE_DIRECTORIES(argmax-pooling-2d-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(argmax-pooling-2d-test PRIVATE XNNPACK gtest gtest_main)
+ ADD_TEST(NAME argmax-pooling-2d-test COMMAND argmax-pooling-2d-test)
+-
+- ADD_EXECUTABLE(average-pooling-2d-test test/average-pooling-2d.cc)
+- TARGET_INCLUDE_DIRECTORIES(average-pooling-2d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(average-pooling-2d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME average-pooling-2d-test COMMAND average-pooling-2d-test)
+-
+- ADD_EXECUTABLE(bankers-rounding-test test/bankers-rounding.cc)
+- TARGET_INCLUDE_DIRECTORIES(bankers-rounding-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(bankers-rounding-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME bankers-rounding-test COMMAND bankers-rounding-test)
+-
+- ADD_EXECUTABLE(ceiling-test test/ceiling.cc)
+- TARGET_INCLUDE_DIRECTORIES(ceiling-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(ceiling-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME ceiling-test COMMAND ceiling-test)
+-
+- ADD_EXECUTABLE(clamp-test test/clamp.cc)
+- TARGET_INCLUDE_DIRECTORIES(clamp-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(clamp-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME clamp-test COMMAND clamp-test)
+-
+- ADD_EXECUTABLE(concatenate2-test test/concatenate2.cc)
+- TARGET_INCLUDE_DIRECTORIES(concatenate2-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(concatenate2-test PRIVATE XNNPACK gtest gtest_main subgraph)
+- ADD_TEST(NAME concatenate2-test COMMAND concatenate2-test)
+-
+- ADD_EXECUTABLE(concatenate3-test test/concatenate3.cc)
+- TARGET_INCLUDE_DIRECTORIES(concatenate3-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(concatenate3-test PRIVATE XNNPACK gtest gtest_main subgraph)
+- ADD_TEST(NAME concatenate3-test COMMAND concatenate3-test)
+-
+- ADD_EXECUTABLE(concatenate4-test test/concatenate4.cc)
+- TARGET_INCLUDE_DIRECTORIES(concatenate4-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(concatenate4-test PRIVATE XNNPACK gtest gtest_main subgraph)
+- ADD_TEST(NAME concatenate4-test COMMAND concatenate4-test)
+-
+- ADD_EXECUTABLE(convert-test test/convert.cc)
+- TARGET_INCLUDE_DIRECTORIES(convert-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(convert-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME convert-test COMMAND convert-test)
+-
+- ADD_EXECUTABLE(convolution-2d-test test/convolution-2d.cc)
+- SET_TARGET_PROPERTIES(convolution-2d-test PROPERTIES CXX_EXTENSIONS YES)
+- TARGET_INCLUDE_DIRECTORIES(convolution-2d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(convolution-2d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph convolution-test-helpers)
+- ADD_TEST(NAME convolution-2d-test COMMAND convolution-2d-test)
+-
+- ADD_EXECUTABLE(copy-test test/copy.cc)
+- TARGET_INCLUDE_DIRECTORIES(copy-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(copy-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME copy-test COMMAND copy-test)
+-
+- ADD_EXECUTABLE(deconvolution-2d-test test/deconvolution-2d.cc)
+- SET_TARGET_PROPERTIES(deconvolution-2d-test PROPERTIES CXX_EXTENSIONS YES)
+- TARGET_INCLUDE_DIRECTORIES(deconvolution-2d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(deconvolution-2d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph convolution-test-helpers)
+- ADD_TEST(NAME deconvolution-2d-test COMMAND deconvolution-2d-test)
+-
+- ADD_EXECUTABLE(depth-to-space-test test/depth-to-space.cc)
+- TARGET_INCLUDE_DIRECTORIES(depth-to-space-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(depth-to-space-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME depth-to-space-test COMMAND depth-to-space-test)
+-
+- ADD_EXECUTABLE(depthwise-convolution-2d-test test/depthwise-convolution-2d.cc)
+- SET_TARGET_PROPERTIES(depthwise-convolution-2d-test PROPERTIES CXX_EXTENSIONS YES)
+- TARGET_INCLUDE_DIRECTORIES(depthwise-convolution-2d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(depthwise-convolution-2d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph convolution-test-helpers)
+- ADD_TEST(NAME depthwise-convolution-2d-test COMMAND depthwise-convolution-2d-test)
+-
+- ADD_EXECUTABLE(divide2-test test/divide2.cc)
+- TARGET_INCLUDE_DIRECTORIES(divide2-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(divide2-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME divide2-test COMMAND divide2-test)
+-
+- ADD_EXECUTABLE(elu-test test/elu.cc)
+- TARGET_INCLUDE_DIRECTORIES(elu-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(elu-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME elu-test COMMAND elu-test)
+-
+- ADD_EXECUTABLE(even-split2-test test/even-split2.cc)
+- TARGET_INCLUDE_DIRECTORIES(even-split2-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(even-split2-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME even-split2-test COMMAND even-split2-test)
+-
+- ADD_EXECUTABLE(even-split3-test test/even-split3.cc)
+- TARGET_INCLUDE_DIRECTORIES(even-split3-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(even-split3-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME even-split3-test COMMAND even-split3-test)
+-
+- ADD_EXECUTABLE(even-split4-test test/even-split4.cc)
+- TARGET_INCLUDE_DIRECTORIES(even-split4-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(even-split4-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME even-split4-test COMMAND even-split4-test)
+-
+- ADD_EXECUTABLE(floor-test test/floor.cc)
+- TARGET_INCLUDE_DIRECTORIES(floor-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(floor-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME floor-test COMMAND floor-test)
+-
+- ADD_EXECUTABLE(fully-connected-test test/fully-connected.cc)
+- TARGET_INCLUDE_DIRECTORIES(fully-connected-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(fully-connected-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME fully-connected-test COMMAND fully-connected-test)
+-
+- ADD_EXECUTABLE(global-average-pooling-1d-test test/global-average-pooling-1d.cc)
+- TARGET_INCLUDE_DIRECTORIES(global-average-pooling-1d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(global-average-pooling-1d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME global-average-pooling-1d-test COMMAND global-average-pooling-1d-test)
+-
+- ADD_EXECUTABLE(global
This message was truncated. Download the full message here.
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 18/31] gnu: Add tensorpipe.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-18-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (tensorpipe): New variable.
---
gnu/packages/machine-learning.scm | 36 +++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 99d7399c12..a58ccfbd5c 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -90,6 +90,7 @@ (define-module (gnu packages machine-learning)
#:use-module (gnu packages image-processing)
#:use-module (gnu packages imagemagick)
#:use-module (gnu packages jupyter)
+ #:use-module (gnu packages libevent)
#:use-module (gnu packages libffi)
#:use-module (gnu packages linux)
#:use-module (gnu packages llvm)
@@ -4099,6 +4100,41 @@ (define-public xnnpack-for-torch2
"-DXNNPACK_BUILD_TESTS=FALSE" ;FIXME: see below
"-DXNNPACK_BUILD_BENCHMARKS=FALSE"))))))
+(define-public tensorpipe
+ (let ((commit "bb1473a4b38b18268e8693044afdb8635bc8351b")
+ (revision "0"))
+ (package
+ (name "tensorpipe")
+ (version (git-version "0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/pytorch/tensorpipe")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0sbpkd69rzybw2j89sjkf4s0j8vkk96d51bsps28894989a75j6v"))
+ (modules '((guix build utils)))
+ (snippet
+ '(delete-file-recursively "third_party"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ #:configure-flags
+ ''("-DBUILD_SHARED_LIBS=ON")
+ ;; There are no tests
+ #:tests? #f))
+ (inputs (list libuv))
+ (native-inputs (list googletest pkg-config pybind11 libnop))
+ (home-page "https://github.com/pytorch/tensorpipe")
+ (synopsis "Tensor-aware point-to-point communication primitive for
+machine learning")
+ (description "TensorPipe provides a tensor-aware channel to transfer
+rich objects from one process to another while using the fastest transport for
+the tensors contained therein.")
+ (license license:bsd-3))))
+
;; Please also update python-torchvision when updating this package.
(define-public python-pytorch
(package
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 19/31] gnu: Add fbgemm.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-19-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (fbgemm): New variable.
* gnu/packages/patches/fbgemm-use-system-libraries.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
gnu/packages/machine-learning.scm | 31 ++++++++++
.../patches/fbgemm-use-system-libraries.patch | 62 +++++++++++++++++++
3 files changed, 94 insertions(+)
create mode 100644 gnu/packages/patches/fbgemm-use-system-libraries.patch

Toggle diff (124 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index c290d91cbb..f81ea13031 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1172,6 +1172,7 @@ dist_patch_DATA = \
%D%/packages/patches/fasthenry-spUtils.patch \
%D%/packages/patches/fasthenry-spSolve.patch \
%D%/packages/patches/fasthenry-spFactor.patch \
+ %D%/packages/patches/fbgemm-use-system-libraries.patch \
%D%/packages/patches/fbreader-curl-7.62.patch \
%D%/packages/patches/fbreader-fix-icon.patch \
%D%/packages/patches/fenics-dolfin-algorithm.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index a58ccfbd5c..5825955d3e 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4100,6 +4100,37 @@ (define-public xnnpack-for-torch2
"-DXNNPACK_BUILD_TESTS=FALSE" ;FIXME: see below
"-DXNNPACK_BUILD_BENCHMARKS=FALSE"))))))
+(define-public fbgemm
+ (package
+ (name "fbgemm")
+ (version "0.6.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/pytorch/fbgemm")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0mw30v55aicqdbh3xwfj9p8f38nw70ks5cxiwpgwjsk0dylah9rf"))
+ (patches (search-patches "fbgemm-use-system-libraries.patch"))
+ (modules '((guix build utils)))
+ (snippet
+ '(delete-file-recursively "third_party"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ #:configure-flags
+ ''("-DFBGEMM_LIBRARY_TYPE=shared")))
+ (inputs (list asmjit cpuinfo))
+ (native-inputs (list pkg-config python googletest))
+ (home-page "https://github.com/pytorch/fbgemm")
+ (synopsis "Facebook GEneral Matrix Multiplication")
+ (description "Low-precision, high-performance matrix-matrix
+multiplications and convolution library for server-side inference.")
+ (supported-systems '("x86_64-linux"))
+ (license license:bsd-3)))
+
(define-public tensorpipe
(let ((commit "bb1473a4b38b18268e8693044afdb8635bc8351b")
(revision "0"))
diff --git a/gnu/packages/patches/fbgemm-use-system-libraries.patch b/gnu/packages/patches/fbgemm-use-system-libraries.patch
new file mode 100644
index 0000000000..456bc3021c
--- /dev/null
+++ b/gnu/packages/patches/fbgemm-use-system-libraries.patch
@@ -0,0 +1,62 @@
+Use the asmjit and cpuinfo packages.
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 134523e..b88b0e7 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -235,7 +235,7 @@ message(WARNING "CMAKE_CXX_FLAGS_DEBUG is ${CMAKE_CXX_FLAGS_DEBUG}")
+ message(WARNING "CMAKE_CXX_FLAGS_RELEASE is ${CMAKE_CXX_FLAGS_RELEASE}")
+ message(WARNING "==========")
+
+-if(NOT TARGET asmjit)
++if(FALSE)
+ #Download asmjit from github if ASMJIT_SRC_DIR is not specified.
+ if(NOT DEFINED ASMJIT_SRC_DIR)
+ set(ASMJIT_SRC_DIR "${FBGEMM_SOURCE_DIR}/third_party/asmjit"
+@@ -264,7 +264,7 @@ if(NOT TARGET asmjit)
+ endif()
+ endif()
+
+-if(NOT TARGET cpuinfo)
++if(FALSE)
+ #Download cpuinfo from github if CPUINFO_SOURCE_DIR is not specified.
+ if(NOT DEFINED CPUINFO_SOURCE_DIR)
+ set(CPUINFO_SOURCE_DIR "${FBGEMM_SOURCE_DIR}/third_party/cpuinfo"
+@@ -340,9 +340,12 @@ target_include_directories(fbgemm BEFORE
+ target_link_libraries(fbgemm
+ $<BUILD_INTERFACE:asmjit>
+ $<BUILD_INTERFACE:cpuinfo>)
+-add_dependencies(fbgemm
+- asmjit
+- cpuinfo)
++
++find_package(asmjit)
++find_package(PkgConfig REQUIRED)
++pkg_check_modules(libcpuinfo REQUIRED IMPORTED_TARGET libcpuinfo)
++
++target_link_libraries(fbgemm asmjit::asmjit PkgConfig::libcpuinfo)
+
+ if(OpenMP_FOUND)
+ target_link_libraries(fbgemm OpenMP::OpenMP_CXX)
+diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt
+index 2cffddf..bd4d409 100644
+--- a/test/CMakeLists.txt
++++ b/test/CMakeLists.txt
+@@ -14,7 +14,7 @@ set(CMAKE_C_STANDARD 11)
+ set(CMAKE_C_EXTENSIONS OFF)
+ set(CMAKE_C_STANDARD_REQUIRED ON)
+
+-if(FBGEMM_BUILD_TESTS AND NOT TARGET gtest)
++if(FALSE)
+ #Download Googletest framework from github if
+ #GOOGLETEST_SOURCE_DIR is not specified.
+ if(NOT DEFINED GOOGLETEST_SOURCE_DIR)
+@@ -78,7 +78,7 @@ macro(add_gtest TESTNAME)
+ target_link_libraries(${TESTNAME} ${OpenMP_CXX_LIBRARIES})
+ endif()
+
+- add_dependencies(${TESTNAME} gtest fbgemm)
++ add_dependencies(${TESTNAME} fbgemm)
+ add_test(${TESTNAME} ${TESTNAME})
+ set_target_properties(${TESTNAME} PROPERTIES FOLDER test)
+ endmacro()
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 20/31] gnu: Add qnnpack.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-20-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (qnnpack): New variable.
---
gnu/packages/machine-learning.scm | 49 +++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)

Toggle diff (62 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 5825955d3e..c883af54d9 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4024,6 +4024,55 @@ (define-public nnpack
(supported-systems '("x86_64-linux" "armhf-linux" "aarch64-linux"))
(license license:bsd-2))))
+(define-public qnnpack
+ (let ((commit "7d2a4e9931a82adc3814275b6219a03e24e36b4c")
+ (revision "0"))
+ (package
+ (name "qnnpack")
+ (version (git-version "0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/pytorch/qnnpack")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1dgzriiaz9arsrfwhx42y4l74wbzn6xvdmllfb66v4pmvi5gpxc5"))
+ (modules '((guix build utils)))
+ (snippet
+ '(delete-file-recursively "deps"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ ;; The shared library build fails with linker errors, so we build the
+ ;; static library with -fPIC as in the bundled PyTorch version.
+ #:configure-flags
+ ''("-DQNNPACK_LIBRARY_TYPE=static"
+ "-DCMAKE_POSITION_INDEPENDENT_CODE=ON")
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'patch-cmake
+ (lambda _
+ (substitute* "CMakeLists.txt"
+ (("IF.*SOURCE_DIR.*")
+ "IF(FALSE)\n")
+ (("IF\\(NOT TARGET.*")
+ "IF(FALSE)\n")
+ (("TARGET_LINK_LIBRARIES.*(fxdiv|psimd|fp16)\\).*")
+ "")
+ (("(TARGET_LINK_LIBRARIES.*) fp16 (.*)" _ before after)
+ (string-append before " " after))))))))
+ (inputs (list clog cpuinfo fp16 fxdiv psimd pthreadpool))
+ (native-inputs (list googletest googlebenchmark))
+ (home-page "https://github.com/pytorch/qnnpack")
+ (synopsis "Quantized Neural Network PACKage")
+ (description "QNNPACK is a library for low-precision neural network
+inference. It contains the implementation of common neural network operators
+on quantized 8-bit tensors.")
+ (license license:bsd-3))))
+
(define-public xnnpack
;; There's currently no tag on this repo.
(let ((version "0.0")
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 21/31] gnu: Add foxi.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-21-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (foxi): New variable.
* gnu/packages/patches/foxi-fix-build.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
gnu/packages/machine-learning.scm | 29 ++++++++++++
gnu/packages/patches/foxi-fix-build.patch | 55 +++++++++++++++++++++++
3 files changed, 85 insertions(+)
create mode 100644 gnu/packages/patches/foxi-fix-build.patch

Toggle diff (115 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index f81ea13031..c88c21ac60 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1204,6 +1204,7 @@ dist_patch_DATA = \
%D%/packages/patches/foobillard++-pkg-config.patch \
%D%/packages/patches/foomatic-filters-CVE-2015-8327.patch \
%D%/packages/patches/foomatic-filters-CVE-2015-8560.patch \
+ %D%/packages/patches/foxi-fix-build.patch \
%D%/packages/patches/fp16-implicit-double.patch \
%D%/packages/patches/fp16-system-libraries.patch \
%D%/packages/patches/fpc-reproducibility.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index c883af54d9..5c21e4ed46 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4215,6 +4215,35 @@ (define-public tensorpipe
the tensors contained therein.")
(license license:bsd-3))))
+(define-public foxi
+ (let
+ ((commit "c278588e34e535f0bb8f00df3880d26928038cad")
+ (revision "0"))
+ (package
+ (name "foxi")
+ (version (git-version "1.4.1" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/houseroad/foxi")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0q3ssm5hmmvwfwx87mnnajbavzgpahybw6rpn8ysr9r095dwgq5a"))
+ (patches (search-patches "foxi-fix-build.patch"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ ;; No tests
+ #:tests? #f))
+ (home-page "https://github.com/houseroad/foxi")
+ (synopsis "ONNXIFI with Facebook Extension")
+ (description "ONNX Interface for Framework Integration is a cross-platform
+API for loading and executing ONNX graphs on optimized backends. This package
+contains facebook extensions and is used by PyTorch.")
+ (license license:expat))))
+
;; Please also update python-torchvision when updating this package.
(define-public python-pytorch
(package
diff --git a/gnu/packages/patches/foxi-fix-build.patch b/gnu/packages/patches/foxi-fix-build.patch
new file mode 100644
index 0000000000..d82090ad38
--- /dev/null
+++ b/gnu/packages/patches/foxi-fix-build.patch
@@ -0,0 +1,55 @@
+Taken from https://github.com/houseroad/foxi/pull/25.
+
+diff --git a/foxi/onnxifi_dummy.c b/foxi/onnxifi_dummy.c
+index 2115af9..73e25fc 100644
+--- a/foxi/onnxifi_dummy.c
++++ b/foxi/onnxifi_dummy.c
+@@ -103,7 +103,10 @@ ONNXIFI_PUBLIC ONNXIFI_CHECK_RESULT onnxStatus ONNXIFI_ABI onnxInitGraph(
+ const void* onnxModel,
+ uint32_t weightCount,
+ const onnxTensorDescriptorV1* weightDescriptors,
+- onnxGraph* graph) {
++ onnxGraph* graph,
++ uint32_t maxSeqLength,
++ void* deferredWeightReader) {
++
+ if (graph == NULL) {
+ return ONNXIFI_STATUS_INVALID_POINTER;
+ }
+@@ -215,6 +218,8 @@ ONNXIFI_PUBLIC ONNXIFI_CHECK_RESULT onnxStatus ONNXIFI_ABI
+ onnxWaitEventFor(onnxEvent event,
+ uint32_t timeoutMs,
+ onnxEventState* eventState,
+- onnxStatus* eventStatus) {
++ onnxStatus* eventStatus,
++ char* message,
++ size_t* messageLength) {
+ return ONNXIFI_STATUS_SUCCESS;
+ }
+\ No newline at end of file
+diff --git a/foxi/onnxifi_wrapper.c b/foxi/onnxifi_wrapper.c
+index 98a9325..abe1440 100644
+--- a/foxi/onnxifi_wrapper.c
++++ b/foxi/onnxifi_wrapper.c
+@@ -761,7 +761,9 @@ ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI onnxInitGraph(
+ const void* onnxModel,
+ uint32_t weightsCount,
+ const onnxTensorDescriptorV1* weightDescriptors,
+- onnxGraph* graph)
++ onnxGraph* graph,
++ uint32_t maxSeqLength,
++ void* deferredWeightReader)
+ {
+ if (graph == NULL) {
+ return ONNXIFI_STATUS_INVALID_POINTER;
+@@ -797,7 +799,9 @@ ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI onnxInitGraph(
+ onnxModel,
+ weightsCount,
+ weightDescriptors,
+- &graph_wrapper->graph);
++ &graph_wrapper->graph,
++ maxSeqLength,
++ deferredWeightReader);
+ switch (status) {
+ case ONNXIFI_STATUS_SUCCESS:
+ case ONNXIFI_STATUS_FALLBACK:
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 25/31] gnu: Add qnnpack-pytorch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-25-david.elsing@posteo.net
This is an internal fork of QNNPACK in the PyTorch source tree.

* gnu/packages/machine-learning.scm (%python-pytorch-version): New variable.
(%python-pytorch-src): New variable.
(qnnpack-pytorch): New variable.
---
gnu/packages/machine-learning.scm | 127 ++++++++++++++++++++++++++++++
1 file changed, 127 insertions(+)

Toggle diff (140 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 0cab068fbd..eed649f118 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4310,6 +4310,133 @@ (define-public ideep-pytorch
PyTorch.")
(license license:expat)))
+(define %python-pytorch-version "2.2.1")
+
+(define %python-pytorch-src
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/pytorch/pytorch")
+ (commit (string-append "v" %python-pytorch-version))))
+ (file-name (git-file-name "python-pytorch" %python-pytorch-version))
+ (sha256
+ (base32
+ "03mm0pwwb5lxdsmmiw3cch9fijgjw81kmmc4ln9rlyazkm7l1r48"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ ;; Bundled or unused code
+ (for-each
+ (lambda (dir)
+ (when (file-exists? dir)
+ (delete-file-recursively dir)))
+ '("android"
+ "aten/src/ATen/native/cuda/cutlass_extensions"
+ "aten/src/ATen/native/quantized/cpu/qnnpack"
+ "caffe2/mobile/contrib/libopencl-stub"
+ "caffe2/mobile/contrib/libvulkan-stub"
+ "third_party"))
+
+ ;; Autogenerated files
+ (for-each
+ delete-file
+ '("aten/src/ATen/nnapi/nnapi_wrapper.cpp"
+ "aten/src/ATen/nnapi/nnapi_wrapper.h"
+ "caffe2/mobile/contrib/ios/mpscnn/mpscnn_kernels.h"
+ "caffe2/proto/caffe2_legacy_pb2.pyi"
+ "caffe2/proto/caffe2_pb2.pyi"
+ "caffe2/proto/hsm_pb2.pyi"
+ "caffe2/proto/metanet_pb2.pyi"
+ "caffe2/proto/predictor_consts_pb2.pyi"
+ "caffe2/proto/prof_dag_pb2.pyi"
+ "caffe2/proto/torch_pb2.pyi"
+ ;; These files contain just lists of floating point values and
+ ;; might be as well hand-written.
+ ;; "test/cpp/api/init_baseline.h"
+ ;; "test/cpp/api/optim_baseline.h"
+ "test/mobile/test_upgrader_bytecode_table_example.cpp"
+ "torch/csrc/jit/mobile/upgrader_mobile.cpp"
+ "torch/csrc/jit/runtime/decomposition_registry_util.cpp"
+ "torch/csrc/jit/runtime/serialized_shape_function_registry.cpp"
+ "torch/csrc/jit/tensorexpr/external_functions_codegen.cpp"
+ "torch/csrc/jit/serialization/mobile_bytecode_generated.h"))
+ (delete-file-recursively ".github")
+ (for-each
+ (lambda (dir)
+ (for-each
+ delete-file
+ (find-files dir "\\.cu$")))
+ '("aten/src/ATen/native/transformers/cuda/flash_attn/kernels"
+ "aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels"))))))
+
+(define-public qnnpack-pytorch
+ (package
+ (inherit qnnpack)
+ (name "qnnpack-pytorch")
+ (version (string-append "pytorch-" %python-pytorch-version))
+ (source
+ (origin
+ (inherit %python-pytorch-src)
+ (patches '())
+ (modules '((guix build utils)
+ (srfi srfi-26)
+ (ice-9 ftw)))
+ (snippet
+ '(begin
+ (rename-file "aten/src/ATen/native/quantized/cpu/qnnpack"
+ "../qnnpack")
+ (let ((outdir (getcwd)))
+ (chdir "..")
+ (rename-file outdir "dummy")
+ (rename-file "qnnpack" outdir)
+ (chdir outdir)
+ (delete-file-recursively "deps"))))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments qnnpack)
+ ((#:phases phases #~%standard-phases)
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'patch-cmake
+ (lambda _
+ (substitute* "CMakeLists.txt"
+ (("project\\(.*" orig)
+ (apply
+ string-append
+ orig "\n"
+ (map (lambda (name)
+ (string-append
+ "option(" name " \"\" ON)\n"))
+ '("USE_SYSTEM_CPUINFO" "USE_SYSTEM_FP16" "USE_SYSTEM_FXDIV"
+ "USE_SYSTEM_PSIMD" "USE_SYSTEM_PTHREADPOOL"))))
+ (("if.*SOURCE_DIR.*")
+ "if(FALSE)\n")
+ (("if\\(NOT TARGET (clog|gtest|benchmark).*")
+ "if(FALSE)\n")
+ (("target_link_libraries.*(fxdiv|psimd|fp16)\\).*")
+ "")
+ (("(target_link_libraries.*) fp16 (.*)" _ before after)
+ (string-append before " " after)))))
+ (add-after 'unpack 'fix-cstring-include
+ (lambda _
+ (substitute* "include/pack_block_sparse.h"
+ (("#include.*<vector>.*" orig)
+ (string-append orig "\n#include <cstring>\n")))))
+ (add-after 'install 'install-missing-headers
+ (lambda _
+ (for-each
+ (lambda (name)
+ (install-file (string-append "../source/include/" name)
+ (string-append #$output "/include")))
+ '("pack_block_sparse.h"
+ "pytorch_qnnpack.h"
+ "qnnpack_func.h"))
+ (copy-recursively
+ "../source/src/qnnpack"
+ (string-append #$output "/include/qnnpack"))))))
+ ;; Some tests occasionally fail on i686 due to floating point rounding.
+ ((#:tests? _ #t)
+ (not (string-prefix? "i686" (or (%current-target-system)
+ (%current-system)))))))))
+
;; Please also update python-torchvision when updating this package.
(define-public python-pytorch
(package
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 24/31] gnu: Remove xnnpack-for-torch2.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-24-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (xnnpack-for-torch2): Delete variable.
* gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch: Remove file.
* gnu/local.mk (dist_patch_DATA): Remove it.
---
gnu/local.mk | 1 -
gnu/packages/machine-learning.scm | 28 -
.../xnnpack-for-torch2-system-libraries.patch | 2660 -----------------
3 files changed, 2689 deletions(-)
delete mode 100644 gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch

Toggle diff (116 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 7b87bf93e5..bba56e7ebe 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -2200,7 +2200,6 @@ dist_patch_DATA = \
%D%/packages/patches/xmonad-dynamic-linking.patch \
%D%/packages/patches/xnnpack-remove-broken-tests.patch \
%D%/packages/patches/xnnpack-system-libraries.patch \
- %D%/packages/patches/xnnpack-for-torch2-system-libraries.patch \
%D%/packages/patches/xplanet-1.3.1-cxx11-eof.patch \
%D%/packages/patches/xplanet-1.3.1-libdisplay_DisplayOutput.cpp.patch \
%D%/packages/patches/xplanet-1.3.1-libimage_gif.c.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 35d96ac012..0cab068fbd 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4190,34 +4190,6 @@ (define-public xnnpack
TensorFlow.js, PyTorch, and MediaPipe.")
(license license:bsd-3))))
-(define-public xnnpack-for-torch2
- ;; There's currently no tag on this repo.
- (let ((version "0.0")
- (commit "51a987591a6fc9f0fc0707077f53d763ac132cbf")
- (revision "3"))
- (package
- (inherit xnnpack)
- (name "xnnpack")
- (version (git-version version revision commit))
- (home-page "https://github.com/google/XNNPACK") ;fork of QNNPACK
- (source (origin
- (method git-fetch)
- (uri (git-reference (url home-page) (commit commit)))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "1rzby82xq8d0rl1d148yz88jh9cpsw5c8b2yw7yg39mi7qmr55rm"))
- (patches (search-patches "xnnpack-for-torch2-system-libraries.patch"))))
- (arguments
- (list
- #:tests? #false
- #:configure-flags '(list "-DXNNPACK_USE_SYSTEM_LIBS=YES"
- "-DBUILD_SHARED_LIBS=ON"
- "-DCMAKE_POSITION_INDEPENDENT_CODE=ON"
- "-DXNNPACK_LIBRARY_TYPE=shared"
- "-DXNNPACK_BUILD_TESTS=FALSE" ;FIXME: see below
- "-DXNNPACK_BUILD_BENCHMARKS=FALSE"))))))
-
(define-public fbgemm
(package
(name "fbgemm")
diff --git a/gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch b/gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch
deleted file mode 100644
index a711aa0d6c..0000000000
--- a/gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch
+++ /dev/null
@@ -1,2660 +0,0 @@
-This patch allows the build process to use the provided dependencies instead
-of adding their source as CMake sub-directories (in which case "make install"
-would install googletest's and googlebenchmark's libraries and headers).
-
-diff a/CMakeLists.txt b/CMakeLists.txt
---- a/CMakeLists.txt
-+++ b/CMakeLists.txt 2023-12-27 13:29:04.608138355 +0100
-@@ -850,25 +850,8 @@
- SET_PROPERTY(TARGET pthreadpool PROPERTY IMPORTED_IMPLIB "${PTHREADPOOL_LIBRARY}")
- ENDIF()
- ENDIF()
--TARGET_LINK_LIBRARIES(microkernels-all PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(microkernels-prod PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(hardware-config PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(indirection PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(jit PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(logging PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(microparams-init PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(normalization PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(packing PRIVATE pthreadpool)
- IF(XNNPACK_BUILD_LIBRARY)
- TARGET_LINK_LIBRARIES(XNNPACK PUBLIC pthreadpool)
-- TARGET_LINK_LIBRARIES(allocator PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(cache PRIVATE memory pthreadpool)
-- TARGET_LINK_LIBRARIES(subgraph PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(operators PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(operator-run PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(operator-utils PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(mutex PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(memory PRIVATE pthreadpool)
- TARGET_LINK_LIBRARIES(post-operation PUBLIC pthreadpool allocator)
- ENDIF()
-
-@@ -889,12 +872,6 @@
- SET_PROPERTY(TARGET fxdiv PROPERTY LINKER_LANGUAGE C)
- ENDIF()
- ENDIF()
--TARGET_LINK_LIBRARIES(microkernels-all PRIVATE fxdiv)
--TARGET_LINK_LIBRARIES(microkernels-prod PRIVATE fxdiv)
--TARGET_LINK_LIBRARIES(indirection PRIVATE fxdiv)
--IF(XNNPACK_BUILD_LIBRARY)
-- TARGET_LINK_LIBRARIES(XNNPACK PRIVATE fxdiv)
--ENDIF()
-
- # ---[ Configure FP16
- IF(NOT TARGET fp16)
-@@ -913,17 +890,7 @@
- SET_PROPERTY(TARGET fp16 PROPERTY LINKER_LANGUAGE C)
- ENDIF()
- ENDIF()
--TARGET_LINK_LIBRARIES(microkernels-all PRIVATE fp16)
--TARGET_LINK_LIBRARIES(microkernels-prod PRIVATE fp16)
--TARGET_LINK_LIBRARIES(microparams-init PRIVATE fp16)
--TARGET_LINK_LIBRARIES(packing PRIVATE fp16)
--TARGET_LINK_LIBRARIES(indirection PRIVATE fp16)
- IF(XNNPACK_BUILD_LIBRARY)
-- TARGET_LINK_LIBRARIES(subgraph PRIVATE fp16)
-- TARGET_LINK_LIBRARIES(operators PRIVATE fp16)
-- TARGET_LINK_LIBRARIES(operator-run PRIVATE fp16)
--
-- TARGET_LINK_LIBRARIES(XNNPACK PRIVATE fp16)
- INSTALL(TARGETS XNNPACK
- LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
- ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
-@@ -943,7 +910,7 @@
- # Helper libraries
- ADD_LIBRARY(gemm-microkernel-tester STATIC test/gemm-microkernel-tester.cc)
- TARGET_INCLUDE_DIRECTORIES(gemm-microkernel-tester PRIVATE . include src test)
-- TARGET_LINK_LIBRARIES(gemm-microkernel-tester PRIVATE fp16 pthreadpool gtest)
-+ TARGET_LINK_LIBRARIES(gemm-microkernel-tester PRIVATE gtest)
- TARGET_LINK_LIBRARIES(gemm-microkernel-tester PRIVATE jit packing)
- IF(XNNPACK_BUILD_LIBRARY)
- TARGET_LINK_LIBRARIES(gemm-microkernel-tester PRIVATE XNNPACK post-operation)
-@@ -960,22 +927,22 @@
- # ---[ Build operator-level unit tests
- ADD_EXECUTABLE(abs-nc-test test/abs-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(abs-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(abs-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(abs-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME abs-nc-test COMMAND abs-nc-test)
-
- ADD_EXECUTABLE(abs-nc-eager-test test/abs-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(abs-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(abs-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(abs-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME abs-nc-eager-test COMMAND abs-nc-eager-test)
-
- ADD_EXECUTABLE(add-nd-test test/add-nd.cc)
- TARGET_INCLUDE_DIRECTORIES(add-nd-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(add-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(add-nd-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME add-nd-test COMMAND add-nd-test)
-
- ADD_EXECUTABLE(add-nd-eager-test test/add-nd-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(add-nd-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(add-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(add-nd-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME add-nd-eager-test COMMAND add-nd-eager-test)
-
- ADD_EXECUTABLE(argmax-pooling-nhwc-test test/argmax-pooling-nhwc.cc)
-@@ -985,27 +952,27 @@
-
- ADD_EXECUTABLE(average-pooling-nhwc-test test/average-pooling-nhwc.cc)
- TARGET_INCLUDE_DIRECTORIES(average-pooling-nhwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(average-pooling-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(average-pooling-nhwc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME average-pooling-nhwc-test COMMAND average-pooling-nhwc-test)
-
- ADD_EXECUTABLE(bankers-rounding-nc-test test/bankers-rounding-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(bankers-rounding-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(bankers-rounding-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(bankers-rounding-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME bankers-rounding-nc-test COMMAND bankers-rounding-nc-test)
-
- ADD_EXECUTABLE(bankers-rounding-nc-eager-test test/bankers-rounding-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(bankers-rounding-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(bankers-rounding-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(bankers-rounding-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME bankers-rounding-nc-eager-test COMMAND bankers-rounding-nc-eager-test)
-
- ADD_EXECUTABLE(ceiling-nc-test test/ceiling-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(ceiling-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(ceiling-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(ceiling-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME ceiling-nc-test COMMAND ceiling-nc-test)
-
- ADD_EXECUTABLE(ceiling-nc-eager-test test/ceiling-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(ceiling-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(ceiling-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(ceiling-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME ceiling-nc-eager-test COMMAND ceiling-nc-eager-test)
-
- ADD_EXECUTABLE(channel-shuffle-nc-test test/channel-shuffle-nc.cc)
-@@ -1015,12 +982,12 @@
-
- ADD_EXECUTABLE(clamp-nc-test test/clamp-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(clamp-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(clamp-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(clamp-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME clamp-nc-test COMMAND clamp-nc-test)
-
- ADD_EXECUTABLE(clamp-nc-eager-test test/clamp-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(clamp-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(clamp-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(clamp-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME clamp-nc-eager-test COMMAND clamp-nc-eager-test)
-
- ADD_EXECUTABLE(constant-pad-nd-test test/constant-pad-nd.cc)
-@@ -1035,25 +1002,25 @@
-
- ADD_EXECUTABLE(convert-nc-test test/convert-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(convert-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(convert-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(convert-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME convert-nc-test COMMAND convert-nc-test)
-
- ADD_EXECUTABLE(convert-nc-eager-test test/convert-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(convert-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(convert-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(convert-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME convert-nc-eager-test COMMAND convert-nc-eager-test)
-
- ADD_EXECUTABLE(convolution-nhwc-test test/convolution-nhwc.cc)
- SET_TARGET_PROPERTIES(convolution-nhwc-test PROPERTIES CXX_EXTENSIONS YES)
- TARGET_INCLUDE_DIRECTORIES(convolution-nhwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(convolution-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(convolution-nhwc-test PRIVATE XNNPACK gtest gtest_main)
- TARGET_LINK_LIBRARIES(convolution-nhwc-test PRIVATE convolution-test-helpers)
- ADD_TEST(NAME convolution-nhwc-test COMMAND convolution-nhwc-test)
-
- ADD_EXECUTABLE(convolution-nchw-test test/convolution-nchw.cc)
- SET_TARGET_PROPERTIES(convolution-nchw-test PROPERTIES CXX_EXTENSIONS YES)
- TARGET_INCLUDE_DIRECTORIES(convolution-nchw-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(convolution-nchw-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(convolution-nchw-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME convolution-nchw-test COMMAND convolution-nchw-test)
-
- ADD_EXECUTABLE(copy-nc-test test/copy-nc.cc)
-@@ -1068,7 +1035,7 @@
-
- ADD_EXECUTABLE(deconvolution-nhwc-test test/deconvolution-nhwc.cc)
- TARGET_INCLUDE_DIRECTORIES(deconvolution-nhwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(deconvolution-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(deconvolution-nhwc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME deconvolution-nhwc-test COMMAND deconvolution-nhwc-test)
-
- ADD_EXECUTABLE(depth-to-space-nchw2nhwc-test test/depth-to-space-nchw2nhwc.cc)
-@@ -1083,197 +1050,197 @@
-
- ADD_EXECUTABLE(divide-nd-test test/divide-nd.cc)
- TARGET_INCLUDE_DIRECTORIES(divide-nd-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(divide-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(divide-nd-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME divide-nd-test COMMAND divide-nd-test)
-
- ADD_EXECUTABLE(divide-nd-eager-test test/divide-nd-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(divide-nd-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(divide-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(divide-nd-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME divide-nd-eager-test COMMAND divide-nd-eager-test)
-
- ADD_EXECUTABLE(elu-nc-test test/elu-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(elu-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(elu-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(elu-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME elu-nc-test COMMAND elu-nc-test)
-
- ADD_EXECUTABLE(elu-nc-eager-test test/elu-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(elu-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(elu-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(elu-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME elu-nc-eager-test COMMAND elu-nc-eager-test)
-
- ADD_EXECUTABLE(fully-connected-nc-test test/fully-connected-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(fully-connected-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(fully-connected-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(fully-connected-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME fully-connected-nc-test COMMAND fully-connected-nc-test)
-
- ADD_EXECUTABLE(floor-nc-test test/floor-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(floor-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(floor-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(floor-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME floor-nc-test COMMAND floor-nc-test)
-
- ADD_EXECUTABLE(floor-nc-eager-test test/floor-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(floor-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(floor-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(floor-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME floor-nc-eager-test COMMAND floor-nc-eager-test)
-
- ADD_EXECUTABLE(global-average-pooling-nwc-test test/global-average-pooling-nwc.cc)
- TARGET_INCLUDE_DIRECTORIES(global-average-pooling-nwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(global-average-pooling-nwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(global-average-pooling-nwc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME global-average-pooling-nwc-test COMMAND global-average-pooling-nwc-test)
-
- ADD_EXECUTABLE(global-average-pooling-ncw-test test/global-average-pooling-ncw.cc)
- TARGET_INCLUDE_DIRECTORIES(global-average-pooling-ncw-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(global-average-pooling-ncw-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(global-average-pooling-ncw-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME global-average-pooling-ncw-test COMMAND global-average-pooling-ncw-test)
-
- ADD_EXECUTABLE(hardswish-nc-test test/hardswish-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(hardswish-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(hardswish-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(hardswish-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME hardswish-nc-test COMMAND hardswish-nc-test)
-
- ADD_EXECUTABLE(hardswish-nc-eager-test test/hardswish-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(hardswish-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(hardswish-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(hardswish-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME hardswish-nc-eager-test COMMAND hardswish-nc-eager-test)
-
- ADD_EXECUTABLE(leaky-relu-nc-test test/leaky-relu-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(leaky-relu-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(leaky-relu-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(leaky-relu-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME leaky-relu-nc-test COMMAND leaky-relu-nc-test)
-
- ADD_EXECUTABLE(leaky-relu-nc-eager-test test/leaky-relu-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(leaky-relu-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(leaky-relu-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(leaky-relu-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME leaky-relu-nc-eager-test COMMAND leaky-relu-nc-eager-test)
-
- ADD_EXECUTABLE(max-pooling-nhwc-test test/max-pooling-nhwc.cc)
- TARGET_INCLUDE_DIRECTORIES(max-pooling-nhwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(max-pooling-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(max-pooling-nhwc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME max-pooling-nhwc-test COMMAND max-pooling-nhwc-test)
-
- ADD_EXECUTABLE(maximum-nd-test test/maximum-nd.cc)
- TARGET_INCLUDE_DIRECTORIES(maximum-nd-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(maximum-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(maximum-nd-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME maximum-nd-test COMMAND maximum-nd-test)
-
- ADD_EXECUTABLE(maximum-nd-eager-test test/maximum-nd-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(maximum-nd-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(maximum-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(maximum-nd-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME maximum-nd-eager-test COMMAND maximum-nd-eager-test)
-
- ADD_EXECUTABLE(minimum-nd-test test/minimum-nd.cc)
- TARGET_INCLUDE_DIRECTORIES(minimum-nd-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(minimum-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(minimum-nd-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME minimum-nd-test COMMAND minimum-nd-test)
-
- ADD_EXECUTABLE(minimum-nd-eager-test test/minimum-nd-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(minimum-nd-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(minimum-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(minimum-nd-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME minim
This message was truncated. Download the full message here.
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 30/31] gnu: Add qnnpack-pytorch-for-r-torch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-30-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (%python-pytorch-for-r-torch-version): New
variable.
(%python-pytorch-for-r-torch-src): New variable.
(qnnpack-pytorch-for-r-torch): New variable.
---
gnu/packages/machine-learning.scm | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)

Toggle diff (41 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 0c46a1ba1f..4413ecad13 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4679,6 +4679,34 @@ (define-public python-pytorch
Note: currently this package does not provide GPU support.")
(license license:bsd-3)))
+(define %python-pytorch-for-r-torch-version "2.0.1")
+
+(define %python-pytorch-for-r-torch-src
+ (origin
+ (inherit %python-pytorch-src)
+ (uri (git-reference
+ (url "https://github.com/pytorch/pytorch")
+ (commit (string-append "v" %python-pytorch-for-r-torch-version))))
+ (file-name (git-file-name "python-pytorch"
+ %python-pytorch-for-r-torch-version))
+ (sha256
+ (base32
+ "0iirrn687i7sfv0p0i7dn89x3rf13a7l8y1y5h190h51yjxpxqxa"))))
+
+(define-public qnnpack-pytorch-for-r-torch
+ (package
+ (inherit qnnpack-pytorch)
+ (version (string-append "pytorch-" %python-pytorch-for-r-torch-version))
+ (source
+ (origin
+ (inherit %python-pytorch-for-r-torch-src)
+ (patches '())
+ (modules '((guix build utils)
+ (srfi srfi-26)
+ (ice-9 ftw)))
+ (snippet
+ (origin-snippet (package-source qnnpack-pytorch)))))))
+
(define-public python-pytorch-for-r-torch
(package
(inherit python-pytorch)
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 26/31] gnu: python-pytorch: Update to 2.2.1 and unbundle dependencies.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-26-david.elsing@posteo.net
Autogenerated files are also regenerated. The tests can be run, but are
disabled, as they require a long time.

* gnu/packages/machine-learning.scm (python-pytorch): Update to 2.2.1.
[version]: Use %python-pytorch-version.
[source]: Use %python-pytorch-src.
[arguments]: Remove 'make-things-writable phase. Add 'cmake-patches,
'set-max-jobs, 'codegen1, 'codegen2, 'build2, 'install2 phases. Adjust
'use-system-libraries and 'check phases.
[native-inputs]: Add doxygen, ideep-pytorch, pocketfft-cpp, python-expecttest,
python-pytest-flakefinder, python-pytest-rerunfailures-13,
python-pytest-shard, python-pytest-xdist, python-hypothesis,
python-types-dataclasses, python-typing-extensions-4.10 and valgrind.
[inputs]: Add asmjit, clog, flatbuffers-next, foxi, fxdiv, libuv,
miniz-for-pytorch, qnnpack, qnnpack-pytorch, fbgemm and oneapi-dnnl. Use
nnpack, fbgemm and oneapi-dnnl only for supported systems.
[propagated-inputs]: Add python-filelock, python-fsspec, python-jinja2,
python-networkx, python-optree, python-packaging, python-psutil and
python-sympy. Remove python-cffi and python-six.
(%python-pytorch-src)[source]: Add patches.
* gnu/packages/patches/python-pytorch-runpath.patch: Adjust patch.
* gnu/packages/patches/python-pytorch-system-libraries.patch: Adjust patch.
* gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch: Remove file.
* gnu/packages/patches/python-pytorch-fix-codegen.patch: New file.
* gnu/packages/patches/python-pytorch-without-kineto: New file.
* gnu/local.mk (dist_patch_DATA): Register them.
---
gnu/local.mk | 2 +-
gnu/packages/machine-learning.scm | 318 ++++++++++-------
...ython-pytorch-1.9.0-system-libraries.patch | 139 --------
.../patches/python-pytorch-fix-codegen.patch | 167 +++++++++
.../patches/python-pytorch-runpath.patch | 19 +-
.../python-pytorch-system-libraries.patch | 324 +++++++++++++++---
.../python-pytorch-without-kineto.patch | 60 ++++
7 files changed, 720 insertions(+), 309 deletions(-)
delete mode 100644 gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch
create mode 100644 gnu/packages/patches/python-pytorch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-without-kineto.patch

Toggle diff (391 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index bba56e7ebe..84eeb39c8b 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1927,7 +1927,7 @@ dist_patch_DATA = \
%D%/packages/patches/python-pygpgme-fix-pinentry-tests.patch \
%D%/packages/patches/python-pytorch-runpath.patch \
%D%/packages/patches/python-pytorch-system-libraries.patch \
- %D%/packages/patches/python-pytorch-1.9.0-system-libraries.patch \
+ %D%/packages/patches/python-pytorch-without-kineto.patch \
%D%/packages/patches/python-robotframework-atest.patch \
%D%/packages/patches/python-robotframework-source-date-epoch.patch \
%D%/packages/patches/python-robotframework-sshlibrary-rf5-compat.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index eed649f118..f5e19bcaff 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -104,6 +104,7 @@ (define-module (gnu packages machine-learning)
#:use-module (gnu packages parallel)
#:use-module (gnu packages perl)
#:use-module (gnu packages pkg-config)
+ #:use-module (gnu packages pretty-print)
#:use-module (gnu packages protobuf)
#:use-module (gnu packages pulseaudio)
#:use-module (gnu packages python)
@@ -122,6 +123,7 @@ (define-module (gnu packages machine-learning)
#:use-module (gnu packages swig)
#:use-module (gnu packages time)
#:use-module (gnu packages tls)
+ #:use-module (gnu packages valgrind)
#:use-module (gnu packages vulkan)
#:use-module (gnu packages video)
#:use-module (gnu packages web)
@@ -4322,6 +4324,13 @@ (define %python-pytorch-src
(sha256
(base32
"03mm0pwwb5lxdsmmiw3cch9fijgjw81kmmc4ln9rlyazkm7l1r48"))
+ (patches (search-patches "python-pytorch-system-libraries.patch"
+ "python-pytorch-runpath.patch"
+ "python-pytorch-without-kineto.patch"
+ ;; Some autogeneration scripts depend on the
+ ;; compile PyTorch library. Therefore, we create
+ ;; dummy versions which are regenerated later.
+ "python-pytorch-fix-codegen.patch"))
(modules '((guix build utils)))
(snippet
'(begin
@@ -4441,135 +4450,204 @@ (define-public qnnpack-pytorch
(define-public python-pytorch
(package
(name "python-pytorch")
- (version "1.13.1")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/pytorch/pytorch")
- (commit (string-append "v" version))
- (recursive? #t)))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "17yxjzwp4zp75fz7czgz9acijzw7dpyqcza50v8y1x7hfg2gw369"))
- (patches (search-patches "python-pytorch-system-libraries.patch"
- "python-pytorch-runpath.patch"))
- (modules '((guix build utils)))
- (snippet
- '(begin
- ;; XXX: Let's be clear: this package is a bundling fest. We
- ;; delete as much as we can, but there's still a lot left.
- (for-each (lambda (directory)
- (delete-file-recursively
- (string-append "third_party/" directory)))
- '("benchmark" "cpuinfo" "eigen"
-
- ;; FIXME: QNNPACK (of which XNNPACK is a fork)
- ;; needs these.
- ;; "FP16" "FXdiv" "gemmlowp" "psimd"
-
- "gloo" "googletest" "ios-cmake" "NNPACK"
- "onnx" "protobuf" "pthreadpool"
- "pybind11" "python-enum" "python-peachpy"
- "python-six" "tbb" "XNNPACK" "zstd"))
- (substitute* "functorch/CMakeLists.txt"
- (("\\$\\{_rpath_portable_origin\\}/../torch/lib")
- "$ORIGIN/../torch/lib"))))))
+ (version %python-pytorch-version)
+ (source %python-pytorch-src)
(build-system python-build-system)
(arguments
- '(#:phases (modify-phases %standard-phases
- (add-before 'build 'use-system-libraries
- (lambda* (#:key outputs #:allow-other-keys)
- ;; Tell 'setup.py' to let 'CMakeLists.txt' know that we
- ;; want to use "system libraries" instead of the bundled
- ;; ones.
- (setenv "USE_SYSTEM_LIBS" "1")
-
- (substitute* "cmake/Dependencies.cmake"
- (("if\\(USE_SYSTEM_BIND11\\)")
- "if(TRUE)"))
-
- ;; XXX: Disable that for simplicity for now.
- (setenv "USE_FBGEMM" "0")))
- (add-before 'build 'make-things-writable
- (lambda _
- ;; The 'build_caffe2' function in
- ;; 'tools/build_pytorch_libs.py', called from the
- ;; top-level 'setup.py', needs write access to this
- ;; directory.
- (for-each make-file-writable
- (find-files "caffe2/proto" "."
- #:directories? #t))))
- (replace 'check
- (lambda* (#:key inputs outputs tests? #:allow-other-keys)
- ;; Run the test suite following the instructions in
- ;; 'CONTRIBUTING.md'. XXX: Unfortunately this doesn't
- ;; work, unless you set GUIX_PYTHONPATH presumably.
- (when tests?
- (add-installed-pythonpath inputs outputs)
- (invoke "python" "test/run_test.py"))))
- (add-after 'install 'remove-test-executables
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Remove test executables, but keep other executables
- ;; such as 'torch_shm_manager' and and .so files such as
- ;; 'libtorch_global_deps.so'.
- (let ((python-site (site-packages inputs outputs)))
- (for-each delete-file
- (find-files python-site
- "(^test_cpp_rpc|_test)$")))))
- (add-after 'install 'remove-caffe2-onnx-scripts
- (lambda* (#:key outputs #:allow-other-keys)
- (let* ((out (assoc-ref outputs "out"))
- (bin (string-append out "/bin")))
- ;; Remove 'convert-caffe2-to-onnx' and
- ;; 'convert-onnx-to-caffe2': they seem to be
- ;; deprecated and they cause a failure of the
- ;; 'sanity-check' phase:
- ;;
- ;; ImportError: cannot import name 'metanet_pb2' from partially initialized module 'caffe2.proto' (most likely due to a circular import)
- (for-each delete-file
- (find-files bin "^convert-.*caffe2"))
-
- (substitute* (find-files out "^entry_points\\.txt$")
- (("^convert-.*" all)
- (string-append "# " all "\n")))))))
-
- ;; XXX: Tests attempt to download data such as
- ;; <https://raw.githubusercontent.com/pytorch/test-infra/master/stats/slow-tests.json>.
- ;; We're also missing some Python modules, such as expecttest.
- #:tests? #f))
+ (list
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'cmake-patches
+ (lambda _
+ (substitute* "cmake/Dependencies.cmake"
+ (("#POCKETFFT_INCLUDE_DIR")
+ (string-append
+ #$(this-package-native-input "pocketfft-cpp") "/include"))
+ (("#FP16_INCLUDE_DIR")
+ (string-append
+ #$(this-package-input "fp16") "/include")))))
+ (add-before 'build 'use-system-libraries
+ (lambda _
+ (substitute* '("caffe2/serialize/crc.cc"
+ "caffe2/serialize/inline_container.cc")
+ (("\"miniz\\.h\"") "<miniz/miniz.h>"))
+ (substitute*
+ (cons*
+ "torch/csrc/Module.cpp"
+ (map
+ (lambda (name)
+ (string-append
+ "torch/utils/benchmark/utils/valgrind_wrapper/"
+ name))
+ '("compat_bindings.cpp" "timer_callgrind_template.cpp")))
+ (("<callgrind.h>") "<valgrind/callgrind.h>"))
+ ;; Tell 'setup.py' to let 'CMakeLists.txt' know that we
+ ;; want to use "system libraries" instead of the bundled
+ ;; ones.
+ (setenv "USE_SYSTEM_LIBS" "1")
+ ;; For oneDNN
+ (setenv "USE_MKLDNN" "1")
+ ;; Only works with CUPTI
+ (setenv "USE_KINETO" "0")
+ ;; Prevent CMake error by disabling explicitely
+ (setenv "USE_ITT" "0")))
+ (add-after 'use-system-libraries 'set-max-jobs
+ (lambda _
+ (setenv "MAX_JOBS" (number->string (parallel-job-count)))))
+ (add-after 'set-max-jobs 'codegen1
+ (lambda _
+ (with-directory-excursion "torch/csrc/jit/tensorexpr"
+ (setenv "PYTHONPATH" "../../../..")
+ (invoke "python3" "codegen_external.py")
+ (setenv "PYTHONPATH" #f))
+
+ (invoke "python3" "aten/src/ATen/nnapi/codegen.py")
+
+ (invoke "bash" "tools/gen_flatbuffers.sh")
+
+ ;; Generate dummy files as the generation depends on the compiled
+ ;; library. They are regenerated later.
+ (setenv "PYTHONPATH" ".")
+ (invoke "python3"
+ "torchgen/operator_versions/gen_mobile_upgraders.py"
+ "dummy")
+ (setenv "PYTHONPATH" #f)
+
+ (invoke "python3"
+ "torchgen/shape_functions/gen_jit_shape_functions.py"
+ "dummy")
+
+ (invoke "python3"
+ "torchgen/decompositions/gen_jit_decompositions.py"
+ "dummy")))
+ ;; Properly generate autogenerated files ...
+ (add-after 'install 'codegen2
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (add-installed-pythonpath inputs outputs)
+ (invoke "python3"
+ "torchgen/operator_versions/gen_mobile_upgraders.py")
+ (invoke "python3"
+ "torchgen/shape_functions/gen_jit_shape_functions.py")
+ (invoke "python3"
+ "torchgen/decompositions/gen_jit_decompositions.py")))
+ ;; ... rebuild their dependencies ...
+ (add-after 'codegen2 'build2
+ (lambda _
+ (invoke "python3" "setup.py" "build")))
+ ;; ... and install again.
+ (add-after 'build2 'install2
+ (lambda _
+ (invoke "python3" "setup.py" "install" (string-append "--prefix=" #$output)
+ "--no-compile" "--single-version-externally-managed" "--root=/")
+ (invoke "python" "-m" "compileall"
+ "--invalidation-mode=unchecked-hash" #$output)))
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ ;; Run the test suite following the instructions in
+ ;; 'CONTRIBUTING.md'. Unfortunately this doesn't work, unless
+ ;; you set PYTHONPATH or GUIX_PYTHONPATH, but this is done in
+ ;; the codegen2 phase already.
+ (when tests?
+ (invoke "python3" "test/run_test.py" "--core"))))
+ (add-after 'install2 'remove-test-executables
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ ;; Remove test executables, but keep other executables
+ ;; such as 'torch_shm_manager' and and .so files such as
+ ;; 'libtorch_global_deps.so'.
+ (let ((python-site (site-packages inputs outputs)))
+ (for-each delete-file
+ (find-files python-site
+ "(^test_cpp_rpc|_test)$")))))
+ (add-after 'install2 'remove-caffe2-onnx-scripts
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin")))
+ ;; Remove 'convert-caffe2-to-onnx' and
+ ;; 'convert-onnx-to-caffe2': they seem to be
+ ;; deprecated and they cause a failure of the
+ ;; 'sanity-check' phase:
+ ;;
+ ;; ImportError: cannot import name 'metanet_pb2' from partially initialized module 'caffe2.proto' (most likely due to a circular import)
+ (for-each delete-file
+ (find-files bin "^convert-.*caffe2"))
+
+ (substitute* (find-files out "^entry_points\\.txt$")
+ (("^convert-.*" all)
+ (string-append "# " all "\n")))))))
+
+ ;; Even only the core tests take a very long time to run.
+ #:tests? #f))
(native-inputs
- (list cmake ninja))
+ (list cmake
+ doxygen
+ ideep-pytorch
+ ninja
+ pocketfft-cpp
+ python-expecttest
+ python-pytest-flakefinder
+ python-pytest-rerunfailures-13
+ python-pytest-shard
+ python-pytest-xdist
+ python-hypothesis
+ python-types-dataclasses
+ python-typing-extensions-4.10
+ valgrind))
(inputs
- (list eigen
- ;; ("fmt" ,fmt)
- fp16
- gemmlowp
- googletest
- googlebenchmark
- gloo
- nnpack
- openblas
- openmpi
- pthreadpool
- protobuf
- pybind11
- sleef
- xnnpack
- zstd))
+ (append
+ (list asmjit
+ clog
+ eigen
+ flatbuffers-next
+ fmt
+ foxi
+ fp16
+ fxdiv
+ gemmlowp
+ googletest
+ googlebenchmark
+ gloo
+ libuv
+ miniz-for-pytorch
+ openblas
+ openmpi
+ pthreadpool
+ protobuf
+ pybind11
+ qnnpack
+ qnnpack-pytorch
+ sleef
+ tensorpipe
+ xnnpack
+ zstd)
+ ;; TODO: fix build on 32 bit systems
+ (filter
+ (lambda (pkg)
+ (member (or (%current-target-system)
+ (%current-system))
+ (package-transitive-supported-systems pkg)))
+ (list
+ nnpack
+ fbgemm
+ oneapi-dnnl))))
(propagated-inputs
- (list python-astunparse
+ (list cpuinfo
+ onnx ;propagated for its Python modules
+ onnx-optimizer
+ python-astunparse
python-click
+ python-filelock
+ python-fsspec
+ python-future
+ python-jinja2
+ python-networkx
python-numpy
+ python-optree
+ python-packaging
+ python-psutil
python-pyyaml
- python-cffi
- python-typing-extensions
- python-future
- python-six
python-requests
- onnx ;propagated for its Python modules
- onnx-optimizer
- cpuinfo))
+ python-sympy
+ python-typing-extensions))
(home-page "https://pytorch.org/")
(synopsis "Python library for tensor computation and deep neural networks")
(description
diff --git a/gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch b/gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch
deleted file mode 100644
index 76c06520f0..0000000000
--- a/gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch
+++ /dev/null
@@ -1,139 +0,0 @@
-Use our own googletest rather than the bundled one.
-Get NNPACK to use our own PeachPy rather than the bundled one.
-
-diff --git a/cmake/Dependencies.cmake b/cmake/Dependencies.cmake
-index 5d57b9ca78..620cca4e60 100644
---- a/cmake/Dependencies.cmake
-+++ b/cmake/Dependencies.cmake
-@@ -644,11 +644,6 @@ if(BUILD_TEST OR BUILD_MOBILE_BENCHMARK OR BUILD_MOBILE_TEST)
- # this shouldn't be necessary anymore.
- get_property(INC_DIR_temp DIRECTORY PROPERTY INCLUDE_DIRECTORIES)
- set_property(DIRECTORY PROPERTY INCLUDE_DIRECTORIES "")
-- add_subdirectory(${CMAKE_CURRENT_LIST_DIR}/../third_party/googletest)
-- set_property(DIRECTORY PROPERTY INCLUDE_DIRECTORIES ${INC_DIR_temp})
--
-- include_directories(BEFORE SYSTEM ${CMAKE_CURRENT_LIST_DIR}/../third_party/googletest/googletest/include)
-- include_directories(BEFORE SYSTEM ${CMAKE_CURRENT_LIST_DIR}/../third_party/googletest/googlemock/include)
-
- # We will not need to test benchmark lib itself.
- set(B
This message was truncated. Download the full message here.
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 27/31] gnu: python-torchvision: Update to 0.17.1.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-27-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (python-torchvision): Update to 0.17.1.
[source]: Add snippet to remove "android" and "ios" directories.
---
gnu/packages/machine-learning.scm | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

Toggle diff (29 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index f5e19bcaff..02e9d6758f 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -5118,7 +5118,7 @@ (define-public python-torchmetrics
(define-public python-torchvision
(package
(name "python-torchvision")
- (version "0.15.2")
+ (version "0.17.1")
(source (origin
(method git-fetch)
(uri (git-reference
@@ -5128,7 +5128,12 @@ (define-public python-torchvision
(file-name (git-file-name name version))
(sha256
(base32
- "1cq2s13vkgg9rljjbrm4g33yxq7q5zqp7f4xm5cq624gvs0wxmi8"))))
+ "094jz0ryzh0yjxf687r61r482fdh3bax8ix2csghraps0z1sns1b"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file-recursively "android")
+ (delete-file-recursively "ios")))))
(build-system pyproject-build-system)
(arguments
(list #:tests? #false)) ;the test suite is expensive and there is no easy
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 29/31] gnu: Add oneapi-dnnl-for-r-torch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-29-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (oneapi-dnnl-for-r-torch): New variable.
---
gnu/packages/machine-learning.scm | 14 ++++++++++++++
1 file changed, 14 insertions(+)

Toggle diff (27 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 64b62e86d4..0c46a1ba1f 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -5833,6 +5833,20 @@ (define-public oneapi-dnnl
(supported-systems %64bit-supported-systems)
(license license:asl2.0)))
+(define-public oneapi-dnnl-for-r-torch
+ (package
+ (inherit oneapi-dnnl)
+ (version "2.7.3")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/oneapi-src/oneDNN")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name (package-name oneapi-dnnl) version))
+ (sha256
+ (base32 "1zyw5rd8x346bb7gac9a7x3saviw3zvp6aqz2z1l9sv163vmjfz6"))))))
+
(define-public python-gguf
(package
(name "python-gguf")
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 28/31] gnu: Add ideep-pytorch-for-r-torch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-28-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (ideep-pytorch-for-r-torch): New variable.
---
gnu/packages/machine-learning.scm | 15 +++++++++++++++
1 file changed, 15 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 02e9d6758f..64b62e86d4 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4312,6 +4312,21 @@ (define-public ideep-pytorch
PyTorch.")
(license license:expat)))
+(define-public ideep-pytorch-for-r-torch
+ (package
+ (inherit ideep-pytorch)
+ (version "2.7.3-1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/intel/ideep")
+ (commit (string-append "pytorch-rls-v" version))))
+ (file-name (git-file-name (package-name ideep-pytorch) version))
+ (sha256
+ (base32
+ "0hdpkhcjry22fjx2zg2r48v7f4ljrclzj0li2pgk76kvyblfbyvm"))))))
+
(define %python-pytorch-version "2.2.1")
(define %python-pytorch-src
--
2.41.0
D
D
David Elsing wrote on 6 Mar 20:40 +0100
[PATCH 31/31] gnu: python-pytorch-for-r-torch: Adjust to new python-pytorch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240306194037.17992-31-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (%python-pytorch-for-r-torch-src): Add
patches.
(python-pytorch-for-r-torch)[version]: Use
%python-pytorch-for-r-torch-version.
[source]: Use %python-pytorch-for-r-torch-src.
[native-inputs]: Use 'modify-inputs'.
[inputs]: Don't replace xnnpack. Replace qnnpack-pytorch with
qnnpack-pytorch-for-r-torch and oneapi-dnnl with oneapi-dnnl-for-r-torch.
[propagated-inputs]: Don't replace onnx and onnx-optimizer.
[supported-systems]: New field.
* gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen: New file.
* gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch: New file.
* gnu/packages/patches/python-pytorch2-system-libraries.patch: Remove file.
* gnu/local.mk (dist_patch_DATA): Register them.
---
gnu/local.mk | 1 -
gnu/packages/machine-learning.scm | 61 +--
...thon-pytorch-for-r-torch-fix-codegen.patch | 167 ++++++++
...pytorch-for-r-torch-system-libraries.patch | 390 ++++++++++++++++++
.../python-pytorch2-system-libraries.patch | 156 -------
5 files changed, 576 insertions(+), 199 deletions(-)
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch
delete mode 100644 gnu/packages/patches/python-pytorch2-system-libraries.patch

Toggle diff (407 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 84eeb39c8b..6b3af883b6 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1809,7 +1809,6 @@ dist_patch_DATA = \
%D%/packages/patches/python-pyreadstat-link-libiconv.patch \
%D%/packages/patches/python-pyls-black-41.patch \
%D%/packages/patches/python-pypdf-annotate-tests-appropriately.patch \
- %D%/packages/patches/python-pytorch2-system-libraries.patch \
%D%/packages/patches/python-sip-include-dirs.patch \
%D%/packages/patches/python-sgmllib3k-assertions.patch \
%D%/packages/patches/python-sphinx-prompt-docutils-0.19.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 4413ecad13..f85e0b5a93 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4691,7 +4691,15 @@ (define %python-pytorch-for-r-torch-src
%python-pytorch-for-r-torch-version))
(sha256
(base32
- "0iirrn687i7sfv0p0i7dn89x3rf13a7l8y1y5h190h51yjxpxqxa"))))
+ "0iirrn687i7sfv0p0i7dn89x3rf13a7l8y1y5h190h51yjxpxqxa"))
+ (patches (search-patches
+ "python-pytorch-for-r-torch-system-libraries.patch"
+ "python-pytorch-runpath.patch"
+ "python-pytorch-without-kineto.patch"
+ ;; Some autogeneration scripts depend on the
+ ;; compile PyTorch library. Therefore, we create
+ ;; dummy versions which are regenerated later.
+ "python-pytorch-for-r-torch-fix-codegen.patch"))))
(define-public qnnpack-pytorch-for-r-torch
(package
@@ -4707,59 +4715,28 @@ (define-public qnnpack-pytorch-for-r-torch
(snippet
(origin-snippet (package-source qnnpack-pytorch)))))))
+;; Keep in sync with r-torch
(define-public python-pytorch-for-r-torch
(package
(inherit python-pytorch)
(name "python-pytorch")
- (version "2.0.1")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/pytorch/pytorch")
- (commit (string-append "v" version))
- (recursive? #t)))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "14m7v54zyd2qg2xk9mqdpbf4ps7091mdzinzh4vq9p5k4bpznj65"))
- (patches (search-patches "python-pytorch2-system-libraries.patch"
- "python-pytorch-runpath.patch"))
- (modules '((guix build utils)))
- (snippet
- '(begin
- ;; XXX: Let's be clear: this package is a bundling fest. We
- ;; delete as much as we can, but there's still a lot left.
- (for-each (lambda (directory)
- (delete-file-recursively
- (string-append "third_party/" directory)))
- '("benchmark" "cpuinfo" "eigen"
-
- ;; FIXME: QNNPACK (of which XNNPACK is a fork)
- ;; needs these.
- ;; "FP16" "FXdiv" "gemmlowp" "psimd"
-
- "gloo" "googletest" "ios-cmake" "NNPACK"
- "onnx" "protobuf" "pthreadpool"
- "pybind11" "python-enum" "python-peachpy"
- "python-six" "tbb" "XNNPACK" "zstd"))
- (substitute* "caffe2/CMakeLists.txt"
- (("target_link_libraries\\(\\$\\{test_name\\}_\\$\\{CPU_CAPABILITY\\} c10 sleef gtest_main\\)")
- "target_link_libraries(${test_name}_${CPU_CAPABILITY} c10 sleef gtest gtest_main)"))
- (substitute* "functorch/CMakeLists.txt"
- (("\\$\\{_rpath_portable_origin\\}/../torch/lib")
- "$ORIGIN/../torch/lib"))))))
+ (version %python-pytorch-for-r-torch-version)
+ (source %python-pytorch-for-r-torch-src)
+ (native-inputs
+ (modify-inputs (package-native-inputs python-pytorch)
+ (replace "ideep-pytorch" ideep-pytorch-for-r-torch)))
(inputs
(modify-inputs (package-inputs python-pytorch)
- (replace "xnnpack" xnnpack-for-torch2)))
+ (replace "qnnpack-pytorch" qnnpack-pytorch-for-r-torch)
+ (replace "oneapi-dnnl" oneapi-dnnl-for-r-torch)))
(propagated-inputs
(modify-inputs (package-propagated-inputs python-pytorch)
(append python-filelock
python-jinja2
python-networkx
python-opt-einsum
- python-sympy)
- (replace "onnx" onnx-for-torch2)
- (replace "onnx-optimizer" onnx-optimizer-for-torch2)))))
+ python-sympy)))
+ (supported-systems %64bit-supported-systems)))
(define-public python-pytorch-geometric
(package
diff --git a/gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch b/gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch
new file mode 100644
index 0000000000..8515e5ab13
--- /dev/null
+++ b/gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch
@@ -0,0 +1,167 @@
+This patch fixes some scripts for generating source files. For
+gen_jit_decompositions.py, gen_mobile_upgraders.py and
+gen_jit_shape_functions.py, which depend on the compiled PyTorch library, the
+option to generate "dummy" source files is added for the initial build, which
+is later corrected. codegen_external.py is patched to avoid duplicate
+functions and add the static keyword as in the existing generated file.
+
+diff --git a/tools/gen_flatbuffers.sh b/tools/gen_flatbuffers.sh
+index cc0263dbbf..ac34e84b82 100644
+--- a/tools/gen_flatbuffers.sh
++++ b/tools/gen_flatbuffers.sh
+@@ -1,13 +1,13 @@
+ #!/bin/bash
+ ROOT=$(pwd)
+-FF_LOCATION="$ROOT/third_party/flatbuffers"
+-cd "$FF_LOCATION" || exit
+-mkdir build
+-cd build || exit
+-cmake ..
+-cmake --build . --target flatc
+-mkdir -p "$ROOT/build/torch/csrc/jit/serialization"
+-./flatc --cpp --gen-mutable --scoped-enums \
++#FF_LOCATION="$ROOT/third_party/flatbuffers"
++#cd "$FF_LOCATION" || exit
++#mkdir build
++#cd build || exit
++#cmake ..
++#cmake --build . --target flatc
++#mkdir -p "$ROOT/build/torch/csrc/jit/serialization"
++flatc --cpp --gen-mutable --scoped-enums \
+ -o "$ROOT/torch/csrc/jit/serialization" \
+ -c "$ROOT/torch/csrc/jit/serialization/mobile_bytecode.fbs"
+ echo '// @generated' >> "$ROOT/torch/csrc/jit/serialization/mobile_bytecode_generated.h"
+diff --git a/torch/csrc/jit/tensorexpr/codegen_external.py b/torch/csrc/jit/tensorexpr/codegen_external.py
+index 120520b139..0c8587f02d 100644
+--- a/torch/csrc/jit/tensorexpr/codegen_external.py
++++ b/torch/csrc/jit/tensorexpr/codegen_external.py
+@@ -16,9 +16,14 @@ def gen_external(native_functions_path, tags_path, external_path):
+ native_functions = parse_native_yaml(native_functions_path, tags_path)
+ func_decls = []
+ func_registrations = []
+- for func in native_functions:
++ done_names = set()
++ for func in native_functions[0]:
+ schema = func.func
+ name = schema.name.name.base
++ if name in done_names:
++ continue
++ else:
++ done_names.add(name)
+ args = schema.arguments
+ # Only supports extern calls for functions with out variants
+ if not schema.is_out_fn():
+@@ -48,7 +53,7 @@ def gen_external(native_functions_path, tags_path, external_path):
+
+ # print(tensor_decls, name, arg_names)
+ func_decl = f"""\
+-void nnc_aten_{name}(
++static void nnc_aten_{name}(
+ int64_t bufs_num,
+ void** buf_data,
+ int64_t* buf_ranks,
+diff --git a/torchgen/decompositions/gen_jit_decompositions.py b/torchgen/decompositions/gen_jit_decompositions.py
+index 7cfbb803f9..2e69bb1868 100644
+--- a/torchgen/decompositions/gen_jit_decompositions.py
++++ b/torchgen/decompositions/gen_jit_decompositions.py
+@@ -1,8 +1,12 @@
+ #!/usr/bin/env python3
+ import os
+ from pathlib import Path
++import sys
+
+-from torch.jit._decompositions import decomposition_table
++if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ from torch.jit._decompositions import decomposition_table
++else:
++ decomposition_table = {}
+
+ # from torchgen.code_template import CodeTemplate
+
+@@ -85,7 +89,7 @@ def write_decomposition_util_file(path: str) -> None:
+
+
+ def main() -> None:
+- pytorch_dir = Path(__file__).resolve().parents[3]
++ pytorch_dir = Path(__file__).resolve().parents[2]
+ upgrader_path = pytorch_dir / "torch" / "csrc" / "jit" / "runtime"
+ write_decomposition_util_file(str(upgrader_path))
+
+diff --git a/torchgen/operator_versions/gen_mobile_upgraders.py b/torchgen/operator_versions/gen_mobile_upgraders.py
+index e5287cffc5..57f3c38096 100644
+--- a/torchgen/operator_versions/gen_mobile_upgraders.py
++++ b/torchgen/operator_versions/gen_mobile_upgraders.py
+@@ -2,10 +2,12 @@
+ import os
+ from enum import Enum
+ from pathlib import Path
++import sys
+ from typing import Any, Dict, List
+
+-import torch
+-from torch.jit.generate_bytecode import generate_upgraders_bytecode
++if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ import torch
++ from torch.jit.generate_bytecode import generate_upgraders_bytecode
+
+ from torchgen.code_template import CodeTemplate
+ from torchgen.operator_versions.gen_mobile_upgraders_constant import (
+@@ -262,7 +264,10 @@ def construct_register_size(register_size_from_yaml: int) -> str:
+ def construct_version_maps(
+ upgrader_bytecode_function_to_index_map: Dict[str, Any]
+ ) -> str:
+- version_map = torch._C._get_operator_version_map()
++ if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ version_map = torch._C._get_operator_version_map()
++ else:
++ version_map = {}
+ sorted_version_map_ = sorted(version_map.items(), key=lambda item: item[0]) # type: ignore[no-any-return]
+ sorted_version_map = {name: lst for name, lst in sorted_version_map_}
+
+@@ -379,7 +384,10 @@ def sort_upgrader(upgrader_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+
+ def main() -> None:
+
+- upgrader_list = generate_upgraders_bytecode()
++ if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ upgrader_list = generate_upgraders_bytecode()
++ else:
++ upgrader_list = []
+ sorted_upgrader_list = sort_upgrader(upgrader_list)
+ for up in sorted_upgrader_list:
+ print("after sort upgrader : ", next(iter(up)))
+diff --git a/torchgen/shape_functions/gen_jit_shape_functions.py b/torchgen/shape_functions/gen_jit_shape_functions.py
+index c6336a6951..34e394d818 100644
+--- a/torchgen/shape_functions/gen_jit_shape_functions.py
++++ b/torchgen/shape_functions/gen_jit_shape_functions.py
+@@ -18,16 +18,20 @@ you are in the root directory of the Pytorch git repo"""
+ if not file_path.exists():
+ raise Exception(err_msg)
+
+-spec = importlib.util.spec_from_file_location(module_name, file_path)
+-assert spec is not None
+-module = importlib.util.module_from_spec(spec)
+-sys.modules[module_name] = module
+-assert spec.loader is not None
+-assert module is not None
+-spec.loader.exec_module(module)
+-
+-bounded_compute_graph_mapping = module.bounded_compute_graph_mapping
+-shape_compute_graph_mapping = module.shape_compute_graph_mapping
++if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ spec = importlib.util.spec_from_file_location(module_name, file_path)
++ assert spec is not None
++ module = importlib.util.module_from_spec(spec)
++ sys.modules[module_name] = module
++ assert spec.loader is not None
++ assert module is not None
++ spec.loader.exec_module(module)
++
++ bounded_compute_graph_mapping = module.bounded_compute_graph_mapping
++ shape_compute_graph_mapping = module.shape_compute_graph_mapping
++else:
++ bounded_compute_graph_mapping = {}
++ shape_compute_graph_mapping = {}
+
+
+ SHAPE_HEADER = r"""
diff --git a/gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch b/gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch
new file mode 100644
index 0000000000..6e448ce5c5
--- /dev/null
+++ b/gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch
@@ -0,0 +1,390 @@
+Patch build files to also system libraries instead of bundled ones for the
+libraries not supported or working only by specifying USE_SYSTEM_LIBS. This
+includes using the clog, cpuinfo, fbgemm, foxi, fp16, fxdiv, googletest,
+ideep, miniz, nnpack, oneapi-dnnl, pocketfft, pthreadpool, qnnpack,
+qnnpack-pytorch, tensorpipe, valgrind and xnnpack packages.
+For QNNPACK, two versions were bundled and are required: The upstream one and
+an internal fork (now in the package qnnpack-pytorch).
+
+diff --git a/aten/src/ATen/CMakeLists.txt b/aten/src/ATen/CMakeLists.txt
+index 96fc29782b..7f27b6654b 100644
+--- a/aten/src/ATen/CMakeLists.txt
++++ b/aten/src/ATen/CMakeLists.txt
+@@ -362,9 +362,9 @@ if(AT_NNPACK_ENABLED)
+ list(APPEND ATen_CPU_DEPENDENCY_LIBS nnpack) # cpuinfo is added below
+ endif()
+
+-if(MKLDNN_FOUND)
+- list(APPEND ATen_CPU_DEPENDENCY_LIBS ${MKLDNN_LIBRARIES})
+-endif(MKLDNN_FOUND)
++if(USE_MKLDNN)
++ list(APPEND ATen_CPU_DEPENDENCY_LIBS DNNL::dnnl)
++endif(USE_MKLDNN)
+
+ list(APPEND ATen_CPU_DEPENDENCY_LIBS cpuinfo)
+
+diff --git a/caffe2/CMakeLists.txt b/caffe2/CMakeLists.txt
+index 221e3f32b2..3bad9d2f9e 100644
+--- a/caffe2/CMakeLists.txt
++++ b/caffe2/CMakeLists.txt
+@@ -110,9 +110,6 @@ if(NOT MSVC AND USE_XNNPACK)
+ if(NOT TARGET fxdiv)
+ set(FXDIV_BUILD_TESTS OFF CACHE BOOL "")
+ set(FXDIV_BUILD_BENCHMARKS OFF CACHE BOOL "")
+- add_subdirectory(
+- "${FXDIV_SOURCE_DIR}"
+- "${CMAKE_BINARY_DIR}/FXdiv")
+ endif()
+ endif()
+
+@@ -975,7 +972,6 @@ elseif(USE_CUDA)
+ endif()
+
+ if(NOT MSVC AND USE_XNNPACK)
+- TARGET_LINK_LIBRARIES(torch_cpu PRIVATE fxdiv)
+ endif()
+
+ # ==========================================================
+@@ -1314,6 +1310,7 @@ target_link_libraries(torch_cpu PUBLIC c10)
+ target_link_libraries(torch_cpu PUBLIC ${Caffe2_PUBLIC_DEPENDENCY_LIBS})
+ target_link_libraries(torch_cpu PRIVATE ${Caffe2_DEPENDENCY_LIBS})
+ target_link_libraries(torch_cpu PRIVATE ${Caffe2_DEPENDENCY_WHOLE_LINK_LIBS})
++target_link_libraries(torch_cpu PRIVATE miniz clog)
+ target_include_directories(torch_cpu INTERFACE $<INSTALL_INTERFACE:include>)
+ target_include_directories(torch_cpu PRIVATE ${Caffe2_CPU_INCLUDE})
+ target_include_directories(torch_cpu SYSTEM PRIVATE "${Caffe2_DEPENDENCY_INCLUDE}")
+@@ -1570,7 +1567,7 @@ if(BUILD_STATIC_RUNTIME_BENCHMARK)
+ add_executable(static_runtime_bench "${STATIC_RUNTIME_BENCHMARK_SRCS}")
+ add_executable(static_runtime_test "${STATIC_RUNTIME_TEST_SRCS}")
+ target_link_libraries(static_runtime_bench torch_library benchmark)
+- target_link_libraries(static_runtime_test torch_library gtest_main)
++ target_link_libraries(static_runtime_test torch_library gtest_main gtest)
+ endif()
+
+ if(BUILD_TENSOREXPR_BENCHMARK)
+@@ -1601,7 +1598,7 @@ if(BUILD_MOBILE_TEST)
+ foreach(test_src ${ATen_MOBILE_TEST_SRCS})
+ get_filename_component(test_name ${test_src} NAME_WE)
+ add_executable(${test_name} "${test_src}")
+- target_link_libraries(${test_name} torch_library gtest_main)
++ target_link_libraries(${test_name} torch_library gtest_main gtest)
+ target_include_directories(${test_name} PRIVATE $<INSTALL_INTERFACE:include>)
+ target_include_directories(${test_name} PRIVATE $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/include>)
+ target_include_directories(${test_name} PRIVATE ${ATen_CPU_INCLUDE})
+@@ -1622,13 +1619,13 @@ if(BUILD_TEST)
+ if(NOT MSVC)
+ add_executable(${test_name}_${CPU_CAPABILITY} "${test_src}" ../aten/src/ATen/native/quantized/AffineQuantizerBase.cpp)
+ # TODO: Get rid of c10 dependency (which is only needed for the implementation of AT_ERROR)
+- target_link_libraries(${test_name}_${CPU_CAPABILITY} c10 sleef gtest_main)
++ target_link_libraries(${test_name}_${CPU_CAPABILITY} c10 sleef gtest_main gtest)
+ if(USE_FBGEMM)
+ target_link_libraries(${test_name}_${CPU_CAPABILITY} fbgemm)
+ endif()
+ else()
+ add_executable(${test_name}_${CPU_CAPABILITY} "${test_src}")
+- target_link_libraries(${test_name}_${CPU_CAPABILITY} torch_library gtest_main)
++ target_link_libraries(${test_name}_${CPU_CAPABILITY} torch_library gtest_main gtest)
+ endif()
+ target_include_directories(${test_name}_${CPU_CAPABILITY} PRIVATE $<INSTALL_INTERFACE:include>)
+ target_include_directories(${test_name}_${CPU_CAPABILITY} PRIVATE $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/include>)
+@@ -1645,7 +1642,7 @@ if(BUILD_TEST)
+ foreach(test_src ${Caffe2_CPU_TEST_SRCS})
+ get_filename_component(test_name ${test_src} NAME_WE)
+ add_executable(${test_name} "${test_src}")
+- target_link_libraries(${test_name} torch_library gtest_main)
++ target_link_libraries(${test_name} torch_library gtest_main gtest)
+ target_include_directories(${test_name} PRIVATE $<INSTALL_INTERFACE:include>)
+ target_include_directories(${test_name} PRIVATE $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/include>)
+ target_include_directories(${test_name} PRIVATE ${Caffe2_CPU_INCLUDE})
+diff --git a/caffe2/serialize/CMakeLists.txt b/caffe2/serialize/CMakeLists.txt
+index 1552b59d0d..67e1a9a1a3 100644
+--- a/caffe2/serialize/CMakeLists.txt
++++ b/caffe2/serialize/CMakeLists.txt
+@@ -2,7 +2,6 @@ file(GLOB tmp *_test.cc)
+
+ set(Caffe2_CPU_TEST_SRCS ${Caffe2_CPU_TEST_SRCS} ${tmp})
+ list(APPEND Caffe2_CPU_SRCS
+- ${PROJECT_SOURCE_DIR}/third_party/miniz-2.1.0/miniz.c
+ ${CMAKE_CURRENT_SOURCE_DIR}/inline_container.cc
+ ${CMAKE_CURRENT_SOURCE_DIR}/istream_adapter.cc
+ ${CMAKE_CURRENT_SOURCE_DIR}/file_adapter.cc
+diff --git a/cmake/Dependencies.cmake b/cmake/Dependencies.cmake
+index 8c0e3c24bc..d65576ada2 100644
+--- a/cmake/Dependencies.cmake
++++ b/cmake/Dependencies.cmake
+@@ -298,7 +298,7 @@ endif()
+ # --- [ PocketFFT
+ set(AT_POCKETFFT_ENABLED 0)
+ if(NOT AT_MKL_ENABLED)
+- set(POCKETFFT_INCLUDE_DIR "${Torch_SOURCE_DIR}/third_party/pocketfft/")
++ set(POCKETFFT_INCLUDE_DIR "#POCKETFFT_INCLUDE_DIR")
+ if(NOT EXISTS "${POCKETFFT_INCLUDE_DIR}")
+ message(FATAL_ERROR "pocketfft directory not found, expected ${POCKETFFT_INCLUDE_DIR}")
+ elif(NOT EXISTS "${POCKETFFT_INCLUDE_DIR}/pocketfft_hdronly.h")
+@@ -501,19 +501,6 @@ if(USE_QNNPACK)
+ set(QNNPACK_BUILD_T
This message was truncated. Download the full message here.
D
D
David Elsing wrote on 12 Mar 23:46 +0100
[PATCH v2 00/31] Unbundle and update python-pytorch
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312224712.15578-1-david.elsing@posteo.net
In the previous version, FFmpeg, Vulkan and OpenCV were not included, so I
added them to the inputs and enabled the corresponding build options. In a
basic test, the Vulkan backend seemed to work.
I also adjusted the copyright headers, which I forgot ealier.

David Elsing (31):
gnu: asmjit: Update to commit 3ca5c18.
gnu: Add python-typing-extensions-4.10.
gnu: Add python-optree.
gnu: Add python-pytest-flakefinder.
gnu: Add python-pytest-shard.
gnu: Add python-expecttest.
gnu: Add python-pytest-rerunfailures-13.
gnu: Add miniz.
gnu: Add miniz-for-pytorch.
gnu: Add libnop.
gnu: Remove flatbuffers-next-shared.
gnu: python-flatbuffers-next: Update to 23.5.26.
gnu: pthreadpool: Update to commit 178e3e0.
gnu: cpuinfo: Update to commit aa4b216.
gnu: clog: Add "-DUSE_SYSTEM_LIBS=ON" configure flag.
gnu: nnpack: Update to commit 70a77f4.
gnu: oneapi-dnnl: Update to 3.3.5.
gnu: Add tensorpipe.
gnu: Add fbgemm.
gnu: Add qnnpack.
gnu: Add foxi.
gnu: Add ideep-pytorch.
gnu: xnnpack: Update to commit 51a9875.
gnu: Remove xnnpack-for-torch2.
gnu: Add qnnpack-pytorch.
gnu: python-pytorch: Update to 2.2.1 and unbundle dependencies.
gnu: python-torchvision: Update to 0.17.1.
gnu: Add ideep-pytorch-for-r-torch.
gnu: Add oneapi-dnnl-for-r-torch.
gnu: Add qnnpack-pytorch-for-r-torch.
gnu: python-pytorch-for-r-torch: Adjust to new python-pytorch.

gnu/local.mk | 10 +-
gnu/packages/compression.scm | 50 +
gnu/packages/cpp.scm | 6 +-
gnu/packages/machine-learning.scm | 890 +++-
gnu/packages/parallel.scm | 31 +-
.../patches/clog-fix-shared-build.patch | 39 +-
.../patches/cpuinfo-system-libraries.patch | 50 -
.../patches/fbgemm-use-system-libraries.patch | 62 +
gnu/packages/patches/foxi-fix-build.patch | 55 +
gnu/packages/patches/miniz-for-pytorch.patch | 47 +
.../patches/python-optree-fix-32-bit.patch | 122 +
...ython-pytorch-1.9.0-system-libraries.patch | 139 -
.../patches/python-pytorch-fix-codegen.patch | 167 +
...thon-pytorch-for-r-torch-fix-codegen.patch | 167 +
...pytorch-for-r-torch-system-libraries.patch | 399 ++
.../patches/python-pytorch-runpath.patch | 19 +-
.../python-pytorch-system-libraries.patch | 317 +-
.../python-pytorch-without-kineto.patch | 60 +
.../python-pytorch2-system-libraries.patch | 156 -
.../xnnpack-for-torch2-system-libraries.patch | 2660 -----------
.../patches/xnnpack-remove-broken-tests.patch | 335 ++
.../patches/xnnpack-system-libraries.patch | 4133 ++++++++++-------
gnu/packages/python-build.scm | 14 +-
gnu/packages/python-check.scm | 89 +
gnu/packages/python-xyz.scm | 30 +
gnu/packages/serialization.scm | 54 +-
26 files changed, 5144 insertions(+), 4957 deletions(-)
delete mode 100644 gnu/packages/patches/cpuinfo-system-libraries.patch
create mode 100644 gnu/packages/patches/fbgemm-use-system-libraries.patch
create mode 100644 gnu/packages/patches/foxi-fix-build.patch
create mode 100644 gnu/packages/patches/miniz-for-pytorch.patch
create mode 100644 gnu/packages/patches/python-optree-fix-32-bit.patch
delete mode 100644 gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch
create mode 100644 gnu/packages/patches/python-pytorch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch
create mode 100644 gnu/packages/patches/python-pytorch-without-kineto.patch
delete mode 100644 gnu/packages/patches/python-pytorch2-system-libraries.patch
delete mode 100644 gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch
create mode 100644 gnu/packages/patches/xnnpack-remove-broken-tests.patch

--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 01/31] gnu: asmjit: Update to commit 3ca5c18.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-1-david.elsing@posteo.net
* gnu/packages/cpp.scm (asmjit): Update to commit 3ca5c18.
---
gnu/packages/cpp.scm | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Toggle diff (26 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index d451eea2fd..78c3a195a6 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -154,8 +154,8 @@ (define-public argagg
(license license:expat))))
(define-public asmjit
- (let ((commit "4ec760a3d1f69e32ba460ecd2513f29b8428700b")
- (revision "0"))
+ (let ((commit "3ca5c186bf8922e5fe3018432e93651fd2fa4053")
+ (revision "1"))
(package
(name "asmjit")
(version (git-version "0.0.0" revision commit))
@@ -168,7 +168,7 @@ (define-public asmjit
(commit commit)))
(file-name (git-file-name name version))
(sha256
- (base32 "0skgccbpamcbg1byawfq5n6jzxgj64hnc7jznvk35nkskaaz1nlb"))))
+ (base32 "10k1zc0w8m0vnh52id9qlm1sb99qmpvr6k0ha8ag2h223n0d591g"))))
(build-system cmake-build-system)
(arguments
(list #:configure-flags #~(list "-DASMJIT_TEST=TRUE")))
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 02/31] gnu: Add python-typing-extensions-4.10.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-2-david.elsing@posteo.net
* gnu/packages/python-build.scm (python-typing-extensions-4.10): New variable.
---
gnu/packages/python-build.scm | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)

Toggle diff (34 lines)
diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 2ea457cdba..dfd59d5e25 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -10,7 +10,7 @@
;;; Copyright © 2020, 2021, 2022, 2023 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2022 Garek Dyszel <garekdyszel@disroot.org>
;;; Copyright © 2022 Greg Hogan <code@greghogan.com>
-
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -252,6 +252,18 @@ (define-public python-typing-extensions
@end enumerate\n")
(license license:psfl)))
+(define-public python-typing-extensions-4.10
+ (package
+ (inherit python-typing-extensions)
+ (version "4.10.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "typing_extensions" version))
+ (sha256
+ (base32
+ "1jxkj4pni8pdyrn79sq441lsp40xzw363n0qvfc6zfcgkv4dgaxh"))))))
+
;;;
;;; Python builder packages.
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 03/31] gnu: Add python-optree.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-3-david.elsing@posteo.net
* gnu/packages/python-xyz.scm (python-optree): New variable.
---
.../patches/python-optree-fix-32-bit.patch | 122 ++++++++++++++++++
gnu/packages/python-xyz.scm | 30 +++++
2 files changed, 152 insertions(+)
create mode 100644 gnu/packages/patches/python-optree-fix-32-bit.patch

Toggle diff (178 lines)
diff --git a/gnu/packages/patches/python-optree-fix-32-bit.patch b/gnu/packages/patches/python-optree-fix-32-bit.patch
new file mode 100644
index 0000000000..6a32c39bd8
--- /dev/null
+++ b/gnu/packages/patches/python-optree-fix-32-bit.patch
@@ -0,0 +1,122 @@
+In include/utils.h, ssize_t is an alias for py::ssize_t, which is an alias for
+Py_ssize_t in Python, which is an alias for the system ssize_t.
+The latter is defined in glibc as int if __WORDSIZE == 32 and as long int if
+__WORDSIZE == 64. Therefore, we need to remove the explicit template
+specialization for int in the first case.
+
+diff --git a/include/utils.h b/include/utils.h
+index 950a02b..82a9591 100644
+--- a/include/utils.h
++++ b/include/utils.h
+@@ -141,10 +141,12 @@ template <>
+ inline py::handle GET_ITEM_HANDLE<py::tuple>(const py::handle& container, const size_t& item) {
+ return PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::handle GET_ITEM_HANDLE<py::tuple>(const py::handle& container, const int& item) {
+ return PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#endif
+ template <>
+ inline py::handle GET_ITEM_HANDLE<py::list>(const py::handle& container, const ssize_t& item) {
+ return PyList_GET_ITEM(container.ptr(), item);
+@@ -153,10 +155,12 @@ template <>
+ inline py::handle GET_ITEM_HANDLE<py::list>(const py::handle& container, const size_t& item) {
+ return PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::handle GET_ITEM_HANDLE<py::list>(const py::handle& container, const int& item) {
+ return PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#endif
+
+ template <typename Container, typename Item>
+ inline py::object GET_ITEM_BORROW(const py::handle& container, const Item& item) {
+@@ -171,11 +175,13 @@ inline py::object GET_ITEM_BORROW<py::tuple>(const py::handle& container, const
+ return py::reinterpret_borrow<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_BORROW<py::tuple>(const py::handle& container, const int& item) {
+ return py::reinterpret_borrow<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+ template <>
+ inline py::object GET_ITEM_BORROW<py::list>(const py::handle& container, const ssize_t& item) {
+ return py::reinterpret_borrow<py::object>(PyList_GET_ITEM(container.ptr(), item));
+@@ -185,11 +191,13 @@ inline py::object GET_ITEM_BORROW<py::list>(const py::handle& container, const s
+ return py::reinterpret_borrow<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_BORROW<py::list>(const py::handle& container, const int& item) {
+ return py::reinterpret_borrow<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+
+ template <typename Container, typename Item>
+ inline py::object GET_ITEM_STEAL(const py::handle& container, const Item& item) {
+@@ -204,11 +212,13 @@ inline py::object GET_ITEM_STEAL<py::tuple>(const py::handle& container, const s
+ return py::reinterpret_steal<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_STEAL<py::tuple>(const py::handle& container, const int& item) {
+ return py::reinterpret_steal<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+ template <>
+ inline py::object GET_ITEM_STEAL<py::list>(const py::handle& container, const ssize_t& item) {
+ return py::reinterpret_steal<py::object>(PyList_GET_ITEM(container.ptr(), item));
+@@ -218,11 +228,13 @@ inline py::object GET_ITEM_STEAL<py::list>(const py::handle& container, const si
+ return py::reinterpret_steal<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_STEAL<py::list>(const py::handle& container, const int& item) {
+ return py::reinterpret_steal<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+
+ template <typename Container, typename Item>
+ inline void SET_ITEM(const py::handle& container, const Item& item, const py::handle& value) {
+@@ -240,12 +252,14 @@ inline void SET_ITEM<py::tuple>(const py::handle& container,
+ const py::handle& value) {
+ PyTuple_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline void SET_ITEM<py::tuple>(const py::handle& container,
+ const int& item,
+ const py::handle& value) {
+ PyTuple_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#endif
+ template <>
+ inline void SET_ITEM<py::list>(const py::handle& container,
+ const ssize_t& item,
+@@ -258,12 +272,14 @@ inline void SET_ITEM<py::list>(const py::handle& container,
+ const py::handle& value) {
+ PyList_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline void SET_ITEM<py::list>(const py::handle& container,
+ const int& item,
+ const py::handle& value) {
+ PyList_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#endif
+
+ template <typename PyType>
+ inline void AssertExact(const py::handle& object) {
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 0c72caeca0..b664aac3b7 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -148,6 +148,7 @@
;;; Copyright © 2023, 2024 Troy Figiel <troy@troyfigiel.com>
;;; Copyright © 2024 Timothee Mathieu <timothee.mathieu@inria.fr>
;;; Copyright © 2024 Ian Eure <ian@retrospec.tv>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -11636,6 +11637,35 @@ (define-public python-treelib
"This package provides a Python implementation of a tree structure.")
(license license:asl2.0)))
+(define-public python-optree
+ (package
+ (name "python-optree")
+ (version "0.10.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/metaopt/optree")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1q3wljk7cyl5rsam02sfsj8zjrqx4c3x9vic8j6xx13p8czpsisg"))
+ (patches (search-patches "python-optree-fix-32-bit.patch"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs (list python-typing-extensions))
+ (native-inputs
+ (list python-pytest
+ python-pytest-cov
+ python-pytest-xdist
+ cmake
+ pybind11))
+ (home-page "https://github.com/metaopt/optree")
+ (synopsis "Optimized PyTree Utilities")
+ (description "This package contains operations on PyTrees (a tree made of
+container data structures in Python).")
+ (license license:asl2.0)))
+
(define-public python-jupyter-core
(package
(name "python-jupyter-core")
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 04/31] gnu: Add python-pytest-flakefinder.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-4-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-pytest-flakefinder): New variable.
---
gnu/packages/python-check.scm | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)

Toggle diff (39 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 51a38bbcbe..247f8af98b 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -18,6 +18,7 @@
;;; Copyright © 2022 Tomasz Jeneralczyk <tj@schwi.pl>
;;; Copyright © 2022 jgart <jgart@dismail.de>
;;; Copyright © 2024 Troy Figiel <troy@troyfigiel.com>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -191,6 +192,24 @@ (define-public python-pytest-csv
it adds to the Pytest command line interface (CLI).")
(license license:gpl3+)))
+(define-public python-pytest-flakefinder
+ (package
+ (name "python-pytest-flakefinder")
+ (version "1.1.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "pytest-flakefinder" version))
+ (sha256
+ (base32 "03iy80xlkpgzjs2kxa9rrj8dbnp9awyhpcl3hy8fgf5x40cjlhg2"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs (list python-pytest))
+ (home-page "https://github.com/dropbox/pytest-flakefinder")
+ (synopsis "Pytest plugin for finding flaky tests")
+ (description "This package provides a Pytest plugin to run tests multiple
+times and detect flakyness.")
+ (license license:asl2.0)))
+
(define-public python-testfixtures
(package
(name "python-testfixtures")
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 05/31] gnu: Add python-pytest-shard.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-5-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-pytest-shard): New variable.
---
gnu/packages/python-check.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 247f8af98b..f876088488 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -210,6 +210,30 @@ (define-public python-pytest-flakefinder
times and detect flakyness.")
(license license:asl2.0)))
+(define-public python-pytest-shard
+ (let ((commit "64610a08dac6b0511b6d51cf895d0e1040d162ad")
+ (revision "0"))
+ (package
+ (name "python-pytest-shard")
+ (version (git-version "0.1.2" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/AdamGleave/pytest-shard")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1h31m68igz670bzl307hazjrfbr8pk14mxflllar18ydmlrnl677"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs (list python-pytest))
+ (home-page "https://github.com/AdamGleave/pytest-shard")
+ (synopsis "Pytest plugin for sharding tests")
+ (description "This package provides a Pytest extension for sharding
+tests at the granularity of individual test cases, which can be run in
+parallel and on multiple machines.")
+ (license license:expat))))
+
(define-public python-testfixtures
(package
(name "python-testfixtures")
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 07/31] gnu: Add python-pytest-rerunfailures-13.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-7-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-pytest-rerunfailures-13): New variable.
---
gnu/packages/python-check.scm | 11 +++++++++++
1 file changed, 11 insertions(+)

Toggle diff (24 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index dbca80652c..347e4a4701 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -2464,6 +2464,17 @@ (define-public python-pytest-rerunfailures
eliminate flaky failures.")
(license license:mpl2.0)))
+(define-public python-pytest-rerunfailures-13
+ (package
+ (inherit python-pytest-rerunfailures)
+ (version "13.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "pytest-rerunfailures" version))
+ (sha256
+ (base32 "16cin0chv59w4rvnd6r0fisp0s8avmp07rwn9da6yixw43jdncp1"))))))
+
(define-public python-xunitparser
(package
(name "python-xunitparser")
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 08/31] gnu: Add miniz.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-8-david.elsing@posteo.net
* gnu/packages/compression.scm (miniz): New variable.
---
gnu/packages/compression.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (46 lines)
diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 5de17b6b51..fcfc2c30f3 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -38,6 +38,7 @@
;;; Copyright © 2022 Greg Hogan <code@greghogan.com>
;;; Copyright © 2022 Zhu Zihao <all_but_last@163.com>
;;; Copyright © 2021 Foo Chuan Wei <chuanwei.foo@hotmail.com>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -2838,3 +2839,31 @@ (define-public unrar-free
libarchive. It does not rival the non-free @code{unrar} in terms of features,
but special care has been taken to ensure it meets most user's needs.")
(license license:gpl2+)))
+
+(define-public miniz
+ (package
+ (name "miniz")
+ (version "3.0.2")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/richgel999/miniz")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0672q35vjrpakmsr1gwj9k5fwv5ihzhahm19bq4y74wqpn91p7fw"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ #:configure-flags
+ ''("-DBUILD_SHARED_LIBS=ON")
+ ;; No tests
+ #:tests? #f))
+ (home-page "https://github.com/richgel999/miniz")
+ (synopsis "Independent implementation of zlib and Deflate compression")
+ (description "Miniz is a lossless data compression library that implements
+the zlib (RFC 1950) and Deflate (RFC 1951) compressed data format
+specification standards. It supports the most commonly used functions
+exported by the zlib library.")
+ (license license:expat)))
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 10/31] gnu: Add libnop.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-10-david.elsing@posteo.net
* gnu/packages/serialization.scm (libnop): New variable.
---
gnu/packages/serialization.scm | 39 ++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 331fae80cf..bfee0dbfc6 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -19,6 +19,7 @@
;;; Copyright © 2023 Zheng Junjie <873216071@qq.com>
;;; Copyright © 2024 Paul A. Patience <paul@apatience.com>
;;; Copyright © 2024 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -936,3 +937,41 @@ (define-public python-feather-format
(description "This package provides a Python wrapper library to the
Apache Arrow-based Feather binary columnar serialization data frame format.")
(license license:asl2.0)))
+
+(define-public libnop
+ (let ((commit "35e800d81f28c632956c5a592e3cbe8085ecd430")
+ (revision "0"))
+ (package
+ (name "libnop")
+ (version (git-version "0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/google/libnop")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0qqbaljq54qiq0dky9nj47igfcs065ry526jg9a0aafbfl9krpy2"))))
+ (build-system gnu-build-system)
+ (arguments
+ (list
+ #:test-target "test"
+ #:phases
+ #~(modify-phases %standard-phases
+ (delete 'configure)
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ (invoke "out/test"))))
+ (replace 'install
+ (lambda _
+ (copy-recursively
+ "include" (string-append #$output "/include")))))))
+ (native-inputs (list googletest))
+ (home-page "https://github.com/google/libnop")
+ (synopsis "C++ Native Object Protocols")
+ (description "@code{libnop} is a header-only library for serializing and
+deserializing C++ data types without external code generators or runtime
+support libraries.")
+ (license license:asl2.0))))
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 06/31] gnu: Add python-expecttest.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-6-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-expecttest): New variable.
---
gnu/packages/python-check.scm | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index f876088488..dbca80652c 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -2078,6 +2078,41 @@ (define-public python-eradicate
valid Python syntax that are likely to be commented out code.")
(license license:expat)))
+(define-public python-expecttest
+ (let ((commit "683b09a352cc426851adc2e3a9f46e0ab25e4dee")
+ (revision "0"))
+ (package
+ (name "python-expecttest")
+ (version (git-version "0.2.1" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/ezyang/expecttest")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1djwxp9x1hczzxbimv1b1bmd083am88v27l82nmlkhvzyg2cmpvv"))))
+ (build-system pyproject-build-system)
+ (arguments
+ (list
+ #:phases
+ #~(modify-phases %standard-phases
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ ;; The test runs tests expected to fail, so the output is
+ ;; confusing
+ (invoke "python3" "test_expecttest.py")))))))
+ (native-inputs (list python-hypothesis poetry))
+ (home-page "https://github.com/ezyang/expecttest")
+ (synopsis "Python module for expect tests")
+ (description "@code{expecttest} is a Python module for expect tests, where
+the initial expected value of a test can be automatically set by running the
+test itself.")
+ (license license:expat))))
+
(define-public python-robber
(package
(name "python-robber")
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 11/31] gnu: Remove flatbuffers-next-shared.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-11-david.elsing@posteo.net
* gnu/packages/serialization.scm (flatbuffers-next-shared): Delete variable.
* gnu/packages/machine-learning.scm (tensorflow-lite)[arguments]: Use
flatbuffers-next instead of flatbuffers-next-shared.
---
gnu/packages/machine-learning.scm | 5 +++--
gnu/packages/serialization.scm | 11 -----------
2 files changed, 3 insertions(+), 13 deletions(-)

Toggle diff (54 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 5c18a2e9d5..6da189616d 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -27,6 +27,7 @@
;;; Copyright © 2024 David Pflug <david@pflug.io>
;;; Copyright © 2024 Timothee Mathieu <timothee.mathieu@inria.fr>
;;; Copyright © 2024 Spencer King <spencer.king@geneoscopy.com>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -3337,7 +3338,7 @@ (define-public tensorflow-lite
(string-append "-DEigen3_DIR=" #$(this-package-input "eigen")
"/share/eigen3/cmake")
(string-append "-DFlatBuffers_DIR="
- #$(this-package-input "flatbuffers-shared")
+ #$(this-package-input "flatbuffers")
"/lib/cmake/flatbuffers")
(string-append "-DNEON_2_SSE_DIR=" #$(this-package-input "neon2sse")
"/lib/cmake/NEON_2_SSE")
@@ -3424,7 +3425,7 @@ (define-public tensorflow-lite
("cpuinfo" ,cpuinfo)
("eigen" ,eigen)
("fp16" ,fp16)
- ("flatbuffers-shared" ,flatbuffers-next-shared)
+ ("flatbuffers" ,flatbuffers-next)
("gemmlowp" ,gemmlowp)
("mesa-headers" ,mesa-headers)
("neon2sse" ,neon2sse)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index bfee0dbfc6..405c4211da 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -890,17 +890,6 @@ (define-public flatbuffers-next
(base32
"1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
-(define-public flatbuffers-next-shared
- (package
- (inherit flatbuffers-next)
- (name "flatbuffers-shared")
- (version "23.1.21")
- (arguments
- (substitute-keyword-arguments (package-arguments flatbuffers-next)
- ((#:configure-flags flags)
- ;; Compile with -fPIC, needed for shared lib.
- #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags))))))
-
(define-public python-flatbuffers
(package
(name "python-flatbuffers")
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 12/31] gnu: python-flatbuffers-next: Update to 23.5.26.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-12-david.elsing@posteo.net
* gnu/packages/serialization.scm (python-flatbuffers-next): Update to 23.5.26.
---
gnu/packages/serialization.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 405c4211da..bea96f2a92 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -879,7 +879,7 @@ (define-public flatbuffers
(define-public flatbuffers-next
(package
(inherit flatbuffers)
- (version "23.1.21")
+ (version "23.5.26")
(source (origin
(method git-fetch)
(uri (git-reference
@@ -888,7 +888,7 @@ (define-public flatbuffers-next
(file-name (git-file-name "flatbuffers" version))
(sha256
(base32
- "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
+ "0cd12dvkzqdafz46q4302mzgpzbz589zmmiga7bq07f2sqy4vrvv"))))))
(define-public python-flatbuffers
(package
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 23/31] gnu: xnnpack: Update to commit 51a9875.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-23-david.elsing@posteo.net
The tests which fail to build if building the shared library are removed by a
patch. Autogenerated files are removed by a snippet and generated in a
separate build phase.

* gnu/packages/machine-learning.scm (xnnpack): Update to commit 51a9875.
[source]: Add patch for tests. Add snippet to remove autogenerated files.
[arguments]: New modules and phase arguments.
[inputs]: Add clog and cpuinfo.
[native-inputs]: New field.
* gnu/packages/patches/xnnpack-remove-broken-tests.patch: Add file.
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/patches/xnnpack-system-libraries.patch: Update patch.
---
gnu/local.mk | 1 +
gnu/packages/machine-learning.scm | 106 +-
.../patches/xnnpack-remove-broken-tests.patch | 335 ++
.../patches/xnnpack-system-libraries.patch | 4133 ++++++++++-------
4 files changed, 2922 insertions(+), 1653 deletions(-)
create mode 100644 gnu/packages/patches/xnnpack-remove-broken-tests.patch

Toggle diff (351 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index becd39a58d..6d5001c617 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -2204,6 +2204,7 @@ dist_patch_DATA = \
%D%/packages/patches/xfce4-settings-defaults.patch \
%D%/packages/patches/xgboost-use-system-dmlc-core.patch \
%D%/packages/patches/xmonad-dynamic-linking.patch \
+ %D%/packages/patches/xnnpack-remove-broken-tests.patch \
%D%/packages/patches/xnnpack-system-libraries.patch \
%D%/packages/patches/xnnpack-for-torch2-system-libraries.patch \
%D%/packages/patches/xplanet-1.3.1-cxx11-eof.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 287f9b1397..ca3772aca8 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4078,41 +4078,109 @@ (define-public qnnpack
(define-public xnnpack
;; There's currently no tag on this repo.
(let ((version "0.0")
- (commit "ae108ef49aa5623b896fc93d4298c49d1750d9ba")
- (revision "2"))
+ (commit "51a987591a6fc9f0fc0707077f53d763ac132cbf")
+ (revision "3"))
(package
(name "xnnpack")
(version (git-version version revision commit))
(home-page "https://github.com/google/XNNPACK") ;fork of QNNPACK
- (source (origin
- (method git-fetch)
- (uri (git-reference (url home-page) (commit commit)))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "0q68q2jxiiiblx45q4337k13ppgh5vqjwrwznchcnpb8hawjj3zl"))
- (patches (search-patches "xnnpack-system-libraries.patch"))))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference (url home-page) (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1rzby82xq8d0rl1d148yz88jh9cpsw5c8b2yw7yg39mi7qmr55rm"))
+ ;; Some tests fail to link as they use internal symbols, which are
+ ;; not included in the shared library.
+ ;; XXX: Additionally, these tests fail on i686 due to incorrect results:
+ ;; 171 - f32-vlrelu-test (Failed)
+ ;; 211 - qs8-gavgpool-minmax-fp32-test (Failed)
+ ;; 224 - qu8-avgpool-minmax-fp32-test (Failed)
+ ;; 228 - qu8-gavgpool-minmax-fp32-test (Failed)
+ ;; 263 - x32-packx-test (Failed)
+ (patches (search-patches "xnnpack-remove-broken-tests.patch"))
+ (modules '((guix build utils)
+ (ice-9 ftw)
+ (ice-9 textual-ports)
+ (srfi srfi-26)))
+ (snippet
+ '(begin
+ ;; Remove autogenerated files
+ (for-each
+ (lambda (dir)
+ (let ((gendir (string-append "src/" dir "/gen")))
+ (when (file-exists? gendir)
+ (delete-file-recursively gendir)
+ ;; Needed for the scripts generating the files
+ (mkdir gendir))))
+ (scandir "src" (negate (cut member <> '("." "..")))))
+ (delete-file-recursively "google3")
+ (delete-file "cmake/microkernels.cmake")
+ ;; Additional autogenerated files which contain the string "Auto-generated file"
+ (for-each
+ (lambda (dir)
+ (for-each
+ (lambda (name)
+ (let ((path (string-append dir "/" name)))
+ (when (call-with-input-file path
+ (lambda (port)
+ (string-contains
+ (get-string-all port)
+ "Auto-generated file")))
+ (delete-file path))))
+ (scandir dir (negate (cut member <> '("." ".."))))))
+ '("test" "bench" "eval" "models" "src/enums" "src/xnnpack"))))))
(build-system cmake-build-system)
(arguments
- '(#:configure-flags '("-DXNNPACK_USE_SYSTEM_LIBS=YES"
+ (list
+ #:build-type "Release" ;; Debugging symbols require a lot of disk space
+ #:configure-flags ''("-DXNNPACK_USE_SYSTEM_LIBS=YES"
"-DBUILD_SHARED_LIBS=ON"
+ "-DCMAKE_POSITION_INDEPENDENT_CODE=ON"
"-DXNNPACK_LIBRARY_TYPE=shared"
- "-DXNNPACK_BUILD_TESTS=FALSE" ;FIXME: see below
"-DXNNPACK_BUILD_BENCHMARKS=FALSE")
-
- ;; FIXME: Building tests leads to a CMake error:
- ;;
- ;; ADD_LIBRARY cannot create target "all_microkernels" because
- ;; another target with the same name already exists.
- #:tests? #f))
+ #:modules '((ice-9 ftw)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'fix-cmake
+ (lambda _
+ (substitute* "CMakeLists.txt"
+ (("TARGET_INCLUDE_DIRECTORIES\\((pthreadpool|cpuinfo).*") "")
+ ((".*IF\\(NOT TARGET gtest\\).*")
+ "IF(FALSE)\n")
+ (("SET\\(CMAKE_CXX_STANDARD 11\\)")
+ "SET(CMAKE_CXX_STANDARD 14)")
+ (("AMD64") "x86_64"))))
+ (add-after 'patch-source-shebangs 'generate-files
+ (lambda _
+ (for-each
+ (lambda (name)
+ (when (and (string-prefix? "generate" name)
+ (string-suffix? ".sh" name)
+ (not (equal? "generate-amalgamation.sh" name)))
+ (display (string-append name "\n"))
+ (invoke "bash" (string-append "scripts/" name))))
+ (scandir "scripts"))
+ ;; These need to run after the above scripts
+ (display "Remaining files\n")
+ (invoke "python3" "tools/update-microkernels.py")
+ (substitute* "tools/amalgamate-microkernels.py"
+ (("BUILD") "BUILD.bazel"))
+ (invoke "bash" "scripts/generate-amalgamation.sh"))))))
(inputs
- (list cpuinfo
+ (list clog
+ cpuinfo
pthreadpool
googletest
googlebenchmark
fxdiv
fp16
psimd))
+ (native-inputs (list python-pyyaml python-wrapper))
(synopsis "Optimized floating-point neural network inference operators")
(description
"XNNPACK is a highly optimized library of floating-point neural network
diff --git a/gnu/packages/patches/xnnpack-remove-broken-tests.patch b/gnu/packages/patches/xnnpack-remove-broken-tests.patch
new file mode 100644
index 0000000000..529dd2590c
--- /dev/null
+++ b/gnu/packages/patches/xnnpack-remove-broken-tests.patch
@@ -0,0 +1,335 @@
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 95df99eae..ab57d5efa 100755
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -978,16 +978,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(add-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
+ ADD_TEST(NAME add-nd-eager-test COMMAND add-nd-eager-test)
+
+- ADD_EXECUTABLE(argmax-pooling-nhwc-test test/argmax-pooling-nhwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(argmax-pooling-nhwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(argmax-pooling-nhwc-test PRIVATE XNNPACK gtest gtest_main)
+- ADD_TEST(NAME argmax-pooling-nhwc-test COMMAND argmax-pooling-nhwc-test)
+-
+- ADD_EXECUTABLE(average-pooling-nhwc-test test/average-pooling-nhwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(average-pooling-nhwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(average-pooling-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME average-pooling-nhwc-test COMMAND average-pooling-nhwc-test)
+-
+ ADD_EXECUTABLE(bankers-rounding-nc-test test/bankers-rounding-nc.cc)
+ TARGET_INCLUDE_DIRECTORIES(bankers-rounding-nc-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(bankers-rounding-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+@@ -1066,11 +1056,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(copy-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
+ ADD_TEST(NAME copy-nc-eager-test COMMAND copy-nc-eager-test)
+
+- ADD_EXECUTABLE(deconvolution-nhwc-test test/deconvolution-nhwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(deconvolution-nhwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(deconvolution-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME deconvolution-nhwc-test COMMAND deconvolution-nhwc-test)
+-
+ ADD_EXECUTABLE(depth-to-space-nchw2nhwc-test test/depth-to-space-nchw2nhwc.cc)
+ TARGET_INCLUDE_DIRECTORIES(depth-to-space-nchw2nhwc-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(depth-to-space-nchw2nhwc-test PRIVATE XNNPACK gtest gtest_main)
+@@ -1116,11 +1101,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(floor-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
+ ADD_TEST(NAME floor-nc-eager-test COMMAND floor-nc-eager-test)
+
+- ADD_EXECUTABLE(global-average-pooling-nwc-test test/global-average-pooling-nwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(global-average-pooling-nwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(global-average-pooling-nwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME global-average-pooling-nwc-test COMMAND global-average-pooling-nwc-test)
+-
+ ADD_EXECUTABLE(global-average-pooling-ncw-test test/global-average-pooling-ncw.cc)
+ TARGET_INCLUDE_DIRECTORIES(global-average-pooling-ncw-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(global-average-pooling-ncw-test PRIVATE XNNPACK fp16 gtest gtest_main)
+@@ -1146,11 +1126,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(leaky-relu-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
+ ADD_TEST(NAME leaky-relu-nc-eager-test COMMAND leaky-relu-nc-eager-test)
+
+- ADD_EXECUTABLE(max-pooling-nhwc-test test/max-pooling-nhwc.cc)
+- TARGET_INCLUDE_DIRECTORIES(max-pooling-nhwc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(max-pooling-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME max-pooling-nhwc-test COMMAND max-pooling-nhwc-test)
+-
+ ADD_EXECUTABLE(maximum-nd-test test/maximum-nd.cc)
+ TARGET_INCLUDE_DIRECTORIES(maximum-nd-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(maximum-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
+@@ -1191,11 +1166,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(negate-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
+ ADD_TEST(NAME negate-nc-eager-test COMMAND negate-nc-eager-test)
+
+- ADD_EXECUTABLE(prelu-nc-test test/prelu-nc.cc)
+- TARGET_INCLUDE_DIRECTORIES(prelu-nc-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(prelu-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+- ADD_TEST(NAME prelu-nc-test COMMAND prelu-nc-test)
+-
+ ADD_EXECUTABLE(resize-bilinear-nhwc-test test/resize-bilinear-nhwc.cc)
+ TARGET_INCLUDE_DIRECTORIES(resize-bilinear-nhwc-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(resize-bilinear-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
+@@ -1312,16 +1282,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_LINK_LIBRARIES(fusion-test PRIVATE XNNPACK gtest gtest_main)
+ ADD_TEST(NAME fusion-test COMMAND fusion-test)
+
+- ADD_EXECUTABLE(memory-planner-test test/memory-planner-test.cc)
+- TARGET_INCLUDE_DIRECTORIES(memory-planner-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(memory-planner-test PRIVATE XNNPACK gtest gtest_main)
+- ADD_TEST(NAME memory-planner-test COMMAND memory-planner-test)
+-
+- ADD_EXECUTABLE(subgraph-fp16-test test/subgraph-fp16.cc)
+- TARGET_INCLUDE_DIRECTORIES(subgraph-fp16-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(subgraph-fp16-test PRIVATE XNNPACK gtest gtest_main)
+- ADD_TEST(NAME subgraph-fp16-test COMMAND subgraph-fp16-test)
+-
+ ADD_EXECUTABLE(subgraph-nchw-test test/subgraph-nchw.cc)
+ TARGET_INCLUDE_DIRECTORIES(subgraph-nchw-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(subgraph-nchw-test PRIVATE XNNPACK gtest gtest_main)
+@@ -1347,226 +1307,6 @@ IF(XNNPACK_BUILD_TESTS)
+ TARGET_INCLUDE_DIRECTORIES(argmax-pooling-2d-test PRIVATE src test)
+ TARGET_LINK_LIBRARIES(argmax-pooling-2d-test PRIVATE XNNPACK gtest gtest_main)
+ ADD_TEST(NAME argmax-pooling-2d-test COMMAND argmax-pooling-2d-test)
+-
+- ADD_EXECUTABLE(average-pooling-2d-test test/average-pooling-2d.cc)
+- TARGET_INCLUDE_DIRECTORIES(average-pooling-2d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(average-pooling-2d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME average-pooling-2d-test COMMAND average-pooling-2d-test)
+-
+- ADD_EXECUTABLE(bankers-rounding-test test/bankers-rounding.cc)
+- TARGET_INCLUDE_DIRECTORIES(bankers-rounding-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(bankers-rounding-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME bankers-rounding-test COMMAND bankers-rounding-test)
+-
+- ADD_EXECUTABLE(ceiling-test test/ceiling.cc)
+- TARGET_INCLUDE_DIRECTORIES(ceiling-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(ceiling-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME ceiling-test COMMAND ceiling-test)
+-
+- ADD_EXECUTABLE(clamp-test test/clamp.cc)
+- TARGET_INCLUDE_DIRECTORIES(clamp-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(clamp-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME clamp-test COMMAND clamp-test)
+-
+- ADD_EXECUTABLE(concatenate2-test test/concatenate2.cc)
+- TARGET_INCLUDE_DIRECTORIES(concatenate2-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(concatenate2-test PRIVATE XNNPACK gtest gtest_main subgraph)
+- ADD_TEST(NAME concatenate2-test COMMAND concatenate2-test)
+-
+- ADD_EXECUTABLE(concatenate3-test test/concatenate3.cc)
+- TARGET_INCLUDE_DIRECTORIES(concatenate3-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(concatenate3-test PRIVATE XNNPACK gtest gtest_main subgraph)
+- ADD_TEST(NAME concatenate3-test COMMAND concatenate3-test)
+-
+- ADD_EXECUTABLE(concatenate4-test test/concatenate4.cc)
+- TARGET_INCLUDE_DIRECTORIES(concatenate4-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(concatenate4-test PRIVATE XNNPACK gtest gtest_main subgraph)
+- ADD_TEST(NAME concatenate4-test COMMAND concatenate4-test)
+-
+- ADD_EXECUTABLE(convert-test test/convert.cc)
+- TARGET_INCLUDE_DIRECTORIES(convert-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(convert-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME convert-test COMMAND convert-test)
+-
+- ADD_EXECUTABLE(convolution-2d-test test/convolution-2d.cc)
+- SET_TARGET_PROPERTIES(convolution-2d-test PROPERTIES CXX_EXTENSIONS YES)
+- TARGET_INCLUDE_DIRECTORIES(convolution-2d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(convolution-2d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph convolution-test-helpers)
+- ADD_TEST(NAME convolution-2d-test COMMAND convolution-2d-test)
+-
+- ADD_EXECUTABLE(copy-test test/copy.cc)
+- TARGET_INCLUDE_DIRECTORIES(copy-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(copy-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME copy-test COMMAND copy-test)
+-
+- ADD_EXECUTABLE(deconvolution-2d-test test/deconvolution-2d.cc)
+- SET_TARGET_PROPERTIES(deconvolution-2d-test PROPERTIES CXX_EXTENSIONS YES)
+- TARGET_INCLUDE_DIRECTORIES(deconvolution-2d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(deconvolution-2d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph convolution-test-helpers)
+- ADD_TEST(NAME deconvolution-2d-test COMMAND deconvolution-2d-test)
+-
+- ADD_EXECUTABLE(depth-to-space-test test/depth-to-space.cc)
+- TARGET_INCLUDE_DIRECTORIES(depth-to-space-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(depth-to-space-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME depth-to-space-test COMMAND depth-to-space-test)
+-
+- ADD_EXECUTABLE(depthwise-convolution-2d-test test/depthwise-convolution-2d.cc)
+- SET_TARGET_PROPERTIES(depthwise-convolution-2d-test PROPERTIES CXX_EXTENSIONS YES)
+- TARGET_INCLUDE_DIRECTORIES(depthwise-convolution-2d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(depthwise-convolution-2d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph convolution-test-helpers)
+- ADD_TEST(NAME depthwise-convolution-2d-test COMMAND depthwise-convolution-2d-test)
+-
+- ADD_EXECUTABLE(divide2-test test/divide2.cc)
+- TARGET_INCLUDE_DIRECTORIES(divide2-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(divide2-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME divide2-test COMMAND divide2-test)
+-
+- ADD_EXECUTABLE(elu-test test/elu.cc)
+- TARGET_INCLUDE_DIRECTORIES(elu-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(elu-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME elu-test COMMAND elu-test)
+-
+- ADD_EXECUTABLE(even-split2-test test/even-split2.cc)
+- TARGET_INCLUDE_DIRECTORIES(even-split2-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(even-split2-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME even-split2-test COMMAND even-split2-test)
+-
+- ADD_EXECUTABLE(even-split3-test test/even-split3.cc)
+- TARGET_INCLUDE_DIRECTORIES(even-split3-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(even-split3-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME even-split3-test COMMAND even-split3-test)
+-
+- ADD_EXECUTABLE(even-split4-test test/even-split4.cc)
+- TARGET_INCLUDE_DIRECTORIES(even-split4-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(even-split4-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME even-split4-test COMMAND even-split4-test)
+-
+- ADD_EXECUTABLE(floor-test test/floor.cc)
+- TARGET_INCLUDE_DIRECTORIES(floor-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(floor-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME floor-test COMMAND floor-test)
+-
+- ADD_EXECUTABLE(fully-connected-test test/fully-connected.cc)
+- TARGET_INCLUDE_DIRECTORIES(fully-connected-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(fully-connected-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME fully-connected-test COMMAND fully-connected-test)
+-
+- ADD_EXECUTABLE(global-average-pooling-1d-test test/global-average-pooling-1d.cc)
+- TARGET_INCLUDE_DIRECTORIES(global-average-pooling-1d-test PRIVATE src test)
+- TARGET_LINK_LIBRARIES(global-average-pooling-1d-test PRIVATE XNNPACK fp16 gtest gtest_main subgraph)
+- ADD_TEST(NAME global-average-pooling-1d-test COMMAND global-average-pooling-1d-test)
+-
+- ADD_EXECUTABLE(global
This message was truncated. Download the full message here.
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 13/31] gnu: pthreadpool: Update to commit 178e3e0.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-13-david.elsing@posteo.net
* gnu/packages/parallel.scm (pthreadpool): Update to commit 178e3e0.
---
gnu/packages/parallel.scm | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)

Toggle diff (35 lines)
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index ddf4ad016b..e1cbe52934 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -12,6 +12,7 @@
;;; Copyright © 2019-2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2020 Roel Janssen <roel@gnu.org>
;;; Copyright © 2021 Stefan Reichör <stefan@xsteve.at>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -449,9 +450,9 @@ (define-public python-slurm-magic
(define-public pthreadpool
;; This repository has only one tag, 0.1, which is older than what users
;; such as XNNPACK expect.
- (let ((commit "1787867f6183f056420e532eec640cba25efafea")
+ (let ((commit "178e3e0646cc671708bf78e77c273940130ac637")
(version "0.1")
- (revision "1"))
+ (revision "2"))
(package
(name "pthreadpool")
(version (git-version version revision commit))
@@ -462,7 +463,7 @@ (define-public pthreadpool
(file-name (git-file-name name version))
(sha256
(base32
- "02hdvxfn5krw8zivkgjx3b4rk9p02yr4mpdjlp75lsv6z1xf5yrx"))
+ "1s86lnq9bahacf5wxn7y14w70jh3g9lq1l7y16ijwhifd01nc2km"))
(patches (search-patches "pthreadpool-system-libraries.patch"))))
(build-system cmake-build-system)
(arguments '(#:configure-flags '("-DBUILD_SHARED_LIBS=ON")))
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 09/31] gnu: Add miniz-for-pytorch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-9-david.elsing@posteo.net
* gnu/packages/compression.scm (miniz-for-pytorch): New variable.
* gnu/packages/patches/miniz-for-pytorch.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
gnu/packages/compression.scm | 21 +++++++++
gnu/packages/patches/miniz-for-pytorch.patch | 47 ++++++++++++++++++++
3 files changed, 69 insertions(+)
create mode 100644 gnu/packages/patches/miniz-for-pytorch.patch

Toggle diff (96 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index e8b0a0e7be..43da38f4fd 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1689,6 +1689,7 @@ dist_patch_DATA = \
%D%/packages/patches/mingw-w64-reproducible-gendef.patch \
%D%/packages/patches/minisat-friend-declaration.patch \
%D%/packages/patches/minisat-install.patch \
+ %D%/packages/patches/miniz-for-pytorch.patch \
%D%/packages/patches/mit-krb5-hurd.patch \
%D%/packages/patches/mixxx-link-qtscriptbytearray-qtscript.patch \
%D%/packages/patches/mixxx-system-googletest-benchmark.patch \
diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index fcfc2c30f3..589c9bb834 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -2867,3 +2867,24 @@ (define-public miniz
specification standards. It supports the most commonly used functions
exported by the zlib library.")
(license license:expat)))
+
+(define-public miniz-for-pytorch
+ (package
+ (inherit miniz)
+ (version "pytorch-2.2.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/richgel999/miniz")
+ (commit "2.2.0")))
+ (file-name (git-file-name (package-name miniz) version))
+ (sha256
+ (base32
+ "09j9ihigfsavgcmk8l36zmbjvdf1x1w7h2v4rkww1qk1sb43y5zf"))
+ (patches (search-patches "miniz-for-pytorch.patch"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments miniz)
+ ((#:configure-flags flags '())
+ ;; The changes break the examples.
+ `(cons "-DBUILD_EXAMPLES=OFF" ,flags))))
+ (properties '((hidden? . #t)))))
diff --git a/gnu/packages/patches/miniz-for-pytorch.patch b/gnu/packages/patches/miniz-for-pytorch.patch
new file mode 100644
index 0000000000..19fa705a31
--- /dev/null
+++ b/gnu/packages/patches/miniz-for-pytorch.patch
@@ -0,0 +1,47 @@
+Modifications from the internal fork of the python-pytorch package.
+
+diff --git a/miniz.h b/miniz.h
+index 42e1ea2..5869ab9 100644
+--- a/miniz.h
++++ b/miniz.h
+@@ -123,7 +123,7 @@
+ /* If MINIZ_NO_TIME is specified then the ZIP archive functions will not be able to get the current time, or */
+ /* get/set file times, and the C run-time funcs that get/set times won't be called. */
+ /* The current downside is the times written to your archives will be from 1979. */
+-/*#define MINIZ_NO_TIME */
++#define MINIZ_NO_TIME
+
+ /* Define MINIZ_NO_ARCHIVE_APIS to disable all ZIP archive API's. */
+ /*#define MINIZ_NO_ARCHIVE_APIS */
+@@ -135,7 +135,7 @@
+ /*#define MINIZ_NO_ZLIB_APIS */
+
+ /* Define MINIZ_NO_ZLIB_COMPATIBLE_NAME to disable zlib names, to prevent conflicts against stock zlib. */
+-/*#define MINIZ_NO_ZLIB_COMPATIBLE_NAMES */
++#define MINIZ_NO_ZLIB_COMPATIBLE_NAMES
+
+ /* Define MINIZ_NO_MALLOC to disable all calls to malloc, free, and realloc.
+ Note if MINIZ_NO_MALLOC is defined then the user must always provide custom user alloc/free/realloc
+@@ -172,8 +172,7 @@
+ #if !defined(MINIZ_USE_UNALIGNED_LOADS_AND_STORES)
+ #if MINIZ_X86_OR_X64_CPU
+ /* Set MINIZ_USE_UNALIGNED_LOADS_AND_STORES to 1 on CPU's that permit efficient integer loads and stores from unaligned addresses. */
+-#define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 1
+-#define MINIZ_UNALIGNED_USE_MEMCPY
++#define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 0
+ #else
+ #define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 0
+ #endif
+diff --git a/miniz_zip.h b/miniz_zip.h
+index 82502bd..3d92255 100644
+--- a/miniz_zip.h
++++ b/miniz_zip.h
+@@ -150,7 +150,7 @@ typedef enum {
+ MZ_ZIP_TOTAL_ERRORS
+ } mz_zip_error;
+
+-typedef struct
++typedef struct mz_zip_archive
+ {
+ mz_uint64 m_archive_size;
+ mz_uint64 m_central_directory_file_ofs;
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 16/31] gnu: nnpack: Update to commit 70a77f4.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-16-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (nnpack): Update to commit 70a77f4.
[supported-systems]: New field.
---
gnu/packages/machine-learning.scm | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)

Toggle diff (35 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 6da189616d..ea2b7ca648 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -3975,8 +3975,8 @@ (define-public python-umap-learn
(define-public nnpack
(let ((version "0.0")
- (commit "c07e3a0400713d546e0dea2d5466dd22ea389c73")
- (revision "1"))
+ (commit "70a77f485e8b934224f3a79efd8edcd84cd377b8")
+ (revision "2"))
(package
(name "nnpack")
(version (git-version version revision commit))
@@ -3987,7 +3987,7 @@ (define-public nnpack
(file-name (git-file-name name version))
(sha256
(base32
- "0s0kk3a35w3yzf0q447p72350sbsh4qhg6vm3y2djbj4xpg7jc8v"))
+ "0c4pw926279s3rlx7mg4l4vhnfy6dh374n6w7zqhcn0bxpym1hv1"))
(patches (search-patches "nnpack-system-libraries.patch"))))
(build-system cmake-build-system)
;; XXX: The test suite runs but it's very expensive, and on x86_64 CPUs
@@ -4020,6 +4020,8 @@ (define-public nnpack
googletest))
(native-inputs
(list python python-peachpy python-six))
+ ;; Supported for Linux
+ (supported-systems '("x86_64-linux" "armhf-linux" "aarch64-linux"))
(license license:bsd-2))))
(define-public xnnpack
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 14/31] gnu: cpuinfo: Update to commit aa4b216.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-14-david.elsing@posteo.net
* gnu/packages/parallel.scm (cpuinfo): Update to commit aa4b216.
[arguments]: Set the "-DUSE_SYSTEM_LIBS=ON" configure flag instead of using a
patch.
* gnu/packages/patches/cpuinfo-system-libraries.patch: Remove file.
* gnu/local.mk (dist_patch_DATA): Remove it.
---
gnu/local.mk | 1 -
gnu/packages/parallel.scm | 11 ++--
.../patches/cpuinfo-system-libraries.patch | 50 -------------------
3 files changed, 6 insertions(+), 56 deletions(-)
delete mode 100644 gnu/packages/patches/cpuinfo-system-libraries.patch

Toggle diff (87 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 43da38f4fd..75295a8250 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1067,7 +1067,6 @@ dist_patch_DATA = \
%D%/packages/patches/cool-retro-term-wctype.patch \
%D%/packages/patches/coreutils-gnulib-tests.patch \
%D%/packages/patches/cppcheck-disable-char-signedness-test.patch \
- %D%/packages/patches/cpuinfo-system-libraries.patch \
%D%/packages/patches/cpulimit-with-glib-2.32.patch \
%D%/packages/patches/crawl-upgrade-saves.patch \
%D%/packages/patches/crc32c-unbundle-googletest.patch \
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index e1cbe52934..a210ca0313 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -480,8 +480,8 @@ (define-public pthreadpool
(define-public cpuinfo
;; There's currently no tag on this repo.
(let ((version "0.0")
- (revision "2")
- (commit "53298db833c5c5a1598639e9b47cc1a602bbac26"))
+ (revision "3")
+ (commit "aa4b2163b99ac9534194520f70b93eeefb0b3b4e"))
(package
(name "cpuinfo")
(version (git-version version revision commit))
@@ -492,12 +492,13 @@ (define-public cpuinfo
(file-name (git-file-name name version))
(sha256
(base32
- "01kfgxya2w32dz9bd3qm3i2d6nffw0qfyql11rxl7d3g830brj5k"))
- (patches (search-patches "cpuinfo-system-libraries.patch"))))
+ "12x4krkyzxngf1l2ck33lnsp8pyzf6gyjj9mp9cnka9mw3h6617m"))))
(build-system cmake-build-system)
(arguments
(list
- #:configure-flags '(list "-DBUILD_SHARED_LIBS=ON")
+ #:configure-flags
+ '(list "-DBUILD_SHARED_LIBS=ON"
+ "-DUSE_SYSTEM_LIBS=ON")
#:phases
'(modify-phases %standard-phases
(add-after 'unpack 'skip-bad-test
diff --git a/gnu/packages/patches/cpuinfo-system-libraries.patch b/gnu/packages/patches/cpuinfo-system-libraries.patch
deleted file mode 100644
index e25446e9da..0000000000
--- a/gnu/packages/patches/cpuinfo-system-libraries.patch
+++ /dev/null
@@ -1,50 +0,0 @@
-This patch allows the build process to use the provided dependencies instead
-of adding their source as CMake sub-directories (in which case "make install"
-would install googletest's and googlebenchmark's libraries and headers).
-
-diff --git a/CMakeLists.txt b/CMakeLists.txt
-index 57abc26..761c612 100644
---- a/CMakeLists.txt
-+++ b/CMakeLists.txt
-@@ -93,7 +93,7 @@ SET(CONFU_DEPENDENCIES_BINARY_DIR ${CMAKE_BINARY_DIR}/deps
- CACHE PATH "Confu-style dependencies binary directory")
-
- IF(CPUINFO_BUILD_MOCK_TESTS OR CPUINFO_BUILD_UNIT_TESTS)
-- IF(CPUINFO_SUPPORTED_PLATFORM AND NOT DEFINED GOOGLETEST_SOURCE_DIR)
-+ IF(FALSE)
- MESSAGE(STATUS "Downloading Google Test to ${CONFU_DEPENDENCIES_SOURCE_DIR}/googletest (define GOOGLETEST_SOURCE_DIR to avoid it)")
- CONFIGURE_FILE(cmake/DownloadGoogleTest.cmake "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download/CMakeLists.txt")
- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
-@@ -105,7 +105,7 @@ IF(CPUINFO_BUILD_MOCK_TESTS OR CPUINFO_BUILD_UNIT_TESTS)
- ENDIF()
-
- IF(CPUINFO_BUILD_BENCHMARKS)
-- IF(CPUINFO_SUPPORTED_PLATFORM AND NOT DEFINED GOOGLEBENCHMARK_SOURCE_DIR)
-+ IF(FALSE)
- MESSAGE(STATUS "Downloading Google Benchmark to ${CONFU_DEPENDENCIES_SOURCE_DIR}/googlebenchmark (define GOOGLEBENCHMARK_SOURCE_DIR to avoid it)")
- CONFIGURE_FILE(cmake/DownloadGoogleBenchmark.cmake "${CONFU_DEPENDENCIES_BINARY_DIR}/googlebenchmark-download/CMakeLists.txt")
- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
-@@ -271,14 +271,6 @@ INSTALL(TARGETS cpuinfo
-
- # ---[ cpuinfo micro-benchmarks
- IF(CPUINFO_SUPPORTED_PLATFORM AND CPUINFO_BUILD_BENCHMARKS)
-- # ---[ Build google benchmark
-- IF(NOT TARGET benchmark)
-- SET(BENCHMARK_ENABLE_TESTING OFF CACHE BOOL "")
-- ADD_SUBDIRECTORY(
-- "${GOOGLEBENCHMARK_SOURCE_DIR}"
-- "${CONFU_DEPENDENCIES_BINARY_DIR}/googlebenchmark")
-- ENDIF()
--
- IF(CMAKE_SYSTEM_NAME MATCHES "^(Linux|Android)$")
- ADD_EXECUTABLE(get-current-bench bench/get-current.cc)
- TARGET_LINK_LIBRARIES(get-current-bench cpuinfo benchmark)
-@@ -289,7 +281,7 @@ IF(CPUINFO_SUPPORTED_PLATFORM AND CPUINFO_BUILD_BENCHMARKS)
- ENDIF()
-
- IF(CPUINFO_SUPPORTED_PLATFORM)
-- IF(CPUINFO_BUILD_MOCK_TESTS OR CPUINFO_BUILD_UNIT_TESTS)
-+ IF(FALSE)
- # ---[ Build google test
- IF(NOT TARGET gtest)
- IF(MSVC AND NOT CPUINFO_RUNTIME_TYPE STREQUAL "static")
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 18/31] gnu: Add tensorpipe.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-18-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (tensorpipe): New variable.
---
gnu/packages/machine-learning.scm | 36 +++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 7f39b4ea13..ad8b8c9960 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -91,6 +91,7 @@ (define-module (gnu packages machine-learning)
#:use-module (gnu packages image-processing)
#:use-module (gnu packages imagemagick)
#:use-module (gnu packages jupyter)
+ #:use-module (gnu packages libevent)
#:use-module (gnu packages libffi)
#:use-module (gnu packages linux)
#:use-module (gnu packages llvm)
@@ -4100,6 +4101,41 @@ (define-public xnnpack-for-torch2
"-DXNNPACK_BUILD_TESTS=FALSE" ;FIXME: see below
"-DXNNPACK_BUILD_BENCHMARKS=FALSE"))))))
+(define-public tensorpipe
+ (let ((commit "bb1473a4b38b18268e8693044afdb8635bc8351b")
+ (revision "0"))
+ (package
+ (name "tensorpipe")
+ (version (git-version "0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/pytorch/tensorpipe")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0sbpkd69rzybw2j89sjkf4s0j8vkk96d51bsps28894989a75j6v"))
+ (modules '((guix build utils)))
+ (snippet
+ '(delete-file-recursively "third_party"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ #:configure-flags
+ ''("-DBUILD_SHARED_LIBS=ON")
+ ;; There are no tests
+ #:tests? #f))
+ (inputs (list libuv))
+ (native-inputs (list googletest pkg-config pybind11 libnop))
+ (home-page "https://github.com/pytorch/tensorpipe")
+ (synopsis "Tensor-aware point-to-point communication primitive for
+machine learning")
+ (description "TensorPipe provides a tensor-aware channel to transfer
+rich objects from one process to another while using the fastest transport for
+the tensors contained therein.")
+ (license license:bsd-3))))
+
;; Please also update python-torchvision when updating this package.
(define-public python-pytorch
(package
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 27/31] gnu: python-torchvision: Update to 0.17.1.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-27-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (python-torchvision): Update to 0.17.1.
[source]: Add snippet to remove "android" and "ios" directories.
---
gnu/packages/machine-learning.scm | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

Toggle diff (29 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index c1fe0e2812..bdcde1b1b3 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -5134,7 +5134,7 @@ (define-public python-torchmetrics
(define-public python-torchvision
(package
(name "python-torchvision")
- (version "0.15.2")
+ (version "0.17.1")
(source (origin
(method git-fetch)
(uri (git-reference
@@ -5144,7 +5144,12 @@ (define-public python-torchvision
(file-name (git-file-name name version))
(sha256
(base32
- "1cq2s13vkgg9rljjbrm4g33yxq7q5zqp7f4xm5cq624gvs0wxmi8"))))
+ "094jz0ryzh0yjxf687r61r482fdh3bax8ix2csghraps0z1sns1b"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file-recursively "android")
+ (delete-file-recursively "ios")))))
(build-system pyproject-build-system)
(arguments
(list #:tests? #false)) ;the test suite is expensive and there is no easy
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 17/31] gnu: oneapi-dnnl: Update to 3.3.5.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-17-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (oneapi-dnnl): Update to 3.3.5.
[supported-systems]: New field.
---
gnu/packages/machine-learning.scm | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index ea2b7ca648..7f39b4ea13 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -5376,7 +5376,7 @@ (define-public python-brian2tools
(define-public oneapi-dnnl
(package
(name "oneapi-dnnl")
- (version "3.1")
+ (version "3.3.5")
(source
(origin
(method git-fetch)
@@ -5385,7 +5385,7 @@ (define-public oneapi-dnnl
(commit (string-append "v" version))))
(file-name (git-file-name name version))
(sha256
- (base32 "1jgmb5kl0bf4a2zfn94zlb117672r9lvvkkmwl86ihlyr1mpr3d0"))))
+ (base32 "05ra5zziys2hvn29y6ysiqzsf4jr9bf2bci5sc3swvf3bs2y5ihf"))))
(build-system cmake-build-system)
(arguments (if (target-riscv64?)
(list #:configure-flags #~'("-DDNNL_CPU_RUNTIME=SEQ"))
@@ -5395,6 +5395,7 @@ (define-public oneapi-dnnl
(description
"OneAPI Deep Neural Network Library (oneDNN) is a cross-platform
performance library of basic building blocks for deep learning applications.")
+ (supported-systems %64bit-supported-systems)
(license license:asl2.0)))
(define-public python-gguf
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 15/31] gnu: clog: Add "-DUSE_SYSTEM_LIBS=ON" configure flag.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-15-david.elsing@posteo.net
* gnu/packages/parallel.scm (clog)[arguments]: Add "-DUSE_SYSTEM_LIBS=ON"
to #:configure-flags.
* gnu/packages/patches/clog-fix-shared-build.patch: Adjust patch.
---
gnu/packages/parallel.scm | 13 ++++---
.../patches/clog-fix-shared-build.patch | 39 ++-----------------
2 files changed, 12 insertions(+), 40 deletions(-)

Toggle diff (56 lines)
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index a210ca0313..93d82a7d72 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -524,11 +524,14 @@ (define-public clog
(inherit (package-source cpuinfo))
(patches (search-patches "clog-fix-shared-build.patch"))))
(arguments
- (list #:configure-flags #~(list "-DBUILD_SHARED_LIBS=ON")
- #:phases #~(modify-phases %standard-phases
- (add-after 'unpack 'chdir
- (lambda _
- (chdir "deps/clog"))))))
+ (list
+ #:configure-flags
+ ''("-DBUILD_SHARED_LIBS=ON"
+ "-DUSE_SYSTEM_LIBS=ON")
+ #:phases #~(modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _
+ (chdir "deps/clog"))))))
(native-inputs (list googletest))
(inputs '())
(synopsis "C-style logging library based on printf")
diff --git a/gnu/packages/patches/clog-fix-shared-build.patch b/gnu/packages/patches/clog-fix-shared-build.patch
index bf80544b90..eb305339f5 100644
--- a/gnu/packages/patches/clog-fix-shared-build.patch
+++ b/gnu/packages/patches/clog-fix-shared-build.patch
@@ -1,50 +1,19 @@
Author: Antero Mejr <antero@mailbox.org>
-Notes: Disabled function visibility hacks and googletest download. Enabled
-non-static builds.
+Notes: Disabled function visibility hacks. Enabled non-static builds.
diff --git a/deps/clog/CMakeLists.txt b/deps/clog/CMakeLists.txt
-index 083f519..b7b225a 100644
+index 6e50c41..db02e95 100644
--- a/deps/clog/CMakeLists.txt
+++ b/deps/clog/CMakeLists.txt
-@@ -38,20 +38,8 @@ SET(CONFU_DEPENDENCIES_SOURCE_DIR ${CMAKE_SOURCE_DIR}/deps
- SET(CONFU_DEPENDENCIES_BINARY_DIR ${CMAKE_BINARY_DIR}/deps
- CACHE PATH "Confu-style dependencies binary directory")
+@@ -55,7 +55,7 @@ IF(CLOG_BUILD_TESTS)
+ ENDIF()
--IF(CLOG_BUILD_TESTS)
-- IF(NOT DEFINED GOOGLETEST_SOURCE_DIR)
-- MESSAGE(STATUS "Downloading Google Test to ${CONFU_DEPENDENCIES_SOURCE_DIR}/googletest (define GOOGLETEST_SOURCE_DIR to avoid it)")
-- CONFIGURE_FILE(cmake/DownloadGoogleTest.cmake "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download/CMakeLists.txt")
-- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
-- WORKING_DIRECTORY "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download")
-- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" --build .
-- WORKING_DIRECTORY "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download")
-- SET(GOOGLETEST_SOURCE_DIR "${CONFU_DEPENDENCIES_SOURCE_DIR}/googletest" CACHE STRING "Google Test source directory")
-- ENDIF()
--ENDIF()
--
# ---[ clog library
-ADD_LIBRARY(clog STATIC src/clog.c)
+ADD_LIBRARY(clog src/clog.c)
SET_TARGET_PROPERTIES(clog PROPERTIES
C_STANDARD 99
C_EXTENSIONS NO)
-@@ -74,16 +62,6 @@ INSTALL(TARGETS clog
-
- # ---[ clog tests
- IF(CLOG_BUILD_TESTS)
-- # ---[ Build google test
-- IF(NOT TARGET gtest)
-- IF(MSVC AND NOT CLOG_RUNTIME_TYPE STREQUAL "static")
-- SET(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
-- ENDIF()
-- ADD_SUBDIRECTORY(
-- "${GOOGLETEST_SOURCE_DIR}"
-- "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest")
-- ENDIF()
--
- ADD_EXECUTABLE(clog-test test/clog.cc)
- SET_TARGET_PROPERTIES(clog-test PROPERTIES
- CXX_STANDARD 11
diff --git a/deps/clog/include/clog.h b/deps/clog/include/clog.h
index 4143761..aa9000f 100644
--- a/deps/clog/include/clog.h
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 20/31] gnu: Add qnnpack.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-20-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (qnnpack): New variable.
---
gnu/packages/machine-learning.scm | 49 +++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)

Toggle diff (62 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 72b11cca5f..bc1aa308a9 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4025,6 +4025,55 @@ (define-public nnpack
(supported-systems '("x86_64-linux" "armhf-linux" "aarch64-linux"))
(license license:bsd-2))))
+(define-public qnnpack
+ (let ((commit "7d2a4e9931a82adc3814275b6219a03e24e36b4c")
+ (revision "0"))
+ (package
+ (name "qnnpack")
+ (version (git-version "0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/pytorch/qnnpack")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1dgzriiaz9arsrfwhx42y4l74wbzn6xvdmllfb66v4pmvi5gpxc5"))
+ (modules '((guix build utils)))
+ (snippet
+ '(delete-file-recursively "deps"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ ;; The shared library build fails with linker errors, so we build the
+ ;; static library with -fPIC as in the bundled PyTorch version.
+ #:configure-flags
+ ''("-DQNNPACK_LIBRARY_TYPE=static"
+ "-DCMAKE_POSITION_INDEPENDENT_CODE=ON")
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'patch-cmake
+ (lambda _
+ (substitute* "CMakeLists.txt"
+ (("IF.*SOURCE_DIR.*")
+ "IF(FALSE)\n")
+ (("IF\\(NOT TARGET.*")
+ "IF(FALSE)\n")
+ (("TARGET_LINK_LIBRARIES.*(fxdiv|psimd|fp16)\\).*")
+ "")
+ (("(TARGET_LINK_LIBRARIES.*) fp16 (.*)" _ before after)
+ (string-append before " " after))))))))
+ (inputs (list clog cpuinfo fp16 fxdiv psimd pthreadpool))
+ (native-inputs (list googletest googlebenchmark))
+ (home-page "https://github.com/pytorch/qnnpack")
+ (synopsis "Quantized Neural Network PACKage")
+ (description "QNNPACK is a library for low-precision neural network
+inference. It contains the implementation of common neural network operators
+on quantized 8-bit tensors.")
+ (license license:bsd-3))))
+
(define-public xnnpack
;; There's currently no tag on this repo.
(let ((version "0.0")
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 29/31] gnu: Add oneapi-dnnl-for-r-torch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-29-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (oneapi-dnnl-for-r-torch): New variable.
---
gnu/packages/machine-learning.scm | 14 ++++++++++++++
1 file changed, 14 insertions(+)

Toggle diff (27 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 999172fb28..0bcbaab67f 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -5849,6 +5849,20 @@ (define-public oneapi-dnnl
(supported-systems %64bit-supported-systems)
(license license:asl2.0)))
+(define-public oneapi-dnnl-for-r-torch
+ (package
+ (inherit oneapi-dnnl)
+ (version "2.7.3")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/oneapi-src/oneDNN")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name (package-name oneapi-dnnl) version))
+ (sha256
+ (base32 "1zyw5rd8x346bb7gac9a7x3saviw3zvp6aqz2z1l9sv163vmjfz6"))))))
+
(define-public python-gguf
(package
(name "python-gguf")
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 19/31] gnu: Add fbgemm.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-19-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (fbgemm): New variable.
* gnu/packages/patches/fbgemm-use-system-libraries.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
gnu/packages/machine-learning.scm | 31 ++++++++++
.../patches/fbgemm-use-system-libraries.patch | 62 +++++++++++++++++++
3 files changed, 94 insertions(+)
create mode 100644 gnu/packages/patches/fbgemm-use-system-libraries.patch

Toggle diff (124 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 75295a8250..2f685e2f7d 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1174,6 +1174,7 @@ dist_patch_DATA = \
%D%/packages/patches/fasthenry-spUtils.patch \
%D%/packages/patches/fasthenry-spSolve.patch \
%D%/packages/patches/fasthenry-spFactor.patch \
+ %D%/packages/patches/fbgemm-use-system-libraries.patch \
%D%/packages/patches/fbreader-curl-7.62.patch \
%D%/packages/patches/fbreader-fix-icon.patch \
%D%/packages/patches/fenics-dolfin-algorithm.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index ad8b8c9960..72b11cca5f 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4101,6 +4101,37 @@ (define-public xnnpack-for-torch2
"-DXNNPACK_BUILD_TESTS=FALSE" ;FIXME: see below
"-DXNNPACK_BUILD_BENCHMARKS=FALSE"))))))
+(define-public fbgemm
+ (package
+ (name "fbgemm")
+ (version "0.6.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/pytorch/fbgemm")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0mw30v55aicqdbh3xwfj9p8f38nw70ks5cxiwpgwjsk0dylah9rf"))
+ (patches (search-patches "fbgemm-use-system-libraries.patch"))
+ (modules '((guix build utils)))
+ (snippet
+ '(delete-file-recursively "third_party"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ #:configure-flags
+ ''("-DFBGEMM_LIBRARY_TYPE=shared")))
+ (inputs (list asmjit cpuinfo))
+ (native-inputs (list pkg-config python googletest))
+ (home-page "https://github.com/pytorch/fbgemm")
+ (synopsis "Facebook GEneral Matrix Multiplication")
+ (description "Low-precision, high-performance matrix-matrix
+multiplications and convolution library for server-side inference.")
+ (supported-systems '("x86_64-linux"))
+ (license license:bsd-3)))
+
(define-public tensorpipe
(let ((commit "bb1473a4b38b18268e8693044afdb8635bc8351b")
(revision "0"))
diff --git a/gnu/packages/patches/fbgemm-use-system-libraries.patch b/gnu/packages/patches/fbgemm-use-system-libraries.patch
new file mode 100644
index 0000000000..456bc3021c
--- /dev/null
+++ b/gnu/packages/patches/fbgemm-use-system-libraries.patch
@@ -0,0 +1,62 @@
+Use the asmjit and cpuinfo packages.
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 134523e..b88b0e7 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -235,7 +235,7 @@ message(WARNING "CMAKE_CXX_FLAGS_DEBUG is ${CMAKE_CXX_FLAGS_DEBUG}")
+ message(WARNING "CMAKE_CXX_FLAGS_RELEASE is ${CMAKE_CXX_FLAGS_RELEASE}")
+ message(WARNING "==========")
+
+-if(NOT TARGET asmjit)
++if(FALSE)
+ #Download asmjit from github if ASMJIT_SRC_DIR is not specified.
+ if(NOT DEFINED ASMJIT_SRC_DIR)
+ set(ASMJIT_SRC_DIR "${FBGEMM_SOURCE_DIR}/third_party/asmjit"
+@@ -264,7 +264,7 @@ if(NOT TARGET asmjit)
+ endif()
+ endif()
+
+-if(NOT TARGET cpuinfo)
++if(FALSE)
+ #Download cpuinfo from github if CPUINFO_SOURCE_DIR is not specified.
+ if(NOT DEFINED CPUINFO_SOURCE_DIR)
+ set(CPUINFO_SOURCE_DIR "${FBGEMM_SOURCE_DIR}/third_party/cpuinfo"
+@@ -340,9 +340,12 @@ target_include_directories(fbgemm BEFORE
+ target_link_libraries(fbgemm
+ $<BUILD_INTERFACE:asmjit>
+ $<BUILD_INTERFACE:cpuinfo>)
+-add_dependencies(fbgemm
+- asmjit
+- cpuinfo)
++
++find_package(asmjit)
++find_package(PkgConfig REQUIRED)
++pkg_check_modules(libcpuinfo REQUIRED IMPORTED_TARGET libcpuinfo)
++
++target_link_libraries(fbgemm asmjit::asmjit PkgConfig::libcpuinfo)
+
+ if(OpenMP_FOUND)
+ target_link_libraries(fbgemm OpenMP::OpenMP_CXX)
+diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt
+index 2cffddf..bd4d409 100644
+--- a/test/CMakeLists.txt
++++ b/test/CMakeLists.txt
+@@ -14,7 +14,7 @@ set(CMAKE_C_STANDARD 11)
+ set(CMAKE_C_EXTENSIONS OFF)
+ set(CMAKE_C_STANDARD_REQUIRED ON)
+
+-if(FBGEMM_BUILD_TESTS AND NOT TARGET gtest)
++if(FALSE)
+ #Download Googletest framework from github if
+ #GOOGLETEST_SOURCE_DIR is not specified.
+ if(NOT DEFINED GOOGLETEST_SOURCE_DIR)
+@@ -78,7 +78,7 @@ macro(add_gtest TESTNAME)
+ target_link_libraries(${TESTNAME} ${OpenMP_CXX_LIBRARIES})
+ endif()
+
+- add_dependencies(${TESTNAME} gtest fbgemm)
++ add_dependencies(${TESTNAME} fbgemm)
+ add_test(${TESTNAME} ${TESTNAME})
+ set_target_properties(${TESTNAME} PROPERTIES FOLDER test)
+ endmacro()
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 30/31] gnu: Add qnnpack-pytorch-for-r-torch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-30-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (%python-pytorch-for-r-torch-version): New
variable.
(%python-pytorch-for-r-torch-src): New variable.
(qnnpack-pytorch-for-r-torch): New variable.
---
gnu/packages/machine-learning.scm | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)

Toggle diff (41 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 0bcbaab67f..4a8aa13d50 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4695,6 +4695,34 @@ (define-public python-pytorch
Note: currently this package does not provide GPU support.")
(license license:bsd-3)))
+(define %python-pytorch-for-r-torch-version "2.0.1")
+
+(define %python-pytorch-for-r-torch-src
+ (origin
+ (inherit %python-pytorch-src)
+ (uri (git-reference
+ (url "https://github.com/pytorch/pytorch")
+ (commit (string-append "v" %python-pytorch-for-r-torch-version))))
+ (file-name (git-file-name "python-pytorch"
+ %python-pytorch-for-r-torch-version))
+ (sha256
+ (base32
+ "0iirrn687i7sfv0p0i7dn89x3rf13a7l8y1y5h190h51yjxpxqxa"))))
+
+(define-public qnnpack-pytorch-for-r-torch
+ (package
+ (inherit qnnpack-pytorch)
+ (version (string-append "pytorch-" %python-pytorch-for-r-torch-version))
+ (source
+ (origin
+ (inherit %python-pytorch-for-r-torch-src)
+ (patches '())
+ (modules '((guix build utils)
+ (srfi srfi-26)
+ (ice-9 ftw)))
+ (snippet
+ (origin-snippet (package-source qnnpack-pytorch)))))))
+
(define-public python-pytorch-for-r-torch
(package
(inherit python-pytorch)
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 22/31] gnu: Add ideep-pytorch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-22-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (ideep-pytorch): New variable.
---
gnu/packages/machine-learning.scm | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)

Toggle diff (46 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 759eb7f66f..287f9b1397 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -52,6 +52,7 @@ (define-module (gnu packages machine-learning)
#:use-module (guix download)
#:use-module (guix svn-download)
#:use-module (guix build-system cmake)
+ #:use-module (guix build-system copy)
#:use-module (guix build-system gnu)
#:use-module (guix build-system ocaml)
#:use-module (guix build-system pyproject)
@@ -4245,6 +4246,31 @@ (define-public foxi
contains facebook extensions and is used by PyTorch.")
(license license:expat))))
+(define-public ideep-pytorch
+ (package
+ (name "ideep-pytorch")
+ (version "3.3.2")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/intel/ideep")
+ (commit (string-append "pytorch-rls-v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0y6r938qryi3bnf15rp0fbilsfimdcgmvsa0ygwrn3zifw6386rb"))))
+ (build-system copy-build-system)
+ (arguments
+ (list
+ #:install-plan
+ ''(("include" "include"))))
+ (home-page "https://github.com/intel/ideep")
+ (synopsis "Ideep headers for interal use by PyTorch")
+ (description "This library is used internally as header-only library by
+PyTorch.")
+ (license license:expat)))
+
;; Please also update python-torchvision when updating this package.
(define-public python-pytorch
(package
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 21/31] gnu: Add foxi.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-21-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (foxi): New variable.
* gnu/packages/patches/foxi-fix-build.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
gnu/packages/machine-learning.scm | 29 ++++++++++++
gnu/packages/patches/foxi-fix-build.patch | 55 +++++++++++++++++++++++
3 files changed, 85 insertions(+)
create mode 100644 gnu/packages/patches/foxi-fix-build.patch

Toggle diff (115 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 2f685e2f7d..becd39a58d 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1206,6 +1206,7 @@ dist_patch_DATA = \
%D%/packages/patches/foobillard++-pkg-config.patch \
%D%/packages/patches/foomatic-filters-CVE-2015-8327.patch \
%D%/packages/patches/foomatic-filters-CVE-2015-8560.patch \
+ %D%/packages/patches/foxi-fix-build.patch \
%D%/packages/patches/fp16-implicit-double.patch \
%D%/packages/patches/fp16-system-libraries.patch \
%D%/packages/patches/fpc-reproducibility.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index bc1aa308a9..759eb7f66f 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4216,6 +4216,35 @@ (define-public tensorpipe
the tensors contained therein.")
(license license:bsd-3))))
+(define-public foxi
+ (let
+ ((commit "c278588e34e535f0bb8f00df3880d26928038cad")
+ (revision "0"))
+ (package
+ (name "foxi")
+ (version (git-version "1.4.1" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/houseroad/foxi")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0q3ssm5hmmvwfwx87mnnajbavzgpahybw6rpn8ysr9r095dwgq5a"))
+ (patches (search-patches "foxi-fix-build.patch"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ ;; No tests
+ #:tests? #f))
+ (home-page "https://github.com/houseroad/foxi")
+ (synopsis "ONNXIFI with Facebook Extension")
+ (description "ONNX Interface for Framework Integration is a cross-platform
+API for loading and executing ONNX graphs on optimized backends. This package
+contains facebook extensions and is used by PyTorch.")
+ (license license:expat))))
+
;; Please also update python-torchvision when updating this package.
(define-public python-pytorch
(package
diff --git a/gnu/packages/patches/foxi-fix-build.patch b/gnu/packages/patches/foxi-fix-build.patch
new file mode 100644
index 0000000000..d82090ad38
--- /dev/null
+++ b/gnu/packages/patches/foxi-fix-build.patch
@@ -0,0 +1,55 @@
+Taken from https://github.com/houseroad/foxi/pull/25.
+
+diff --git a/foxi/onnxifi_dummy.c b/foxi/onnxifi_dummy.c
+index 2115af9..73e25fc 100644
+--- a/foxi/onnxifi_dummy.c
++++ b/foxi/onnxifi_dummy.c
+@@ -103,7 +103,10 @@ ONNXIFI_PUBLIC ONNXIFI_CHECK_RESULT onnxStatus ONNXIFI_ABI onnxInitGraph(
+ const void* onnxModel,
+ uint32_t weightCount,
+ const onnxTensorDescriptorV1* weightDescriptors,
+- onnxGraph* graph) {
++ onnxGraph* graph,
++ uint32_t maxSeqLength,
++ void* deferredWeightReader) {
++
+ if (graph == NULL) {
+ return ONNXIFI_STATUS_INVALID_POINTER;
+ }
+@@ -215,6 +218,8 @@ ONNXIFI_PUBLIC ONNXIFI_CHECK_RESULT onnxStatus ONNXIFI_ABI
+ onnxWaitEventFor(onnxEvent event,
+ uint32_t timeoutMs,
+ onnxEventState* eventState,
+- onnxStatus* eventStatus) {
++ onnxStatus* eventStatus,
++ char* message,
++ size_t* messageLength) {
+ return ONNXIFI_STATUS_SUCCESS;
+ }
+\ No newline at end of file
+diff --git a/foxi/onnxifi_wrapper.c b/foxi/onnxifi_wrapper.c
+index 98a9325..abe1440 100644
+--- a/foxi/onnxifi_wrapper.c
++++ b/foxi/onnxifi_wrapper.c
+@@ -761,7 +761,9 @@ ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI onnxInitGraph(
+ const void* onnxModel,
+ uint32_t weightsCount,
+ const onnxTensorDescriptorV1* weightDescriptors,
+- onnxGraph* graph)
++ onnxGraph* graph,
++ uint32_t maxSeqLength,
++ void* deferredWeightReader)
+ {
+ if (graph == NULL) {
+ return ONNXIFI_STATUS_INVALID_POINTER;
+@@ -797,7 +799,9 @@ ONNXIFI_PUBLIC onnxStatus ONNXIFI_ABI onnxInitGraph(
+ onnxModel,
+ weightsCount,
+ weightDescriptors,
+- &graph_wrapper->graph);
++ &graph_wrapper->graph,
++ maxSeqLength,
++ deferredWeightReader);
+ switch (status) {
+ case ONNXIFI_STATUS_SUCCESS:
+ case ONNXIFI_STATUS_FALLBACK:
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 24/31] gnu: Remove xnnpack-for-torch2.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-24-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (xnnpack-for-torch2): Delete variable.
* gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch: Remove file.
* gnu/local.mk (dist_patch_DATA): Remove it.
---
gnu/local.mk | 1 -
gnu/packages/machine-learning.scm | 28 -
.../xnnpack-for-torch2-system-libraries.patch | 2660 -----------------
3 files changed, 2689 deletions(-)
delete mode 100644 gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch

Toggle diff (116 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 6d5001c617..a66d00919f 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -2206,7 +2206,6 @@ dist_patch_DATA = \
%D%/packages/patches/xmonad-dynamic-linking.patch \
%D%/packages/patches/xnnpack-remove-broken-tests.patch \
%D%/packages/patches/xnnpack-system-libraries.patch \
- %D%/packages/patches/xnnpack-for-torch2-system-libraries.patch \
%D%/packages/patches/xplanet-1.3.1-cxx11-eof.patch \
%D%/packages/patches/xplanet-1.3.1-libdisplay_DisplayOutput.cpp.patch \
%D%/packages/patches/xplanet-1.3.1-libimage_gif.c.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index ca3772aca8..24422e6965 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4191,34 +4191,6 @@ (define-public xnnpack
TensorFlow.js, PyTorch, and MediaPipe.")
(license license:bsd-3))))
-(define-public xnnpack-for-torch2
- ;; There's currently no tag on this repo.
- (let ((version "0.0")
- (commit "51a987591a6fc9f0fc0707077f53d763ac132cbf")
- (revision "3"))
- (package
- (inherit xnnpack)
- (name "xnnpack")
- (version (git-version version revision commit))
- (home-page "https://github.com/google/XNNPACK") ;fork of QNNPACK
- (source (origin
- (method git-fetch)
- (uri (git-reference (url home-page) (commit commit)))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "1rzby82xq8d0rl1d148yz88jh9cpsw5c8b2yw7yg39mi7qmr55rm"))
- (patches (search-patches "xnnpack-for-torch2-system-libraries.patch"))))
- (arguments
- (list
- #:tests? #false
- #:configure-flags '(list "-DXNNPACK_USE_SYSTEM_LIBS=YES"
- "-DBUILD_SHARED_LIBS=ON"
- "-DCMAKE_POSITION_INDEPENDENT_CODE=ON"
- "-DXNNPACK_LIBRARY_TYPE=shared"
- "-DXNNPACK_BUILD_TESTS=FALSE" ;FIXME: see below
- "-DXNNPACK_BUILD_BENCHMARKS=FALSE"))))))
-
(define-public fbgemm
(package
(name "fbgemm")
diff --git a/gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch b/gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch
deleted file mode 100644
index a711aa0d6c..0000000000
--- a/gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch
+++ /dev/null
@@ -1,2660 +0,0 @@
-This patch allows the build process to use the provided dependencies instead
-of adding their source as CMake sub-directories (in which case "make install"
-would install googletest's and googlebenchmark's libraries and headers).
-
-diff a/CMakeLists.txt b/CMakeLists.txt
---- a/CMakeLists.txt
-+++ b/CMakeLists.txt 2023-12-27 13:29:04.608138355 +0100
-@@ -850,25 +850,8 @@
- SET_PROPERTY(TARGET pthreadpool PROPERTY IMPORTED_IMPLIB "${PTHREADPOOL_LIBRARY}")
- ENDIF()
- ENDIF()
--TARGET_LINK_LIBRARIES(microkernels-all PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(microkernels-prod PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(hardware-config PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(indirection PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(jit PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(logging PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(microparams-init PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(normalization PRIVATE pthreadpool)
--TARGET_LINK_LIBRARIES(packing PRIVATE pthreadpool)
- IF(XNNPACK_BUILD_LIBRARY)
- TARGET_LINK_LIBRARIES(XNNPACK PUBLIC pthreadpool)
-- TARGET_LINK_LIBRARIES(allocator PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(cache PRIVATE memory pthreadpool)
-- TARGET_LINK_LIBRARIES(subgraph PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(operators PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(operator-run PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(operator-utils PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(mutex PRIVATE pthreadpool)
-- TARGET_LINK_LIBRARIES(memory PRIVATE pthreadpool)
- TARGET_LINK_LIBRARIES(post-operation PUBLIC pthreadpool allocator)
- ENDIF()
-
-@@ -889,12 +872,6 @@
- SET_PROPERTY(TARGET fxdiv PROPERTY LINKER_LANGUAGE C)
- ENDIF()
- ENDIF()
--TARGET_LINK_LIBRARIES(microkernels-all PRIVATE fxdiv)
--TARGET_LINK_LIBRARIES(microkernels-prod PRIVATE fxdiv)
--TARGET_LINK_LIBRARIES(indirection PRIVATE fxdiv)
--IF(XNNPACK_BUILD_LIBRARY)
-- TARGET_LINK_LIBRARIES(XNNPACK PRIVATE fxdiv)
--ENDIF()
-
- # ---[ Configure FP16
- IF(NOT TARGET fp16)
-@@ -913,17 +890,7 @@
- SET_PROPERTY(TARGET fp16 PROPERTY LINKER_LANGUAGE C)
- ENDIF()
- ENDIF()
--TARGET_LINK_LIBRARIES(microkernels-all PRIVATE fp16)
--TARGET_LINK_LIBRARIES(microkernels-prod PRIVATE fp16)
--TARGET_LINK_LIBRARIES(microparams-init PRIVATE fp16)
--TARGET_LINK_LIBRARIES(packing PRIVATE fp16)
--TARGET_LINK_LIBRARIES(indirection PRIVATE fp16)
- IF(XNNPACK_BUILD_LIBRARY)
-- TARGET_LINK_LIBRARIES(subgraph PRIVATE fp16)
-- TARGET_LINK_LIBRARIES(operators PRIVATE fp16)
-- TARGET_LINK_LIBRARIES(operator-run PRIVATE fp16)
--
-- TARGET_LINK_LIBRARIES(XNNPACK PRIVATE fp16)
- INSTALL(TARGETS XNNPACK
- LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
- ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
-@@ -943,7 +910,7 @@
- # Helper libraries
- ADD_LIBRARY(gemm-microkernel-tester STATIC test/gemm-microkernel-tester.cc)
- TARGET_INCLUDE_DIRECTORIES(gemm-microkernel-tester PRIVATE . include src test)
-- TARGET_LINK_LIBRARIES(gemm-microkernel-tester PRIVATE fp16 pthreadpool gtest)
-+ TARGET_LINK_LIBRARIES(gemm-microkernel-tester PRIVATE gtest)
- TARGET_LINK_LIBRARIES(gemm-microkernel-tester PRIVATE jit packing)
- IF(XNNPACK_BUILD_LIBRARY)
- TARGET_LINK_LIBRARIES(gemm-microkernel-tester PRIVATE XNNPACK post-operation)
-@@ -960,22 +927,22 @@
- # ---[ Build operator-level unit tests
- ADD_EXECUTABLE(abs-nc-test test/abs-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(abs-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(abs-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(abs-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME abs-nc-test COMMAND abs-nc-test)
-
- ADD_EXECUTABLE(abs-nc-eager-test test/abs-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(abs-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(abs-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(abs-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME abs-nc-eager-test COMMAND abs-nc-eager-test)
-
- ADD_EXECUTABLE(add-nd-test test/add-nd.cc)
- TARGET_INCLUDE_DIRECTORIES(add-nd-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(add-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(add-nd-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME add-nd-test COMMAND add-nd-test)
-
- ADD_EXECUTABLE(add-nd-eager-test test/add-nd-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(add-nd-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(add-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(add-nd-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME add-nd-eager-test COMMAND add-nd-eager-test)
-
- ADD_EXECUTABLE(argmax-pooling-nhwc-test test/argmax-pooling-nhwc.cc)
-@@ -985,27 +952,27 @@
-
- ADD_EXECUTABLE(average-pooling-nhwc-test test/average-pooling-nhwc.cc)
- TARGET_INCLUDE_DIRECTORIES(average-pooling-nhwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(average-pooling-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(average-pooling-nhwc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME average-pooling-nhwc-test COMMAND average-pooling-nhwc-test)
-
- ADD_EXECUTABLE(bankers-rounding-nc-test test/bankers-rounding-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(bankers-rounding-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(bankers-rounding-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(bankers-rounding-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME bankers-rounding-nc-test COMMAND bankers-rounding-nc-test)
-
- ADD_EXECUTABLE(bankers-rounding-nc-eager-test test/bankers-rounding-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(bankers-rounding-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(bankers-rounding-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(bankers-rounding-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME bankers-rounding-nc-eager-test COMMAND bankers-rounding-nc-eager-test)
-
- ADD_EXECUTABLE(ceiling-nc-test test/ceiling-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(ceiling-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(ceiling-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(ceiling-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME ceiling-nc-test COMMAND ceiling-nc-test)
-
- ADD_EXECUTABLE(ceiling-nc-eager-test test/ceiling-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(ceiling-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(ceiling-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(ceiling-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME ceiling-nc-eager-test COMMAND ceiling-nc-eager-test)
-
- ADD_EXECUTABLE(channel-shuffle-nc-test test/channel-shuffle-nc.cc)
-@@ -1015,12 +982,12 @@
-
- ADD_EXECUTABLE(clamp-nc-test test/clamp-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(clamp-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(clamp-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(clamp-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME clamp-nc-test COMMAND clamp-nc-test)
-
- ADD_EXECUTABLE(clamp-nc-eager-test test/clamp-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(clamp-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(clamp-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(clamp-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME clamp-nc-eager-test COMMAND clamp-nc-eager-test)
-
- ADD_EXECUTABLE(constant-pad-nd-test test/constant-pad-nd.cc)
-@@ -1035,25 +1002,25 @@
-
- ADD_EXECUTABLE(convert-nc-test test/convert-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(convert-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(convert-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(convert-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME convert-nc-test COMMAND convert-nc-test)
-
- ADD_EXECUTABLE(convert-nc-eager-test test/convert-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(convert-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(convert-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(convert-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME convert-nc-eager-test COMMAND convert-nc-eager-test)
-
- ADD_EXECUTABLE(convolution-nhwc-test test/convolution-nhwc.cc)
- SET_TARGET_PROPERTIES(convolution-nhwc-test PROPERTIES CXX_EXTENSIONS YES)
- TARGET_INCLUDE_DIRECTORIES(convolution-nhwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(convolution-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(convolution-nhwc-test PRIVATE XNNPACK gtest gtest_main)
- TARGET_LINK_LIBRARIES(convolution-nhwc-test PRIVATE convolution-test-helpers)
- ADD_TEST(NAME convolution-nhwc-test COMMAND convolution-nhwc-test)
-
- ADD_EXECUTABLE(convolution-nchw-test test/convolution-nchw.cc)
- SET_TARGET_PROPERTIES(convolution-nchw-test PROPERTIES CXX_EXTENSIONS YES)
- TARGET_INCLUDE_DIRECTORIES(convolution-nchw-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(convolution-nchw-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(convolution-nchw-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME convolution-nchw-test COMMAND convolution-nchw-test)
-
- ADD_EXECUTABLE(copy-nc-test test/copy-nc.cc)
-@@ -1068,7 +1035,7 @@
-
- ADD_EXECUTABLE(deconvolution-nhwc-test test/deconvolution-nhwc.cc)
- TARGET_INCLUDE_DIRECTORIES(deconvolution-nhwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(deconvolution-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(deconvolution-nhwc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME deconvolution-nhwc-test COMMAND deconvolution-nhwc-test)
-
- ADD_EXECUTABLE(depth-to-space-nchw2nhwc-test test/depth-to-space-nchw2nhwc.cc)
-@@ -1083,197 +1050,197 @@
-
- ADD_EXECUTABLE(divide-nd-test test/divide-nd.cc)
- TARGET_INCLUDE_DIRECTORIES(divide-nd-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(divide-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(divide-nd-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME divide-nd-test COMMAND divide-nd-test)
-
- ADD_EXECUTABLE(divide-nd-eager-test test/divide-nd-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(divide-nd-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(divide-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(divide-nd-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME divide-nd-eager-test COMMAND divide-nd-eager-test)
-
- ADD_EXECUTABLE(elu-nc-test test/elu-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(elu-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(elu-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(elu-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME elu-nc-test COMMAND elu-nc-test)
-
- ADD_EXECUTABLE(elu-nc-eager-test test/elu-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(elu-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(elu-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(elu-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME elu-nc-eager-test COMMAND elu-nc-eager-test)
-
- ADD_EXECUTABLE(fully-connected-nc-test test/fully-connected-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(fully-connected-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(fully-connected-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(fully-connected-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME fully-connected-nc-test COMMAND fully-connected-nc-test)
-
- ADD_EXECUTABLE(floor-nc-test test/floor-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(floor-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(floor-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(floor-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME floor-nc-test COMMAND floor-nc-test)
-
- ADD_EXECUTABLE(floor-nc-eager-test test/floor-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(floor-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(floor-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(floor-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME floor-nc-eager-test COMMAND floor-nc-eager-test)
-
- ADD_EXECUTABLE(global-average-pooling-nwc-test test/global-average-pooling-nwc.cc)
- TARGET_INCLUDE_DIRECTORIES(global-average-pooling-nwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(global-average-pooling-nwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(global-average-pooling-nwc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME global-average-pooling-nwc-test COMMAND global-average-pooling-nwc-test)
-
- ADD_EXECUTABLE(global-average-pooling-ncw-test test/global-average-pooling-ncw.cc)
- TARGET_INCLUDE_DIRECTORIES(global-average-pooling-ncw-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(global-average-pooling-ncw-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(global-average-pooling-ncw-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME global-average-pooling-ncw-test COMMAND global-average-pooling-ncw-test)
-
- ADD_EXECUTABLE(hardswish-nc-test test/hardswish-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(hardswish-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(hardswish-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(hardswish-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME hardswish-nc-test COMMAND hardswish-nc-test)
-
- ADD_EXECUTABLE(hardswish-nc-eager-test test/hardswish-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(hardswish-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(hardswish-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(hardswish-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME hardswish-nc-eager-test COMMAND hardswish-nc-eager-test)
-
- ADD_EXECUTABLE(leaky-relu-nc-test test/leaky-relu-nc.cc)
- TARGET_INCLUDE_DIRECTORIES(leaky-relu-nc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(leaky-relu-nc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(leaky-relu-nc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME leaky-relu-nc-test COMMAND leaky-relu-nc-test)
-
- ADD_EXECUTABLE(leaky-relu-nc-eager-test test/leaky-relu-nc-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(leaky-relu-nc-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(leaky-relu-nc-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(leaky-relu-nc-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME leaky-relu-nc-eager-test COMMAND leaky-relu-nc-eager-test)
-
- ADD_EXECUTABLE(max-pooling-nhwc-test test/max-pooling-nhwc.cc)
- TARGET_INCLUDE_DIRECTORIES(max-pooling-nhwc-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(max-pooling-nhwc-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(max-pooling-nhwc-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME max-pooling-nhwc-test COMMAND max-pooling-nhwc-test)
-
- ADD_EXECUTABLE(maximum-nd-test test/maximum-nd.cc)
- TARGET_INCLUDE_DIRECTORIES(maximum-nd-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(maximum-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(maximum-nd-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME maximum-nd-test COMMAND maximum-nd-test)
-
- ADD_EXECUTABLE(maximum-nd-eager-test test/maximum-nd-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(maximum-nd-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(maximum-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(maximum-nd-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME maximum-nd-eager-test COMMAND maximum-nd-eager-test)
-
- ADD_EXECUTABLE(minimum-nd-test test/minimum-nd.cc)
- TARGET_INCLUDE_DIRECTORIES(minimum-nd-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(minimum-nd-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(minimum-nd-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME minimum-nd-test COMMAND minimum-nd-test)
-
- ADD_EXECUTABLE(minimum-nd-eager-test test/minimum-nd-eager.cc)
- TARGET_INCLUDE_DIRECTORIES(minimum-nd-eager-test PRIVATE src test)
-- TARGET_LINK_LIBRARIES(minimum-nd-eager-test PRIVATE XNNPACK fp16 gtest gtest_main)
-+ TARGET_LINK_LIBRARIES(minimum-nd-eager-test PRIVATE XNNPACK gtest gtest_main)
- ADD_TEST(NAME minim
This message was truncated. Download the full message here.
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 31/31] gnu: python-pytorch-for-r-torch: Adjust to new python-pytorch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-31-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (%python-pytorch-for-r-torch-src): Add
patches.
(python-pytorch-for-r-torch)[version]: Use
%python-pytorch-for-r-torch-version.
[source]: Use %python-pytorch-for-r-torch-src.
[native-inputs]: Use 'modify-inputs'.
[inputs]: Don't replace xnnpack. Replace qnnpack-pytorch with
qnnpack-pytorch-for-r-torch and oneapi-dnnl with oneapi-dnnl-for-r-torch.
[propagated-inputs]: Don't replace onnx and onnx-optimizer.
[supported-systems]: New field.
* gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen: New file.
* gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch: New file.
* gnu/packages/patches/python-pytorch2-system-libraries.patch: Remove file.
* gnu/local.mk (dist_patch_DATA): Register them.
---
gnu/local.mk | 1 -
gnu/packages/machine-learning.scm | 61 +--
...thon-pytorch-for-r-torch-fix-codegen.patch | 167 ++++++++
...pytorch-for-r-torch-system-libraries.patch | 399 ++++++++++++++++++
.../python-pytorch2-system-libraries.patch | 156 -------
5 files changed, 585 insertions(+), 199 deletions(-)
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch
delete mode 100644 gnu/packages/patches/python-pytorch2-system-libraries.patch

Toggle diff (404 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index c0b9b3a150..19d71d1b41 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1812,7 +1812,6 @@ dist_patch_DATA = \
%D%/packages/patches/python-pyreadstat-link-libiconv.patch \
%D%/packages/patches/python-pyls-black-41.patch \
%D%/packages/patches/python-pypdf-annotate-tests-appropriately.patch \
- %D%/packages/patches/python-pytorch2-system-libraries.patch \
%D%/packages/patches/python-sip-include-dirs.patch \
%D%/packages/patches/python-sgmllib3k-assertions.patch \
%D%/packages/patches/python-sphinx-prompt-docutils-0.19.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 4a8aa13d50..213260d0b5 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4707,7 +4707,15 @@ (define %python-pytorch-for-r-torch-src
%python-pytorch-for-r-torch-version))
(sha256
(base32
- "0iirrn687i7sfv0p0i7dn89x3rf13a7l8y1y5h190h51yjxpxqxa"))))
+ "0iirrn687i7sfv0p0i7dn89x3rf13a7l8y1y5h190h51yjxpxqxa"))
+ (patches (search-patches
+ "python-pytorch-for-r-torch-system-libraries.patch"
+ "python-pytorch-runpath.patch"
+ "python-pytorch-without-kineto.patch"
+ ;; Some autogeneration scripts depend on the
+ ;; compile PyTorch library. Therefore, we create
+ ;; dummy versions which are regenerated later.
+ "python-pytorch-for-r-torch-fix-codegen.patch"))))
(define-public qnnpack-pytorch-for-r-torch
(package
@@ -4723,59 +4731,28 @@ (define-public qnnpack-pytorch-for-r-torch
(snippet
(origin-snippet (package-source qnnpack-pytorch)))))))
+;; Keep in sync with r-torch
(define-public python-pytorch-for-r-torch
(package
(inherit python-pytorch)
(name "python-pytorch")
- (version "2.0.1")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/pytorch/pytorch")
- (commit (string-append "v" version))
- (recursive? #t)))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "14m7v54zyd2qg2xk9mqdpbf4ps7091mdzinzh4vq9p5k4bpznj65"))
- (patches (search-patches "python-pytorch2-system-libraries.patch"
- "python-pytorch-runpath.patch"))
- (modules '((guix build utils)))
- (snippet
- '(begin
- ;; XXX: Let's be clear: this package is a bundling fest. We
- ;; delete as much as we can, but there's still a lot left.
- (for-each (lambda (directory)
- (delete-file-recursively
- (string-append "third_party/" directory)))
- '("benchmark" "cpuinfo" "eigen"
-
- ;; FIXME: QNNPACK (of which XNNPACK is a fork)
- ;; needs these.
- ;; "FP16" "FXdiv" "gemmlowp" "psimd"
-
- "gloo" "googletest" "ios-cmake" "NNPACK"
- "onnx" "protobuf" "pthreadpool"
- "pybind11" "python-enum" "python-peachpy"
- "python-six" "tbb" "XNNPACK" "zstd"))
- (substitute* "caffe2/CMakeLists.txt"
- (("target_link_libraries\\(\\$\\{test_name\\}_\\$\\{CPU_CAPABILITY\\} c10 sleef gtest_main\\)")
- "target_link_libraries(${test_name}_${CPU_CAPABILITY} c10 sleef gtest gtest_main)"))
- (substitute* "functorch/CMakeLists.txt"
- (("\\$\\{_rpath_portable_origin\\}/../torch/lib")
- "$ORIGIN/../torch/lib"))))))
+ (version %python-pytorch-for-r-torch-version)
+ (source %python-pytorch-for-r-torch-src)
+ (native-inputs
+ (modify-inputs (package-native-inputs python-pytorch)
+ (replace "ideep-pytorch" ideep-pytorch-for-r-torch)))
(inputs
(modify-inputs (package-inputs python-pytorch)
- (replace "xnnpack" xnnpack-for-torch2)))
+ (replace "qnnpack-pytorch" qnnpack-pytorch-for-r-torch)
+ (replace "oneapi-dnnl" oneapi-dnnl-for-r-torch)))
(propagated-inputs
(modify-inputs (package-propagated-inputs python-pytorch)
(append python-filelock
python-jinja2
python-networkx
python-opt-einsum
- python-sympy)
- (replace "onnx" onnx-for-torch2)
- (replace "onnx-optimizer" onnx-optimizer-for-torch2)))))
+ python-sympy)))
+ (supported-systems %64bit-supported-systems)))
(define-public python-pytorch-geometric
(package
diff --git a/gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch b/gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch
new file mode 100644
index 0000000000..8515e5ab13
--- /dev/null
+++ b/gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch
@@ -0,0 +1,167 @@
+This patch fixes some scripts for generating source files. For
+gen_jit_decompositions.py, gen_mobile_upgraders.py and
+gen_jit_shape_functions.py, which depend on the compiled PyTorch library, the
+option to generate "dummy" source files is added for the initial build, which
+is later corrected. codegen_external.py is patched to avoid duplicate
+functions and add the static keyword as in the existing generated file.
+
+diff --git a/tools/gen_flatbuffers.sh b/tools/gen_flatbuffers.sh
+index cc0263dbbf..ac34e84b82 100644
+--- a/tools/gen_flatbuffers.sh
++++ b/tools/gen_flatbuffers.sh
+@@ -1,13 +1,13 @@
+ #!/bin/bash
+ ROOT=$(pwd)
+-FF_LOCATION="$ROOT/third_party/flatbuffers"
+-cd "$FF_LOCATION" || exit
+-mkdir build
+-cd build || exit
+-cmake ..
+-cmake --build . --target flatc
+-mkdir -p "$ROOT/build/torch/csrc/jit/serialization"
+-./flatc --cpp --gen-mutable --scoped-enums \
++#FF_LOCATION="$ROOT/third_party/flatbuffers"
++#cd "$FF_LOCATION" || exit
++#mkdir build
++#cd build || exit
++#cmake ..
++#cmake --build . --target flatc
++#mkdir -p "$ROOT/build/torch/csrc/jit/serialization"
++flatc --cpp --gen-mutable --scoped-enums \
+ -o "$ROOT/torch/csrc/jit/serialization" \
+ -c "$ROOT/torch/csrc/jit/serialization/mobile_bytecode.fbs"
+ echo '// @generated' >> "$ROOT/torch/csrc/jit/serialization/mobile_bytecode_generated.h"
+diff --git a/torch/csrc/jit/tensorexpr/codegen_external.py b/torch/csrc/jit/tensorexpr/codegen_external.py
+index 120520b139..0c8587f02d 100644
+--- a/torch/csrc/jit/tensorexpr/codegen_external.py
++++ b/torch/csrc/jit/tensorexpr/codegen_external.py
+@@ -16,9 +16,14 @@ def gen_external(native_functions_path, tags_path, external_path):
+ native_functions = parse_native_yaml(native_functions_path, tags_path)
+ func_decls = []
+ func_registrations = []
+- for func in native_functions:
++ done_names = set()
++ for func in native_functions[0]:
+ schema = func.func
+ name = schema.name.name.base
++ if name in done_names:
++ continue
++ else:
++ done_names.add(name)
+ args = schema.arguments
+ # Only supports extern calls for functions with out variants
+ if not schema.is_out_fn():
+@@ -48,7 +53,7 @@ def gen_external(native_functions_path, tags_path, external_path):
+
+ # print(tensor_decls, name, arg_names)
+ func_decl = f"""\
+-void nnc_aten_{name}(
++static void nnc_aten_{name}(
+ int64_t bufs_num,
+ void** buf_data,
+ int64_t* buf_ranks,
+diff --git a/torchgen/decompositions/gen_jit_decompositions.py b/torchgen/decompositions/gen_jit_decompositions.py
+index 7cfbb803f9..2e69bb1868 100644
+--- a/torchgen/decompositions/gen_jit_decompositions.py
++++ b/torchgen/decompositions/gen_jit_decompositions.py
+@@ -1,8 +1,12 @@
+ #!/usr/bin/env python3
+ import os
+ from pathlib import Path
++import sys
+
+-from torch.jit._decompositions import decomposition_table
++if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ from torch.jit._decompositions import decomposition_table
++else:
++ decomposition_table = {}
+
+ # from torchgen.code_template import CodeTemplate
+
+@@ -85,7 +89,7 @@ def write_decomposition_util_file(path: str) -> None:
+
+
+ def main() -> None:
+- pytorch_dir = Path(__file__).resolve().parents[3]
++ pytorch_dir = Path(__file__).resolve().parents[2]
+ upgrader_path = pytorch_dir / "torch" / "csrc" / "jit" / "runtime"
+ write_decomposition_util_file(str(upgrader_path))
+
+diff --git a/torchgen/operator_versions/gen_mobile_upgraders.py b/torchgen/operator_versions/gen_mobile_upgraders.py
+index e5287cffc5..57f3c38096 100644
+--- a/torchgen/operator_versions/gen_mobile_upgraders.py
++++ b/torchgen/operator_versions/gen_mobile_upgraders.py
+@@ -2,10 +2,12 @@
+ import os
+ from enum import Enum
+ from pathlib import Path
++import sys
+ from typing import Any, Dict, List
+
+-import torch
+-from torch.jit.generate_bytecode import generate_upgraders_bytecode
++if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ import torch
++ from torch.jit.generate_bytecode import generate_upgraders_bytecode
+
+ from torchgen.code_template import CodeTemplate
+ from torchgen.operator_versions.gen_mobile_upgraders_constant import (
+@@ -262,7 +264,10 @@ def construct_register_size(register_size_from_yaml: int) -> str:
+ def construct_version_maps(
+ upgrader_bytecode_function_to_index_map: Dict[str, Any]
+ ) -> str:
+- version_map = torch._C._get_operator_version_map()
++ if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ version_map = torch._C._get_operator_version_map()
++ else:
++ version_map = {}
+ sorted_version_map_ = sorted(version_map.items(), key=lambda item: item[0]) # type: ignore[no-any-return]
+ sorted_version_map = {name: lst for name, lst in sorted_version_map_}
+
+@@ -379,7 +384,10 @@ def sort_upgrader(upgrader_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+
+ def main() -> None:
+
+- upgrader_list = generate_upgraders_bytecode()
++ if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ upgrader_list = generate_upgraders_bytecode()
++ else:
++ upgrader_list = []
+ sorted_upgrader_list = sort_upgrader(upgrader_list)
+ for up in sorted_upgrader_list:
+ print("after sort upgrader : ", next(iter(up)))
+diff --git a/torchgen/shape_functions/gen_jit_shape_functions.py b/torchgen/shape_functions/gen_jit_shape_functions.py
+index c6336a6951..34e394d818 100644
+--- a/torchgen/shape_functions/gen_jit_shape_functions.py
++++ b/torchgen/shape_functions/gen_jit_shape_functions.py
+@@ -18,16 +18,20 @@ you are in the root directory of the Pytorch git repo"""
+ if not file_path.exists():
+ raise Exception(err_msg)
+
+-spec = importlib.util.spec_from_file_location(module_name, file_path)
+-assert spec is not None
+-module = importlib.util.module_from_spec(spec)
+-sys.modules[module_name] = module
+-assert spec.loader is not None
+-assert module is not None
+-spec.loader.exec_module(module)
+-
+-bounded_compute_graph_mapping = module.bounded_compute_graph_mapping
+-shape_compute_graph_mapping = module.shape_compute_graph_mapping
++if len(sys.argv) < 2 or sys.argv[1] != "dummy":
++ spec = importlib.util.spec_from_file_location(module_name, file_path)
++ assert spec is not None
++ module = importlib.util.module_from_spec(spec)
++ sys.modules[module_name] = module
++ assert spec.loader is not None
++ assert module is not None
++ spec.loader.exec_module(module)
++
++ bounded_compute_graph_mapping = module.bounded_compute_graph_mapping
++ shape_compute_graph_mapping = module.shape_compute_graph_mapping
++else:
++ bounded_compute_graph_mapping = {}
++ shape_compute_graph_mapping = {}
+
+
+ SHAPE_HEADER = r"""
diff --git a/gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch b/gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch
new file mode 100644
index 0000000000..e5d647f70d
--- /dev/null
+++ b/gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch
@@ -0,0 +1,399 @@
+Patch build files to also system libraries instead of bundled ones for the
+libraries not supported or working only by specifying USE_SYSTEM_LIBS. This
+includes using the clog, cpuinfo, fbgemm, foxi, fp16, fxdiv, googletest,
+ideep, miniz, nnpack, oneapi-dnnl, pocketfft, pthreadpool, qnnpack,
+qnnpack-pytorch, tensorpipe, valgrind and xnnpack packages.
+For QNNPACK, two versions were bundled and are required: The upstream one and
+an internal fork (now in the package qnnpack-pytorch).
+
+diff --git a/aten/src/ATen/CMakeLists.txt b/aten/src/ATen/CMakeLists.txt
+index 96fc297..7f27b66 100644
+--- a/aten/src/ATen/CMakeLists.txt
++++ b/aten/src/ATen/CMakeLists.txt
+@@ -362,9 +362,9 @@ if(AT_NNPACK_ENABLED)
+ list(APPEND ATen_CPU_DEPENDENCY_LIBS nnpack) # cpuinfo is added below
+ endif()
+
+-if(MKLDNN_FOUND)
+- list(APPEND ATen_CPU_DEPENDENCY_LIBS ${MKLDNN_LIBRARIES})
+-endif(MKLDNN_FOUND)
++if(USE_MKLDNN)
++ list(APPEND ATen_CPU_DEPENDENCY_LIBS DNNL::dnnl)
++endif(USE_MKLDNN)
+
+ list(APPEND ATen_CPU_DEPENDENCY_LIBS cpuinfo)
+
+diff --git a/caffe2/CMakeLists.txt b/caffe2/CMakeLists.txt
+index 221e3f3..417f601 100644
+--- a/caffe2/CMakeLists.txt
++++ b/caffe2/CMakeLists.txt
+@@ -110,9 +110,6 @@ if(NOT MSVC AND USE_XNNPACK)
+ if(NOT TARGET fxdiv)
+ set(FXDIV_BUILD_TESTS OFF CACHE BOOL "")
+ set(FXDIV_BUILD_BENCHMARKS OFF CACHE BOOL "")
+- add_subdirectory(
+- "${FXDIV_SOURCE_DIR}"
+- "${CMAKE_BINARY_DIR}/FXdiv")
+ endif()
+ endif()
+
+@@ -975,7 +972,6 @@ elseif(USE_CUDA)
+ endif()
+
+ if(NOT MSVC AND USE_XNNPACK)
+- TARGET_LINK_LIBRARIES(torch_cpu PRIVATE fxdiv)
+ endif()
+
+ # ==========================================================
+@@ -1314,6 +1310,7 @@ target_link_libraries(torch_cpu PUBLIC c10)
+ target_link_libraries(torch_cpu PUBLIC ${Caffe2_PUBLIC_DEPENDENCY_LIBS})
+ target_link_libraries(torch_cpu PRIVATE ${Caffe2_DEPENDENCY_LIBS})
+ target_link_libraries(torch_cpu PRIVATE ${Caffe2_DEPENDENCY_WHOLE_LINK_LIBS})
++target_link_libraries(torch_cpu PRIVATE miniz clog)
+ target_include_directories(torch_cpu INTERFACE $<INSTALL_INTERFACE:include>)
+ target_include_directories(torch_cpu PRIVATE ${Caffe2_CPU_INCLUDE})
+ target_include_directories(torch_cpu SYSTEM PRIVATE "${Caffe2_DEPENDENCY_INCLUDE}")
+@@ -1570,7 +1567,7 @@ if(BUILD_STATIC_RUNTIME_BENCHMARK)
+ add_executable(static_runtime_bench "${STATIC_RUNTIME_BENCHMARK_SRCS}")
+ add_executable(static_runtime_test "${STATIC_RUNTIME_TEST_SRCS}")
+ target_link_libraries(static_runtime_bench torch_library benchmark)
+- target_link_libraries(static_runtime_test torch_library gtest_main)
++ target_link_libraries(static_runtime_test torch_library gtest_main gtest)
+ endif()
+
+ if(BUILD_TENSOREXPR_BENCHMARK)
+@@ -1601,7 +1598,7 @@ if(BUILD_MOBILE_TEST)
+ foreach(test_src ${ATen_MOBILE_TEST_SRCS})
+ get_filename_component(test_name ${test_src} NAME_WE)
+ add_executable(${test_name} "${test_src}")
+- target_link_libraries(${test_name} torch_library gtest_main)
++ target_link_libraries(${test_name} torch_library gtest_main gtest)
+ target_include_directories(${test_name} PRIVATE $<INSTALL_INTERFACE:include>)
+ target_include_directories(${test_name} PRIVATE $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/include>)
+ target_include_directories(${test_name} PRIVATE ${ATen_CPU_INCLUDE})
+@@ -1622,13 +1619,13 @@ if(BUILD_TEST)
+ if(NOT MSVC)
+ add_executable(${test_name}_${CPU_CAPABILITY} "${test_src}" ../aten/src/ATen/native/quantized/AffineQuantizerBase.cpp)
+ # TODO: Get rid of c10 dependency (which is only needed for the implementation of AT_ERROR)
+- target_link_libraries(${test_name}_${CPU_CAPABILITY} c10 sleef gtest_main)
++ target_link_libraries(${test_name}_${CPU_CAPABILITY} c10 sleef gtest_main gtest)
+ if(USE_FBGEMM)
+ target_link_libraries(${test_name}_${CPU_CAPABILITY} fbgemm)
+ endif()
+ else()
+ add_executable(${test_name}_${CPU_CAPABILITY} "${test_src}")
+- target_link_libraries(${test_name}_${CPU_CAPABILITY} torch_library gtest_main)
++ target_link_libraries(${test_name}_${CPU_CAPABILITY} torch_library gtest_main gtest)
+ endif()
+ target_include_directories(${test_name}_${CPU_CAPABILITY} PRIVATE $<INSTALL_INTERFACE:include>)
+ target_include_directories(${test_name}_${CPU_CAPABILITY} PRIVATE $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/include>)
+@@ -1645,7 +1642,7 @@ if(BUILD_TEST)
+ foreach(test_src ${Caffe2_CPU_TEST_SRCS})
+ get_filename_component(test_name ${test_src} NAME_WE)
+ add_executable(${test_name} "${test_src}")
+- target_link_libraries(${test_name} torch_library gtest_main)
++ target_link_libraries(${test_name} torch_library gtest_main gtest)
+ target_include_directories(${test_name} PRIVATE $<INSTALL_INTERFACE:include>)
+ target_include_directories(${test_name} PRIVATE $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/include>)
+ target_include_directories(${test_name} PRIVATE ${Caffe2_CPU_INCLUDE})
+@@ -1703,7 +1700,7 @@ if(BUILD_TEST)
+ foreach(test_src ${Caffe2_VULKAN_TEST_SRCS})
+ get_filename_component(test_name ${test_src} NAME_WE)
+ add_executable(${test_name} "${test_src}")
+- target_link_libraries(${test_name} torch_library gtest_main)
++ target_link_libraries(${test_name} torch_library gtest_main gtest)
+ target_include_directories(${test_name} PRIVATE $<INSTALL_INTERFACE:include>)
+ target_include_directories(${test_name} PRIVATE ${Caffe2_CPU_INCLUDE})
+ add_test(NAME ${test_name} COMMAND $<TARGET_FILE:${test_name}>)
+diff --git a/caffe2/serialize/CMakeLists.txt b/caffe2/serialize/CMakeLists.txt
+index 1552b59..67e1a9a 100644
+--- a/caffe2/serialize/CMakeLists.txt
++++ b/caffe2/serialize/CMakeLists.txt
+@@ -2,7 +2,6 @@ file(GLOB tmp *_test.cc)
+
+ set(Caffe2_CPU_TEST_SRCS ${Caffe2_CPU_TEST_SRCS} ${tmp})
+ list(APPEND Caffe2_CPU_SRCS
+- ${PROJECT_SOURCE_DIR}/third_party/miniz-2.1.0/miniz.c
+ ${CMAKE_CURRENT_SOURCE_DIR}/inline_container.cc
+ ${CMAKE_CURRENT_SOURCE_DIR}/istream_adapter.cc
+ ${CMAKE_CURRENT_SOURCE_DIR}/file_adapter.cc
+diff --git a/cmake/Dependencies.cmake b/cmake/Dependencies.cmake
+index 8c0e3c2..d65576a 100644
+--- a/cmake
This message was truncated. Download the full message here.
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 28/31] gnu: Add ideep-pytorch-for-r-torch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-28-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (ideep-pytorch-for-r-torch): New variable.
---
gnu/packages/machine-learning.scm | 15 +++++++++++++++
1 file changed, 15 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index bdcde1b1b3..999172fb28 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4313,6 +4313,21 @@ (define-public ideep-pytorch
PyTorch.")
(license license:expat)))
+(define-public ideep-pytorch-for-r-torch
+ (package
+ (inherit ideep-pytorch)
+ (version "2.7.3-1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/intel/ideep")
+ (commit (string-append "pytorch-rls-v" version))))
+ (file-name (git-file-name (package-name ideep-pytorch) version))
+ (sha256
+ (base32
+ "0hdpkhcjry22fjx2zg2r48v7f4ljrclzj0li2pgk76kvyblfbyvm"))))))
+
(define %python-pytorch-version "2.2.1")
(define %python-pytorch-src
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 25/31] gnu: Add qnnpack-pytorch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-25-david.elsing@posteo.net
This is an internal fork of QNNPACK in the PyTorch source tree.

* gnu/packages/machine-learning.scm (%python-pytorch-version): New variable.
(%python-pytorch-src): New variable.
(qnnpack-pytorch): New variable.
---
gnu/packages/machine-learning.scm | 127 ++++++++++++++++++++++++++++++
1 file changed, 127 insertions(+)

Toggle diff (140 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 24422e6965..6b62d7016b 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4311,6 +4311,133 @@ (define-public ideep-pytorch
PyTorch.")
(license license:expat)))
+(define %python-pytorch-version "2.2.1")
+
+(define %python-pytorch-src
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/pytorch/pytorch")
+ (commit (string-append "v" %python-pytorch-version))))
+ (file-name (git-file-name "python-pytorch" %python-pytorch-version))
+ (sha256
+ (base32
+ "03mm0pwwb5lxdsmmiw3cch9fijgjw81kmmc4ln9rlyazkm7l1r48"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ ;; Bundled or unused code
+ (for-each
+ (lambda (dir)
+ (when (file-exists? dir)
+ (delete-file-recursively dir)))
+ '("android"
+ "aten/src/ATen/native/cuda/cutlass_extensions"
+ "aten/src/ATen/native/quantized/cpu/qnnpack"
+ "caffe2/mobile/contrib/libopencl-stub"
+ "caffe2/mobile/contrib/libvulkan-stub"
+ "third_party"))
+
+ ;; Autogenerated files
+ (for-each
+ delete-file
+ '("aten/src/ATen/nnapi/nnapi_wrapper.cpp"
+ "aten/src/ATen/nnapi/nnapi_wrapper.h"
+ "caffe2/mobile/contrib/ios/mpscnn/mpscnn_kernels.h"
+ "caffe2/proto/caffe2_legacy_pb2.pyi"
+ "caffe2/proto/caffe2_pb2.pyi"
+ "caffe2/proto/hsm_pb2.pyi"
+ "caffe2/proto/metanet_pb2.pyi"
+ "caffe2/proto/predictor_consts_pb2.pyi"
+ "caffe2/proto/prof_dag_pb2.pyi"
+ "caffe2/proto/torch_pb2.pyi"
+ ;; These files contain just lists of floating point values and
+ ;; might be as well hand-written.
+ ;; "test/cpp/api/init_baseline.h"
+ ;; "test/cpp/api/optim_baseline.h"
+ "test/mobile/test_upgrader_bytecode_table_example.cpp"
+ "torch/csrc/jit/mobile/upgrader_mobile.cpp"
+ "torch/csrc/jit/runtime/decomposition_registry_util.cpp"
+ "torch/csrc/jit/runtime/serialized_shape_function_registry.cpp"
+ "torch/csrc/jit/tensorexpr/external_functions_codegen.cpp"
+ "torch/csrc/jit/serialization/mobile_bytecode_generated.h"))
+ (delete-file-recursively ".github")
+ (for-each
+ (lambda (dir)
+ (for-each
+ delete-file
+ (find-files dir "\\.cu$")))
+ '("aten/src/ATen/native/transformers/cuda/flash_attn/kernels"
+ "aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels"))))))
+
+(define-public qnnpack-pytorch
+ (package
+ (inherit qnnpack)
+ (name "qnnpack-pytorch")
+ (version (string-append "pytorch-" %python-pytorch-version))
+ (source
+ (origin
+ (inherit %python-pytorch-src)
+ (patches '())
+ (modules '((guix build utils)
+ (srfi srfi-26)
+ (ice-9 ftw)))
+ (snippet
+ '(begin
+ (rename-file "aten/src/ATen/native/quantized/cpu/qnnpack"
+ "../qnnpack")
+ (let ((outdir (getcwd)))
+ (chdir "..")
+ (rename-file outdir "dummy")
+ (rename-file "qnnpack" outdir)
+ (chdir outdir)
+ (delete-file-recursively "deps"))))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments qnnpack)
+ ((#:phases phases #~%standard-phases)
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'patch-cmake
+ (lambda _
+ (substitute* "CMakeLists.txt"
+ (("project\\(.*" orig)
+ (apply
+ string-append
+ orig "\n"
+ (map (lambda (name)
+ (string-append
+ "option(" name " \"\" ON)\n"))
+ '("USE_SYSTEM_CPUINFO" "USE_SYSTEM_FP16" "USE_SYSTEM_FXDIV"
+ "USE_SYSTEM_PSIMD" "USE_SYSTEM_PTHREADPOOL"))))
+ (("if.*SOURCE_DIR.*")
+ "if(FALSE)\n")
+ (("if\\(NOT TARGET (clog|gtest|benchmark).*")
+ "if(FALSE)\n")
+ (("target_link_libraries.*(fxdiv|psimd|fp16)\\).*")
+ "")
+ (("(target_link_libraries.*) fp16 (.*)" _ before after)
+ (string-append before " " after)))))
+ (add-after 'unpack 'fix-cstring-include
+ (lambda _
+ (substitute* "include/pack_block_sparse.h"
+ (("#include.*<vector>.*" orig)
+ (string-append orig "\n#include <cstring>\n")))))
+ (add-after 'install 'install-missing-headers
+ (lambda _
+ (for-each
+ (lambda (name)
+ (install-file (string-append "../source/include/" name)
+ (string-append #$output "/include")))
+ '("pack_block_sparse.h"
+ "pytorch_qnnpack.h"
+ "qnnpack_func.h"))
+ (copy-recursively
+ "../source/src/qnnpack"
+ (string-append #$output "/include/qnnpack"))))))
+ ;; Some tests occasionally fail on i686 due to floating point rounding.
+ ((#:tests? _ #t)
+ (not (string-prefix? "i686" (or (%current-target-system)
+ (%current-system)))))))))
+
;; Please also update python-torchvision when updating this package.
(define-public python-pytorch
(package
--
2.41.0
D
D
David Elsing wrote on 12 Mar 23:51 +0100
[PATCH v2 26/31] gnu: python-pytorch: Update to 2.2.1 and unbundle dependencies.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240312225211.16427-26-david.elsing@posteo.net
Autogenerated files are also regenerated. The tests can be run, but are
disabled, as they require a long time.

* gnu/packages/machine-learning.scm (python-pytorch): Update to 2.2.1.
[version]: Use %python-pytorch-version.
[source]: Use %python-pytorch-src.
[arguments]: Remove 'make-things-writable phase. Add 'cmake-patches,
'set-max-jobs, 'codegen1, 'codegen2, 'build2, 'install2 phases. Adjust
'use-system-libraries and 'check phases.
[native-inputs]: Add doxygen, ideep-pytorch, pocketfft-cpp, python-expecttest,
python-pytest-flakefinder, python-pytest-rerunfailures-13,
python-pytest-shard, python-pytest-xdist, python-hypothesis,
python-types-dataclasses, python-typing-extensions-4.10 and valgrind.
[inputs]: Add asmjit, clog, flatbuffers-next, foxi, fxdiv, libuv,
miniz-for-pytorch, qnnpack, qnnpack-pytorch, fbgemm and oneapi-dnnl. Use
nnpack, fbgemm and oneapi-dnnl only for supported systems.
[propagated-inputs]: Add python-filelock, python-fsspec, python-jinja2,
python-networkx, python-optree, python-packaging, python-psutil and
python-sympy. Remove python-cffi and python-six.
(%python-pytorch-src)[source]: Add patches.
* gnu/packages/patches/python-pytorch-runpath.patch: Adjust patch.
* gnu/packages/patches/python-pytorch-system-libraries.patch: Adjust patch.
* gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch: Remove file.
* gnu/packages/patches/python-pytorch-fix-codegen.patch: New file.
* gnu/packages/patches/python-pytorch-without-kineto: New file.
* gnu/local.mk (dist_patch_DATA): Register them.
---
gnu/local.mk | 3 +-
gnu/packages/machine-learning.scm | 333 +++++++++++-------
...ython-pytorch-1.9.0-system-libraries.patch | 139 --------
.../patches/python-pytorch-fix-codegen.patch | 167 +++++++++
.../patches/python-pytorch-runpath.patch | 19 +-
.../python-pytorch-system-libraries.patch | 317 +++++++++++++++--
.../python-pytorch-without-kineto.patch | 60 ++++
7 files changed, 737 insertions(+), 301 deletions(-)
delete mode 100644 gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch
create mode 100644 gnu/packages/patches/python-pytorch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-without-kineto.patch

Toggle diff (400 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index a66d00919f..c0b9b3a150 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1929,9 +1929,10 @@ dist_patch_DATA = \
%D%/packages/patches/python-pyan3-fix-absolute-path-bug.patch \
%D%/packages/patches/python-pyan3-fix-positional-arguments.patch \
%D%/packages/patches/python-pygpgme-fix-pinentry-tests.patch \
+ %D%/packages/patches/python-pytorch-fix-codegen.patch \
%D%/packages/patches/python-pytorch-runpath.patch \
%D%/packages/patches/python-pytorch-system-libraries.patch \
- %D%/packages/patches/python-pytorch-1.9.0-system-libraries.patch \
+ %D%/packages/patches/python-pytorch-without-kineto.patch \
%D%/packages/patches/python-robotframework-atest.patch \
%D%/packages/patches/python-robotframework-source-date-epoch.patch \
%D%/packages/patches/python-robotframework-sshlibrary-rf5-compat.patch \
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 6b62d7016b..c1fe0e2812 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -105,6 +105,7 @@ (define-module (gnu packages machine-learning)
#:use-module (gnu packages parallel)
#:use-module (gnu packages perl)
#:use-module (gnu packages pkg-config)
+ #:use-module (gnu packages pretty-print)
#:use-module (gnu packages protobuf)
#:use-module (gnu packages pulseaudio)
#:use-module (gnu packages python)
@@ -123,6 +124,7 @@ (define-module (gnu packages machine-learning)
#:use-module (gnu packages swig)
#:use-module (gnu packages time)
#:use-module (gnu packages tls)
+ #:use-module (gnu packages valgrind)
#:use-module (gnu packages vulkan)
#:use-module (gnu packages video)
#:use-module (gnu packages web)
@@ -4323,6 +4325,13 @@ (define %python-pytorch-src
(sha256
(base32
"03mm0pwwb5lxdsmmiw3cch9fijgjw81kmmc4ln9rlyazkm7l1r48"))
+ (patches (search-patches "python-pytorch-system-libraries.patch"
+ "python-pytorch-runpath.patch"
+ "python-pytorch-without-kineto.patch"
+ ;; Some autogeneration scripts depend on the
+ ;; compile PyTorch library. Therefore, we create
+ ;; dummy versions which are regenerated later.
+ "python-pytorch-fix-codegen.patch"))
(modules '((guix build utils)))
(snippet
'(begin
@@ -4442,135 +4451,219 @@ (define-public qnnpack-pytorch
(define-public python-pytorch
(package
(name "python-pytorch")
- (version "1.13.1")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/pytorch/pytorch")
- (commit (string-append "v" version))
- (recursive? #t)))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "17yxjzwp4zp75fz7czgz9acijzw7dpyqcza50v8y1x7hfg2gw369"))
- (patches (search-patches "python-pytorch-system-libraries.patch"
- "python-pytorch-runpath.patch"))
- (modules '((guix build utils)))
- (snippet
- '(begin
- ;; XXX: Let's be clear: this package is a bundling fest. We
- ;; delete as much as we can, but there's still a lot left.
- (for-each (lambda (directory)
- (delete-file-recursively
- (string-append "third_party/" directory)))
- '("benchmark" "cpuinfo" "eigen"
-
- ;; FIXME: QNNPACK (of which XNNPACK is a fork)
- ;; needs these.
- ;; "FP16" "FXdiv" "gemmlowp" "psimd"
-
- "gloo" "googletest" "ios-cmake" "NNPACK"
- "onnx" "protobuf" "pthreadpool"
- "pybind11" "python-enum" "python-peachpy"
- "python-six" "tbb" "XNNPACK" "zstd"))
- (substitute* "functorch/CMakeLists.txt"
- (("\\$\\{_rpath_portable_origin\\}/../torch/lib")
- "$ORIGIN/../torch/lib"))))))
+ (version %python-pytorch-version)
+ (source %python-pytorch-src)
(build-system python-build-system)
(arguments
- '(#:phases (modify-phases %standard-phases
- (add-before 'build 'use-system-libraries
- (lambda* (#:key outputs #:allow-other-keys)
- ;; Tell 'setup.py' to let 'CMakeLists.txt' know that we
- ;; want to use "system libraries" instead of the bundled
- ;; ones.
- (setenv "USE_SYSTEM_LIBS" "1")
-
- (substitute* "cmake/Dependencies.cmake"
- (("if\\(USE_SYSTEM_BIND11\\)")
- "if(TRUE)"))
-
- ;; XXX: Disable that for simplicity for now.
- (setenv "USE_FBGEMM" "0")))
- (add-before 'build 'make-things-writable
- (lambda _
- ;; The 'build_caffe2' function in
- ;; 'tools/build_pytorch_libs.py', called from the
- ;; top-level 'setup.py', needs write access to this
- ;; directory.
- (for-each make-file-writable
- (find-files "caffe2/proto" "."
- #:directories? #t))))
- (replace 'check
- (lambda* (#:key inputs outputs tests? #:allow-other-keys)
- ;; Run the test suite following the instructions in
- ;; 'CONTRIBUTING.md'. XXX: Unfortunately this doesn't
- ;; work, unless you set GUIX_PYTHONPATH presumably.
- (when tests?
- (add-installed-pythonpath inputs outputs)
- (invoke "python" "test/run_test.py"))))
- (add-after 'install 'remove-test-executables
- (lambda* (#:key inputs outputs #:allow-other-keys)
- ;; Remove test executables, but keep other executables
- ;; such as 'torch_shm_manager' and and .so files such as
- ;; 'libtorch_global_deps.so'.
- (let ((python-site (site-packages inputs outputs)))
- (for-each delete-file
- (find-files python-site
- "(^test_cpp_rpc|_test)$")))))
- (add-after 'install 'remove-caffe2-onnx-scripts
- (lambda* (#:key outputs #:allow-other-keys)
- (let* ((out (assoc-ref outputs "out"))
- (bin (string-append out "/bin")))
- ;; Remove 'convert-caffe2-to-onnx' and
- ;; 'convert-onnx-to-caffe2': they seem to be
- ;; deprecated and they cause a failure of the
- ;; 'sanity-check' phase:
- ;;
- ;; ImportError: cannot import name 'metanet_pb2' from partially initialized module 'caffe2.proto' (most likely due to a circular import)
- (for-each delete-file
- (find-files bin "^convert-.*caffe2"))
-
- (substitute* (find-files out "^entry_points\\.txt$")
- (("^convert-.*" all)
- (string-append "# " all "\n")))))))
-
- ;; XXX: Tests attempt to download data such as
- ;; <https://raw.githubusercontent.com/pytorch/test-infra/master/stats/slow-tests.json>.
- ;; We're also missing some Python modules, such as expecttest.
- #:tests? #f))
+ (list
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'cmake-patches
+ (lambda _
+ (substitute* "cmake/Dependencies.cmake"
+ (("#POCKETFFT_INCLUDE_DIR")
+ (string-append
+ #$(this-package-native-input "pocketfft-cpp") "/include"))
+ (("#FP16_INCLUDE_DIR")
+ (string-append
+ #$(this-package-input "fp16") "/include")))))
+ (add-before 'build 'use-system-libraries
+ (lambda _
+ (substitute* '("caffe2/serialize/crc.cc"
+ "caffe2/serialize/inline_container.cc")
+ (("\"miniz\\.h\"") "<miniz/miniz.h>"))
+ (substitute* "aten/src/ATen/native/vulkan/api/Allocator.h"
+ (("<include/vk_mem_alloc.h>")
+ "<vk_mem_alloc.h>"))
+ ;; For Vulkan
+ (substitute* "CMakeLists.txt"
+ (("append_cxx_flag.*-Werror=(return-type|range-loop-construct).*") ""))
+ (substitute*
+ (cons*
+ "torch/csrc/Module.cpp"
+ (map
+ (lambda (name)
+ (string-append
+ "torch/utils/benchmark/utils/valgrind_wrapper/"
+ name))
+ '("compat_bindings.cpp" "timer_callgrind_template.cpp")))
+ (("<callgrind.h>") "<valgrind/callgrind.h>"))
+ (setenv "USE_FFMPEG" "1")
+ (setenv "USE_VULKAN" "1")
+ (setenv "USE_OPENCV" "1")
+ ;; Tell 'setup.py' to let 'CMakeLists.txt' know that we
+ ;; want to use "system libraries" instead of the bundled
+ ;; ones.
+ (setenv "USE_SYSTEM_LIBS" "1")
+ ;; For oneDNN
+ (setenv "USE_MKLDNN" "1")
+ ;; Only works with CUPTI
+ (setenv "USE_KINETO" "0")
+ ;; Prevent CMake error by disabling explicitely
+ (setenv "USE_ITT" "0")))
+ (add-after 'use-system-libraries 'set-max-jobs
+ (lambda _
+ (setenv "MAX_JOBS" (number->string (parallel-job-count)))))
+ (add-after 'set-max-jobs 'codegen1
+ (lambda _
+ (with-directory-excursion "torch/csrc/jit/tensorexpr"
+ (setenv "PYTHONPATH" "../../../..")
+ (invoke "python3" "codegen_external.py")
+ (setenv "PYTHONPATH" #f))
+
+ (invoke "python3" "aten/src/ATen/nnapi/codegen.py")
+
+ (invoke "bash" "tools/gen_flatbuffers.sh")
+
+ ;; Generate dummy files as the generation depends on the compiled
+ ;; library. They are regenerated later.
+ (setenv "PYTHONPATH" ".")
+ (invoke "python3"
+ "torchgen/operator_versions/gen_mobile_upgraders.py"
+ "dummy")
+ (setenv "PYTHONPATH" #f)
+
+ (invoke "python3"
+ "torchgen/shape_functions/gen_jit_shape_functions.py"
+ "dummy")
+
+ (invoke "python3"
+ "torchgen/decompositions/gen_jit_decompositions.py"
+ "dummy")))
+ ;; Properly generate autogenerated files ...
+ (add-after 'install 'codegen2
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (add-installed-pythonpath inputs outputs)
+ (invoke "python3"
+ "torchgen/operator_versions/gen_mobile_upgraders.py")
+ (invoke "python3"
+ "torchgen/shape_functions/gen_jit_shape_functions.py")
+ (invoke "python3"
+ "torchgen/decompositions/gen_jit_decompositions.py")))
+ ;; ... rebuild their dependencies ...
+ (add-after 'codegen2 'build2
+ (lambda _
+ (invoke "python3" "setup.py" "build")))
+ ;; ... and install again.
+ (add-after 'build2 'install2
+ (lambda _
+ (invoke "python3" "setup.py" "install" (string-append "--prefix=" #$output)
+ "--no-compile" "--single-version-externally-managed" "--root=/")
+ (invoke "python" "-m" "compileall"
+ "--invalidation-mode=unchecked-hash" #$output)))
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ ;; Run the test suite following the instructions in
+ ;; 'CONTRIBUTING.md'. Unfortunately this doesn't work, unless
+ ;; you set PYTHONPATH or GUIX_PYTHONPATH, but this is done in
+ ;; the codegen2 phase already.
+ (when tests?
+ (invoke "python3" "test/run_test.py" "--core"))))
+ (add-after 'install2 'remove-test-executables
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ ;; Remove test executables, but keep other executables
+ ;; such as 'torch_shm_manager' and and .so files such as
+ ;; 'libtorch_global_deps.so'.
+ (let ((python-site (site-packages inputs outputs)))
+ (for-each delete-file
+ (find-files python-site
+ "(^test_cpp_rpc|_test)$")))))
+ (add-after 'install2 'remove-caffe2-onnx-scripts
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin")))
+ ;; Remove 'convert-caffe2-to-onnx' and
+ ;; 'convert-onnx-to-caffe2': they seem to be
+ ;; deprecated and they cause a failure of the
+ ;; 'sanity-check' phase:
+ ;;
+ ;; ImportError: cannot import name 'metanet_pb2' from partially initialized module 'caffe2.proto' (most likely due to a circular import)
+ (for-each delete-file
+ (find-files bin "^convert-.*caffe2"))
+
+ (substitute* (find-files out "^entry_points\\.txt$")
+ (("^convert-.*" all)
+ (string-append "# " all "\n")))))))
+
+ ;; Even only the core tests take a very long time to run.
+ #:tests? #f))
(native-inputs
- (list cmake ninja))
+ (list cmake
+ doxygen
+ ideep-pytorch
+ ninja
+ pocketfft-cpp
+ python-expecttest
+ python-pytest-flakefinder
+ python-pytest-rerunfailures-13
+ python-pytest-shard
+ python-pytest-xdist
+ python-hypothesis
+ python-types-dataclasses
+ python-typing-extensions-4.10
+ shaderc
+ valgrind))
(inputs
- (list eigen
- ;; ("fmt" ,fmt)
- fp16
- gemmlowp
- googletest
- googlebenchmark
- gloo
- nnpack
- openblas
- openmpi
- pthreadpool
- protobuf
- pybind11
- sleef
- xnnpack
- zstd))
+ (append
+ (list asmjit
+ clog
+ eigen
+ ffmpeg
+ flatbuffers-next
+ fmt
+ foxi
+ fp16
+ fxdiv
+ gemmlowp
+ gloo
+ googletest
+ googlebenchmark
+ libuv
+ miniz-for-pytorch
+ openblas
+ opencv
+ openmpi
+ pthreadpool
+ protobuf
+ pybind11
+ qnnpack
+ qnnpack-pytorch
+ sleef
+ tensorpipe
+ vulkan-headers
+ vulkan-loader
+ vulkan-memory-allocator
+ xnnpack
+ zstd)
+ ;; TODO: fix build on 32 bit systems
+ (filter
+ (lambda (pkg)
+ (member (or (%current-target-system)
+ (%current-system))
+ (package-transitive-supported-systems pkg)))
+ (list
+ nnpack
+ fbgemm
+ oneapi-dnnl))))
(propagated-inputs
- (list python-astunparse
+ (list cpuinfo
+ onnx ;propagated for its Python modules
+ onnx-optimizer
+ python-astunparse
python-click
+ python-filelock
+ python-fsspec
+ python-future
+ python-jinja2
+ python-networkx
python-numpy
+ python-optree
+ python-packaging
+ python-psutil
python-pyyaml
- python-cffi
- python-typing-extensions
- python-future
- python-six
python-requests
- onnx ;propagated for its Python modules
- onnx-optimizer
- cpuinfo))
+ python-sympy
+ python-typing-extensions))
(home-page "https://pytorch.org/")
(synopsis "Python library for tensor computation and deep neural networks")
(description
diff --git a/gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch b/gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch
deleted file mode 100644
index 76c06520f0..0000000000
--- a/gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch
+++ /dev/null
@@ -1,139 +0,0 @@
-Use our own googletest rather than the bundled one.
-Get NNPACK to use our own PeachPy rather than the bundled one.
-
-diff --git a/cmake/Dependencies.cmake b/cmake/Dependencies.cma
This message was truncated. Download the full message here.
D
D
David Elsing wrote on 20 Mar 23:37 +0100
[PATCH v3 00/32] Unbundle and update python-pytorch
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223737.12883-1-david.elsing@posteo.net
In the QA, several builds failed, mostly because some libraries are not
supported on all architectures, so I added the supported-systems fields
and only include oneapi-dnnl, qnnpack, qnnpack-pytorch, xnnpack, nnpack
and cpuinfo for supported systems as input for python-pytorch.
For x86_64, nnpack requires AVX2 or AVX-512 instructions and fbgemm is
only available for x86_64 with AVX2 or AVX-512 instructions.
Therefore, I removed them from the inputs for x86_64 (fbgemm for all)
and only include them in the python-pytorch-avx package variant.

asmjit fails to build on armhf because GCC runs out of memory. What
should be done here? It's only used by blacksmith, fbgemm and
python-pytorch*.

cpuinfo has a bug on aarch64, which was reported upstream:
I therefore excluded aarch64-linux from supported-systems.

Due to these problems, the python-pytorch package only builds on
x86_64-linux, but it doesn't seem to have been different before.

David Elsing (32):
gnu: asmjit: Update to commit 3ca5c18.
gnu: Add python-typing-extensions-4.10.
gnu: Add python-optree.
gnu: Add python-pytest-flakefinder.
gnu: Add python-pytest-shard.
gnu: Add python-expecttest.
gnu: Add python-pytest-rerunfailures-13.
gnu: Add miniz.
gnu: Add miniz-for-pytorch.
gnu: Add libnop.
gnu: Remove flatbuffers-next-shared.
gnu: python-flatbuffers-next: Update to 23.5.26.
gnu: pthreadpool: Update to commit 178e3e0.
gnu: cpuinfo: Update to commit aa4b216.
gnu: clog: Add "-DUSE_SYSTEM_LIBS=ON" configure flag.
gnu: nnpack: Update to commit 70a77f4.
gnu: oneapi-dnnl: Update to 3.3.5.
gnu: Add tensorpipe.
gnu: Add fbgemm.
gnu: Add qnnpack.
gnu: Add foxi.
gnu: Add ideep-pytorch.
gnu: xnnpack: Update to commit 51a9875.
gnu: Remove xnnpack-for-torch2.
gnu: Add qnnpack-pytorch.
gnu: python-pytorch: Update to 2.2.1 and unbundle dependencies.
gnu: Add python-pytorch-avx.
gnu: python-torchvision: Update to 0.17.1.
gnu: Add ideep-pytorch-for-r-torch.
gnu: Add oneapi-dnnl-for-r-torch.
gnu: Add qnnpack-pytorch-for-r-torch.
gnu: python-pytorch-for-r-torch: Adjust to new python-pytorch.

gnu/local.mk | 10 +-
gnu/packages/compression.scm | 50 +
gnu/packages/cpp.scm | 6 +-
gnu/packages/machine-learning.scm | 964 +++-
gnu/packages/parallel.scm | 34 +-
.../patches/clog-fix-shared-build.patch | 39 +-
.../patches/cpuinfo-system-libraries.patch | 50 -
.../patches/fbgemm-use-system-libraries.patch | 62 +
gnu/packages/patches/foxi-fix-build.patch | 55 +
gnu/packages/patches/miniz-for-pytorch.patch | 47 +
.../patches/python-optree-fix-32-bit.patch | 122 +
...ython-pytorch-1.9.0-system-libraries.patch | 139 -
.../patches/python-pytorch-fix-codegen.patch | 167 +
...thon-pytorch-for-r-torch-fix-codegen.patch | 167 +
...pytorch-for-r-torch-system-libraries.patch | 399 ++
.../patches/python-pytorch-runpath.patch | 19 +-
.../python-pytorch-system-libraries.patch | 317 +-
.../python-pytorch-without-kineto.patch | 60 +
.../python-pytorch2-system-libraries.patch | 156 -
.../xnnpack-for-torch2-system-libraries.patch | 2660 -----------
.../patches/xnnpack-remove-broken-tests.patch | 337 ++
.../patches/xnnpack-system-libraries.patch | 4133 ++++++++++-------
gnu/packages/python-build.scm | 14 +-
gnu/packages/python-check.scm | 89 +
gnu/packages/python-xyz.scm | 30 +
gnu/packages/serialization.scm | 54 +-
26 files changed, 5209 insertions(+), 4971 deletions(-)
delete mode 100644 gnu/packages/patches/cpuinfo-system-libraries.patch
create mode 100644 gnu/packages/patches/fbgemm-use-system-libraries.patch
create mode 100644 gnu/packages/patches/foxi-fix-build.patch
create mode 100644 gnu/packages/patches/miniz-for-pytorch.patch
create mode 100644 gnu/packages/patches/python-optree-fix-32-bit.patch
delete mode 100644 gnu/packages/patches/python-pytorch-1.9.0-system-libraries.patch
create mode 100644 gnu/packages/patches/python-pytorch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-fix-codegen.patch
create mode 100644 gnu/packages/patches/python-pytorch-for-r-torch-system-libraries.patch
create mode 100644 gnu/packages/patches/python-pytorch-without-kineto.patch
delete mode 100644 gnu/packages/patches/python-pytorch2-system-libraries.patch
delete mode 100644 gnu/packages/patches/xnnpack-for-torch2-system-libraries.patch
create mode 100644 gnu/packages/patches/xnnpack-remove-broken-tests.patch

--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 01/32] gnu: asmjit: Update to commit 3ca5c18.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-1-david.elsing@posteo.net
* gnu/packages/cpp.scm (asmjit): Update to commit 3ca5c18.
---
gnu/packages/cpp.scm | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Toggle diff (26 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index d451eea2fd..78c3a195a6 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -154,8 +154,8 @@ (define-public argagg
(license license:expat))))
(define-public asmjit
- (let ((commit "4ec760a3d1f69e32ba460ecd2513f29b8428700b")
- (revision "0"))
+ (let ((commit "3ca5c186bf8922e5fe3018432e93651fd2fa4053")
+ (revision "1"))
(package
(name "asmjit")
(version (git-version "0.0.0" revision commit))
@@ -168,7 +168,7 @@ (define-public asmjit
(commit commit)))
(file-name (git-file-name name version))
(sha256
- (base32 "0skgccbpamcbg1byawfq5n6jzxgj64hnc7jznvk35nkskaaz1nlb"))))
+ (base32 "10k1zc0w8m0vnh52id9qlm1sb99qmpvr6k0ha8ag2h223n0d591g"))))
(build-system cmake-build-system)
(arguments
(list #:configure-flags #~(list "-DASMJIT_TEST=TRUE")))
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 02/32] gnu: Add python-typing-extensions-4.10.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-2-david.elsing@posteo.net
* gnu/packages/python-build.scm (python-typing-extensions-4.10): New variable.
---
gnu/packages/python-build.scm | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)

Toggle diff (34 lines)
diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 2ea457cdba..dfd59d5e25 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -10,7 +10,7 @@
;;; Copyright © 2020, 2021, 2022, 2023 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2022 Garek Dyszel <garekdyszel@disroot.org>
;;; Copyright © 2022 Greg Hogan <code@greghogan.com>
-
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -252,6 +252,18 @@ (define-public python-typing-extensions
@end enumerate\n")
(license license:psfl)))
+(define-public python-typing-extensions-4.10
+ (package
+ (inherit python-typing-extensions)
+ (version "4.10.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "typing_extensions" version))
+ (sha256
+ (base32
+ "1jxkj4pni8pdyrn79sq441lsp40xzw363n0qvfc6zfcgkv4dgaxh"))))))
+
;;;
;;; Python builder packages.
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 04/32] gnu: Add python-pytest-flakefinder.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-4-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-pytest-flakefinder): New variable.
---
gnu/packages/python-check.scm | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)

Toggle diff (39 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 51a38bbcbe..247f8af98b 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -18,6 +18,7 @@
;;; Copyright © 2022 Tomasz Jeneralczyk <tj@schwi.pl>
;;; Copyright © 2022 jgart <jgart@dismail.de>
;;; Copyright © 2024 Troy Figiel <troy@troyfigiel.com>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -191,6 +192,24 @@ (define-public python-pytest-csv
it adds to the Pytest command line interface (CLI).")
(license license:gpl3+)))
+(define-public python-pytest-flakefinder
+ (package
+ (name "python-pytest-flakefinder")
+ (version "1.1.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "pytest-flakefinder" version))
+ (sha256
+ (base32 "03iy80xlkpgzjs2kxa9rrj8dbnp9awyhpcl3hy8fgf5x40cjlhg2"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs (list python-pytest))
+ (home-page "https://github.com/dropbox/pytest-flakefinder")
+ (synopsis "Pytest plugin for finding flaky tests")
+ (description "This package provides a Pytest plugin to run tests multiple
+times and detect flakyness.")
+ (license license:asl2.0)))
+
(define-public python-testfixtures
(package
(name "python-testfixtures")
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 03/32] gnu: Add python-optree.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-3-david.elsing@posteo.net
* gnu/packages/python-xyz.scm (python-optree): New variable.
---
.../patches/python-optree-fix-32-bit.patch | 122 ++++++++++++++++++
gnu/packages/python-xyz.scm | 30 +++++
2 files changed, 152 insertions(+)
create mode 100644 gnu/packages/patches/python-optree-fix-32-bit.patch

Toggle diff (178 lines)
diff --git a/gnu/packages/patches/python-optree-fix-32-bit.patch b/gnu/packages/patches/python-optree-fix-32-bit.patch
new file mode 100644
index 0000000000..6a32c39bd8
--- /dev/null
+++ b/gnu/packages/patches/python-optree-fix-32-bit.patch
@@ -0,0 +1,122 @@
+In include/utils.h, ssize_t is an alias for py::ssize_t, which is an alias for
+Py_ssize_t in Python, which is an alias for the system ssize_t.
+The latter is defined in glibc as int if __WORDSIZE == 32 and as long int if
+__WORDSIZE == 64. Therefore, we need to remove the explicit template
+specialization for int in the first case.
+
+diff --git a/include/utils.h b/include/utils.h
+index 950a02b..82a9591 100644
+--- a/include/utils.h
++++ b/include/utils.h
+@@ -141,10 +141,12 @@ template <>
+ inline py::handle GET_ITEM_HANDLE<py::tuple>(const py::handle& container, const size_t& item) {
+ return PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::handle GET_ITEM_HANDLE<py::tuple>(const py::handle& container, const int& item) {
+ return PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#endif
+ template <>
+ inline py::handle GET_ITEM_HANDLE<py::list>(const py::handle& container, const ssize_t& item) {
+ return PyList_GET_ITEM(container.ptr(), item);
+@@ -153,10 +155,12 @@ template <>
+ inline py::handle GET_ITEM_HANDLE<py::list>(const py::handle& container, const size_t& item) {
+ return PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::handle GET_ITEM_HANDLE<py::list>(const py::handle& container, const int& item) {
+ return PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item));
+ }
++#endif
+
+ template <typename Container, typename Item>
+ inline py::object GET_ITEM_BORROW(const py::handle& container, const Item& item) {
+@@ -171,11 +175,13 @@ inline py::object GET_ITEM_BORROW<py::tuple>(const py::handle& container, const
+ return py::reinterpret_borrow<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_BORROW<py::tuple>(const py::handle& container, const int& item) {
+ return py::reinterpret_borrow<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+ template <>
+ inline py::object GET_ITEM_BORROW<py::list>(const py::handle& container, const ssize_t& item) {
+ return py::reinterpret_borrow<py::object>(PyList_GET_ITEM(container.ptr(), item));
+@@ -185,11 +191,13 @@ inline py::object GET_ITEM_BORROW<py::list>(const py::handle& container, const s
+ return py::reinterpret_borrow<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_BORROW<py::list>(const py::handle& container, const int& item) {
+ return py::reinterpret_borrow<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+
+ template <typename Container, typename Item>
+ inline py::object GET_ITEM_STEAL(const py::handle& container, const Item& item) {
+@@ -204,11 +212,13 @@ inline py::object GET_ITEM_STEAL<py::tuple>(const py::handle& container, const s
+ return py::reinterpret_steal<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_STEAL<py::tuple>(const py::handle& container, const int& item) {
+ return py::reinterpret_steal<py::object>(
+ PyTuple_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+ template <>
+ inline py::object GET_ITEM_STEAL<py::list>(const py::handle& container, const ssize_t& item) {
+ return py::reinterpret_steal<py::object>(PyList_GET_ITEM(container.ptr(), item));
+@@ -218,11 +228,13 @@ inline py::object GET_ITEM_STEAL<py::list>(const py::handle& container, const si
+ return py::reinterpret_steal<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline py::object GET_ITEM_STEAL<py::list>(const py::handle& container, const int& item) {
+ return py::reinterpret_steal<py::object>(
+ PyList_GET_ITEM(container.ptr(), py::ssize_t_cast(item)));
+ }
++#endif
+
+ template <typename Container, typename Item>
+ inline void SET_ITEM(const py::handle& container, const Item& item, const py::handle& value) {
+@@ -240,12 +252,14 @@ inline void SET_ITEM<py::tuple>(const py::handle& container,
+ const py::handle& value) {
+ PyTuple_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline void SET_ITEM<py::tuple>(const py::handle& container,
+ const int& item,
+ const py::handle& value) {
+ PyTuple_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#endif
+ template <>
+ inline void SET_ITEM<py::list>(const py::handle& container,
+ const ssize_t& item,
+@@ -258,12 +272,14 @@ inline void SET_ITEM<py::list>(const py::handle& container,
+ const py::handle& value) {
+ PyList_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#if __WORDSIZE != 32
+ template <>
+ inline void SET_ITEM<py::list>(const py::handle& container,
+ const int& item,
+ const py::handle& value) {
+ PyList_SET_ITEM(container.ptr(), py::ssize_t_cast(item), value.inc_ref().ptr());
+ }
++#endif
+
+ template <typename PyType>
+ inline void AssertExact(const py::handle& object) {
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 0ac9d669d3..bac7d79af9 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -150,6 +150,7 @@
;;; Copyright © 2024 Timothee Mathieu <timothee.mathieu@inria.fr>
;;; Copyright © 2024 Ian Eure <ian@retrospec.tv>
;;; Copyright © 2024 Adriel Dumas--Jondeau <leirda@disroot.org>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -11741,6 +11742,35 @@ (define-public python-treelib
"This package provides a Python implementation of a tree structure.")
(license license:asl2.0)))
+(define-public python-optree
+ (package
+ (name "python-optree")
+ (version "0.10.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/metaopt/optree")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1q3wljk7cyl5rsam02sfsj8zjrqx4c3x9vic8j6xx13p8czpsisg"))
+ (patches (search-patches "python-optree-fix-32-bit.patch"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs (list python-typing-extensions))
+ (native-inputs
+ (list python-pytest
+ python-pytest-cov
+ python-pytest-xdist
+ cmake
+ pybind11))
+ (home-page "https://github.com/metaopt/optree")
+ (synopsis "Optimized PyTree Utilities")
+ (description "This package contains operations on PyTrees (a tree made of
+container data structures in Python).")
+ (license license:asl2.0)))
+
(define-public python-jupyter-core
(package
(name "python-jupyter-core")
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 05/32] gnu: Add python-pytest-shard.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-5-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-pytest-shard): New variable.
---
gnu/packages/python-check.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 247f8af98b..f876088488 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -210,6 +210,30 @@ (define-public python-pytest-flakefinder
times and detect flakyness.")
(license license:asl2.0)))
+(define-public python-pytest-shard
+ (let ((commit "64610a08dac6b0511b6d51cf895d0e1040d162ad")
+ (revision "0"))
+ (package
+ (name "python-pytest-shard")
+ (version (git-version "0.1.2" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/AdamGleave/pytest-shard")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1h31m68igz670bzl307hazjrfbr8pk14mxflllar18ydmlrnl677"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs (list python-pytest))
+ (home-page "https://github.com/AdamGleave/pytest-shard")
+ (synopsis "Pytest plugin for sharding tests")
+ (description "This package provides a Pytest extension for sharding
+tests at the granularity of individual test cases, which can be run in
+parallel and on multiple machines.")
+ (license license:expat))))
+
(define-public python-testfixtures
(package
(name "python-testfixtures")
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 07/32] gnu: Add python-pytest-rerunfailures-13.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-7-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-pytest-rerunfailures-13): New variable.
---
gnu/packages/python-check.scm | 11 +++++++++++
1 file changed, 11 insertions(+)

Toggle diff (24 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index dbca80652c..347e4a4701 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -2464,6 +2464,17 @@ (define-public python-pytest-rerunfailures
eliminate flaky failures.")
(license license:mpl2.0)))
+(define-public python-pytest-rerunfailures-13
+ (package
+ (inherit python-pytest-rerunfailures)
+ (version "13.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (pypi-uri "pytest-rerunfailures" version))
+ (sha256
+ (base32 "16cin0chv59w4rvnd6r0fisp0s8avmp07rwn9da6yixw43jdncp1"))))))
+
(define-public python-xunitparser
(package
(name "python-xunitparser")
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 06/32] gnu: Add python-expecttest.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-6-david.elsing@posteo.net
* gnu/packages/python-check.scm (python-expecttest): New variable.
---
gnu/packages/python-check.scm | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index f876088488..dbca80652c 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -2078,6 +2078,41 @@ (define-public python-eradicate
valid Python syntax that are likely to be commented out code.")
(license license:expat)))
+(define-public python-expecttest
+ (let ((commit "683b09a352cc426851adc2e3a9f46e0ab25e4dee")
+ (revision "0"))
+ (package
+ (name "python-expecttest")
+ (version (git-version "0.2.1" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/ezyang/expecttest")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1djwxp9x1hczzxbimv1b1bmd083am88v27l82nmlkhvzyg2cmpvv"))))
+ (build-system pyproject-build-system)
+ (arguments
+ (list
+ #:phases
+ #~(modify-phases %standard-phases
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ ;; The test runs tests expected to fail, so the output is
+ ;; confusing
+ (invoke "python3" "test_expecttest.py")))))))
+ (native-inputs (list python-hypothesis poetry))
+ (home-page "https://github.com/ezyang/expecttest")
+ (synopsis "Python module for expect tests")
+ (description "@code{expecttest} is a Python module for expect tests, where
+the initial expected value of a test can be automatically set by running the
+test itself.")
+ (license license:expat))))
+
(define-public python-robber
(package
(name "python-robber")
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 08/32] gnu: Add miniz.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-8-david.elsing@posteo.net
* gnu/packages/compression.scm (miniz): New variable.
---
gnu/packages/compression.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (46 lines)
diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index 5de17b6b51..fcfc2c30f3 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -38,6 +38,7 @@
;;; Copyright © 2022 Greg Hogan <code@greghogan.com>
;;; Copyright © 2022 Zhu Zihao <all_but_last@163.com>
;;; Copyright © 2021 Foo Chuan Wei <chuanwei.foo@hotmail.com>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -2838,3 +2839,31 @@ (define-public unrar-free
libarchive. It does not rival the non-free @code{unrar} in terms of features,
but special care has been taken to ensure it meets most user's needs.")
(license license:gpl2+)))
+
+(define-public miniz
+ (package
+ (name "miniz")
+ (version "3.0.2")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/richgel999/miniz")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0672q35vjrpakmsr1gwj9k5fwv5ihzhahm19bq4y74wqpn91p7fw"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ #:configure-flags
+ ''("-DBUILD_SHARED_LIBS=ON")
+ ;; No tests
+ #:tests? #f))
+ (home-page "https://github.com/richgel999/miniz")
+ (synopsis "Independent implementation of zlib and Deflate compression")
+ (description "Miniz is a lossless data compression library that implements
+the zlib (RFC 1950) and Deflate (RFC 1951) compressed data format
+specification standards. It supports the most commonly used functions
+exported by the zlib library.")
+ (license license:expat)))
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 09/32] gnu: Add miniz-for-pytorch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-9-david.elsing@posteo.net
* gnu/packages/compression.scm (miniz-for-pytorch): New variable.
* gnu/packages/patches/miniz-for-pytorch.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
gnu/packages/compression.scm | 21 +++++++++
gnu/packages/patches/miniz-for-pytorch.patch | 47 ++++++++++++++++++++
3 files changed, 69 insertions(+)
create mode 100644 gnu/packages/patches/miniz-for-pytorch.patch

Toggle diff (96 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 7f1006010b..ca83a84ea2 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1691,6 +1691,7 @@ dist_patch_DATA = \
%D%/packages/patches/mingw-w64-reproducible-gendef.patch \
%D%/packages/patches/minisat-friend-declaration.patch \
%D%/packages/patches/minisat-install.patch \
+ %D%/packages/patches/miniz-for-pytorch.patch \
%D%/packages/patches/mit-krb5-hurd.patch \
%D%/packages/patches/mixxx-link-qtscriptbytearray-qtscript.patch \
%D%/packages/patches/mixxx-system-googletest-benchmark.patch \
diff --git a/gnu/packages/compression.scm b/gnu/packages/compression.scm
index fcfc2c30f3..589c9bb834 100644
--- a/gnu/packages/compression.scm
+++ b/gnu/packages/compression.scm
@@ -2867,3 +2867,24 @@ (define-public miniz
specification standards. It supports the most commonly used functions
exported by the zlib library.")
(license license:expat)))
+
+(define-public miniz-for-pytorch
+ (package
+ (inherit miniz)
+ (version "pytorch-2.2.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/richgel999/miniz")
+ (commit "2.2.0")))
+ (file-name (git-file-name (package-name miniz) version))
+ (sha256
+ (base32
+ "09j9ihigfsavgcmk8l36zmbjvdf1x1w7h2v4rkww1qk1sb43y5zf"))
+ (patches (search-patches "miniz-for-pytorch.patch"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments miniz)
+ ((#:configure-flags flags '())
+ ;; The changes break the examples.
+ `(cons "-DBUILD_EXAMPLES=OFF" ,flags))))
+ (properties '((hidden? . #t)))))
diff --git a/gnu/packages/patches/miniz-for-pytorch.patch b/gnu/packages/patches/miniz-for-pytorch.patch
new file mode 100644
index 0000000000..19fa705a31
--- /dev/null
+++ b/gnu/packages/patches/miniz-for-pytorch.patch
@@ -0,0 +1,47 @@
+Modifications from the internal fork of the python-pytorch package.
+
+diff --git a/miniz.h b/miniz.h
+index 42e1ea2..5869ab9 100644
+--- a/miniz.h
++++ b/miniz.h
+@@ -123,7 +123,7 @@
+ /* If MINIZ_NO_TIME is specified then the ZIP archive functions will not be able to get the current time, or */
+ /* get/set file times, and the C run-time funcs that get/set times won't be called. */
+ /* The current downside is the times written to your archives will be from 1979. */
+-/*#define MINIZ_NO_TIME */
++#define MINIZ_NO_TIME
+
+ /* Define MINIZ_NO_ARCHIVE_APIS to disable all ZIP archive API's. */
+ /*#define MINIZ_NO_ARCHIVE_APIS */
+@@ -135,7 +135,7 @@
+ /*#define MINIZ_NO_ZLIB_APIS */
+
+ /* Define MINIZ_NO_ZLIB_COMPATIBLE_NAME to disable zlib names, to prevent conflicts against stock zlib. */
+-/*#define MINIZ_NO_ZLIB_COMPATIBLE_NAMES */
++#define MINIZ_NO_ZLIB_COMPATIBLE_NAMES
+
+ /* Define MINIZ_NO_MALLOC to disable all calls to malloc, free, and realloc.
+ Note if MINIZ_NO_MALLOC is defined then the user must always provide custom user alloc/free/realloc
+@@ -172,8 +172,7 @@
+ #if !defined(MINIZ_USE_UNALIGNED_LOADS_AND_STORES)
+ #if MINIZ_X86_OR_X64_CPU
+ /* Set MINIZ_USE_UNALIGNED_LOADS_AND_STORES to 1 on CPU's that permit efficient integer loads and stores from unaligned addresses. */
+-#define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 1
+-#define MINIZ_UNALIGNED_USE_MEMCPY
++#define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 0
+ #else
+ #define MINIZ_USE_UNALIGNED_LOADS_AND_STORES 0
+ #endif
+diff --git a/miniz_zip.h b/miniz_zip.h
+index 82502bd..3d92255 100644
+--- a/miniz_zip.h
++++ b/miniz_zip.h
+@@ -150,7 +150,7 @@ typedef enum {
+ MZ_ZIP_TOTAL_ERRORS
+ } mz_zip_error;
+
+-typedef struct
++typedef struct mz_zip_archive
+ {
+ mz_uint64 m_archive_size;
+ mz_uint64 m_central_directory_file_ofs;
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 10/32] gnu: Add libnop.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-10-david.elsing@posteo.net
* gnu/packages/serialization.scm (libnop): New variable.
---
gnu/packages/serialization.scm | 39 ++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 331fae80cf..bfee0dbfc6 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -19,6 +19,7 @@
;;; Copyright © 2023 Zheng Junjie <873216071@qq.com>
;;; Copyright © 2024 Paul A. Patience <paul@apatience.com>
;;; Copyright © 2024 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -936,3 +937,41 @@ (define-public python-feather-format
(description "This package provides a Python wrapper library to the
Apache Arrow-based Feather binary columnar serialization data frame format.")
(license license:asl2.0)))
+
+(define-public libnop
+ (let ((commit "35e800d81f28c632956c5a592e3cbe8085ecd430")
+ (revision "0"))
+ (package
+ (name "libnop")
+ (version (git-version "0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/google/libnop")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0qqbaljq54qiq0dky9nj47igfcs065ry526jg9a0aafbfl9krpy2"))))
+ (build-system gnu-build-system)
+ (arguments
+ (list
+ #:test-target "test"
+ #:phases
+ #~(modify-phases %standard-phases
+ (delete 'configure)
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ (invoke "out/test"))))
+ (replace 'install
+ (lambda _
+ (copy-recursively
+ "include" (string-append #$output "/include")))))))
+ (native-inputs (list googletest))
+ (home-page "https://github.com/google/libnop")
+ (synopsis "C++ Native Object Protocols")
+ (description "@code{libnop} is a header-only library for serializing and
+deserializing C++ data types without external code generators or runtime
+support libraries.")
+ (license license:asl2.0))))
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 12/32] gnu: python-flatbuffers-next: Update to 23.5.26.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-12-david.elsing@posteo.net
* gnu/packages/serialization.scm (python-flatbuffers-next): Update to 23.5.26.
---
gnu/packages/serialization.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 405c4211da..bea96f2a92 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -879,7 +879,7 @@ (define-public flatbuffers
(define-public flatbuffers-next
(package
(inherit flatbuffers)
- (version "23.1.21")
+ (version "23.5.26")
(source (origin
(method git-fetch)
(uri (git-reference
@@ -888,7 +888,7 @@ (define-public flatbuffers-next
(file-name (git-file-name "flatbuffers" version))
(sha256
(base32
- "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
+ "0cd12dvkzqdafz46q4302mzgpzbz589zmmiga7bq07f2sqy4vrvv"))))))
(define-public python-flatbuffers
(package
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 11/32] gnu: Remove flatbuffers-next-shared.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-11-david.elsing@posteo.net
* gnu/packages/serialization.scm (flatbuffers-next-shared): Delete variable.
* gnu/packages/machine-learning.scm (tensorflow-lite)[arguments]: Use
flatbuffers-next instead of flatbuffers-next-shared.
---
gnu/packages/machine-learning.scm | 5 +++--
gnu/packages/serialization.scm | 11 -----------
2 files changed, 3 insertions(+), 13 deletions(-)

Toggle diff (54 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 5c18a2e9d5..6da189616d 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -27,6 +27,7 @@
;;; Copyright © 2024 David Pflug <david@pflug.io>
;;; Copyright © 2024 Timothee Mathieu <timothee.mathieu@inria.fr>
;;; Copyright © 2024 Spencer King <spencer.king@geneoscopy.com>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -3337,7 +3338,7 @@ (define-public tensorflow-lite
(string-append "-DEigen3_DIR=" #$(this-package-input "eigen")
"/share/eigen3/cmake")
(string-append "-DFlatBuffers_DIR="
- #$(this-package-input "flatbuffers-shared")
+ #$(this-package-input "flatbuffers")
"/lib/cmake/flatbuffers")
(string-append "-DNEON_2_SSE_DIR=" #$(this-package-input "neon2sse")
"/lib/cmake/NEON_2_SSE")
@@ -3424,7 +3425,7 @@ (define-public tensorflow-lite
("cpuinfo" ,cpuinfo)
("eigen" ,eigen)
("fp16" ,fp16)
- ("flatbuffers-shared" ,flatbuffers-next-shared)
+ ("flatbuffers" ,flatbuffers-next)
("gemmlowp" ,gemmlowp)
("mesa-headers" ,mesa-headers)
("neon2sse" ,neon2sse)
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index bfee0dbfc6..405c4211da 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -890,17 +890,6 @@ (define-public flatbuffers-next
(base32
"1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz"))))))
-(define-public flatbuffers-next-shared
- (package
- (inherit flatbuffers-next)
- (name "flatbuffers-shared")
- (version "23.1.21")
- (arguments
- (substitute-keyword-arguments (package-arguments flatbuffers-next)
- ((#:configure-flags flags)
- ;; Compile with -fPIC, needed for shared lib.
- #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags))))))
-
(define-public python-flatbuffers
(package
(name "python-flatbuffers")
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 15/32] gnu: clog: Add "-DUSE_SYSTEM_LIBS=ON" configure flag.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-15-david.elsing@posteo.net
* gnu/packages/parallel.scm (clog)[arguments]: Add "-DUSE_SYSTEM_LIBS=ON"
to #:configure-flags.
* gnu/packages/patches/clog-fix-shared-build.patch: Adjust patch.
---
gnu/packages/parallel.scm | 13 ++++---
.../patches/clog-fix-shared-build.patch | 39 ++-----------------
2 files changed, 12 insertions(+), 40 deletions(-)

Toggle diff (56 lines)
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index 4238312367..f591834047 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -527,11 +527,14 @@ (define-public clog
(inherit (package-source cpuinfo))
(patches (search-patches "clog-fix-shared-build.patch"))))
(arguments
- (list #:configure-flags #~(list "-DBUILD_SHARED_LIBS=ON")
- #:phases #~(modify-phases %standard-phases
- (add-after 'unpack 'chdir
- (lambda _
- (chdir "deps/clog"))))))
+ (list
+ #:configure-flags
+ ''("-DBUILD_SHARED_LIBS=ON"
+ "-DUSE_SYSTEM_LIBS=ON")
+ #:phases #~(modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _
+ (chdir "deps/clog"))))))
(native-inputs (list googletest))
(inputs '())
(synopsis "C-style logging library based on printf")
diff --git a/gnu/packages/patches/clog-fix-shared-build.patch b/gnu/packages/patches/clog-fix-shared-build.patch
index bf80544b90..eb305339f5 100644
--- a/gnu/packages/patches/clog-fix-shared-build.patch
+++ b/gnu/packages/patches/clog-fix-shared-build.patch
@@ -1,50 +1,19 @@
Author: Antero Mejr <antero@mailbox.org>
-Notes: Disabled function visibility hacks and googletest download. Enabled
-non-static builds.
+Notes: Disabled function visibility hacks. Enabled non-static builds.
diff --git a/deps/clog/CMakeLists.txt b/deps/clog/CMakeLists.txt
-index 083f519..b7b225a 100644
+index 6e50c41..db02e95 100644
--- a/deps/clog/CMakeLists.txt
+++ b/deps/clog/CMakeLists.txt
-@@ -38,20 +38,8 @@ SET(CONFU_DEPENDENCIES_SOURCE_DIR ${CMAKE_SOURCE_DIR}/deps
- SET(CONFU_DEPENDENCIES_BINARY_DIR ${CMAKE_BINARY_DIR}/deps
- CACHE PATH "Confu-style dependencies binary directory")
+@@ -55,7 +55,7 @@ IF(CLOG_BUILD_TESTS)
+ ENDIF()
--IF(CLOG_BUILD_TESTS)
-- IF(NOT DEFINED GOOGLETEST_SOURCE_DIR)
-- MESSAGE(STATUS "Downloading Google Test to ${CONFU_DEPENDENCIES_SOURCE_DIR}/googletest (define GOOGLETEST_SOURCE_DIR to avoid it)")
-- CONFIGURE_FILE(cmake/DownloadGoogleTest.cmake "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download/CMakeLists.txt")
-- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
-- WORKING_DIRECTORY "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download")
-- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" --build .
-- WORKING_DIRECTORY "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download")
-- SET(GOOGLETEST_SOURCE_DIR "${CONFU_DEPENDENCIES_SOURCE_DIR}/googletest" CACHE STRING "Google Test source directory")
-- ENDIF()
--ENDIF()
--
# ---[ clog library
-ADD_LIBRARY(clog STATIC src/clog.c)
+ADD_LIBRARY(clog src/clog.c)
SET_TARGET_PROPERTIES(clog PROPERTIES
C_STANDARD 99
C_EXTENSIONS NO)
-@@ -74,16 +62,6 @@ INSTALL(TARGETS clog
-
- # ---[ clog tests
- IF(CLOG_BUILD_TESTS)
-- # ---[ Build google test
-- IF(NOT TARGET gtest)
-- IF(MSVC AND NOT CLOG_RUNTIME_TYPE STREQUAL "static")
-- SET(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
-- ENDIF()
-- ADD_SUBDIRECTORY(
-- "${GOOGLETEST_SOURCE_DIR}"
-- "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest")
-- ENDIF()
--
- ADD_EXECUTABLE(clog-test test/clog.cc)
- SET_TARGET_PROPERTIES(clog-test PROPERTIES
- CXX_STANDARD 11
diff --git a/deps/clog/include/clog.h b/deps/clog/include/clog.h
index 4143761..aa9000f 100644
--- a/deps/clog/include/clog.h
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 13/32] gnu: pthreadpool: Update to commit 178e3e0.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-13-david.elsing@posteo.net
* gnu/packages/parallel.scm (pthreadpool): Update to commit 178e3e0.
---
gnu/packages/parallel.scm | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)

Toggle diff (35 lines)
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index ddf4ad016b..e1cbe52934 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -12,6 +12,7 @@
;;; Copyright © 2019-2023 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2020 Roel Janssen <roel@gnu.org>
;;; Copyright © 2021 Stefan Reichör <stefan@xsteve.at>
+;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -449,9 +450,9 @@ (define-public python-slurm-magic
(define-public pthreadpool
;; This repository has only one tag, 0.1, which is older than what users
;; such as XNNPACK expect.
- (let ((commit "1787867f6183f056420e532eec640cba25efafea")
+ (let ((commit "178e3e0646cc671708bf78e77c273940130ac637")
(version "0.1")
- (revision "1"))
+ (revision "2"))
(package
(name "pthreadpool")
(version (git-version version revision commit))
@@ -462,7 +463,7 @@ (define-public pthreadpool
(file-name (git-file-name name version))
(sha256
(base32
- "02hdvxfn5krw8zivkgjx3b4rk9p02yr4mpdjlp75lsv6z1xf5yrx"))
+ "1s86lnq9bahacf5wxn7y14w70jh3g9lq1l7y16ijwhifd01nc2km"))
(patches (search-patches "pthreadpool-system-libraries.patch"))))
(build-system cmake-build-system)
(arguments '(#:configure-flags '("-DBUILD_SHARED_LIBS=ON")))
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 14/32] gnu: cpuinfo: Update to commit aa4b216.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-14-david.elsing@posteo.net
* gnu/packages/parallel.scm (cpuinfo): Update to commit aa4b216.
[arguments]: Set the "-DUSE_SYSTEM_LIBS=ON" configure flag instead of using a
patch.
[supported-systems]: New field.
* gnu/packages/patches/cpuinfo-system-libraries.patch: Remove file.
* gnu/local.mk (dist_patch_DATA): Remove it.
---
gnu/local.mk | 1 -
gnu/packages/parallel.scm | 14 ++++--
.../patches/cpuinfo-system-libraries.patch | 50 -------------------
3 files changed, 9 insertions(+), 56 deletions(-)
delete mode 100644 gnu/packages/patches/cpuinfo-system-libraries.patch

Toggle diff (97 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index ca83a84ea2..474260b764 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1068,7 +1068,6 @@ dist_patch_DATA = \
%D%/packages/patches/cool-retro-term-wctype.patch \
%D%/packages/patches/coreutils-gnulib-tests.patch \
%D%/packages/patches/cppcheck-disable-char-signedness-test.patch \
- %D%/packages/patches/cpuinfo-system-libraries.patch \
%D%/packages/patches/cpulimit-with-glib-2.32.patch \
%D%/packages/patches/crawl-upgrade-saves.patch \
%D%/packages/patches/crc32c-unbundle-googletest.patch \
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index e1cbe52934..4238312367 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -480,8 +480,8 @@ (define-public pthreadpool
(define-public cpuinfo
;; There's currently no tag on this repo.
(let ((version "0.0")
- (revision "2")
- (commit "53298db833c5c5a1598639e9b47cc1a602bbac26"))
+ (revision "3")
+ (commit "aa4b2163b99ac9534194520f70b93eeefb0b3b4e"))
(package
(name "cpuinfo")
(version (git-version version revision commit))
@@ -492,12 +492,13 @@ (define-public cpuinfo
(file-name (git-file-name name version))
(sha256
(base32
- "01kfgxya2w32dz9bd3qm3i2d6nffw0qfyql11rxl7d3g830brj5k"))
- (patches (search-patches "cpuinfo-system-libraries.patch"))))
+ "12x4krkyzxngf1l2ck33lnsp8pyzf6gyjj9mp9cnka9mw3h6617m"))))
(build-system cmake-build-system)
(arguments
(list
- #:configure-flags '(list "-DBUILD_SHARED_LIBS=ON")
+ #:configure-flags
+ '(list "-DBUILD_SHARED_LIBS=ON"
+ "-DUSE_SYSTEM_LIBS=ON")
#:phases
'(modify-phases %standard-phases
(add-after 'unpack 'skip-bad-test
@@ -513,6 +514,9 @@ (define-public cpuinfo
"The cpuinfo library provides a C/C++ and a command-line interface to
obtain information about the CPU being used: supported instruction set,
processor name, cache information, and topology information.")
+ ;; On aarch64-linux, there is a bug reported upstream:
+ ;; https://github.com/pytorch/cpuinfo/issues/14
+ (supported-systems '("armv7-linux" "i686-linux" "x86_64-linux"))
(license license:bsd-2))))
(define-public clog
diff --git a/gnu/packages/patches/cpuinfo-system-libraries.patch b/gnu/packages/patches/cpuinfo-system-libraries.patch
deleted file mode 100644
index e25446e9da..0000000000
--- a/gnu/packages/patches/cpuinfo-system-libraries.patch
+++ /dev/null
@@ -1,50 +0,0 @@
-This patch allows the build process to use the provided dependencies instead
-of adding their source as CMake sub-directories (in which case "make install"
-would install googletest's and googlebenchmark's libraries and headers).
-
-diff --git a/CMakeLists.txt b/CMakeLists.txt
-index 57abc26..761c612 100644
---- a/CMakeLists.txt
-+++ b/CMakeLists.txt
-@@ -93,7 +93,7 @@ SET(CONFU_DEPENDENCIES_BINARY_DIR ${CMAKE_BINARY_DIR}/deps
- CACHE PATH "Confu-style dependencies binary directory")
-
- IF(CPUINFO_BUILD_MOCK_TESTS OR CPUINFO_BUILD_UNIT_TESTS)
-- IF(CPUINFO_SUPPORTED_PLATFORM AND NOT DEFINED GOOGLETEST_SOURCE_DIR)
-+ IF(FALSE)
- MESSAGE(STATUS "Downloading Google Test to ${CONFU_DEPENDENCIES_SOURCE_DIR}/googletest (define GOOGLETEST_SOURCE_DIR to avoid it)")
- CONFIGURE_FILE(cmake/DownloadGoogleTest.cmake "${CONFU_DEPENDENCIES_BINARY_DIR}/googletest-download/CMakeLists.txt")
- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
-@@ -105,7 +105,7 @@ IF(CPUINFO_BUILD_MOCK_TESTS OR CPUINFO_BUILD_UNIT_TESTS)
- ENDIF()
-
- IF(CPUINFO_BUILD_BENCHMARKS)
-- IF(CPUINFO_SUPPORTED_PLATFORM AND NOT DEFINED GOOGLEBENCHMARK_SOURCE_DIR)
-+ IF(FALSE)
- MESSAGE(STATUS "Downloading Google Benchmark to ${CONFU_DEPENDENCIES_SOURCE_DIR}/googlebenchmark (define GOOGLEBENCHMARK_SOURCE_DIR to avoid it)")
- CONFIGURE_FILE(cmake/DownloadGoogleBenchmark.cmake "${CONFU_DEPENDENCIES_BINARY_DIR}/googlebenchmark-download/CMakeLists.txt")
- EXECUTE_PROCESS(COMMAND "${CMAKE_COMMAND}" -G "${CMAKE_GENERATOR}" .
-@@ -271,14 +271,6 @@ INSTALL(TARGETS cpuinfo
-
- # ---[ cpuinfo micro-benchmarks
- IF(CPUINFO_SUPPORTED_PLATFORM AND CPUINFO_BUILD_BENCHMARKS)
-- # ---[ Build google benchmark
-- IF(NOT TARGET benchmark)
-- SET(BENCHMARK_ENABLE_TESTING OFF CACHE BOOL "")
-- ADD_SUBDIRECTORY(
-- "${GOOGLEBENCHMARK_SOURCE_DIR}"
-- "${CONFU_DEPENDENCIES_BINARY_DIR}/googlebenchmark")
-- ENDIF()
--
- IF(CMAKE_SYSTEM_NAME MATCHES "^(Linux|Android)$")
- ADD_EXECUTABLE(get-current-bench bench/get-current.cc)
- TARGET_LINK_LIBRARIES(get-current-bench cpuinfo benchmark)
-@@ -289,7 +281,7 @@ IF(CPUINFO_SUPPORTED_PLATFORM AND CPUINFO_BUILD_BENCHMARKS)
- ENDIF()
-
- IF(CPUINFO_SUPPORTED_PLATFORM)
-- IF(CPUINFO_BUILD_MOCK_TESTS OR CPUINFO_BUILD_UNIT_TESTS)
-+ IF(FALSE)
- # ---[ Build google test
- IF(NOT TARGET gtest)
- IF(MSVC AND NOT CPUINFO_RUNTIME_TYPE STREQUAL "static")
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 27/32] gnu: Add python-pytorch-avx.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-27-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (python-pytorch-avx): New variable.
---
gnu/packages/machine-learning.scm | 15 +++++++++++++++
1 file changed, 15 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index a062a2398b..ea023ad749 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4712,6 +4712,21 @@ (define-public python-pytorch
Note: currently this package does not provide GPU support.")
(license license:bsd-3)))
+;; This package variant includes the dependencies requiring at least AVX2 or
+;; AVX-512.
+(define-public python-pytorch-avx
+ (package/inherit python-pytorch
+ (name "python-pytorch-avx")
+ (inputs
+ (modify-inputs (package-inputs python-pytorch)
+ (append fbgemm nnpack)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments python-pytorch)
+ ((#:phases phases)
+ #~(modify-phases #$phases
+ (delete 'disable-avx-dependencies)))))
+ (supported-systems '("x86_64-linux"))))
+
(define-public python-pytorch-for-r-torch
(package
(inherit python-pytorch)
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 18/32] gnu: Add tensorpipe.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-18-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (tensorpipe): New variable.
---
gnu/packages/machine-learning.scm | 36 +++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 5806de8891..c39dccbe77 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -91,6 +91,7 @@ (define-module (gnu packages machine-learning)
#:use-module (gnu packages image-processing)
#:use-module (gnu packages imagemagick)
#:use-module (gnu packages jupyter)
+ #:use-module (gnu packages libevent)
#:use-module (gnu packages libffi)
#:use-module (gnu packages linux)
#:use-module (gnu packages llvm)
@@ -4093,6 +4094,41 @@ (define-public xnnpack-for-torch2
"-DXNNPACK_BUILD_TESTS=FALSE" ;FIXME: see below
"-DXNNPACK_BUILD_BENCHMARKS=FALSE"))))))
+(define-public tensorpipe
+ (let ((commit "bb1473a4b38b18268e8693044afdb8635bc8351b")
+ (revision "0"))
+ (package
+ (name "tensorpipe")
+ (version (git-version "0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/pytorch/tensorpipe")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0sbpkd69rzybw2j89sjkf4s0j8vkk96d51bsps28894989a75j6v"))
+ (modules '((guix build utils)))
+ (snippet
+ '(delete-file-recursively "third_party"))))
+ (build-system cmake-build-system)
+ (arguments
+ (list
+ #:configure-flags
+ ''("-DBUILD_SHARED_LIBS=ON")
+ ;; There are no tests
+ #:tests? #f))
+ (inputs (list libuv))
+ (native-inputs (list googletest pkg-config pybind11 libnop))
+ (home-page "https://github.com/pytorch/tensorpipe")
+ (synopsis "Tensor-aware point-to-point communication primitive for
+machine learning")
+ (description "TensorPipe provides a tensor-aware channel to transfer
+rich objects from one process to another while using the fastest transport for
+the tensors contained therein.")
+ (license license:bsd-3))))
+
;; Please also update python-torchvision when updating this package.
(define-public python-pytorch
(package
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 16/32] gnu: nnpack: Update to commit 70a77f4.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-16-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (nnpack): Update to commit 70a77f4.
[supported-systems]: New field.
---
gnu/packages/machine-learning.scm | 21 ++++++++-------------
1 file changed, 8 insertions(+), 13 deletions(-)

Toggle diff (52 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 6da189616d..4b3ebff474 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -3973,10 +3973,11 @@ (define-public python-umap-learn
also for general non-linear dimension reduction.")
(license license:bsd-3)))
+;; Requires AVX2 on x86_64-linux.
(define-public nnpack
(let ((version "0.0")
- (commit "c07e3a0400713d546e0dea2d5466dd22ea389c73")
- (revision "1"))
+ (commit "70a77f485e8b934224f3a79efd8edcd84cd377b8")
+ (revision "2"))
(package
(name "nnpack")
(version (git-version version revision commit))
@@ -3987,19 +3988,11 @@ (define-public nnpack
(file-name (git-file-name name version))
(sha256
(base32
- "0s0kk3a35w3yzf0q447p72350sbsh4qhg6vm3y2djbj4xpg7jc8v"))
+ "0c4pw926279s3rlx7mg4l4vhnfy6dh374n6w7zqhcn0bxpym1hv1"))
(patches (search-patches "nnpack-system-libraries.patch"))))
(build-system cmake-build-system)
- ;; XXX: The test suite runs but it's very expensive, and on x86_64 CPUs
- ;; that lack the right ISA extensions, tests fail with:
- ;;
- ;; Expected equality of these values:
- ;; nnp_status_success
- ;; Which is: 0
- ;; status
- ;; Which is: 51
- ;;
- ;; where 51 is 'nnp_status_unsupported_hardware'.
+ ;; XXX: The test suite runs but it's very expensive. On x86_64-linux, it
+ ;; requires AVX2 instructions.
(arguments '(#:tests? #f))
(synopsis "Acceleration package for neural network computations")
(description
@@ -4020,6 +4013,8 @@ (define-public nnpack
googletest))
(native-inputs
(list python python-peachpy python-six))
+ ;; Supported for Linux.
+ (supported-systems '("x86_64-linux" "armhf-linux" "aarch64-linux"))
(license license:bsd-2))))
(define-public xnnpack
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 17/32] gnu: oneapi-dnnl: Update to 3.3.5.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-17-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (oneapi-dnnl): Update to 3.3.5.
[supported-systems]: New field.
---
gnu/packages/machine-learning.scm | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 4b3ebff474..5806de8891 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -5369,7 +5369,7 @@ (define-public python-brian2tools
(define-public oneapi-dnnl
(package
(name "oneapi-dnnl")
- (version "3.1")
+ (version "3.3.5")
(source
(origin
(method git-fetch)
@@ -5378,7 +5378,7 @@ (define-public oneapi-dnnl
(commit (string-append "v" version))))
(file-name (git-file-name name version))
(sha256
- (base32 "1jgmb5kl0bf4a2zfn94zlb117672r9lvvkkmwl86ihlyr1mpr3d0"))))
+ (base32 "05ra5zziys2hvn29y6ysiqzsf4jr9bf2bci5sc3swvf3bs2y5ihf"))))
(build-system cmake-build-system)
(arguments (if (target-riscv64?)
(list #:configure-flags #~'("-DDNNL_CPU_RUNTIME=SEQ"))
@@ -5388,6 +5388,7 @@ (define-public oneapi-dnnl
(description
"OneAPI Deep Neural Network Library (oneDNN) is a cross-platform
performance library of basic building blocks for deep learning applications.")
+ (supported-systems %64bit-supported-systems)
(license license:asl2.0)))
(define-public python-gguf
--
2.41.0
D
D
David Elsing wrote on 20 Mar 23:38 +0100
[PATCH v3 29/32] gnu: Add ideep-pytorch-for-r-torch.
(address . 69591@debbugs.gnu.org)(name . David Elsing)(address . david.elsing@posteo.net)
20240320223906.13214-29-david.elsing@posteo.net
* gnu/packages/machine-learning.scm (ideep-pytorch-for-r-torch): New variable.
---
gnu/packages/machine-learning.scm | 15 +++++++++++++++
1 file changed, 15 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index c816b202e2..939e798f45 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -4315,6 +4315,21 @@ (define-public ideep-pytorch
PyTorch.")
(license license:expat)))
+(define-public ideep-pytorch-for-r-torch
+ (package
+ (inherit ideep-pytorch)
+ (version "2.7.3-1")
+ (