[PATCH 0/7] Add toil

  • Open
  • quality assurance status badge
Details
One participant
  • Arun Isaac
Owner
unassigned
Submitted by
Arun Isaac
Severity
normal
A
A
Arun Isaac wrote on 11 Mar 19:11 +0100
(address . guix-patches@gnu.org)(name . Arun Isaac)(address . arunisaac@systemreboot.net)
cover.1710180345.git.arunisaac@systemreboot.net
This patchset adds toil.

The linter warns that there is a collision between different versions of
python-pyparsing. However, it is not possible to avoid that collision without
upgrading python-pyparsing. And, the python-pyparsing upgrade is very
complex. It requires rebuilding the world. It also introduces a dependency
cycle since the latest version of python-pyparsing switches to the
pyproject-build-system.

Arun Isaac (7):
gnu: Add python-typing-extensions-4.10.
gnu: Add python-pypubsub.
gnu: Add python-addict.
gnu: Add python-prefixed.
gnu: Add python-enlighten.
gnu: Add python-cwltool.
gnu: Add python-toil.

gnu/packages/bioinformatics.scm | 74 +++++++++++++++++++++++++++++
gnu/packages/python-build.scm | 13 ++++++
gnu/packages/python-xyz.scm | 83 +++++++++++++++++++++++++++++++++
3 files changed, 170 insertions(+)


base-commit: 001dfb89579856b8695b33b59f484f36e313347f
--
2.41.0
A
A
Arun Isaac wrote on 11 Mar 19:22 +0100
[PATCH 1/7] gnu: Add python-typing-extensions-4.10.
(address . 69736@debbugs.gnu.org)(name . Arun Isaac)(address . arunisaac@systemreboot.net)
8df7adb0e828150216c4356f360c75646f465a29.1710180345.git.arunisaac@systemreboot.net
* gnu/packages/python-build.scm (python-typing-extensions-4.10): New variable.

Change-Id: I82d9538e76353b39cce082020a939e1a8fb7badc
---
gnu/packages/python-build.scm | 13 +++++++++++++
1 file changed, 13 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm
index 2ea457cdba..adf26f7f8b 100644
--- a/gnu/packages/python-build.scm
+++ b/gnu/packages/python-build.scm
@@ -10,6 +10,7 @@
;;; Copyright © 2020, 2021, 2022, 2023 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2022 Garek Dyszel <garekdyszel@disroot.org>
;;; Copyright © 2022 Greg Hogan <code@greghogan.com>
+;;; Copyright © 2024 Arun Isaac <arunisaac@systemreboot.net>
;;;
;;; This file is part of GNU Guix.
@@ -252,6 +253,18 @@ (define-public python-typing-extensions
@end enumerate\n")
(license license:psfl)))
+;; TODO: Remove this package once python-typing-extensions is updated.
+(define-public python-typing-extensions-4.10
+ (package
+ (inherit python-typing-extensions)
+ (version "4.10.0")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "typing_extensions" version))
+ (sha256
+ (base32
+ "1jxkj4pni8pdyrn79sq441lsp40xzw363n0qvfc6zfcgkv4dgaxh"))))))
+
;;;
;;; Python builder packages.
--
2.41.0
A
A
Arun Isaac wrote on 11 Mar 19:22 +0100
[PATCH 2/7] gnu: Add python-pypubsub.
(address . 69736@debbugs.gnu.org)(name . Arun Isaac)(address . arunisaac@systemreboot.net)
7296b4c41745399c2ca7991e1e4240326d5ba2eb.1710180345.git.arunisaac@systemreboot.net
* gnu/packages/python-xyz.scm (python-pypubsub): New variable.

Change-Id: Id8f06bd592b6273cff282aa628f05503a67519b0
---
gnu/packages/python-xyz.scm | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)

Toggle diff (39 lines)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 0c72caeca0..ed9b1c9140 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -7768,6 +7768,32 @@ (define-public python-blinker
interested parties to subscribe to events, or \"signals\".")
(license license:expat)))
+(define-public python-pypubsub
+ (package
+ (name "python-pypubsub")
+ (version "4.0.3")
+ ;; There is no source tarball on PyPI.
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/schollii/pypubsub")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "02j74w28wzmdvxkk8i561ywjgizjifq3hgcl080yj0rvkd3wivlb"))))
+ (build-system python-build-system)
+ (home-page "https://github.com/schollii/pypubsub")
+ (synopsis "Python publish-subscribe library")
+ (description "@code{python-pypubsub} provides a pure Python
+publish-subscribe API to facilitate event-based or message-based architecture
+in a single-process application. It is centered on the notion of a topic;
+senders publish messages of a given topic, and listeners subscribe to messages
+of a given topic, all inside the same process. The package also supports a
+variety of advanced features that facilitate debugging and maintaining topics
+and messages in larger desktop or server-based applications.")
+ (license license:bsd-2)))
+
(define-public pelican
(package
(name "pelican")
--
2.41.0
A
A
Arun Isaac wrote on 11 Mar 19:22 +0100
[PATCH 3/7] gnu: Add python-addict.
(address . 69736@debbugs.gnu.org)(name . Arun Isaac)(address . arunisaac@systemreboot.net)
4aaac246971a498c94a8ff50de693ff3ad2fb993.1710180345.git.arunisaac@systemreboot.net
* gnu/packages/python-xyz.scm (python-addict): New variable.

Change-Id: I1554d17495bf9337067422f2118f8b46e51abead
---
gnu/packages/python-xyz.scm | 17 +++++++++++++++++
1 file changed, 17 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index ed9b1c9140..34b894a351 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -23058,6 +23058,23 @@ (define-public python-attrdict3
both as keys and as attributes.")
(license license:expat)))
+(define-public python-addict
+ (package
+ (name "python-addict")
+ (version "2.4.0")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "addict" version))
+ (sha256
+ (base32
+ "1574sicy5ydx9pvva3lbx8qp56z9jbdwbj26aqgjhyh61q723cmk"))))
+ (build-system pyproject-build-system)
+ (home-page "https://github.com/mewwts/addict")
+ (synopsis "Python dictionary with attribute accessible values")
+ (description "@code{python-addict} provides dictionaries whose items can
+be get and set using both attribute and item syntax.")
+ (license license:expat)))
+
(define-public python-attrs
(package
(name "python-attrs")
--
2.41.0
A
A
Arun Isaac wrote on 11 Mar 19:22 +0100
[PATCH 4/7] gnu: Add python-prefixed.
(address . 69736@debbugs.gnu.org)(name . Arun Isaac)(address . arunisaac@systemreboot.net)
6bcc582851650c6bdf19b944b92a8ac174728a25.1710180345.git.arunisaac@systemreboot.net
* gnu/packages/python-xyz.scm (python-prefixed): New variable.

Change-Id: I1398953059feb740dfdd9abe12bd0aa7abe3714e
---
gnu/packages/python-xyz.scm | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 34b894a351..87204b392f 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -32516,6 +32516,24 @@ (define-public python-quicktions
is the cythonized version of @code{fractions.Fraction}.")
(license license:psfl)))
+(define-public python-prefixed
+ (package
+ (name "python-prefixed")
+ (version "0.7.0")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "prefixed" version))
+ (sha256
+ (base32
+ "1sdvxwy4kvcxvnq1nx70j1ccx5ga6wdb478vqd5azf1fc1gd2m0b"))))
+ (build-system pyproject-build-system)
+ (home-page "https://github.com/Rockhopper-Technologies/prefixed")
+ (synopsis "Prefixed alternative numeric library")
+ (description "@code{python-prefixed} provides an alternative
+implementation of the built-in float which supports formatted output with
+SI (decimal) and IEC (binary) prefixes.")
+ (license license:mpl2.0)))
+
(define-public python-pathvalidate
(package
(name "python-pathvalidate")
--
2.41.0
A
A
Arun Isaac wrote on 11 Mar 19:22 +0100
[PATCH 5/7] gnu: Add python-enlighten.
(address . 69736@debbugs.gnu.org)(name . Arun Isaac)(address . arunisaac@systemreboot.net)
e898c033567c0fbb277bea27759d64f774a473af.1710180345.git.arunisaac@systemreboot.net
* gnu/packages/python-xyz.scm (python-enlighten): New variable.

Change-Id: Ifcc1e71cf012eb3b7e2c0ceef7d288cea9fa6a91
---
gnu/packages/python-xyz.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 87204b392f..755f64e361 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -17869,6 +17869,28 @@ (define-public python-progressbar33
;; Either or both of these licenses may be selected.
(license (list license:lgpl2.1+ license:bsd-3))))
+(define-public python-enlighten
+ (package
+ (name "python-enlighten")
+ (version "1.12.4")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "enlighten" version))
+ (sha256
+ (base32
+ "0psvglhi0c7d7pdk9rfb8scnv3xbq2fz78619x2mxvz094mxkwvm"))))
+ (build-system pyproject-build-system)
+ (propagated-inputs
+ (list python-blessed python-prefixed))
+ (home-page "https://github.com/Rockhopper-Technologies/enlighten")
+ (synopsis "Console progress bar library")
+ (description "@code{python-enlighten} is a console progress bar library
+for Python. The main advantage of Enlighten is that it allows writing to
+stdout and stderr without any redirection or additional code---just print or
+log as you normally would. Enlighten also includes experimental support for
+Jupyter Notebooks.")
+ (license license:mpl2.0)))
+
(define-public python-colorama
(package
(name "python-colorama")
--
2.41.0
A
A
Arun Isaac wrote on 11 Mar 19:22 +0100
[PATCH 6/7] gnu: Add python-cwltool.
(address . 69736@debbugs.gnu.org)(name . Arun Isaac)(address . arunisaac@systemreboot.net)
14b0eae1181ad4bc93beaabd2aa1f2ad0d0884ae.1710180345.git.arunisaac@systemreboot.net
* gnu/packages/bioinformatics.scm (python-cwltool): New variable.

Change-Id: Ic374d4c02b69c4e6c923d1cc17e629d4a3a6221c
---
gnu/packages/bioinformatics.scm | 12 ++++++++++++
1 file changed, 12 insertions(+)

Toggle diff (25 lines)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index b8ecea013c..68577ded17 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -5326,6 +5326,18 @@ (define-public cwltool
CWL descriptions.")
(license license:asl2.0)))
+;; python-toil tightly integrates with cwltool using it as a library. So,
+;; create a library version of cwltool where inputs become propagated inputs.
+(define-public python-cwltool
+ (package
+ (inherit cwltool)
+ (name "python-cwltool")
+ (inputs
+ (list node))
+ (propagated-inputs
+ (modify-inputs (package-inputs cwltool)
+ (delete "node")))))
+
(define-public python-dendropy
(package
(name "python-dendropy")
--
2.41.0
A
A
Arun Isaac wrote on 11 Mar 19:22 +0100
[PATCH 7/7] gnu: Add python-toil.
(address . 69736@debbugs.gnu.org)(name . Arun Isaac)(address . arunisaac@systemreboot.net)
79a9044f3ced28df0b987c383f25b7fb657aac6e.1710180345.git.arunisaac@systemreboot.net
* gnu/packages/bioinformatics.scm (python-toil): New variable.

Change-Id: I1a6c071eaa9169c1e0cd7c74d1117490ee7bf267
---
gnu/packages/bioinformatics.scm | 62 +++++++++++++++++++++++++++++++++
1 file changed, 62 insertions(+)

Toggle diff (82 lines)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 68577ded17..aed57796cd 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -87,6 +87,7 @@ (define-module (gnu packages bioinformatics)
#:use-module (gnu packages databases)
#:use-module (gnu packages datastructures)
#:use-module (gnu packages dlang)
+ #:use-module (gnu packages docker)
#:use-module (gnu packages file)
#:use-module (gnu packages flex)
#:use-module (gnu packages gawk)
@@ -5338,6 +5339,67 @@ (define-public python-cwltool
(modify-inputs (package-inputs cwltool)
(delete "node")))))
+(define-public python-toil
+ (package
+ (name "python-toil")
+ (version "6.1.0")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "toil" version))
+ (sha256
+ (base32
+ "0hwaihnncsfxw0sf3iigvgw6nylrb56lpk8qjadkgazr98dsp4ha"))))
+ (build-system pyproject-build-system)
+ (arguments
+ (list
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'disable-optional-features
+ (lambda _
+ (substitute* "requirements-cwl.txt"
+ ;; Loosen version restrictions on ruamel.yaml.
+ (("ruamel.yaml[<>=.,[:digit:]]*\n") "ruamel.yaml\n")
+ ;; Remove optional dependency on galaxy-util and
+ ;; galaxy-tool-util. TODO: Package these and restore these
+ ;; dependencies.
+ (("galaxy-util[^\n]*") "")
+ (("galaxy-tool-util[^\n]*") ""))
+ ;; Disable optional wdl, wes and mesos features. We don't yet
+ ;; have dependencies packaged for them. TODO: Package these
+ ;; dependencies and enable these features.
+ (substitute* "setup.py"
+ (("\"wdl\",") "")
+ (("toil-cwl-runner = toil.cwl.cwltoil:main \\[cwl\\]")
+ "toil-cwl-runner = toil.cwl.cwltoil:main")
+ (("'toil-wdl-runner = toil.wdl.wdltoil:main \\[wdl\\]',") "")
+ (("'toil-wes-cwl-runner = toil.server.cli.wes_cwl_runner:main \\[server\\]',") "")
+ (("'_toil_mesos_executor = toil.batchSystems.mesos.executor:main \\[mesos\\]',") "")))))))
+ (propagated-inputs
+ (list python-addict
+ python-cachecontrol
+ python-configargparse
+ python-cwltool
+ python-dateutil
+ python-dill
+ python-docker
+ python-enlighten
+ python-psutil
+ python-pypubsub
+ python-pytz
+ python-pyyaml
+ python-requests
+ python-ruamel.yaml
+ python-typing-extensions-4.10
+ python-urllib3))
+ (home-page "https://github.com/DataBiosphere/toil")
+ (synopsis "Scalable, efficient and cross-platform workflow engine")
+ (description "Toil is a scalable, efficient, cross-platform pipeline
+management system, written entirely in Python, and designed around the
+principles of functional programming.")
+ (license (list license:asl2.0 ;; main license
+ license:expat)))) ;; src/toil/batchSystems/lsf.py and
+ ;; src/toil/batchSystems/lsfHelper.py
+
(define-public python-dendropy
(package
(name "python-dendropy")
--
2.41.0
?