[PATCH 0/4]: gnu: Add parfive

  • Done
  • quality assurance status badge
Details
2 participants
  • Christopher Baines
  • Sharlatan Hellseher
Owner
unassigned
Submitted by
Sharlatan Hellseher
Severity
normal
S
S
Sharlatan Hellseher wrote on 14 Oct 2022 23:08
(address . guix-patches@gnu.org)
CAO+9K5qffko8R+3E3WO91yKC3sv4VoXcFUydNTxgB++=VZSxgQ@mail.gmail.com
Hi Guix team!
This parch series adds parfive - Python module and CLI command for
parallel download from HTTP and FTP. This project comes as part of
inputs to some of Astronomical package I'm working on right now and
useful by it's own.
One thing to mention, tests for aioftp and siosocks produces
depreciation warnings which may be solved by upgrading
python-pytest-asynio to 0.19.0 (I'm not dare to touch as it's part of
600+ inputs), see the issues:
Toggle quote (1 lines)
> ./pre-inst-env guix build --rounds=2 python-siosocks python-aioftp python-pytest-socket parfive
/gnu/store/6kj27ljj8js9cn94kfzqn7ld1q5p7kql-parfive-2.0.1
/gnu/store/bzzzi0h42yqvb67x6d486gnwxxdpbrcr-python-pytest-socket-0.5.1
/gnu/store/iyh72cbvq4k6zcsyadpfp1y28nyz27wz-python-aioftp-0.21.4
/gnu/store/1vv1fipiv7vjc4g5akrx8rl2c1p8bz0v-python-siosocks-0.3.0
Sharlatan Hellseher (4):
gnu: Add siosocks
gnu: Add aioftp
gnu: Add pytest-socket
gnu: Add parfive
gnu/packages/python-check.scm | 49 +++++++++++++++
gnu/packages/python-web.scm | 115 ++++++++++++++++++++++++++++++++++
2 files changed, 164 insertions(+)
base-commit: cd0d27f30dc2274fbf8477f44275d9184521f044
--
… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
?????????? ?? ? ????????? ??????? ????? ? ?????????????????.
S
S
Sharlatan Hellseher wrote on 14 Oct 2022 23:11
[PATCH 1/4]: gnu: Add siosocks
(address . 58530@debbugs.gnu.org)
CAO+9K5rNgKH_6JgK7t_-RWBRAGR5xi71LGAJa-0Jk--zaDYwXg@mail.gmail.com
--
… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
?????????? ?? ? ????????? ??????? ????? ? ?????????????????.
From ac31ebaf1111f89802a33ed1b0fe85e8a49299b3 Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Fri, 14 Oct 2022 20:59:11 +0100
Subject: [PATCH 1/4] gnu: Add siosocks

* gnu/packages/python-web.scm (python-siosocks): New variable.
---
gnu/packages/python-web.scm | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)

Toggle diff (50 lines)
diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 6a9d6ee61d..a00c8b7683 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -6674,6 +6674,43 @@ (define-public python-socksio
SOCKS protocols. It can be paired with any I/O library.")
(license license:expat)))
+(define-public python-siosocks
+ (package
+ (name "python-siosocks")
+ (version "0.3.0")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "siosocks" version))
+ (sha256
+ (base32
+ "0qqxy8wl5mrmlkblzjq9nsg0cbm5jwgj409mhnhq6gd1ypvbndms"))))
+ (build-system python-build-system)
+ (arguments
+ (list #:phases #~(modify-phases %standard-phases
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ (invoke "pytest" "-vvv")))))))
+ (native-inputs (list python-pytest python-pytest-asyncio python-pytest-cov
+ python-pytest-trio))
+ (propagated-inputs (list python-trio))
+ (home-page "https://github.com/pohmelie/siosocks")
+ (synopsis "SOCKSv4 & SOCKSv5 TCP proxy protocol implementation in Python")
+ (description
+ "This package provides a Python module and framework for sans-io socks proxy
+client/server with couple io backends.
+
+Features:
+@itemize
+@item Only TCP connect (no BIND, no UDP)
+@item Both client and server
+@item SOCKS versions: 4, 4a, 5
+@item SOCKSv5 auth: no auth, username/password
+@item Couple io backends: @code{asyncio}, @code{trio}, @code{socketserver}
+@item One-shot socks server (@code{python -m siosocks})
+@end itemize")
+ (license license:expat)))
+
(define-public python-msrest
(package
(name "python-msrest")
--
2.37.3
S
S
Sharlatan Hellseher wrote on 14 Oct 2022 23:11
[PATCH 2/4]: gnu: Add aioftp
(address . 58530@debbugs.gnu.org)
CAO+9K5qA8LK19hnODuWsmcvRCF5rd2KvrajeDczqru3g_V+B0g@mail.gmail.com
--
… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
?????????? ?? ? ????????? ??????? ????? ? ?????????????????.
From 274047ed5622650a6e1d232cd5cec39cd9a4f516 Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Fri, 14 Oct 2022 21:04:56 +0100
Subject: [PATCH 2/4] gnu: Add aioftp

* gnu/packages/python-web.scm (python-aioftp): New variable.
---
gnu/packages/python-web.scm | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)

Toggle diff (49 lines)
diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index a00c8b7683..3c4d179dcc 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -3037,6 +3037,42 @@ (define-public python-requests-ftp
adapter for use with the Requests library.")
(license license:asl2.0)))
+(define-public python-aioftp
+ (package
+ (name "python-aioftp")
+ (version "0.21.4")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "aioftp" version))
+ (sha256
+ (base32
+ "1f8vql2j2b3ykqyh5bxzsp8x5f2if2c1ya232ld3hz3cc7a2dfr8"))))
+ (build-system python-build-system)
+ (arguments
+ (list #:phases #~(modify-phases %standard-phases
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ (invoke "pytest" "-vvv")))))))
+ (native-inputs (list python-async-timeout python-pytest
+ python-pytest-asyncio python-pytest-cov
+ python-trustme))
+ (propagated-inputs (list python-siosocks))
+ (home-page "https://aioftp.readthedocs.io/")
+ (synopsis "FTP client/server for asyncio in Python")
+ (description
+ "FTP client and server for asyncio (Python 3) Library implementing FTP
+protocol, both client and server for Python asyncio module.
+
+ Supported commands as client: USER, PASS, ACCT, PWD, CWD, CDUP, MKD, RMD,
+ MLSD, MLST, RNFR, RNTO, DELE, STOR, APPE, RETR, TYPE, PASV, ABOR, QUIT,
+ REST, LIST (as fallback).
+
+ Supported commands as server: USER, PASS, QUIT, PWD, CWD, CDUP, MKD, RMD,
+ MLSD, LIST (non-standard), MLST, RNFR, RNTO, DELE, STOR, RETR,
+ TYPE (\"I\" and \"A\"), PASV, ABOR, APPE, REST.")
+ (license license:asl2.0)))
+
(define-public python-msal
(package
(name "python-msal")
--
2.37.3
S
S
Sharlatan Hellseher wrote on 14 Oct 2022 23:12
[PATCH 3/4]: gnu: Add pytest-socket
(address . 58530@debbugs.gnu.org)
CAO+9K5o+MNvaWUqruy+rhQF5kBrD1oBKWV1be6-3=3YrkoU=xw@mail.gmail.com
--
… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
?????????? ?? ? ????????? ??????? ????? ? ?????????????????.
From f5e774473b65f1973756d197d09db415bcb0311c Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Fri, 14 Oct 2022 21:42:06 +0100
Subject: [PATCH 3/4] gnu: Add pytest-socket

* gnu/packages/python-check.scm (python-pytest-socket): New variable.
---
gnu/packages/python-check.scm | 49 +++++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)

Toggle diff (62 lines)
diff --git a/gnu/packages/python-check.scm b/gnu/packages/python-check.scm
index 2358e7448d..fd88fc480e 100644
--- a/gnu/packages/python-check.scm
+++ b/gnu/packages/python-check.scm
@@ -395,6 +395,55 @@ (define-public python-vcrpy
interactions, which will update them to correspond to the new API.")
(license license:expat)))
+(define-public python-pytest-socket
+ (package
+ (name "python-pytest-socket")
+ (version "0.5.1")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "pytest-socket" version))
+ (sha256
+ (base32
+ "1dkr86nxkxc0ka3rdnpmk335m8gl1zh1sy8i7w4w1jsidbf82jvw"))))
+ (build-system python-build-system)
+ (arguments
+ ;; FIXME: Tests fail a lot, probably requiring Internet access.
+ (list #:tests? #f
+ #:phases #~(modify-phases %standard-phases
+ (replace 'build
+ (lambda _
+ (setenv "SETUPTOOLS_SCM_PRETEND_VERSION"
+ #$version)
+ (setenv "SOURCE_DATE_EPOCH" "315532800")
+ (invoke "python"
+ "-m"
+ "build"
+ "--wheel"
+ "--no-isolation"
+ ".")))
+ (add-before 'check 'disable-unsupported-test
+ (lambda _
+ (substitute* "tests/test_async.py"
+ (("def test_asynctest")
+ "def __off_test_asynctest"))))
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ (invoke "python" "-m" "pytest" "-vvv")))))))
+ (native-inputs (list python-httpx
+ python-poetry-core
+ python-pypa-build
+ python-pytest
+ python-pytest-httpbin
+ python-pytest-randomly
+ python-starlette))
+ (home-page "https://pypi.org/project/pytest-socket/")
+ (synopsis "Pytest plugin to disable socket calls during tests")
+ (description
+ "This package provides Pytest extension which disables all network calls flowing
+through Python's socket interface")
+ (license license:expat)))
+
(define-public python-pytest-ordering
(package
(name "python-pytest-ordering")
--
2.37.3
S
S
Sharlatan Hellseher wrote on 14 Oct 2022 23:12
[PATCH 4/4]: gnu: Add parfive
(address . 58530@debbugs.gnu.org)
CAO+9K5qTFYEJXWKZqzk_OhHTX5jH+u4kU0GxjJaKiGeY4+CUdA@mail.gmail.com
--
… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
?????????? ?? ? ????????? ??????? ????? ? ?????????????????.
From c51bba6b106a31f773d9dfe31dc4bc6d3c1050aa Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Fri, 14 Oct 2022 21:45:52 +0100
Subject: [PATCH 4/4] gnu: Add parfive

* gnu/packages/python-web.scm (parfive): Newv variable
---
gnu/packages/python-web.scm | 42 +++++++++++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)

Toggle diff (55 lines)
diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 3c4d179dcc..c29be41eff 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -983,6 +983,48 @@ (define-public httpie
(properties `((lint-hidden-cve . ("CVE-2019-10751"))))
(license license:bsd-3)))
+(define-public parfive
+ (package
+ (name "parfive")
+ (version "2.0.1")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "parfive" version))
+ (sha256
+ (base32
+ "19dcbb6g56l5s3ih0bhs3p4acgc0gf4zdzpj4w87m69li2nhmgpx"))))
+ (build-system python-build-system)
+ (arguments
+ (list #:phases #~(modify-phases %standard-phases
+ (add-before 'check 'disable-test-requiring-network
+ (lambda _
+ (substitute* "parfive/tests/test_downloader.py"
+ (("def test_ftp")
+ "def __off_test_ftp"))))
+ (replace 'check
+ (lambda* (#:key tests? #:allow-other-keys)
+ (when tests?
+ (invoke "python" "-m" "pytest" "-vvv" "parfive")))))))
+ (propagated-inputs (list python-aiofiles python-aioftp python-aiohttp
+ python-tqdm))
+ (native-inputs (list python-pytest
+ python-pytest-asyncio
+ python-pytest-cov
+ python-pytest-localserver
+ python-pytest-socket
+ python-setuptools-scm))
+ (home-page "https://parfive.readthedocs.io/")
+ (synopsis "HTTP and FTP parallel file downloader")
+ (description
+ "This package provides CLI tool and Python library @code{parallel} file
+downloader using asyncio. parfive can handle downloading multiple files in
+parallel as well as downloading each file in a number of chunks.
+
+asciicast demo of parfive parfive works by creating a downloader object,
+appending files to it and then running the download. parfive has a synchronous
+API, but uses asyncio to paralellise downloading the files.")
+ (license license:expat)))
+
(define-public python-html2text
(package
(name "python-html2text")
--
2.37.3
S
S
Sharlatan Hellseher wrote on 6 Nov 2022 21:47
(address . 58530@debbugs.gnu.org)
CAO+9K5rx6cZGAsbHhUAP=6juKwaO3ZPZdQCVM=D0_3ugfor4jA@mail.gmail.com
Hi,
A gentle ping on status for these patches.
I've stabilized build for SunPy, the astronomical project for data
analysis for the Sun activity research which has parfive as a hard
dependence.
Thanks,
Oleg
--
… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
?????????? ?? ? ????????? ??????? ????? ? ?????????????????.
C
C
Christopher Baines wrote on 7 Nov 2022 20:55
(name . Sharlatan Hellseher)(address . sharlatanus@gmail.com)
87zgd2o7vp.fsf@cbaines.net
Sharlatan Hellseher <sharlatanus@gmail.com> writes:

Toggle quote (6 lines)
> A gentle ping on status for these patches.
>
> I've stabilized build for SunPy, the astronomical project for data
> analysis for the Sun activity research which has parfive as a hard
> dependence.

Apologies for the delay, I've had a look now and they seem good to me.

I tweaked the commit messages, adding some fullstops in places (just to
match the general Guix convention) and pushed to master as
472917ea6c479b3546ffaea02f755ed9c6e8d46b.

Thanks,

Chris
-----BEGIN PGP SIGNATURE-----

iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmNpYxpfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XeyJA/+JM9uYO1ORDZDwiBVlDvAzVYY7TEa2tZf
rpe6r2oQnn3WZ7ehZkTtB5tvnnwQhgQUOboPIyEvM1tXtx7I6/VyYhT95hHz0QfY
CBxRgQMHKm2kuO/q+runCjIW5/Mfo4sWho1AbvtZ59NrAj3o7FpxfhSXAYk8YELZ
GM2y23zbx85+lKb1P3fbejyf9/umwDZumfmOKB5HOlEFSWuWjKlUPks11JgCrYZ0
as0fzRijTCdKA7jVmEFmADokZe5VTmXZtYXbnDaqsNkmXFrQk9MZQ6Oryl3nUouQ
/Os3jqgN4AcFhiA9liSQDldFxsUvFbnA17pU0Gs89UJdoNIMUoLIOuGsd6QfBdkH
LMEaBmfbZPhbhPIifagJZ/b0tpveU2sCRNSKrwo4p+fQxS/aCK0XLJ3zZ/qcrO3t
UugwHysMvn/xOz3iolYCN31jUpH/ljHqBHJAuGdBGGnLQsRoEbA4uRThs3w0lZTl
JZOv3bsef3Wc2mmL6BcnFJFhvoi5cz+DSLLRmbCZA1TPUYsuRaL3QXcLj6qbpvnh
RSvX9YufgJd1H3yFCa0K5DAtfUCF15GLXKG2iuTo45C87OPMv4fh1mJG2yRNoYA7
FFYZakOj9ZV/Pd5J8Hh4zAzeAnQ5jgEyVsKcdI5pbdZoWLOrPHI+PdEUPoQVF3Np
sIS/tt9tNr0=
=OSx4
-----END PGP SIGNATURE-----

?