python-anaconda-client and conda fail

  • Done
  • quality assurance status badge
Details
3 participants
  • Etienne B. Roesch
  • Hugo Buddelmeijer
  • Simon Tournier
Owner
unassigned
Submitted by
Etienne B. Roesch
Severity
normal
E
E
Etienne B. Roesch wrote on 4 May 2023 16:46
(address . bug-guix@gnu.org)
CAPX-MzA4uZv5Heu2O6n-=7oRLFGfUWdy+6r9VYDCmF6XOeJVJg@mail.gmail.com
Hi,

I am new to guix, but it looks like both python-anaconda-client and conda
fail to build, as ci.guix also confirms. I wish I could readily help, but I
am really new to this :)

Etienne
Attachment: file
H
H
Hugo Buddelmeijer wrote on 8 May 2023 13:31
(address . 63277@debbugs.gnu.org)
CA+Jv8O02Aqc-y8WmkXR3=zFDjBmN+-SnFt6Ca+eCkQaRa70HJw@mail.gmail.com
Hi Etienne,

I've noticed that conda (22.9.0) does not build as well, and have
resolved the issue in my local guix clone.

It required these changes:
- Add python-conda-package-streaming 0.7.0.
- Upgrade python-conda-package-handling from 1.7.3 to 2.1.0 (which has
python-conda-package-streaming as a dependency).
- Remove python-anaconda-client as a dependency of conda.
- Make conda a dependency of python-anaconda-client.

I'm also relatively new to guix. I think it is prefered to have small
patches, so I will send four patches for each of the above steps to
this issue as described in [1]. Then hopefully someone will review
them and merge them. Then after that it might be worthwhile to update
conda itself too.

Some notes:
- python-conda-package-streaming has conda itself as an optional
dependency for testing. I have therefore disabled all the tests.
- python-conda-package-streaming also removes the need for libarchive,
which apparently required some workarounds, so those are also removed.
- It is not clear to me why python-anaconda-client was a dependency of
conda in the first place, so maybe I'm missing something.
- I've also fixed some linting warnings on the existing packages.

Cheers,
Hugo

H
H
Hugo Buddelmeijer wrote on 8 May 2023 14:04
[PATCH 1/5] gnu: Add python-conda-package-streaming.
(address . 63277@debbugs.gnu.org)(name . Hugo Buddelmeijer)(address . hugo@buddelmeijer.nl)
324846ea5414ea4c382de41e604c19d8566c7cf6.1683547087.git.hugo@buddelmeijer.nl
* gnu/packages/python-xyz.scm (python-conda-package-streaming): New variable.
---
gnu/packages/python-xyz.scm | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)

Toggle diff (46 lines)
diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index a5c99a6f20..c627e456e1 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -1688,6 +1688,37 @@ (define-public python-clyent
by @code{binstar}, @code{binstar-build}, and @code{chalmers}.")
(license license:bsd-3)))
+(define-public python-conda-package-streaming
+ (package
+ (name "python-conda-package-streaming")
+ (version "0.7.0")
+ (source (origin
+ (method url-fetch)
+ (uri (pypi-uri "conda_package_streaming" version))
+ (sha256
+ (base32
+ "01a8yhfiww3ac0glnb41iqcrz5n6ya1k0w4vwc5wg06qcp6ipbq2"))))
+ (build-system pyproject-build-system)
+ (arguments
+ ;; All tests require conda to be installed. However conda requires
+ ;; python-conda-package-handling, which requires
+ ;; python-conda-package-streaming (this package), so the tests
+ ;; cannot be run.
+ `(#:tests? #f))
+ (propagated-inputs (list python-requests python-zstandard))
+ (native-inputs (list python-pytest
+ python-pytest-cov
+ python-pytest-mock
+ python-flit
+ python-bottle
+ python-boto3))
+ (home-page "https://github.com/conda/conda-package-streaming")
+ (synopsis
+ "Download metadata from conda packages without transferring entire file.")
+ (description
+ "Download conda metadata from packages without transferring entire file. Get metadata from local .tar.bz2 packages without reading entire files.")
+ (license license:bsd-3)))
+
(define-public python-babel
(package
(name "python-babel")

base-commit: d1aba42ad4e1909faa21d484975c5954c778e002
--
2.39.2
H
H
Hugo Buddelmeijer wrote on 8 May 2023 14:04
[PATCH 2/5] gnu: python-conda-package-handling: Update to 2.1.0
(address . 63277@debbugs.gnu.org)(name . Hugo Buddelmeijer)(address . hugo@buddelmeijer.nl)
fdb18beeabd100567599263bb76f75d43ef19e10.1683547087.git.hugo@buddelmeijer.nl
* gnu/packages/package-management.scm (python-conda-package-handling): Update to 2.1.0
---
gnu/packages/package-management.scm | 21 ++++++++-------------
1 file changed, 8 insertions(+), 13 deletions(-)

Toggle diff (48 lines)
diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 9869dfaacf..1ebc95077b 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -952,7 +952,7 @@ (define-public python-anaconda-client
(define-public python-conda-package-handling
(package
(name "python-conda-package-handling")
- (version "1.7.3")
+ (version "2.1.0")
(source
(origin
(method git-fetch)
@@ -962,25 +962,20 @@ (define-public python-conda-package-handling
(file-name (git-file-name name version))
(sha256
(base32
- "1dq6f5ks3cinb355x712bls9bvv6bli6x3c43sdkqvawdw8xgv9j"))))
+ "17k363s2ad8cfngyrazzqvq287dab64ssah2rhzj4h7v1mfy47bm"))))
(build-system python-build-system)
(arguments
`(#:phases
(modify-phases %standard-phases
- (add-after 'unpack 'use-unmodified-libarchive
- (lambda _
- (substitute* "setup.py"
- (("archive_and_deps") "archive"))))
(replace 'check
- (lambda* (#:key inputs outputs #:allow-other-keys)
- (add-installed-pythonpath inputs outputs)
- (invoke "pytest" "-vv" "tests"))))))
+ (lambda* (#:key tests? inputs outputs #:allow-other-keys)
+ (when tests?
+ (add-installed-pythonpath inputs outputs)
+ (invoke "pytest" "-vv" "tests")))))))
(propagated-inputs
- (list python-six python-tqdm))
- (inputs
- (list libarchive))
+ (list python-conda-package-streaming))
(native-inputs
- (list python-cython python-pytest python-pytest-cov
+ (list python-wheel python-pytest python-pytest-cov
python-pytest-mock python-mock))
(home-page "https://conda.io")
(synopsis "Create and extract conda packages of various formats")
--
2.39.2
H
H
Hugo Buddelmeijer wrote on 8 May 2023 14:04
[PATCH 3/5] gnu: conda: Remove python-anaconda-client dependency.
(address . 63277@debbugs.gnu.org)(name . Hugo Buddelmeijer)(address . hugo@buddelmeijer.nl)
49e908fe2967fc31209c1fbce36370969b209daa.1683547087.git.hugo@buddelmeijer.nl
* gnu/packages/package-management.scm (conda): Remove python-anaconda-client dependency.
---
gnu/packages/package-management.scm | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)

Toggle diff (34 lines)
diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 1ebc95077b..2e2c116c33 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -1101,23 +1101,21 @@ (define-public conda
(inputs
(list python-wrapper))
(propagated-inputs
- (list python-anaconda-client
- python-boto3
+ (list python-boto3
python-conda-package-handling
python-cytoolz
python-pluggy
python-pycosat
- python-pytest
python-pyyaml
python-requests
python-responses
python-ruamel.yaml
- python-tqdm
- ;; XXX: This is dragged in by libarchive and is needed at runtime.
- zstd))
+ python-tqdm))
(native-inputs
(list python-coverage
python-flaky
+ python-mock
+ python-pytest
python-pytest-timeout
python-pytest-xprocess))
(home-page "https://github.com/conda/conda")
--
2.39.2
H
H
Hugo Buddelmeijer wrote on 8 May 2023 14:04
[PATCH 4/5] gnu: python-defusedxml: Update to 0.7.1.
(address . 63277@debbugs.gnu.org)(name . Hugo Buddelmeijer)(address . hugo@buddelmeijer.nl)
2834aeee4ebcbd41a9f7e15ea291dad459c38bca.1683547087.git.hugo@buddelmeijer.nl
* gnu/packages/xml.scm (defused-python): Update to 0.7.1.
---
gnu/packages/xml.scm | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Toggle diff (26 lines)
diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm
index 2bc4f66c01..c4ab832bc5 100644
--- a/gnu/packages/xml.scm
+++ b/gnu/packages/xml.scm
@@ -1439,16 +1439,16 @@ (define-public xlsx2csv
(define-public python-defusedxml
(package
(name "python-defusedxml")
- (version "0.6.0")
+ (version "0.7.1")
(source
(origin
(method url-fetch)
(uri (pypi-uri "defusedxml" version))
(sha256
(base32
- "1xbp8fivl3wlbyg2jrvs4lalaqv1xp9a9f29p75wdx2s2d6h717n"))))
+ "0s9ym98jrd819v4arv9gmcr6mgljhxd9q866sxi5p4c5n4nh7cqv"))))
(build-system python-build-system)
- (home-page "https://bitbucket.org/tiran/defusedxml")
+ (home-page "https://github.com/tiran/defusedxml")
(synopsis "XML bomb protection for Python stdlib modules")
(description
"Defusedxml provides XML bomb protection for Python stdlib modules.")
--
2.39.2
H
H
Hugo Buddelmeijer wrote on 8 May 2023 14:04
[PATCH 5/5] gnu: python-anaconda-client: Update to 1.11.2
(address . 63277@debbugs.gnu.org)(name . Hugo Buddelmeijer)(address . hugo@buddelmeijer.nl)
c9d38caf6137502e1848eb9a203c4f73a3bdcf92.1683547087.git.hugo@buddelmeijer.nl
* gnu/packages/package-management.scm (python-anaconda-client): Update to 1.11.2.
---
gnu/packages/package-management.scm | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)

Toggle diff (46 lines)
diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 2e2c116c33..ea5679fb44 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -900,7 +900,7 @@ (define-public rpm
(define-public python-anaconda-client
(package
(name "python-anaconda-client")
- (version "1.8.0")
+ (version "1.11.2")
(source
(origin
(method git-fetch)
@@ -910,10 +910,10 @@ (define-public python-anaconda-client
(file-name (git-file-name name version))
(sha256
(base32
- "1vyk0g0gci4z9psisb8h50zi3j1nwfdg1jw3j76cxv0brln0v3fw"))))
+ "1cd1ycpa9b4i5hn7fqksddk6iky2ap6gw6f3l8fy95ypdjyk2z3d"))))
(build-system python-build-system)
(propagated-inputs
- (list python-clyent python-nbformat python-pyyaml python-requests))
+ (list python-clyent python-nbformat python-pyyaml python-requests python-requests-toolbelt python-tqdm python-defusedxml python-conda-package-handling conda))
(native-inputs
(list python-coverage
python-dateutil
@@ -934,12 +934,11 @@ (define-public python-anaconda-client
"tests/test_authorizations.py"
"tests/test_login.py"
"tests/test_whoami.py"
- "utils/notebook/tests/test_data_uri.py"
- "utils/notebook/tests/test_base.py"
- "utils/notebook/tests/test_downloader.py"
- "inspect_package/tests/test_conda.py")))
- (with-directory-excursion "binstar_client"
- (for-each delete-file network-tests)))
+ "tests/utils/notebook/test_data_uri.py"
+ "tests/utils/notebook/test_base.py"
+ "tests/utils/notebook/test_downloader.py"
+ "tests/utils/test_conda.py")))
+ (for-each delete-file network-tests))
#t)))))
(home-page "https://github.com/Anaconda-Platform/anaconda-client")
(synopsis "Anaconda Cloud command line client library")
--
2.39.2
H
H
Hugo Buddelmeijer wrote on 8 May 2023 14:22
python-anaconda-client and conda fail
(address . 63277@debbugs.gnu.org)
CA+Jv8O20DgGHqd7JwEcb77P3MouiTLw3M3LLj-cT7y+0CGKPEQ@mail.gmail.com
Sending the patches seems to have worked out just fine. Except that
number 2 got in my own spam folder.

I forgot that I also needed to update python-defusedxml to 0.7.1.

Using guix does feel a bit like a moving target. It is good to learn
how to fix such issues yourself, because they will pop-up often. The
Contributing section [1] of the manual is a good place to start.

Something unrelated: my desktop (gnome) looks sooo much sharper it
seems, so kudos to whoever improved that!

Cheers,
Hugo

H
H
Hugo Buddelmeijer wrote on 11 Sep 2023 10:08
(address . 63277@debbugs.gnu.org)
CA+Jv8O3_Zf4CEq6V87oZ96iLcS5OB33UVaaG-hMBfFc_TwdcQw@mail.gmail.com
The conda package seems to be fixed, and python-anaconda-client also
builds. So I think this can be closed.

On Mon, 8 May 2023 at 14:22, Hugo Buddelmeijer <hugo@buddelmeijer.nl> wrote:
Toggle quote (17 lines)
>
> Sending the patches seems to have worked out just fine. Except that
> number 2 got in my own spam folder.
>
> I forgot that I also needed to update python-defusedxml to 0.7.1.
>
> Using guix does feel a bit like a moving target. It is good to learn
> how to fix such issues yourself, because they will pop-up often. The
> Contributing section [1] of the manual is a good place to start.
>
> Something unrelated: my desktop (gnome) looks sooo much sharper it
> seems, so kudos to whoever improved that!
>
> Cheers,
> Hugo
>
> [1] https://guix.gnu.org/manual/en/html_node/Contributing.html
S
S
Simon Tournier wrote on 11 Sep 2023 18:21
Re: bug#63277: python-anaconda-client and conda fail
87jzswhe9d.fsf@gmail.com
Hi,

On Mon, 11 Sep 2023 at 10:08, Hugo Buddelmeijer <hugo@buddelmeijer.nl> wrote:

Toggle quote (3 lines)
> The conda package seems to be fixed, and python-anaconda-client also
> builds. So I think this can be closed.

Thanks for the follow-up. I am closing.

Feel free to reopen if I misread.

Cheers,
simon
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 63277@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 63277
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch