pypi importer generates wrong pypi-uri without underscore

OpenSubmitted by Ricardo Wurmus.
Details
3 participants
  • Maxim Cournoyer
  • Xinglu Chen
  • Ricardo Wurmus
Owner
unassigned
Severity
normal
R
R
Ricardo Wurmus wrote on 13 Apr 10:17 +0200
(address . bug-guix@gnu.org)
87pmyytx8d.fsf@elephly.net
When importing a package from Pypi whose name includes an underscore, the importer will generate a pypi-uri expression where the underscore has been replaced with a dash.
-- Ricardo
X
X
Xinglu Chen wrote on 13 Apr 20:54 +0200
87y2dmf20v.fsf@yoctocell.xyz
On Tue, Apr 13 2021, Ricardo Wurmus wrote:
Toggle quote (4 lines)> When importing a package from Pypi whose name includes an > underscore, the importer will generate a pypi-uri expression where > the underscore has been replaced with a dash.
Hmm, I just grabbed a random package from PyPi, and the URI seems fineto me.
Toggle snippet (30 lines)~/src/guix $ ./pre-inst-env guix import pypi angreal_data_science_projectStarting download of /tmp/guix-file.4MH43LFromhttps://files.pythonhosted.org/packages/28/5a/6c77f8f5c2333ab2d3e83c27615358a0badcfeb6513a36079ba0fee78524/angreal_data_science_project-0.5.5.tar.gz... …_project-0.5.5.tar.gz 15KiB 378KiB/s 00:00 [##################] 100.0%
Starting download of /tmp/guix-file.hrC4cLFromhttps://files.pythonhosted.org/packages/f4/18/e4747e2f3232ca1fbbd1286455fe57f3390a94b14c26a37dba72eb829523/angreal_data_science_project-0.5.5-py3-none-any.whl... …ect-0.5.5-py3-none-any.whl 28KiB 244KiB/s 00:00 [##################] 100.0%(package (name "python-angreal-data-science-project") (version "0.5.5") (source (origin (method url-fetch) (uri (pypi-uri "angreal_data_science_project" version)) (sha256 (base32 "1krvhwmlmv77j8w3sxaxlkmhd0xzmr24f4n873p00631lacwmpck")))) (build-system python-build-system) (home-page "https://gitlab.com/angreal/data-science-project") (synopsis "An angreal template for data science projects.") (description "An angreal template for data science projects.") (license #f))
How would I reproduce the bug?
R
R
Ricardo Wurmus wrote on 13 Apr 23:39 +0200
(name . Xinglu Chen)(address . public@yoctocell.xyz)(address . 47746@debbugs.gnu.org)
87h7k9uanf.fsf@elephly.net
Xinglu Chen <public@yoctocell.xyz> writes:
Toggle quote (12 lines)> On Tue, Apr 13 2021, Ricardo Wurmus wrote:>>> When importing a package from Pypi whose name includes an >> underscore, the importer will generate a pypi-uri expression >> where >> the underscore has been replaced with a dash.>> Hmm, I just grabbed a random package from PyPi, and the URI > seems fine> to me. […]> How would I reproduce the bug?
You’re right, this only affects some packages. Here are two examples:
Toggle snippet (6 lines)./pre-inst-env guix import pypi -r jupyterlab_widgetsfollowing redirection to `https://pypi.org/pypi/jupyterlab-widgets/json'...[…] (uri (pypi-uri "jupyterlab-widgets" version)) […]
Toggle snippet (6 lines)./pre-inst-env guix import pypi -r jupyter_packagingfollowing redirection to `https://pypi.org/pypi/jupyter-packaging/json'...[…] (uri (pypi-uri "jupyter-packaging" version)) […]
They both have a redirection in common, but the pypi-uri is incorrect as it needs to have the underscore.
-- Ricardo
X
X
Xinglu Chen wrote on 14 Apr 14:45 +0200
(name . Ricardo Wurmus)(address . rekado@elephly.net)(address . 47746@debbugs.gnu.org)
871rbdhw6g.fsf@yoctocell.xyz
On Tue, Apr 13 2021, Ricardo Wurmus wrote:
Toggle quote (8 lines)> You’re right, this only affects some packages. Here are two > examples:>> [...]>> They both have a redirection in common, but the pypi-uri is > incorrect as it needs to have the underscore.
Looking at the JSON object athttps://pypi.org/pypi/jupyterlab-widgets/json, it specifies“jupyterlabs-widgets” as ‘info.downloads.name’, which is what the PyPiimporter reads to determine the PyPi URI.
Would it make sense to just make it read the name that the userspecifies in the shell, i.e. ‘./pre-inst-env guix import pypiPACKAGE-NAME’ (see the patch below)? Or will this potentially mess upother PyPi URIs?
From ec5612e9d4c2026541874c6a3eaf7c9009ce4a31 Mon Sep 17 00:00:00 2001Message-Id: <ec5612e9d4c2026541874c6a3eaf7c9009ce4a31.1618404153.git.public@yoctocell.xyz>From: Xinglu Chen <public@yoctocell.xyz>Date: Wed, 14 Apr 2021 14:37:01 +0200Subject: [PATCH] import: pypi: Use package name from the shell to construct the URI.
Some packages will list an incorrect package name in the JSON object,resulting in PyPi URIs that lead to nowhere.
* guix/import/pypi.scm (pypi->guix-package): Use the package name specified inthe shell (guix import pypi PACKAGE-NAME) to construct the PyPi URI.
Fixes https://issues.guix.gnu.org/47746.Reported by Ricardo Wurmus <rekado@elephly.net>.--- guix/import/pypi.scm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Toggle diff (25 lines)diff --git a/guix/import/pypi.scm b/guix/import/pypi.scmindex bf4dc50138..be9470ad26 100644--- a/guix/import/pypi.scm+++ b/guix/import/pypi.scm@@ -9,6 +9,7 @@ ;;; Copyright © 2020 Lars-Dominik Braun <ldb@leibniz-psychology.org> ;;; Copyright © 2020 Arun Isaac <arunisaac@systemreboot.net> ;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>+;;; Copyright © 2021 Xinglu Chen <public@yoctocell.xyz> ;;; ;;; This file is part of GNU Guix. ;;;@@ -483,7 +484,7 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE." (leave (G_ "no source release for pypi package ~a ~a~%") (project-info-name info) (project-info-version info)))))- (make-pypi-sexp (project-info-name info)+ (make-pypi-sexp package-name (project-info-version info) (and=> (latest-source-release project) distribution-url)
base-commit: 82543e9649da2da9a5285ede4ec4f718fd740fcb-- 2.31.1
M
M
Maxim Cournoyer wrote on 19 Apr 04:37 +0200
(name . Xinglu Chen)(address . public@yoctocell.xyz)
87pmyrhueb.fsf@gmail.com
Hi,
Xinglu Chen <public@yoctocell.xyz> writes:
Toggle quote (30 lines)> On Tue, Apr 13 2021, Ricardo Wurmus wrote:>>> You’re right, this only affects some packages. Here are two >> examples:>>>> [...]>>>> They both have a redirection in common, but the pypi-uri is >> incorrect as it needs to have the underscore.>> Looking at the JSON object at> <https://pypi.org/pypi/jupyterlab-widgets/json>, it specifies> “jupyterlabs-widgets” as ‘info.downloads.name’, which is what the PyPi> importer reads to determine the PyPi URI.>> Would it make sense to just make it read the name that the user> specifies in the shell, i.e. ‘./pre-inst-env guix import pypi> PACKAGE-NAME’ (see the patch below)? Or will this potentially mess up> other PyPi URIs?>>>From ec5612e9d4c2026541874c6a3eaf7c9009ce4a31 Mon Sep 17 00:00:00 2001> Message-Id: <ec5612e9d4c2026541874c6a3eaf7c9009ce4a31.1618404153.git.public@yoctocell.xyz>> From: Xinglu Chen <public@yoctocell.xyz>> Date: Wed, 14 Apr 2021 14:37:01 +0200> Subject: [PATCH] import: pypi: Use package name from the shell to construct> the URI.>> Some packages will list an incorrect package name in the JSON object,> resulting in PyPi URIs that lead to nowhere.
In my opinion, if a package contains wrong values in their metadata,it's a problem that should be reported to that package. As with otherthings, garbage in -> garbage out; Guix cannot do much here. I seem torecall that some older PyPI mirror had that problem. It may be thattheir metadata name -> URI is not consistent. We could try to find whatthat 2nd scheme is and attempt both, although that's not very elegantand it probably points to packages being very outdated to start with.
Thanks,
Maxim
?