Should ‘guix import pypi’ get dependencies from pyproject files?

  • Done
  • quality assurance status badge
Details
4 participants
  • Lars-Dominik Braun
  • Ludovic Courtès
  • Sharlatan Hellseher
  • Tanguy LE CARROUR
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 25 Mar 2024 12:06
(address . bug-guix@gnu.org)
877chqtvsk.fsf@inria.fr
Hello Python team!

Should ‘guix import pypi’ attempt to get dependency information from
‘pyproject.toml’, in addition to ‘requirements.txt’ and wheel ‘METADATA’
as it already does?

It might be more complicated than we’d like: in some cases, that file
seems to be used as a “trampoline” to Poetry. For instance, in
python-pypugjs, the ‘requires’ bit delegates everything to Poetry:

Toggle snippet (26 lines)
[tool.poetry]
name = "pypugjs"
version = "5.9.4"
description = ""
authors = ["Andy Grabow <andy@freilandkiwis.de>"]
license = "MIT"

[tool.poetry.dependencies]
python = "^3.8"
Jinja2 = "^3.1.1"
Mako = "^1.1.3"
tornado = "^6.0.4"
six = "^1.15.0"
coverage = "^6.3.2"
nose = "^1.3.7"
Flask = "^2.1.1"
charset-normalizer = "^2.1.0"
flake8 = "^4.0.1"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Thoughts?

Ludo’.
S
S
Sharlatan Hellseher wrote on 25 Mar 2024 20:28
Re: bug#69997: Should ‘guix import pypi’ get dep endencies from pyproject files?
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 69997@debbugs.gnu.org)
CAO+9K5o4xpKnhXcBX81LQ3ct6oS+1y-7mr5JVT=4fVwLE9QkMA@mail.gmail.com
Hi Lido!

Toggle quote (4 lines)
> Should ‘guix import pypi’ attempt to get dependency
> information fromn‘pyproject.toml’, in addition to
> ‘requirements.txt’ and wheel ‘METADATA’ as it already does?

It's quite a common practice in modern Python just to include
pyproject.toml, that fact makes importing long chains problematic.

It would be nice to have common yaml/toml parser for that task.

Oleg
Attachment: file
T
T
Tanguy LE CARROUR wrote on 26 Mar 2024 08:54
Re: bug#69997: Should ‘guix import pypi’ get dependencies from pyproject files?
171143965734.4362.8405481724513962855@bioneland.org
Hi Ludo’,


Quoting Ludovic Courtès (2024-03-25 12:06:51)
Toggle quote (8 lines)
> Should ‘guix import pypi’ attempt to get dependency information from
> ‘pyproject.toml’, in addition to ‘requirements.txt’ and wheel ‘METADATA’
> as it already does?
>
> It might be more complicated than we’d like: in some cases, that file
> seems to be used as a “trampoline” to Poetry. For instance, in
> python-pypugjs, the ‘requires’ bit delegates everything to Poetry:

Short answer: no! ?

I’m pretty sure you know everything that I’m about to write, but better
say it out loud…

For a "standard modern" project managed with Poetry, the Python source
package contains `PKG-INFO` and `pyproject.toml ` that both contain
the run time dependencies. The wheel package only contains `METADATA` that
lists the dependencies. The source only contains a `pyproject.toml`.
To make the installed package as small as possible, tests files and
uncompiled assets are not (should not be) included.
From a Guix stand point, it’s better to build from source to be able to
run the test suite.

For the `python-pypugjs` you used as an example, we build from source,
so I guess the question does not arise. If we were to use the packages
available on PyPI, what I said above is *NOT* confirmed ?:
- wheel (`.whl`) only contains `METADATA` with the dependencies; **BUT**
- source (`.tar.gz`) contains `PKG-INFO` (without dependency information),
`pyproject.toml` (with dep’) and `setup.py` (also with dep’).

… "fun" fact, the information in `pyproject.toml` are **NOT** the same as
the one in `setup.py`!? ? `pyproject.toml` says that `nose` is a run time
dependency (which it is not), but `setup.py` properly lists it in `tests_require`.

So, my answer would be: do not import from PyPI! Yes, I know, it’s radical! ?
But if you have to, rely on the wheel’s `METADATA` file.

I hope this make sense. … I’m not really sure any more! ?

Regards,

--
Tanguy
L
L
Ludovic Courtès wrote on 26 Mar 2024 17:04
(name . Tanguy LE CARROUR)(address . tanguy@bioneland.org)
87o7b1ou6z.fsf@inria.fr
Hello!

Tanguy LE CARROUR <tanguy@bioneland.org> skribis:

Toggle quote (3 lines)
> I’m pretty sure you know everything that I’m about to write, but better
> say it out loud…

Nope, I know nothing (I’ve just been told about using ‘pyproject.toml’
and it seemed to kinda make sense. :-))

Toggle quote (20 lines)
> For a "standard modern" project managed with Poetry, the Python source
> package contains `PKG-INFO` and `pyproject.toml ` that both contain
> the run time dependencies. The wheel package only contains `METADATA` that
> lists the dependencies. The source only contains a `pyproject.toml`.
> To make the installed package as small as possible, tests files and
> uncompiled assets are not (should not be) included.
> From a Guix stand point, it’s better to build from source to be able to
> run the test suite.
>
> For the `python-pypugjs` you used as an example, we build from source,
> so I guess the question does not arise. If we were to use the packages
> available on PyPI, what I said above is *NOT* confirmed ?:
> - wheel (`.whl`) only contains `METADATA` with the dependencies; **BUT**
> - source (`.tar.gz`) contains `PKG-INFO` (without dependency information),
> `pyproject.toml` (with dep’) and `setup.py` (also with dep’).
>
> … "fun" fact, the information in `pyproject.toml` are **NOT** the same as
> the one in `setup.py`!? ? `pyproject.toml` says that `nose` is a run time
> dependency (which it is not), but `setup.py` properly lists it in `tests_require`.

Oh my, such a mess.

Toggle quote (5 lines)
> So, my answer would be: do not import from PyPI! Yes, I know, it’s radical! ?
> But if you have to, rely on the wheel’s `METADATA` file.
>
> I hope this make sense. … I’m not really sure any more! ?

It does!

But then I mean, we could offer, say, ‘guix import upstream https://…’,
and that thing could parse ‘setup.py’ or similar to produce a package
definition from that. Maybe that’s what you had in mind: import
straight from upstream rather than via PyPI?

Thanks,
Ludo’.
T
T
Tanguy LE CARROUR wrote on 26 Mar 2024 17:55
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
171147212395.13452.3553266600176809571@bioneland.org
Hi,


Quoting Ludovic Courtès (2024-03-26 17:04:52)
Toggle quote (12 lines)
> Tanguy LE CARROUR <tanguy@bioneland.org> skribis:
> > So, my answer would be: do not import from PyPI! Yes, I know, it’s radical! ?
> > But if you have to, rely on the wheel’s `METADATA` file.
> >
> > I hope this make sense. … I’m not really sure any more! ?
>
> It does!
>
> But then I mean, we could offer, say, ‘guix import upstream https://…’,
> and that thing could parse ‘setup.py’ or similar to produce a package
> definition from that.

I’m not against a good-old-generic-solution®, but this one might be
a bit… too generic. It contains no mention to Python, so the next logical step
would be `guix import URL`. Not that I would not like it, though! ?

So I would say… let’s wait and see what the others think. In the
meantime, I’ll have to dive deeper in the PEP and the actual importer
code.


Toggle quote (2 lines)
> Maybe that’s what you had in mind: import straight from upstream rather than via PyPI?

I actually had nothing in mind! I blacked out after sending this email.
Too many emotions at once, I guess! I had no clue it was such a mess. ?

--
Tanguy
T
T
Tanguy LE CARROUR wrote on 26 Mar 2024 18:14
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
171147325018.15415.4526491903307851998@bioneland.org
Hi,

Quoting Tanguy LE CARROUR (2024-03-26 17:55:23)
Toggle quote (17 lines)
> Quoting Ludovic Courtès (2024-03-26 17:04:52)
> > Tanguy LE CARROUR <tanguy@bioneland.org> skribis:
> > > So, my answer would be: do not import from PyPI! Yes, I know, it’s radical! ?
> > > But if you have to, rely on the wheel’s `METADATA` file.
> > >
> > > I hope this make sense. … I’m not really sure any more! ?
> >
> > It does!
> >
> > But then I mean, we could offer, say, ‘guix import upstream https://…’,
> > and that thing could parse ‘setup.py’ or similar to produce a package
> > definition from that.
> […]
> So I would say… let’s wait and see what the others think. In the
> meantime, I’ll have to dive deeper in the PEP and the actual importer
> code.

According to PEP 427 [1] a.k.a. Binary distribution format [2], if you
go for packaged/PyPI then we should go for `METADATA`.


But, as stated earlier, we should build from source, to make sure we can
run the test suite. Active projects should slowly migrate to PEP 517 [3]
`pyproject.toml`. But, this is not a solution! ? This is actually yet
another problem! ?


Each build system relies on it’s own file organization. For instance, Poetry
looks for a `[tool.poetry.dependencies]` section in the file. So the
importer should be "build system aware", which leads us to… `guix import poetry URL`!?
Not really generic any more! ?

I guess we should sleep on it…

--
Tanguy
L
L
Lars-Dominik Braun wrote on 27 Mar 2024 07:49
Re: bug#69997: Should ‘gui x import pypi’ get dependencies from pyproject files?
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
ZgPBd4TjMRz8q6lq@noor.fritz.box
Hey Ludo,

Toggle quote (4 lines)
> Should ‘guix import pypi’ attempt to get dependency information from
> ‘pyproject.toml’, in addition to ‘requirements.txt’ and wheel ‘METADATA’
> as it already does?

yes it should. It’s the next logical step after having a
pyproject-build-system. The python-team branch (not sure whether Ricardo
had time to merge it yet) adds a TOML parser to Guix as a first step.

Toggle quote (4 lines)
> It might be more complicated than we’d like: in some cases, that file
> seems to be used as a “trampoline” to Poetry. For instance, in
> python-pypugjs, the ‘requires’ bit delegates everything to Poetry:

The file is always a trampoline to other build systems, like setuptools
or poetry. That’s (unfortunately) by design, see the [build-system]
section.

Toggle quote (12 lines)
> [tool.poetry.dependencies]
> python = "^3.8"
> Jinja2 = "^3.1.1"
> Mako = "^1.1.3"
> tornado = "^6.0.4"
> six = "^1.15.0"
> coverage = "^6.3.2"
> nose = "^1.3.7"
> Flask = "^2.1.1"
> charset-normalizer = "^2.1.0"
> flake8 = "^4.0.1"

That’s unfortunate, because the
specification includes a common field for dependencies:
I guess we’ll have to deal with these idiosyncracies as well :(

Lars
L
L
Ludovic Courtès wrote on 28 Mar 2024 19:09
Re: bug#69997: Should ‘guix import pypi’ get dependencies from pyproject files?
(name . Tanguy LE CARROUR)(address . tanguy@bioneland.org)
87msqikz2q.fsf@inria.fr
Hi,

Tanguy LE CARROUR <tanguy@bioneland.org> skribis:

Toggle quote (2 lines)
> Quoting Ludovic Courtès (2024-03-26 17:04:52)

[...]

Toggle quote (8 lines)
>> But then I mean, we could offer, say, ‘guix import upstream https://…’,
>> and that thing could parse ‘setup.py’ or similar to produce a package
>> definition from that.
>
> I’m not against a good-old-generic-solution®, but this one might be
> a bit… too generic. It contains no mention to Python, so the next logical step
> would be `guix import URL`. Not that I would not like it, though! ?

Well, this has been on my mind for a long time. Such a tool could
provide at least a useful package skeleton even for software using CMake
or Autotools.

Toggle quote (4 lines)
> So I would say… let’s wait and see what the others think. In the
> meantime, I’ll have to dive deeper in the PEP and the actual importer
> code.

Looks like consensus among you Pythonistas has yet to be reached
regarding whether ‘pyproject.toml’ data would be a useful addition. :-)

We’ll see!

Ludo’.

PS: I hear more and more long-time Python developers dismayed by the sad
state of packaging and code evolution in Python. In Guile land, we
say: refugees welcome! Come discover a great language and a great
community (together with their own set of problems).
T
T
Tanguy LE CARROUR wrote on 29 Mar 2024 08:46
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
171169836168.1828.14021738276113751443@bioneland.org
Hi Ludo’,


Quoting Ludovic Courtès (2024-03-28 19:09:49)
Toggle quote (25 lines)
> Tanguy LE CARROUR <tanguy@bioneland.org> skribis:
>
> > Quoting Ludovic Courtès (2024-03-26 17:04:52)
>
> [...]
>
> >> But then I mean, we could offer, say, ‘guix import upstream https://…’,
> >> and that thing could parse ‘setup.py’ or similar to produce a package
> >> definition from that.
> >
> > I’m not against a good-old-generic-solution®, but this one might be
> > a bit… too generic. It contains no mention to Python, so the next logical step
> > would be `guix import URL`. Not that I would not like it, though! ?
>
> Well, this has been on my mind for a long time. Such a tool could
> provide at least a useful package skeleton even for software using CMake
> or Autotools.
>
> > So I would say… let’s wait and see what the others think. In the
> > meantime, I’ll have to dive deeper in the PEP and the actual importer
> > code.
>
> Looks like consensus among you Pythonistas has yet to be reached
> regarding whether ‘pyproject.toml’ data would be a useful addition. :-)

I have to admit that the Zen of Python [1] "There should be one-- and preferably only one --
obvious way to do it." has been recently difficult to follow packaging-wise!


Even Poetry, the one I’ve been using for years, as made questionable
decisions [2] and I have to admit that I had a look at the competitors.



Toggle quote (5 lines)
> PS: I hear more and more long-time Python developers dismayed by the sad
> state of packaging and code evolution in Python. In Guile land, we
> say: refugees welcome! Come discover a great language and a great
> community (together with their own set of problems).

Thanks for the kind invitation, but… I have an acute form of parens-itis.
Seeing more that a pair of parenthesis on a single line make my eyes bleed! ?
And… `#` is for comments, every other use is complete heresy and those
who go against the creed should suffer! … isn’t that what the parentheses
are for?! ?

Any way, thanks again for caring about Python!

Regards,

--
Tanguy
L
L
Ludovic Courtès wrote on 29 Mar 2024 10:06
(name . Tanguy LE CARROUR)(address . tanguy@bioneland.org)
87wmpliezs.fsf@inria.fr
Tanguy LE CARROUR <tanguy@bioneland.org> skribis:

Toggle quote (11 lines)
>> PS: I hear more and more long-time Python developers dismayed by the sad
>> state of packaging and code evolution in Python. In Guile land, we
>> say: refugees welcome! Come discover a great language and a great
>> community (together with their own set of problems).
>
> Thanks for the kind invitation, but… I have an acute form of parens-itis.
> Seeing more that a pair of parenthesis on a single line make my eyes bleed! ?
> And… `#` is for comments, every other use is complete heresy and those
> who go against the creed should suffer! … isn’t that what the parentheses
> are for?! ?

Worry not! As part of our refugees-welcome effort, “we” have put
together tools and guides to help you feel at home:


:-)

Ludo’.
T
T
Tanguy LE CARROUR wrote on 29 Mar 2024 11:11
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
171170706932.12538.2476080876569348495@bioneland.org
Quoting Ludovic Courtès (2024-03-29 10:06:31)
Toggle quote (19 lines)
> Tanguy LE CARROUR <tanguy@bioneland.org> skribis:
>
> >> PS: I hear more and more long-time Python developers dismayed by the sad
> >> state of packaging and code evolution in Python. In Guile land, we
> >> say: refugees welcome! Come discover a great language and a great
> >> community (together with their own set of problems).
> >
> > Thanks for the kind invitation, but… I have an acute form of parens-itis.
> > Seeing more that a pair of parenthesis on a single line make my eyes bleed! ?
> > And… `#` is for comments, every other use is complete heresy and those
> > who go against the creed should suffer! … isn’t that what the parentheses
> > are for?! ?
>
> Worry not! As part of our refugees-welcome effort, “we” have put
> together tools and guides to help you feel at home:
>
> https://hg.sr.ht/~arnebab/wisp
> https://www.draketo.de/py2guile

?… ?… ?!

--
Tanguy
L
L
Lars-Dominik Braun wrote on 15 Dec 2024 17:12
Re: Should ‘guix import pypi’ get dependencies from pyproject files?
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
Z17_3ELEm2rj9u2a@noor.fritz.box
Hi,

Toggle quote (4 lines)
> Should ‘guix import pypi’ attempt to get dependency information from
> ‘pyproject.toml’, in addition to ‘requirements.txt’ and wheel ‘METADATA’
> as it already does?

attached patches allow parsing the standardized pyproject.toml fields
for dependencies. This won’t work for poetry (we need a different
version parser for that), but it’s a start.

Lars
Attachment: 0001-import-pypi-Support-extracting-dependencies-from-pyp.patch
From 0abdb392bf10a99291114fc7e162a3845f25c696 Mon Sep 17 00:00:00 2001
Message-ID: <0abdb392bf10a99291114fc7e162a3845f25c696.1734278914.git.lars@6xq.net>
In-Reply-To: <c2e7e07ad407613233edbb7ebfcc6f0c7c0bcc25.1734278914.git.lars@6xq.net>
References: <c2e7e07ad407613233edbb7ebfcc6f0c7c0bcc25.1734278914.git.lars@6xq.net>
From: Lars-Dominik Braun <lars@6xq.net>
Date: Sun, 15 Dec 2024 13:30:59 +0100
Subject: [PATCH 2/4] import: pypi: Add python-wheel to native inputs if
setuptools is used.

* guix/import/pypi.scm (compute-inputs): Add missing python-wheel if
necessary.

Change-Id: Iedad213a6684856e48349289c4d9beba953f396b
---
guix/import/pypi.scm | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)

Toggle diff (26 lines)
diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 7915d65d23..52ec6e4ee6 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -470,12 +470,18 @@ (define (compute-inputs source-url wheel-url archive)
(type type))))
(sort deps string-ci<?)))
+ (define (add-missing-native-inputs inputs)
+ ;; setuptools cannot build wheels without the python-wheel.
+ (if (member "setuptools" inputs)
+ (cons "wheel" inputs)
+ inputs))
+
;; TODO: Record version number ranges in <upstream-input>.
(let ((dependencies (guess-requirements source-url wheel-url archive)))
(match dependencies
((propagated native)
(append (requirements->upstream-inputs propagated 'propagated)
- (requirements->upstream-inputs native 'native))))))
+ (requirements->upstream-inputs (add-missing-native-inputs native) 'native))))))
(define* (pypi-package-inputs pypi-package #:optional version)
"Return the list of <upstream-input> for PYPI-PACKAGE. This procedure
--
2.45.2
From 0c9708bf7b387f2100cdf375353982fbca9b364e Mon Sep 17 00:00:00 2001
Message-ID: <0c9708bf7b387f2100cdf375353982fbca9b364e.1734278914.git.lars@6xq.net>
In-Reply-To: <c2e7e07ad407613233edbb7ebfcc6f0c7c0bcc25.1734278914.git.lars@6xq.net>
References: <c2e7e07ad407613233edbb7ebfcc6f0c7c0bcc25.1734278914.git.lars@6xq.net>
From: Lars-Dominik Braun <lars@6xq.net>
Date: Sun, 15 Dec 2024 16:56:53 +0100
Subject: [PATCH 3/4] import: pypi: Default to setuptools as build system
input.

* guix/import/pypi.scm (guess-requirements): Default to setuptools if
pyproject.toml does not exist.

Change-Id: I600bd0a44342847878e3a2a7041bd7e7c7d30769
---
guix/import/pypi.scm | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)

Toggle diff (30 lines)
diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index 52ec6e4ee6..bba7361307 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -448,15 +448,21 @@ (define (guess-requirements source-url wheel-url archive)
(((first-propagated first-native) (second-propagated second-native))
(list (append first-propagated second-propagated) (append first-native second-native)))))
+ (define default-pyproject.toml-dependencies
+ ;; If there is no pyproject.toml, we assume it’s an old-style setuptools-based project.
+ '(() ("setuptools")))
+
;; requires.txt and the metadata of a wheel contain redundant information,
;; so fetch only one of them, preferring requires.txt from the source
;; distribution, which we always fetch, since the source tarball also
;; contains pyproject.toml.
(match (guess-requirements-from-source)
((from-pyproject.toml #f)
- (merge (or from-pyproject.toml '(() ())) (or (guess-requirements-from-wheel) '(() ()))))
+ (merge (or from-pyproject.toml default-pyproject.toml-dependencies)
+ (or (guess-requirements-from-wheel) '(() ()))))
((from-pyproject.toml from-requires.txt)
- (merge (or from-pyproject.toml '(() ())) from-requires.txt))))
+ (merge (or from-pyproject.toml default-pyproject.toml-dependencies)
+ from-requires.txt))))
(define (compute-inputs source-url wheel-url archive)
"Given the SOURCE-URL and WHEEL-URL of an already downloaded ARCHIVE, return
--
2.45.2
From 8ab434690c870deb95bfbf61adc60a6a38d084bb Mon Sep 17 00:00:00 2001
Message-ID: <8ab434690c870deb95bfbf61adc60a6a38d084bb.1734278914.git.lars@6xq.net>
In-Reply-To: <c2e7e07ad407613233edbb7ebfcc6f0c7c0bcc25.1734278914.git.lars@6xq.net>
References: <c2e7e07ad407613233edbb7ebfcc6f0c7c0bcc25.1734278914.git.lars@6xq.net>
From: Lars-Dominik Braun <lars@6xq.net>
Date: Sun, 15 Dec 2024 17:02:44 +0100
Subject: [PATCH 4/4] import: pypi: Move deduplication to final processing
step.

* guix/import/pypi.scm (parse-requires.txt): Remove deduplication.
(parse-wheel-metadata): Remove deduplication.
(compute-inputs): Instead do it here on all the collected inputs.

Change-Id: I2504cc693e9bf2e4cc44fd37b5823904dbaaa925
---
guix/import/pypi.scm | 16 ++++++----------
1 file changed, 6 insertions(+), 10 deletions(-)

Toggle diff (43 lines)
diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index bba7361307..530b7d6879 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -283,12 +283,7 @@ (define (parse-requires.txt requires.txt)
(let ((line (read-line port)))
(cond
((eof-object? line)
- ;; Duplicates can occur, since the same requirement can be
- ;; listed multiple times with different conditional markers, e.g.
- ;; pytest >= 3 ; python_version >= "3.3"
- ;; pytest < 3 ; python_version < "3.3"
- (map (compose reverse delete-duplicates)
- (list required-deps test-deps)))
+ (list required-deps test-deps))
((or (string-null? line) (comment? line))
(loop required-deps test-deps inside-test-section? optional?))
((section-header? line)
@@ -342,8 +337,7 @@ (define (parse-wheel-metadata metadata)
(let ((line (read-line port)))
(cond
((eof-object? line)
- (map (compose reverse delete-duplicates)
- (list required-deps test-deps)))
+ (list required-deps test-deps))
((and (requires-dist-header? line) (not (extra? line)))
(loop (cons (specification->requirement-name
(requires-dist-value line))
@@ -486,8 +480,10 @@ (define (compute-inputs source-url wheel-url archive)
(let ((dependencies (guess-requirements source-url wheel-url archive)))
(match dependencies
((propagated native)
- (append (requirements->upstream-inputs propagated 'propagated)
- (requirements->upstream-inputs (add-missing-native-inputs native) 'native))))))
+ (append (requirements->upstream-inputs (delete-duplicates propagated)
+ 'propagated)
+ (requirements->upstream-inputs (delete-duplicates (add-missing-native-inputs native))
+ 'native))))))
(define* (pypi-package-inputs pypi-package #:optional version)
"Return the list of <upstream-input> for PYPI-PACKAGE. This procedure
--
2.45.2
S
S
Sharlatan Hellseher wrote on 16 Dec 2024 02:30
Should ‘guix import pypi’ get dependencies from pyproject files?
(address . 69997@debbugs.gnu.org)
CAO+9K5qzJ2Z3t2RuzmUEp9R3=3FXOG64Kg24HgGoW9y0jPU7Ow@mail.gmail.com
Hi,

As it's pypi importer which improving refresh as well, may
it be pushed to master or we need it on team branch first?

Thanks,
Oleg
Attachment: file
L
L
Ludovic Courtès wrote on 16 Dec 2024 10:00
Re: Should ‘guix import pypi ’ get dependencies from pyproject files?
(name . Lars-Dominik Braun)(address . lars@6xq.net)
871py8f1f3.fsf@inria.fr
Hello,

Lars-Dominik Braun <lars@6xq.net> skribis:

Toggle quote (4 lines)
> attached patches allow parsing the standardized pyproject.toml fields
> for dependencies. This won’t work for poetry (we need a different
> version parser for that), but it’s a start.

Excellent! The code LGTM (I can’t really tell about the Python side of
things).

Toggle quote (3 lines)
> + (list (guess-requirements-from-pyproject.toml dir)
> + (guess-requirements-from-requires.txt dir))))

It looks like we still take info from ‘requirements.txt’; is
‘pyproject.toml’ insufficient?

Anyhow, if others on the Python team approve, I guess you can go ahead
and push it.

Thanks,
Ludo’.
S
S
Sharlatan Hellseher wrote on 16 Dec 2024 11:26
Re: Should ‘guix import pypi’ get dependencies f rom pyproject files?
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
CAO+9K5qH9Sd7Lek8SWrs4SVF=A6488tnZHbKO==6+YWakjhUZQ@mail.gmail.com
Hi Ludovic,

Toggle quote (1 lines)
> It looks like we still take info from ‘requirements.txt’; is
Python packaging ecosystem is not standradisied like in Julia
somteimes you may see all at once setup.cfg, setup.py, pyproject.toml,
requirements*.txt (covering in separate files dev/install deps),

Poetry changed the game as it become popular but it does not follow
PEP in pyproject.toml and just
created it's own config field in it not copatible with pip.

This patch would help to refresh easely most of the astronomy packages
as they are on
pyproject.toml and failed to be refresh with guix refresh CLI.

Lars, do you have a commit access?

Toggle quote (1 lines)
> Anyhow, if others on the Python team approve, I guess you can go ahead
and push it.

I vote for merge as well

Thanks,
Oleg
L
L
Lars-Dominik Braun wrote on 16 Dec 2024 19:42
Re: Should ‘guix import pypi’ get dependencies from pyproject files?
Z2B0lbJJ0A6bQbfW@noor.fritz.box
Hi,

Toggle quote (3 lines)
> It looks like we still take info from ‘requirements.txt’; is
> ‘pyproject.toml’ insufficient?

correct. As a first quality of life improvement I would keep the other
information sources, but as we improve the importer further (by adding
poetry support for example), we need to fall back on them less often
I hope.

Toggle quote (2 lines)
> Lars, do you have a commit access?

I have, but please go ahead and merge it into master. My available time
slots to work on Guix are quite limited, so merging my patches – even
though I have commit access – speeds things up.

Lars
S
S
Sharlatan Hellseher wrote on 16 Dec 2024 20:29
Should ‘guix import pypi†™ get dependencies from pyproject files?
(address . 69997-done@debbugs.gnu.org)
87zfkvv32o.fsf@gmail.com
Hi,

Pushed as

a3ffb920f1 * master import: pypi: Move deduplication to final processing step.
c904350a81 * import: pypi: Default to setuptools as build system input.
d7890af335 * import: pypi: Add python-wheel to native inputs if setuptools is used.
8bb3bb19c2 * import: pypi: Support extracting dependencies from pyproject.toml.

to master.

--
Thanks,
Oleg
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEmEeB3micIcJkGAhndtcnv/Ys0rUFAmdgf68ACgkQdtcnv/Ys
0rWg7xAAi7sxWqF8sx2PuHTi+UXXtT16PYmVYJr56D7IMGtmegq4wnet2qc7AMV4
qZhdIobOxVDhsD7YZ9W+1k7dmWnVNyhndwECQBijZvD8M058JgqV+M0/oEgpE/+r
OaEuSypWpEvIMhIPphg/xdLGBAqfjxVamOvydWt46HhJNtUXfMjPUylJVisF6wLY
9RbzU9bcmBfQTQGN4L+t3XxM/W+Lp4tJZb1Oal9bkGIZ9gP5Cg+0rcZQCi6hSSNx
z18US4oxQ9cta/rl75mak13QT961hhFpJnO0g6f5i36GW3Bq/Ems70iW5TvhxkbK
WDdJ7SAdV8U52ybWdBqeA+ILrDI3/TyRKugdPXMnNG6cQS2EeHJ9R4sJiRa/1K7d
xEomVzx+kLNIAd9NTfjA2dYok6AvW5Le5rHfIFlw8Wkl/+dkOgHeqtaCanS/Njdk
y8OhfSdMlZ+e+p2DPIht4+3K/3oIEp7G6W1qZrMWcBnAad/sg/uLGpSHhWtKVtNs
cb4+8nE8wTbzZba48msQrQuCAZkrYYZeEHA8ezTvWinuQRo2BhEYuBwKtbRMLEXN
Dl263zKW/4jg6fwfGoFLQX34umACRVSaA1Gf1AMQE53V2gxiZ8PNLSmbna5hRD9D
fJ2jJSYqcj/pUL1FOT+zMEmbj1UTCW7VQeXuU9a5L43hf0dngzw=
=qSua
-----END PGP SIGNATURE-----

Closed
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 69997@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 69997
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch