Request for merging "python-team" branch

Hi Guix!

I think it's a time to review and merge python-team branch.

It will bring refactored pyproject-build-system, refreshed collection of

packages from python-build module, refreshed python-numpy (1.24),

default python-pytest (8+), fresh python-flask, some clean up from

python-nose which is deprecated for few years.

Please test and report for any issues of the package you

love/need/demand.

--

Oleg

CC Python team for wider spread.

Hey there,

https://qa.guix.gnu.org/says it’s your turn to merge! :-)

Ludo’.

Hi Sharlatan,

Guix cheerleader here. Go for it! Merge ahoy!

??

LGTM

Hi,

python-team branch has been rebased, conflicts resolved (python-redis

and python-pydevd) and merged to master.

Long lists of rebuild are in CI right now:

- https://ci.guix.gnu.org/eval/1903757

- https://ci.guix.gnu.org/eval/1903820

Post merge fix-up might be required like adding missing build time

dependencies (python-setuptools, python-wheel) for

pyproject-build-system packages not refreshed for a while.

--

Thanks,

Oleg

Hi all,

Since this merge landed, the builds for several Python packages in my personal channel broke. Any package using pyproject-build-system for a Python project using setuptools seems to be affected. This python-manhole package[1] is an example. It's about as simple as they can get, with no inputs or custom build steps. It's failing with:

------------------------------------------------------------

starting phase `build'

Using 'setuptools.build_meta' to build wheels, auto-detected '#f', override '#f'.

Prepending '[]' to sys.path, auto-detected '#f', override '#f'.

Traceback (most recent call last):

File "<string>", line 6, in <module>

File "/gnu/store/jjcka1g6sk2cvwx8nm4fdwpdq3vll0v0-python-3.10.7/lib/python3.10/importlib/__init__.py", line 126, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "<frozen importlib._bootstrap>", line 1050, in _gcd_import

File "<frozen importlib._bootstrap>", line 1027, in _find_and_load

File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked

File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed

File "<frozen importlib._bootstrap>", line 1050, in _gcd_import

File "<frozen importlib._bootstrap>", line 1027, in _find_and_load

File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked

ModuleNotFoundError: No module named 'setuptools'

error: in phase 'build': uncaught exception:

%exception #<&invoke-error program: "python" arguments: ("-c" "import sys, importlib, json\nbackend_path = json.loads (sys.argv[1]) or []\nbackend_path.extend (sys.path)\nsys.path = backend_path\nconfig_settings = json.loads (sys.argv[4])\nbuilder = importlib.import_module(sys.argv[2])\nbuilder.build_wheel(sys.argv[3], config_settings=config_settings)" "[]" "setuptools.build_meta" "dist" "{}") exit-status: 1 term-signal: #f stop-signal: #f>

phase `build' failed after 0.0 seconds

command "python" "-c" "import sys, importlib, json\nbackend_path = json.loads (sys.argv[1]) or []\nbackend_path.extend (sys.path)\nsys.path = backend_path\nconfig_settings = json.loads (sys.argv[4])\nbuilder = importlib.import_module(sys.argv[2])\nbuilder.build_wheel(sys.argv[3], config_settings=config_settings)" "[]" "setuptools.build_meta" "dist" "{}" failed with status 1

build process 10 exited with status 256

------------------------------------------------------------

Since it's complaining about setuptools, I thought I might need to add that to the native-inputs. Alas, that fails with a different error:

------------------------------------------------------------

starting phase `build'

Using 'setuptools.build_meta' to build wheels, auto-detected '#f', override '#f'.

Prepending '[]' to sys.path, auto-detected '#f', override '#f'.

usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]

or: -c --help [cmd1 cmd2 ...]

or: -c --help-commands

or: -c cmd --help

error: invalid command 'bdist_wheel'

------------------------------------------------------------

Is there somewhere I can find out how to fix these packages for the updated pyproject-build-system? Should they be getting switched to python-build-system?

Noting also that the PyPI module for `guix import' hardcodes the pyproject-build-system, so it will generate unbuildable definitions for any Python project which uses setuptools.

Thanks,

-- Ian

[1]: https://codeberg.org/ieure/atomized-guix/src/branch/main/atomized/packages/python-xyz.scm#L22

On Fri, Dec 13, 2024, at 5:00 PM, jgart wrote:

Hi Ian,

On Saturday, December 14th, 2024 at 5:00 PM, Ian Eure <ian@retrospec.tv> wrote:

I don't have any comment about the pyproject-build-system issue in general, but I've done enough maintenance of Python packages in general to be familiar with this error. It should be as simple as adding `python-wheel` to the native-inputs--it is the module that allows setuptools to generate .whl files, and I'd be surprised if additional packages were needed (I've encountered having to add explicit setuptools and wheel dependencies to requirements.txt files for pip when changing the build environment of packages such as changing distros or Python versions). HTH!

Cheers,

Kaelyn

Hi Ian,

as Sharlatan Hellseher wrote in https://issues.guix.gnu.org/issue/74715#4,

you need to add python-setuptools and python-wheel to your

setuptools-based packages. The default python toolchain used by

pyproject-build-system (python-sans-pip-wrapper from

gnu/packages/python.scm) does not include these packages any more,

since they are technically not required and declaring them as *real*

inputs allows using different versions of these packages more easily

for packages, which require them. Plus there are quite a few packages,

which build using different build systems nowadays.

The python importer should probably be updated to read pyproject.toml

and parse the [build-system] table (there is a toml parser in Guix now,

so this should be easy).

Lars

Hi,

Maybe we may put some news about changes in pyproject build system?

Thanks

Oleg

Hi Lars-Dominik,

On Sun, Dec 15, 2024, at 6:28 AM, Lars-Dominik Braun wrote:

Thanks, this worked for me. I skimmed the related bug, but missed this comment. I think the docs for pyproject-build-system are likely the best place for this, as they already mention some of the setuptools/pyproject interaction. I sent a patch (#74899) with some draft language, let me know what you think.

Would it be helpful to open a bug about this?

Thanks,

-- Ian

I confirm, this line

(inputs (list python-setuptools python-wheel))

fixes the issue for me.

However, ‘guix lint’ complaints about python-setuptools not necessary.

--

Cayetano Santos

GnuPG Key: https://meta.sr.ht/~csantosb.pgp

FingerPrint: CCB8 1842 F9D7 058E CD67 377A BF5C DF4D F6BF 6682

On Sun, Dec 15, 2024 at 10:29:03AM +0000, Sharlatan Hellseher wrote:

I think that's a great idea.

Hi,

there is https://issues.guix.gnu.org/69997,which includes patches to

add initial pyproject.toml support to `guix import pypi`.

Lars