[PATCH 000/714] python native-inputs

  • Done
  • quality assurance status badge
Details
One participant
  • Nicolas Graves
Owner
unassigned
Submitted by
Nicolas Graves
Severity
normal
N
N
Nicolas Graves wrote on 3 May 18:58 +0200
(address . guix-patches@gnu.org)(address . ngraves@ngraves.fr)
20240503165902.25863-1-ngraves@ngraves.fr
How did I proceed ?

1) Identify packages we often don't need as native-inputs + Patches 70570.
2) Systematically remove these packages when they are in native-inputs.
3) Iterate : try to rebuild needed packages, and fixes along the way (these can be updates, packages rewrites, inputs additions...). Back up from the native-input removal when it's not trivial, except for some specific cases (python-dateutil) or sometimes when the package was updated on master.
4) Stop when everything rebuilds properly, except for some specific cases.

The packages that need rebuilding are identified from :
guix graph --type=reverse-package -M 1 python-codecov python-coverage python-black python-isort python-pycodestyle python-pep8 python-pyflakes python-flake8 python-pylint python-mypy python-coveralls python-twine python-pytest-isort python-pytest-flake8 python-pytest-cov python-pytest-black python-pytest-pep8 python-pytest-mypy pre-commit | grep label | cut -d\" -f4 | cut -d@ -f1 | sort | uniq

Some are removed from this list where the package was propagated rather than native, or when the package is left unchanged (e.g. open-adventure), or for those specific cases :
- "chirp" ; needs to rebuild webkitgtk, no thanks
- "cpplint" ; has been updated on master + requires pytest-runner==5.6
- "python-modin" ; this is broken by pandas upgrade, will fix this indenpendently
- "python-libcst" ; broken upstream but unchanged
- "python-schema-salad" ; broken upstream but unchanged
- "python-dolfin-adjoint" ; timeout, but probably works.
- "reprotest" ; needs to rebuild the whole openjdk, no thanks.

This patch series is built upon python-team's branch, with Vinicius' patch series 69980 and my patch series 40416 and 70570 on top. For said packages to build properly, we also need the commit 789286ebd91dac2f85ffc4aa65612f011a8bebd3 from Ricardo (python-zarr update), so it's probably good to cherry pick it in the branch first (or rebase, but that will require more work).

Now, what about results in package node graph reduction?

Here's a summary of number of packages under guix refresh -l :

| package | master | python-team+patch series |
|----------------------+--------+--------------------------|
| python-codecov | 6 | 2 |
| python-coverage | 2341 | 445 |
| python-black | 326 | 34 |
| python-isort | 268 | 10 |
| python-pycodestyle | 386 | 160 |
| python-pep8 | 217 | 321 |
| python-pyflakes | 386 | 166 |
| python-flake8 | 371 | 151 |
| python-pylint | 229 | 6 |
| python-mypy | 341 | 268 |
| python-coveralls | 23 | 1 |
| python-twine | 26 | 1 |
| python-pytest-isort | 174 | 1 |
| python-pytest-flake8 | 296 | 7 |
| python-pytest-cov | 2293 | 272 |
| python-pytest-black | 272 | 6 |
| python-pytest-pep8 | 31 | 151 |
| python-pytest-mypy | 274 | 1 |
| pre-commit | 123 | 1 |

This means e.g. that an update of python-pylint will trigger 6 packages rebuilds rather than 229. Keep in mind that master isn't yet merged in python-team, so it's normal that some figures might appear higher in the second column (but will diminish once master is merged anyway).

Nicolas Graves (714):
gnu: python-transient: Remove python-black native-input.
gnu: python-ovh: Remove python-black native-input.
gnu: python-funsor: Remove python-black native-input.
gnu: python-imbalanced-learn: Remove python-black native-input.
gnu: python-hyperopt: Remove python-black native-input.
gnu: python-captum: Remove python-black native-input.
gnu: python-pyro-ppl: Remove python-black native-input.
gnu: python-trezor: Remove python-black native-input.
gnu: python-openai: Remove python-black native-input.
gnu: python-starsessions-for-pytorch-lightning: Remove python-black.
gnu: python-graphtools: Remove python-black native-input.
gnu: python-jupytext: Improve package style.
gnu: python-pydantic-cli: Remove python-black native-input.
gnu: python-imageio: Remove python-black native-input.
gnu: python-isort: Improve package style.
gnu: python-imageio: Remove python-black native-input.
gnu: python-minikanren: Remove python-black native-input.
gnu: python-eliot: Remove python-black native-input.
gnu: python-update-checker: Remove python-black native-input.
gnu: python-doubletdetection: Remove python-black native-input.
gnu: python-liana-py: Remove python-black native-input.
gnu: scvelo: Remove python-black native-input.
gnu: python-black: Improve package style.
gnu: calibre: Remove python-flake8 native-input.
gnu: mdpo: Remove python-flake8 native-input.
gnu: python-apprise: Remove python-flake8 native-input.
gnu: python-colorful: Remove python-flake8 native-input.
gnu: python-pymd4c: Remove python-flake8 native-input.
gnu: python-pyls-black: Remove python-flake8 native-input.
gnu: python-pypugjs: Remove python-flake8 native-input.
gnu: python-docx: Remove python-flake8 native-input.
gnu: python-importlib-resources: Remove python-flake8 native-input.
gnu: python-jaraco-test: Remove python-flake8 native-input.
gnu: python-seaborn: Remove python-flake8 native-input.
gnu: python-progressbar2: Remove python-flake8 native-input.
gnu: python-codespell: Remove python-flake8 native-input.
gnu: python-inflect: Remove python-flake8 native-input.
gnu: python-argcomplete: Remove python-mypy native-input.
gnu: python-diff-cover: Remove python-flake8 native-input.
gnu: python-validators: Remove python-flake8 native-input.
gnu: python-marshmallow: Remove python-flake8 native-input.
gnu: python-marshmallow-jsonapi: Remove python-flake8 native-input.
gnu: python-crds: Remove python-flake8 native-input.
gnu: python-doubletdetection: Remove python-flake8 native-input.
gnu: python-biom-format: Remove python-flake8 native-input.
gnu: scvelo: Remove python-flake8 native-input.
gnu: python-docker-pycreds: Remove python-flake8 native-input.
gnu: python-scikit-rf: Remove python-flake8 native-input.
gnu: python-mt-940: Remove python-flake8 native-input.
gnu: python-geopy: Remove python-flake8 native-input.
gnu: python-pygsp: Remove python-flake8 native-input.
gnu: python-uqbar: Remove python-flake8 native-input.
gnu: python-funsor: Remove python-flake8 native-input.
gnu: python-imbalanced-learn: Remove python-flake8 native-input.
gnu: python-captum: Remove python-flake8 native-input.
gnu: python-pyro-api: Remove python-flake8 native-input.
gnu: python-pyro-ppl: Remove python-flake8 native-input.
gnu: python-linear-operator: Remove python-flake8 native-input.
gnu: python-gpytorch: Remove python-flake8 native-input.
gnu: python-mutagen: Remove python-flake8 native-input.
gnu: python-pure-protobuf: Remove python-flake8 native-input.
gnu: python-ovh: Remove python-flake8 native-input.
gnu: python-furl: Remove python-flake8 native-input.
gnu: python-httplib2: Remove python-flake8 native-input.
gnu: python-http-ece: Remove python-flake8 native-input.
gnu: python-woob: Remove python-flake8 native-input.
gnu: python-starsessions-for-pytorch-lightning: Remove python-flake8
native-input.
gnu: python-apispec: Remove python-flake8 native-input.
gnu: python-apispec-webframeworks: Remove python-flake8 native-input.
gnu: python-flasgger: Remove python-flake8 native-input.
gnu: python-yq: Remove python-flake8 native-input.
gnu: python-update-checker: Remove python-flake8 native-input.
gnu: python-eliot: Remove python-flake8 native-input.
gnu: python-dateparser: Remove python-flake8 native-input.
gnu: python-versioneer: Remove python-flake8 native-input.
gnu: python-jinja2-cli: Remove python-flake8 native-input.
gnu: nikola: Remove python-flake8 native-input.
gnu: python-verspec: Remove python-flake8 native-input.
gnu: python-mike: Remove python-flake8 native-input.
gnu: python-arpeggio: Remove python-flake8 native-input.
gnu: python-dolfin-adjoint: Remove python-flake8 native-input.
gnu: python-statsmodels: Remove python-flake8 native-input.
gnu: python-panflute: Remove python-flake8 native-input.
gnu: sshuttle: Remove python-flake8 native-input.
gnu: qtile: Remove python-flake8 native-input.
gnu: python-http-ece: Update to 1.2.0.
gnu: python-androguard: Remove python-coverage native-input.
gnu: python-astropy: Remove python-coverage native-input.
gnu: python-poliastro: Remove python-coverage native-input.
gnu: python-ega-download-client: Remove python-coverage native-input.
gnu: python-scikit-bio: Remove python-coverage native-input.
gnu: python-cooler: Remove python-coverage native-input.
gnu: u-boot-tools: Remove python-coverage native-input.
gnu: python-pytest-randomly: Remove python-coverage native-input.
gnu: python-pytest-flakes: Remove python-coverage native-input.
gnu: python-aiounittest: Remove python-coverage native-input.
gnu: python-aiosqlite: Remove python-coverage native-input.
gnu: reprotest: Remove python-coverage native-input.
gnu: python-django-localflavor: Remove python-coverage native-input.
gnu: python-django-haystack: Remove python-coverage native-input.
gnu: python-scikit-rf: Remove python-coverage native-input.
gnu: python-geopy: Remove python-coverage native-input.
gnu: gpodder: Remove python-coverage native-input.
gnu: python-mygpoclient: Remove python-coverage native-input.
gnu: python-graphtools: Remove python-coverage native-input.
gnu: python-pygsp: Remove python-coverage native-input.
gnu: python-jupyter-packaging: Remove python-coverage native-input.
gnu: python-jupyter-server: Remove python-coverage native-input.
gnu: onnx: Remove python-coverage native-input.
gnu: onnx-optimizer: Remove python-coverage native-input.
gnu: python-lightning-utilities: Remove python-coverage native-input.
gnu: python-pytorch-lightning: Remove python-coverage native-input.
gnu: python-torchmetrics: Remove python-coverage native-input.
gnu: python-gpytorch: Remove python-coverage native-input.
gnu: python-pylast: Remove python-coverage native-input.
gnu: python-anaconda-client: Remove python-coverage native-input.
gnu: conda: Remove python-coverage native-input.
gnu: python-vcrpy: Remove python-coverage native-input.
gnu: python-re-assert: Remove python-coverage native-input.
gnu: python-multivolumefile: Remove python-coverage native-input.
gnu: python-pybcj: Remove python-coverage native-input.
gnu: python-bcj-cffi: Remove python-coverage native-input.
gnu: python-pyppmd: Remove python-coverage native-input.
gnu: python-ppmd-cffi: Remove python-coverage native-input.
gnu: python-py7zr: Remove python-coverage native-input.
gnu: python-ovh: Remove python-coverage native-input.
gnu: python-flask-misaka: Remove python-coverage native-input.
gnu: python-openid: Remove python-coverage native-input.
gnu: python-zope-interface: Remove python-coverage native-input.
gnu: python-zope-i18nmessageid: Update to 5.1.1.
gnu: python-zope-hookable: Remove python-coverage native-input.
gnu: python-url: Remove python-coverage native-input.
gnu: python-transaction: Remove python-coverage native-input.
gnu: python-sanic: Remove python-coverage native-input.
gnu: python-woob: Remove python-coverage native-input.
gnu: python-flask-combo-jsonapi: Remove python-coverage native-input.
gnu: python-apprise: Remove python-coverage native-input.
gnu: python-colorful: Remove python-coverage native-input.
gnu: python-lunr: Remove python-coverage native-input.
gnu: python-mdit-py-plugins: Remove python-coverage native-input.
gnu: python-bidict: Remove python-coverage native-input.
gnu: python-sh: Remove python-coverage native-input.
gnu: python-cftime: Remove python-coverage native-input.
gnu: python-can: Remove python-coverage native-input.
gnu: python-jsonargparse: Remove python-coverage native-input.
gnu: python-extension-helpers: Remove python-coverage native-input.
gnu: python-pyjwt: Remove python-coverage native-input.
gnu: python-virtualenv-clone: Remove python-coverage native-input.
gnu: python-pypugjs: Remove python-coverage native-input.
gnu: python-lsp-server: Remove python-coverage native-input.
gnu: python-pikepdf: Remove python-coverage native-input.
gnu: python-cram: Remove python-coverage native-input.
gnu: python-tasklogger: Remove python-coverage native-input.
gnu: python-notebook: Remove python-coverage native-input.
gnu: python-structlog: Remove python-coverage native-input.
gnu: binwalk: Remove python-coverage native-input.
gnu: python-argcomplete: Remove python-coverage native-input.
gnu: python-attrs: Remove python-coverage native-input.
gnu: python-yq: Remove python-coverage native-input.
gnu: python-stone: Remove python-coverage native-input.
gnu: python-identify: Remove python-coverage native-input.
gnu: python-nodeenv: Remove python-coverage native-input.
gnu: python-anyio: Remove python-coverage native-input.
gnu: python-forbiddenfruit: Remove python-coverage native-input.
gnu: python-watchgod: Remove python-coverage native-input.
gnu: python-eliot: Remove python-coverage native-input.
gnu: python-pem: Remove python-coverage native-input.
gnu: python-scikit-build: Remove python-coverage native-input.
gnu: python-minikanren: Remove python-coverage native-input.
gnu: nikola: Remove python-coverage native-input.
gnu: python-verspec: Remove python-coverage native-input.
gnu: python-mike: Remove python-coverage native-input.
gnu: python-arpeggio: Remove python-coverage native-input.
gnu: python-dolfin-adjoint: Remove python-coverage native-input.
gnu: python-emcee: Remove python-coverage native-input.
gnu: python-rpy2: Remove python-coverage native-input.
gnu: rtv: Remove python-coverage native-input.
gnu: tuir: Remove python-coverage native-input.
gnu: python-halo: Remove python-coverage native-input.
gnu: python-log-symbols: Remove python-coverage native-input.
gnu: python-spinners: Remove python-coverage native-input.
gnu: python-panflute: Remove python-coverage native-input.
gnu: pre-commit: Remove python-coverage native-input.
gnu: python-httpretty: Remove python-coverage native-input.
gnu: python-xmltodict: Remove python-coverage native-input.
gnu: python-affine: Remove python-flake8 native-input.
gnu: python-cartopy: Remove python-coveralls native-input.
gnu: python-graphtools: Remove python-coveralls native-input.
gnu: python-pygsp: Remove python-coveralls native-input.
gnu: python-multivolumefile: Remove python-coveralls native-input.
gnu: python-py7zr: Remove python-coveralls native-input.
gnu: python-trimesh: Remove python-coveralls native-input.
gnu: python-flask-combo-jsonapi: Remove python-coveralls native-input.
gnu: python-affine: Remove python-coveralls native-input.
gnu: python-cftime: Remove python-coveralls native-input.
gnu: python-ddlparse: Remove python-coveralls native-input.
gnu: python-tasklogger: Remove python-coveralls native-input.
gnu: python-minikanren: Remove python-coveralls native-input.
gnu: python-arpeggio: Remove python-coveralls native-input.
gnu: tgcli: Remove python-coveralls native-input.
gnu: rtv: Remove python-coveralls native-input.
gnu: tuir: Remove python-coveralls native-input.
gnu: python-scipy: Remove python-pycodestyle native-input.
gnu: python-jsonargparse: Remove python-pycodestyle native-input.
gnu: python-lsp-server: Remove python-pycodestyle native-input.
gnu: python-orderedmultidict: Remove python-pycodestyle native-input.
gnu: python-progressbar2: Remove python-pycodestyle native-input.
gnu: python-stem: Remove python-pycodestyle native-input.
gnu: python-versioneer: Remove python-pycodestyle native-input.
gnu: python-versioneer: Remove python-pyflakes native-input.
gnu: python-re-assert: Remove python-covdefaults native-input.
gnu: python-can: Remove python-codecov native-input.
gnu: python-androguard: Remove python-codecov native-input.
gnu: python-wiimatch: Remove python-codecov native-input.
gnu: python-ddlparse: Remove python-codecov native-input.
gnu: python-poliastro: Remove python-mypy native-input.
gnu: python-psycopg: Remove python-mypy native-input.
gnu: uranium: Remove python-mypy native-input.
gnu: cura: Remove python-mypy native-input.
gnu: python-uqbar: Remove python-mypy native-input.
gnu: python-imbalanced-learn: Remove python-mypy native-input.
gnu: python-captum: Remove python-mypy native-input.
gnu: python-torchmetrics: Remove python-mypy native-input.
gnu: python-pyro-ppl: Remove python-mypy native-input.
gnu: python-nptyping: Remove python-mypy native-input.
gnu: python-scipy: Remove python-mypy native-input.
gnu: python-huggingface-hub: Remove python-mypy native-input.
gnu: python-asgiref: Remove python-mypy native-input.
gnu: python-starsessions-for-pytorch-lightning: Remove python-mypy
native-input.
gnu: python-fastapi: Remove python-mypy native-input.
gnu: python-pyls-black: Remove python-mypy native-input.
gnu: kalamine: Remove python-mypy native-input.
gnu: python-pydantic-cli: Remove python-mypy native-input.
gnu: python-immutables: Remove python-mypy native-input.
gnu: python-duckdb: Remove python-mypy native-input.
gnu: python-marshmallow: Remove python-mypy native-input.
gnu: python-apispec: Remove python-mypy native-input.
gnu: python-typeguard: Remove python-mypy native-input.
gnu: python-trio-typing: Remove python-mypy native-input.
gnu: python-verspec: Remove python-mypy native-input.
gnu: python-catalogue: Remove python-mypy native-input.
gnu: python-msgspec: Remove python-mypy native-input.
gnu: python-transient: Remove python-mypy native-input.
gnu: python-aionotify: Update to 0.3.0.
gnu: vdirsyncer: Upgrade to pyproject-build-system and enable tests.
gnu: radicale: Remove python-pytest-cov native-input.
gnu: vdirsyncer: Remove python-pytest-cov native-input.
gnu: python-django-contact-form: Remove python-pytest-cov
native-input.
gnu: python-django-localflavor: Improve package style.
gnu: python-docker-pycreds: Improve package style.
gnu: Add xcffibgen.
gnu: python-xcffib: Move to (gnu packages xorg).
gnu: qtile: Update to 0.23.0.
gnu: qtile: Improve package style.
gnu: qtile: Remove python-pytest-cov native-input.
gnu: sshuttle: Upgrade to pyproject-build-system.
gnu: Add ghc-language-python.
gnu: python-aiosignal: Remove python-pytest-cov native-input.
gnu: python-aiostream: Remove python-pytest-cov native-input.
gnu: python-apiron: Remove python-pytest-cov native-input.
gnu: python-venusian: Remove python-pytest-cov native-input.
gnu: python-hupper: Remove python-pytest-cov native-input.
gnu: python-astroml: Replace python-pytest-cov native-input.
gnu: python-casa-formats-io: Remove python-pytest-cov native-input.
gnu: python-cdflib: Remove python-pytest-cov native-input.
gnu: python-czml3: Remove python-pytest-cov native-input.
gnu: python-poliastro: Remove python-pytest-cov native-input.
gnu: python-wiimatch: Remove python-pytest-cov native-input.
gnu: python-pysox: Remove python-pytest-cov native-input.
gnu: borgmatic: Remove python-pytest-cov native-input.
gnu: python-biom-format: Remove python-pytest-cov native-input.
gnu: python-screed: Remove python-pytest-cov native-input.
gnu: python-pyfaidx: Remove python-pytest-cov native-input.
gnu: python-cooler: Remove python-p
This message was truncated. Download the full message here.
N
N
Nicolas Graves wrote on 4 May 00:18 +0200
[PATCH 001/714] gnu: python-transient: Remove python-black native-input.
(address . 70735@debbugs.gnu.org)(address . ngraves@ngraves.fr)
20240503224254.28001-1-ngraves@ngraves.fr
* gnu/packages/virtualization.scm (python-transient):
[native-inputs]: Remove python-black.

Change-Id: Id965a243ebf42ad249e63e9ad6f4fb22e39c3876
---
gnu/packages/virtualization.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (15 lines)
diff --git a/gnu/packages/virtualization.scm b/gnu/packages/virtualization.scm
index f366388623..1bfa31b7d3 100644
--- a/gnu/packages/virtualization.scm
+++ b/gnu/packages/virtualization.scm
@@ -2777,7 +2777,7 @@ (define-public python-transient
python-requests
python-toml))
(native-inputs
- (list python-black python-mypy python-pyhamcrest python-twine))
+ (list python-mypy python-pyhamcrest python-twine))
(home-page "https://github.com/ALSchwalm/transient")
(synopsis "QEMU Wrapper written in Python")
(description
--
2.41.0
N
N
Nicolas Graves wrote on 4 May 00:18 +0200
[PATCH 002/714] gnu: python-ovh: Remove python-black native-input.
(address . 70735@debbugs.gnu.org)(address . ngraves@ngraves.fr)
20240503224254.28001-2-ngraves@ngraves.fr
* gnu/packages/python-web.scm (python-ovh):
[native-inputs]: Remove python-black.

Change-Id: I62a0ac9c37a72ab5e7e979001bb347c2c75a8be9
---
gnu/packages/python-web.scm | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)

Toggle diff (16 lines)
diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index bd9c50a006..3fccdcdf59 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -974,8 +974,7 @@ (define-public python-ovh
(build-system pyproject-build-system)
(arguments (list #:tests? #f)) ; XXX: tests require networking
(propagated-inputs (list python-requests))
- (native-inputs (list python-black
- python-coverage
+ (native-inputs (list python-coverage
python-flake8
python-isort
python-pytest
--
2.41.0
N
N
Nicolas Graves wrote on 4 May 00:18 +0200
[PATCH 003/714] gnu: python-funsor: Remove python-black native-input.
(address . 70735@debbugs.gnu.org)(address . ngraves@ngraves.fr)
20240503224254.28001-3-ngraves@ngraves.fr
* gnu/packages/machine-learning.scm (python-funsor):
[native-inputs]: Remove python-black.

Change-Id: I2049ce0c0701f91a883646d40bd985f8a7bda7d9
---
gnu/packages/machine-learning.scm | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)

Toggle diff (16 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index e11a851105..62fe0d9980 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -173,8 +173,7 @@ (define-public python-funsor
(propagated-inputs (list python-makefun python-multipledispatch
python-numpy python-opt-einsum
python-typing-extensions))
- (native-inputs (list python-black
- python-flake8
+ (native-inputs (list python-flake8
python-isort
python-nbsphinx
python-pandas
--
2.41.0
N
N
Nicolas Graves wrote on 4 May 00:18 +0200
[PATCH 004/714] gnu: python-imbalanced-learn: Remove python-black native-input.
(address . 70735@debbugs.gnu.org)(address . ngraves@ngraves.fr)
20240503224254.28001-4-ngraves@ngraves.fr
* gnu/packages/machine-learning.scm (python-imbalanced-learn):
[native-inputs]: Remove python-black.

Change-Id: I238b32a50b8a025466108e1c836bac7c6e06a3fb
---
gnu/packages/machine-learning.scm | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)

Toggle diff (16 lines)
diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm
index 62fe0d9980..cec9b63f17 100644
--- a/gnu/packages/machine-learning.scm
+++ b/gnu/packages/machine-learning.scm
@@ -1904,8 +1904,7 @@ (define-public python-imbalanced-learn
(getcwd)))))))
(propagated-inputs (list python-joblib python-numpy python-scikit-learn
python-scipy python-threadpoolctl))
- (native-inputs (list python-black
- python-flake8
+ (native-inputs (list python-flake8
python-keras
python-mypy
python-numpydoc
--
2.41.0
N
N
Nicolas Graves wrote on 4 May 00:18 +0200
[PATCH 005/714] gnu: python-hyperopt: Remove python-black native-input.
(address . 70735@debbugs.gnu.org)(address . ngraves@ngraves.fr)</