[PATCH 0/5] gnu: poetry: Update to 1.0.3.

Hi Guix!

Following my update on the work on poetry [1][], here is a patch series

to update poetry and the required dependencies.

[1][]: https://lists.gnu.org/archive/html/guix-devel/2020-02/msg00293.html

To make it work, I had to disable some tests. Some of those failing

tests were reported upstream, some I'll try to fix later or report upstream.

Regards

Tanguy Le Carrour (5):

gnu: python-pexpect: Update to 4.8.0.

gnu: python-jsonschema: Update to 3.2.0.

gnu: python-fakeredis: Update to 1.2.1.

gnu: python-cachy: Update to 0.3.0.

gnu: poetry: Update to 1.0.3.

gnu/packages/databases.scm | 7 +++--

gnu/packages/python-xyz.scm | 59 ++++++++++++++++++++++++++-----------

2 files changed, 47 insertions(+), 19 deletions(-)

--

2.25.1

[arguments]: Disable failing test.

---

gnu/packages/python-xyz.scm | 9 ++++++---

1 file changed, 6 insertions(+), 3 deletions(-)

[propagated-inputs]: Add python-importlib-metadata.

[arguments]: Disable failing test.

---

gnu/packages/python-xyz.scm | 10 ++++++++--

1 file changed, 8 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-sortedcontainers.

---

gnu/packages/databases.scm | 7 +++++--

1 file changed, 5 insertions(+), 2 deletions(-)

[arguments]: Disable failing tests.

---

gnu/packages/python-xyz.scm | 18 ++++++++++++++++--

1 file changed, 16 insertions(+), 2 deletions(-)

[propagated-inputs] Add python-clikit, add python-importlib-metadata,

add python-keyring, add python-pexpect, remove python-msgpack,

remove python-glob2. [home-page] Update URL.

[arguments] Remove distutils patch, patch versions of keyring

and importlib-metadata.

---

gnu/packages/python-xyz.scm | 22 ++++++++++++----------

1 file changed, 12 insertions(+), 10 deletions(-)

Hi Guix!

In the meantime, Poetry 1.0.5 has been released. This V2 patch set updates

some more dependencies and updates poetry to 1.0.5.

Regards

--

Tanguy

[arguments]: Disable failing test.

---

gnu/packages/python-xyz.scm | 9 ++++++---

1 file changed, 6 insertions(+), 3 deletions(-)

[arguments]: Disable failing tests.

---

gnu/packages/python-xyz.scm | 18 ++++++++++++++++--

1 file changed, 16 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-sortedcontainers.

---

gnu/packages/databases.scm | 7 +++++--

1 file changed, 5 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-importlib-metadata.

[arguments]: Disable failing test.

---

gnu/packages/python-xyz.scm | 10 ++++++++--

1 file changed, 8 insertions(+), 2 deletions(-)

---

gnu/packages/graphics.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-msgpack.

---

gnu/packages/python-web.scm | 1 +

1 file changed, 1 insertion(+)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

[propagated-inputs] Add python-clikit, add python-importlib-metadata,

add python-keyring, add python-pexpect, remove python-msgpack,

remove python-glob2. [home-page] Update URL.

[arguments] Remove distutils patch, patch versions of keyring

and importlib-metadata.

---

gnu/packages/python-xyz.scm | 22 ++++++++++++----------

1 file changed, 12 insertions(+), 10 deletions(-)

Hi,

V2 works fine on my computer at home, but when I applied it at work and

build poetry… it failed?! I had to add missing propageted inputs to

(yet) another package!?

Here is the updated patch set…

--

Tanguy

[arguments]: Disable failing test.

---

gnu/packages/python-xyz.scm | 9 ++++++---

1 file changed, 6 insertions(+), 3 deletions(-)

[arguments]: Disable failing tests.

---

gnu/packages/python-xyz.scm | 18 ++++++++++++++++--

1 file changed, 16 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-sortedcontainers.

---

gnu/packages/databases.scm | 7 +++++--

1 file changed, 5 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-importlib-metadata.

[arguments]: Disable failing test.

---

gnu/packages/python-xyz.scm | 10 ++++++++--

1 file changed, 8 insertions(+), 2 deletions(-)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

---

gnu/packages/graphics.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-msgpack.

---

gnu/packages/python-web.scm | 1 +

1 file changed, 1 insertion(+)

[propagated-inputs] Add python-appdirs, python-distlib, python-filelock,

python-importlib-metadata and python-six.

---

gnu/packages/python-xyz.scm | 6 ++++++

1 file changed, 6 insertions(+)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

[propagated-inputs] Add python-clikit, add python-importlib-metadata,

add python-keyring, add python-pexpect, remove python-msgpack,

remove python-glob2. [home-page] Update URL.

[arguments] Remove distutils patch, patch versions of keyring

and importlib-metadata.

---

gnu/packages/python-xyz.scm | 22 ++++++++++++----------

1 file changed, 12 insertions(+), 10 deletions(-)

On Mon, Mar 09, 2020 at 09:18:18AM +0100, Tanguy Le Carrour wrote:

Do you have any idea what's going on with these tests?

We should at least add a comment here, explaining why skipping the tests

is not hiding something that will be a problem while using cachy.

On Mon, Mar 09, 2020 at 09:18:20AM +0100, Tanguy Le Carrour wrote:

Again, we need a comment explaining what's wrong with the test.

Hi Leo,

Thanks for taking the time to read my patches!

Le 03/11, Leo Famulari a écrit :

My bad! Sorry!

I started investigating when I submitted the patch, but… well… I haven't

found a fix yet and I'm not sure who to blame!

The test relies on a call to `subprocess.check_output` to run

`python -m jsonschema --version`, but it behaves like `--version` was

never passed to the command?!

```

145 def test_version(self):

146 version = subprocess.check_output(

147 [sys.executable, "-m", "jsonschema", "--version"],

148 stderr=subprocess.STDOUT,

149 )

```

Replacing `check_output` with `check_call` shows the actual error message:

```

usage: __main__.py [-h] [-i INSTANCES] [-F ERROR_FORMAT] [-V VALIDATOR] schema

__main__.py: error: the following arguments are required: schema

```

Doesn't really look like a problem with jsonschema to me, and… I would not dare

blaming python. ^_^'

Any idea welcome!

--

Tanguy

Le 03/11, Leo Famulari a �crit :

Those ones are out of my comfort zone! :-(

Seems that all the `RedisStoreTestCase` are failing with messages like:

```

ResourceWarning: unclosed <socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 33754)>

if self._get_socket():

ResourceWarning: Enable tracemalloc to get the object allocation traceback

```

Might be related to the Redis server that is supposed to be running

during the tests?!

Again, any idea welcome!

--

Tanguy

On Thu, Mar 12, 2020 at 09:48:28AM +0100, Tanguy Le Carrour wrote:

How do you know it behaves like that? Is there some error output?

Le 03/12, Leo Famulari a écrit :

Actually, you're right, I don't know, because I didn't reproduce the exact same

error!

```

$ ./pre-inst-env guix build python-jsonschema -K

[…]

subprocess.CalledProcessError: Command '['/gnu/store/l8nphg0idd8pfddyad8f92lx8d1hc053-python-wrapper-3.7.4/bin/python', '-m', 'jsonschema', '--version']' returned non-zero exit status 1.

```

The exit status is 1.

To reproduce it in the build directory I did the following:

```

$ cd /tmp/guix-build-python-jsonschema-3.2.0.drv-0/jsonschema-3.2.0/

$ guix environment python-jsonschema --ad-hoc python-importlib-metadata

$ set -x PYTHONPATH .:$PYTHONPATH # I'm using Fish

$ trial jsonschema/tests/test_cli.py [env] GUIX

jsonschema.tests.test_cli

TestCLI

test_draft3_schema_draft4_validator ... [OK]

test_successful_validation ... [OK]

test_unsuccessful_validation ... [OK]

test_unsuccessful_validation_multiple_instances ... [OK]

test_version ... [ERROR]

===============================================================================

[ERROR]

Traceback (most recent call last):

[…]

subprocess.CalledProcessError: Command '['/gnu/store/m4kgg8z52kn6xspmd3brvivd129d4i3s-python-wrapper-3.7.4/bin/python', '-m', 'jsonschema', '--version']' returned non-zero exit status 2.

```

But the exit status was… 2! Which I didn't pay attention at first.

Then, I modified the code of the test to add some log:

```

$ trial jsonschema/tests/test_cli.py [env] GUIX

jsonschema.tests.test_cli

TestCLI

test_draft3_schema_draft4_validator ... [OK]

test_successful_validation ... [OK]

test_unsuccessful_validation ... [OK]

test_unsuccessful_validation_multiple_instances ... [OK]

test_version ... usage: __main__.py [-h] [-i INSTANCES] [-F ERROR_FORMAT] [-V VALIDATOR] schema

__main__.py: error: the following arguments are required: schema

[ERROR]

```

This error message in the output is the one I would get by calling

`python -m jsonschema` without `--version`.

This was the end of my investigation and… a mistake! ^_^'

Right after your question, I tried to do the same but directly from my package

definition by adding a `substitute`:

```

(add-before 'check 'replace-check-output

(lambda _

(substitute* "jsonschema/tests/test_cli.py"

(("check_output") "check_call"))

#t))

```

Re-built the package and… got the exit status 1 with a different error message

in the log:

```

jsonschema.tests.test_cli

TestCLI

test_draft3_schema_draft4_validator ... [OK]

test_successful_validation ... [OK]

test_unsuccessful_validation ... [OK]

test_unsuccessful_validation_multiple_instances ... [OK]

test_version ... /gnu/store/l8nphg0idd8pfddyad8f92lx8d1hc053-python-wrapper-3.7.4/bin/python: No module named jsonschema

[ERROR]

===============================================================================

[ERROR]

Traceback (most recent call last):

[…]

subprocess.CalledProcessError: Command '['/gnu/store/l8nphg0idd8pfddyad8f92lx8d1hc053-python-wrapper-3.7.4/bin/python', '-m', 'jsonschema', '--version']' returned non-zero exit status 1.

```

So, I'm left with the same question:

- why does it fail with `No module named jsonschema`? and with a second one

- why wasn't I able to reproduce the error from the build directory?!

I still have a lot to learn! :-(

Any help welcome!

--

Tanguy

On Thu, Mar 12, 2020 at 09:58:22AM +0100, Tanguy Le Carrour wrote:

Okay, networking is not possible in the build environment, so for the

tests that depend on connecting to a server, you can add a comment like

"These tests require networking.".

Is that all of them?

On Fri, Mar 13, 2020 at 09:45:41AM +0100, Tanguy Le Carrour wrote:

I'm not sure, I'm not that strong with Python.

There are lots of things that could have changed.

First, I would use `guix environment --pure` to try isolating the rest

of the system, beginning to approximate the build chroot. You could even

try `guix environment --container` if your system supports it.

Second, the tests might be idempotent.

You'll have a hard time reproducing the build environment exactly but

it's usually possible to figure out why the test is failing.

On Fri, Mar 13, 2020 at 09:45:41AM +0100, Tanguy Le Carrour wrote:

My understanding is that the test suite of jsonschema can't find

jsonschema. Is that correct? If so, you should look into the

add-installed-pythonpath procedure and some examples of how it's used.

On Thu, Mar 12, 2020 at 09:58:22AM +0100, Tanguy Le Carrour wrote:

I just got word that networking to localhost *is* possible in the build

environment. So it's not as simple as that in this case. Can you check

if the Redis server is running?

Le 03/14, Leo Famulari a �crit :

Good to know, thanks!

I guess so! I'll double check, update the patch and submit it!

Thanks!

--

Tanguy

Le 03/14, Leo Famulari a �crit :

*ARF* should have read all of it! :-(

I'll investigate! Thanks for the advice.

--

Tanguy

Le 03/14, Leo Famulari a écrit :

I'll do that! Thanks!

--

Tanguy

Hi Leo,

Le 03/14, Tanguy Le Carrour a écrit :

It solved the problem! Thanks! I'm moving to fixing the cachy/redis problem

now, and I'll submit a v4 patch set once it's done!

Regards

--

Tanguy

Hi Leo,

Le 03/14, Leo Famulari a écrit :

I checked and this bug has been reported upstream last year and is not

yet fixed.

There's a workaround, but it requires to package `pifpaf` and 5 more

dependencies. I'll eventually do that… later next week!?

In the meantime, I'm pushing v4 of the patch set if you're OK with

disabling the failing tests and adding a comment pointing to the bug report.

If you're not, which is perfectly fine, I've reorganized the stack so that

you could commit the 8 patches up to `python-cachy`.

Regards

--

Tanguy

[arguments]: Disable failing test.

---

gnu/packages/python-xyz.scm | 9 ++++++---

1 file changed, 6 insertions(+), 3 deletions(-)

[propagated-inputs]: Add python-importlib-metadata.

[arguments]: Use add-installed-pythonpath to make a test pass.

---

gnu/packages/python-xyz.scm | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)

---

gnu/packages/graphics.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-msgpack.

---

gnu/packages/python-web.scm | 1 +

1 file changed, 1 insertion(+)

[propagated-inputs] Add python-appdirs, python-distlib, python-filelock,

python-importlib-metadata and python-six.

---

gnu/packages/python-xyz.scm | 6 ++++++

1 file changed, 6 insertions(+)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-sortedcontainers.

---

gnu/packages/databases.scm | 7 +++++--

1 file changed, 5 insertions(+), 2 deletions(-)

[arguments]: Disable failing tests.

---

gnu/packages/python-xyz.scm | 20 ++++++++++++++++++--

1 file changed, 18 insertions(+), 2 deletions(-)

[propagated-inputs] Add python-clikit, add python-importlib-metadata,

add python-keyring, add python-pexpect, remove python-msgpack,

remove python-glob2. [home-page] Update URL.

[arguments] Remove distutils patch, patch versions of keyring

and importlib-metadata.

---

gnu/packages/python-xyz.scm | 22 ++++++++++++----------

1 file changed, 12 insertions(+), 10 deletions(-)

On Thu, Mar 19, 2020 at 08:42:16AM +0100, Tanguy Le Carrour wrote:

I'm glad to hear it, thanks!

On Thu, Mar 19, 2020 at 06:27:05PM +0100, Tanguy Le Carrour wrote:

That sounds fine, thanks!

Le 03/20, Leo Famulari a �crit :

Thanks to you for (kindly) pushing the contributors to keep the package quality

as high as possible! :-)

--

Tanguy

Hi Leo!

I fixed the cachy test problem using pifpaf. The following V5 contains 4

new packages and the modified python-cachy.

Regards

--

Tanguy

[arguments]: Disable failing test.

---

gnu/packages/python-xyz.scm | 9 ++++++---

1 file changed, 6 insertions(+), 3 deletions(-)

[propagated-inputs]: Add python-importlib-metadata.

[arguments]: Use add-installed-pythonpath to make a test pass.

---

gnu/packages/python-xyz.scm | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)

---

gnu/packages/graphics.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

[propagated-inputs]: Add python-msgpack.

---

gnu/packages/python-web.scm | 1 +

1 file changed, 1 insertion(+)

[propagated-inputs] Add python-appdirs, python-distlib, python-filelock,

python-importlib-metadata and python-six.

---

gnu/packages/python-xyz.scm | 6 ++++++

1 file changed, 6 insertions(+)

---

gnu/packages/python-xyz.scm | 4 ++--

1 file changed, 2 insertions(+), 2 deletions(-)

---

gnu/packages/python-xyz.scm | 38 +++++++++++++++++++++++++++++++++++++

1 file changed, 38 insertions(+)

[propagated-inputs]: Add python-sortedcontainers.

---

gnu/packages/databases.scm | 7 +++++--

1 file changed, 5 insertions(+), 2 deletions(-)

---

gnu/packages/python-xyz.scm | 23 +++++++++++++++++++++++

1 file changed, 23 insertions(+)

[arguments]: Run checks using pifpaf. [native-inputs]: Add memcached and

python-pifpaf.

---

gnu/packages/python-xyz.scm | 13 ++++++++++---

1 file changed, 10 insertions(+), 3 deletions(-)

---

gnu/packages/python-xyz.scm | 21 +++++++++++++++++++++

1 file changed, 21 insertions(+)

---

gnu/packages/python-xyz.scm | 26 ++++++++++++++++++++++++++

1 file changed, 26 insertions(+)

[propagated-inputs] Add python-clikit, add python-importlib-metadata,

add python-keyring, add python-pexpect, remove python-msgpack,

remove python-glob2. [home-page] Update URL.

[arguments] Remove distutils patch, patch versions of keyring

and importlib-metadata.

---

gnu/packages/python-xyz.scm | 22 ++++++++++++----------

1 file changed, 12 insertions(+), 10 deletions(-)

Argh ! Sorry Leo, but… someone updated (at least) one of Poetry's

dependency in the meantime and… Poetry doesn't support that version

(yet)!

I'm fixing it and sending an updated patch. Fortunately, it's only the

last patch of the stack that has to be updated!

Sorry for the noise!

--

Tanguy

[propagated-inputs] Add python-clikit, add python-importlib-metadata,

add python-keyring, add python-pexpect, remove python-msgpack,

remove python-glob2. [home-page] Update URL.

[arguments] Remove distutils patch, patch versions of keyring, pyrsistent

and importlib-metadata.

---

gnu/packages/python-xyz.scm | 23 +++++++++++++----------

1 file changed, 13 insertions(+), 10 deletions(-)

On Sat, Mar 21, 2020 at 09:50:25AM +0100, Tanguy Le Carrour wrote:

It's important to find the right balance. I don't want to push

contributors away. Please tell us if you think we are being

unreasonable!

Hi Leo,

Le 03/22, Leo Famulari a �crit :

I really appriciate that you guys take the time to read and discuss

contributr's code.

And you won't scare me away from the project by being picky, honest and polite!

;-)

Best regards

--

Tanguy

On Sat, Mar 21, 2020 at 03:26:09PM +0100, Tanguy Le Carrour wrote:

Alright I pushed as 97b1366d49179f38a365bdfa383a8ccba779f8b8 with some

cosmetic changes to the commit messages, synopses, and descriptions, as

well as a fix for python-msgpack-transitional.

Thanks!