jupyter terminal out of pty devices

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Steffen Rytter Postas
Owner
unassigned
Submitted by
Steffen Rytter Postas
Severity
normal
S
S
Steffen Rytter Postas wrote on 1 Jul 2019 22:54
(address . bug-guix@gnu.org)
618e32ff-190e-7746-cd4f-88ad13149041@scalehost.eu
Hi,

I've been having a few issues with the Jupyter package. First and
foremost, this is my first time trying it out with Guix. Previously I
have had it running using Docker, but I prefer the modularity and
programmability of Guix, not to mention the rest of the amazing concepts
it brings.

However, nothing is perfect, and as we'll see here, that is indeed true.
Let it be known that I've tested this on my desktop and on my work laptop.

On my desktop, I've had no issues getting it running, but the terminal
functionality does not work at all, for several reasons.

There may very well be another issue, that I've yet to reproduce, but
the above I can reproduce easily on my machines.

Both environments use the same command to execute, and are using `guix
environment` to start the Jupyter notebook system:

`LC_ALL=C guix environment -C -N --pure -m env.scm -- jupyter-notebook`

The `env.scm` file used, is as follows:


(define-packages '(
    "glibc-utf8-locales"
    "coreutils-minimal"
    "bash"
    "python"
    "jupyter"
    ))
(use-modules (gnu packages))
(packages->manifest (map specification->package packages))

You may notice that Python is mentioned as an explicit package, along
with coreutils-minimal. These are BOTH workarounds, to even get to the
bugs, mentioned above.

Without python, running the notebook yields the following error:

pkg_resources.DistributionNotFound: The 'six' distribution was not found
and is required by traitlets

However, I have not been able to reproduce this anywhere. I do believe
this is a bug though. Setting python to be a required package gets me
past the above error, however removing python does NOT re-introduce the
error on any of my systems.

Without coreutils-minimal, opening the terminal yields the following error:

[E 18:40:08.400 NotebookApp] Uncaught exception POST /api/terminals
(127.0.0.1)
    HTTPServerRequest(protocol='http', host='localhost:8888',
method='POST', uri='/api/terminals', version='HTTP/1.1',
remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File
"/gnu/store/varva2qk8sdmy68hm05cfk2vxdczzzhf-python-tornado-5.1.1/lib/python3.7/site-packages/tornado/web.py",
line 1590, in _execute
        result = method(*self.path_args, **self.path_kwargs)
      File
"/gnu/store/varva2qk8sdmy68hm05cfk2vxdczzzhf-python-tornado-5.1.1/lib/python3.7/site-packages/tornado/web.py",
line 3006, in wrapper
        return method(self, *args, **kwargs)
      File
"/gnu/store/zgx9l5mypyh70ywn55rjrr085pyzs61z-python-notebook-5.7.4/lib/python3.7/site-packages/notebook/terminal/api_handlers.py",
line 16, in post
        name, _ = self.terminal_manager.new_named_terminal()
      File
"/gnu/store/mna9g0khcz20bpdknh65qq1bzm6rrp18-python-terminado-0.8.1/lib/python3.7/site-packages/terminado/management.py",
line 319, in new_named_terminal
        term = self.new_terminal()
      File
"/gnu/store/mna9g0khcz20bpdknh65qq1bzm6rrp18-python-terminado-0.8.1/lib/python3.7/site-packages/terminado/management.py",
line 171, in new_terminal
        pty = PtyProcessUnicode.spawn(argv, env=env,
cwd=options.get('cwd', None))
      File
"/gnu/store/fj5idczwb1s908b55z6jl64xmagcygva-python-ptyprocess-0.5.2/lib/python3.7/site-packages/ptyprocess/ptyprocess.py",
line 205, in spawn
        command_with_path = which(command)
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/shutil.py",
line 1129, in which
        if os.path.dirname(cmd):
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/posixpath.py",
line 156, in dirname
        p = os.fspath(p)
    TypeError: expected str, bytes or os.PathLike object, not NoneType
[W 18:40:08.402 NotebookApp] Unhandled error

At this point I assumed this was due to missing the `which(1)` command,
or at least some command missing, causing the above codepaths to return
None.
Adding coreutils-minimal gets me a slight bit further, and removing
coreutils-minimal DOES re-introduce the error as well.

However, at this point, Jupyter notebook "works". The script editing,
the interface, all of this works. Opening the terminal however, now
yields the following error:

[E 18:56:52.376 NotebookApp] Uncaught exception POST /api/terminals
(127.0.0.1)
    HTTPServerRequest(protocol='http', host='localhost:8888',
method='POST', uri='/api/terminals', version='HTTP/1.1',
remote_ip='127.0.0.1')
    Traceback (most recent call last):
      File
"/gnu/store/varva2qk8sdmy68hm05cfk2vxdczzzhf-python-tornado-5.1.1/lib/python3.7/site-packages/tornado/web.py",
line 1590, in _execute
        result = method(*self.path_args, **self.path_kwargs)
      File
"/gnu/store/varva2qk8sdmy68hm05cfk2vxdczzzhf-python-tornado-5.1.1/lib/python3.7/site-packages/tornado/web.py",
line 3006, in wrapper
        return method(self, *args, **kwargs)
      File
"/gnu/store/zgx9l5mypyh70ywn55rjrr085pyzs61z-python-notebook-5.7.4/lib/python3.7/site-packages/notebook/terminal/api_handlers.py",
line 16, in post
        name, _ = self.terminal_manager.new_named_terminal()
      File
"/gnu/store/mna9g0khcz20bpdknh65qq1bzm6rrp18-python-terminado-0.8.1/lib/python3.7/site-packages/terminado/management.py",
line 319, in new_named_terminal
        term = self.new_terminal()
      File
"/gnu/store/mna9g0khcz20bpdknh65qq1bzm6rrp18-python-terminado-0.8.1/lib/python3.7/site-packages/terminado/management.py",
line 171, in new_terminal
        pty = PtyProcessUnicode.spawn(argv, env=env,
cwd=options.get('cwd', None))
      File
"/gnu/store/fj5idczwb1s908b55z6jl64xmagcygva-python-ptyprocess-0.5.2/lib/python3.7/site-packages/ptyprocess/ptyprocess.py",
line 222, in spawn
        pid, fd = pty.fork()
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
line 96, in fork
        master_fd, slave_fd = openpty()
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
line 29, in openpty
        master_fd, slave_name = _open_terminal()
      File
"/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
line 59, in _open_terminal
        raise OSError('out of pty devices')
    OSError: out of pty devices
[W 18:56:52.377 NotebookApp] Unhandled error

So it seems that there's an issue with pty devices when running these
systems as a container.

Sincerely,

Steffen Rytter Postas
L
L
Ludovic Courtès wrote on 6 Jul 2019 00:28
(name . Steffen Rytter Postas)(address . nc@scalehost.eu)(address . 36463-done@debbugs.gnu.org)
87ftnkglps.fsf@gnu.org
Hello,

Steffen Rytter Postas <nc@scalehost.eu> skribis:

Toggle quote (18 lines)
> Both environments use the same command to execute, and are using `guix
> environment` to start the Jupyter notebook system:
>
> `LC_ALL=C guix environment -C -N --pure -m env.scm -- jupyter-notebook`
>
> The `env.scm` file used, is as follows:
>
>
> (define-packages '(
>     "glibc-utf8-locales"
>     "coreutils-minimal"
>     "bash"
>     "python"
>     "jupyter"
>     ))
> (use-modules (gnu packages))
> (packages->manifest (map specification->package packages))

Without ‘-C’, the Jupyter terminal is indeed working for me:

guix environment --ad-hoc jupyter python-ipython python-ipykernel \
bash coreutils --pure -- jupyter notebook

(Though for some reason IceCat displays white boxes instead of the
actual characters in the terminal, unless I zoom in or out. But that’s
another problem…)

Toggle quote (17 lines)
> "/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
> line 96, in fork
>         master_fd, slave_fd = openpty()
>       File
> "/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
> line 29, in openpty
>         master_fd, slave_name = _open_terminal()
>       File
> "/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
> line 59, in _open_terminal
>         raise OSError('out of pty devices')
>     OSError: out of pty devices
> [W 18:56:52.377 NotebookApp] Unhandled error
>
> So it seems that there's an issue with pty devices when running these
> systems as a container.

I could reproduce the problem, and AFAICS commit
935e79af61f4d903e562362cdd828be807e15581 fixes it.

Let me know if it’s all fine now!

Thanks,
Ludo’.
Closed
S
S
Steffen Rytter Postas wrote on 6 Jul 2019 01:49
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 36463-done@debbugs.gnu.org)
391e2b54-3c28-6ed9-5d0a-5997553339fa@scalehost.eu
Hi Ludo,

I can confirm that commit 935e79af61f4d903e562362cdd828be807e15581 does
indeed resolve the issue entirely. Thank you so much for the quick
response and action!

Den 06/07/2019 kl. 00.28 skrev Ludovic Courtès:
Toggle quote (54 lines)
> Hello,
>
> Steffen Rytter Postas <nc@scalehost.eu> skribis:
>
>> Both environments use the same command to execute, and are using `guix
>> environment` to start the Jupyter notebook system:
>>
>> `LC_ALL=C guix environment -C -N --pure -m env.scm -- jupyter-notebook`
>>
>> The `env.scm` file used, is as follows:
>>
>>
>> (define-packages '(
>>     "glibc-utf8-locales"
>>     "coreutils-minimal"
>>     "bash"
>>     "python"
>>     "jupyter"
>>     ))
>> (use-modules (gnu packages))
>> (packages->manifest (map specification->package packages))
> Without ‘-C’, the Jupyter terminal is indeed working for me:
>
> guix environment --ad-hoc jupyter python-ipython python-ipykernel \
> bash coreutils --pure -- jupyter notebook
>
> (Though for some reason IceCat displays white boxes instead of the
> actual characters in the terminal, unless I zoom in or out. But that’s
> another problem…)
>
>> "/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
>> line 96, in fork
>>         master_fd, slave_fd = openpty()
>>       File
>> "/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
>> line 29, in openpty
>>         master_fd, slave_name = _open_terminal()
>>       File
>> "/gnu/store/h8l1pby3cm6b4fxsfwwr65b4d1hyh6cs-python-3.7.0/lib/python3.7/pty.py",
>> line 59, in _open_terminal
>>         raise OSError('out of pty devices')
>>     OSError: out of pty devices
>> [W 18:56:52.377 NotebookApp] Unhandled error
>>
>> So it seems that there's an issue with pty devices when running these
>> systems as a container.
> I could reproduce the problem, and AFAICS commit
> 935e79af61f4d903e562362cdd828be807e15581 fixes it.
>
> Let me know if it’s all fine now!
>
> Thanks,
> Ludo’.
>
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 36463@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 36463
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch