sbcl-common-lisp-jupyter does not install kernel.json

  • Open
  • quality assurance status badge
Details
4 participants
  • Guillaume Le Vaillant
  • Jack Hill
  • Sharlatan Hellseher
  • Tarn Burton
Owner
unassigned
Submitted by
Jack Hill
Severity
normal
J
J
Jack Hill wrote on 5 Feb 2021 20:44
(address . bug-guix@gnu.org)
alpine.DEB.2.21.2102051236250.11419@marsh.hcoop.net
Hi Guix,

The sbcl-common-lisp-jupyter package does not install a kernel.json file.
That's the file that tells Jupyter about the kernel and how to run it, and
should be installed in /share/jupyter/kernels/<kernel-name>/kernel.json.

sbcl-common-lisp-jupyter doesn't come with a kernel.json file to install,
but it can generate one with the following command line:

`sbcl --eval '(require "asdf")' --eval '(require :common-lisp-jupyter)'
--eval '(cl-jupyter:install)' --eval '(exit)'`

(please pardon any awkwardness with the sbcl command line, I'm new to
Common Lisp, and just wanted to play around with it in Jupyter)

That produces the following kernel.json in $HOME/.local/… (I've pretty
printed it for clarity here:

```
{
"interrupt_method": "message",
"language": "common-lisp",
"display_name": "Common Lisp",
"argv": [
"sbcl",
"--eval",
"(ql:quickload :common-lisp-jupyter)",
"--eval",
"(jupyter:run-kernel 'common-lisp-jupyter:kernel #\"{connection_file}\")"
]
}
```

Unfortunately that won't work out of the box, as we don't have quicklisp,
but changing it to:

```
{
"interrupt_method": "message",
"language": "common-lisp",
"display_name": "Common Lisp",
"argv": [
"sbcl",
"--eval",
"(require \"asdf\")",
"--eval",
"(require :common-lisp-jupyter)",
"--eval",
"(jupyter:run-kernel 'common-lisp-jupyter:kernel #\"{connection_file}\")"
]
}
```

allows Jupyter to run the kernel. We would of course need to also
substitute the full store path for sbcl as well.

Is it worth having sbcl-common-lisp-jupyter generate the kernel.json, and
then make many changes to it? Perhaps it would be better to just write out
the correct definition of the file from Guix.

A final note is that the other Common Lisp implementation, like
ecl-common-lisp-jupyter, also have this problem because they are created
as transformation of the sbcl package. I'm not sure if the kernel.json is
portable across the implementation or in general how to best to accomplish
this change for our Common Lisp packages.

Best,
Jack
J
J
Jack Hill wrote on 18 May 2021 18:12
alpine.DEB.2.21.2105181208250.2109@marsh.hcoop.net
Sharlatan,

Thanks for your recent work updated sbcl-common-lisp-jupyter. I was
wondering if you had any thoughts on the best way to install the
kernel.json file [0]. The last time I looked at it, I wasn't sure what the
best solution would be. Do you have any ideas?


Best,
Jack
G
G
Guillaume Le Vaillant wrote on 18 May 2021 18:58
(name . Jack Hill)(address . jackhill@jackhill.us)
87lf8cugh1.fsf@kitej
Hi Jack,

I guess it will be easier to just add a phase writing the "kernel.json"
file in the right place. In this build phase, to know if the package is
being built for SBCL or ECL, the '(%lisp-type)' function that will
return "sbcl" or "ecl" can be used. There's an example in the
sbcl-trivial-backtrace package.
-----BEGIN PGP SIGNATURE-----

iIUEAREKAC0WIQTLxZxm7Ce5cXlAaz5r6CCK3yH+PwUCYKPyGg8cZ2x2QHBvc3Rl
by5uZXQACgkQa+ggit8h/j9StAD/Ujy+UgYSmTfbHUO8Ble1isNgDXq2xI2Jf2tH
hjtISBIA/Rq8fH/FB4QaqzSjKvpSKfGFv7I2ZHdS3cgrrosWGsgE
=rwnn
-----END PGP SIGNATURE-----

S
S
Sharlatan Hellseher wrote on 20 May 2021 00:23
(name . Guillaume Le Vaillant)(address . glv@posteo.net)
CAO+9K5qb=f+F5dSujm+8FLCjigcTXSyBbwRfLN=fKZ2GsSDjsg@mail.gmail.com
Hi,

I've checked the r-irkernel and it's coping existing kernelspec ,
which is not useful in this case.

As Guillaume mentioned we could tweak it before installation phase by
using cl-jupyter:install, so here is my draft:

Toggle snippet (33 lines)
(arguments
`(#:phases
(modify-phases %standard-phases
(add-before 'install 'generate-kernelspec
(lambda* (#:key outputs #:allow-other-keys)
(let* ((out (assoc-ref outputs "out"))
(kernelspec (string-append out
"/share/cl-jupyter/kernelspec")))
(mkdir-p kernelspec)
(invoke "sbcl"
"--eval" "\"(require :asdf)\""
"--eval" "\"(require :common-lisp-jupyter)\""
"--eval"
(string-append
"\"(cl-jupyter:install"
":bin-path" (string-append
(assoc-ref %build-inputs "sbcl")
"/bin/sbcl")
":prefix" out ")\"")
"--eval" "\"(exit)\""))
#t))
(add-after 'install 'install-kernelspec
(lambda* (#:key outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out"))
(kernelspec (string-append out
"/share/cl-jupyter/kernelspec")))
(invoke "jupyter" "kernelspec" "install"
"--name" "cl-jupyter"
"--prefix" out
kernelspec)
#t))))))

But there could be a potential blocking issue with :prefix key


On Tue, 18 May 2021 at 16:58, Guillaume Le Vaillant <glv@posteo.net> wrote:
Toggle quote (11 lines)
>
> Hi Jack,
>
> I guess it will be easier to just add a phase writing the "kernel.json"
> file in the right place. In this build phase, to know if the package is
> being built for SBCL or ECL, the '(%lisp-type)' function that will
> return "sbcl" or "ecl" can be used. There's an example in the
> sbcl-trivial-backtrace package.



--

… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
?????????? ?? ? ????????? ??????? ????? ? ?????????????????.
S
S
Sharlatan Hellseher wrote on 24 May 2021 23:28
(name . Guillaume Le Vaillant)(address . glv@posteo.net)
CAO+9K5oz-Cwi_Jh7pkGUKZx3mQTbfxKsqcu0=h-7rMNAGsTtdQ@mail.gmail.com
Hi,

I played with cl-jupyter:install but it heavily depends on Quicklisp
but what basically does - generates simple JSON with CL implementation

First I tried to do the same during build phase by evaluationg
arbitrary Lisp but could not manged it to work.
Then after checking the kernel.json I just simply formated it with
right %lispt-type path and copied with the same evaluation as R
Jupyter package, but I've got some error during coping into Jupyter
directory:

Toggle snippet (79 lines)
phase `generate-kernelspec' succeeded after 0.0 seconds
starting phase `install-kernelspec'
command "jupyter" "kernelspec" "install" "--name" "cl-jupyter"
"--prefix" "/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7"
"/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7/share/cl-jupyter/kernelspec"
failed with status 127
builder for `/gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv'
failed with exit code 1
build of /gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv
failed
View build log at
'/var/log/guix/drvs/az/l65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv.bz2'.
guix build: error: build of
`/gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv'
failed
(guix/linux-gnu)[sharlatan@guxtil ~/Projects/prj/guix-channel]$:
jupyter kernelspec install --name cl-jupyter --prefix
/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7
/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7/share/cl-jupyter/kernelspec
Traceback (most recent call last):
File "/gnu/store/30ydqwp1xccqzn5s4rcq4clpqzcaz3p1-python-jupyter-client-6.1.12/bin/.jupyter-kernelspec-real",
line 11, in <module>
load_entry_point('jupyter-client==6.1.12', 'console_scripts',
'jupyter-kernelspec')()
File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 663, in launch_instance
app.initialize(argv)
File "<decorator-gen-2>", line 2, in initialize
File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 297, in initialize
self.parse_command_line(argv)
File "<decorator-gen-4>", line 2, in parse_command_line
File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 515, in parse_command_line
return self.initialize_subcommand(subc, subargv)
File "<decorator-gen-3>", line 2, in initialize_subcommand
File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 453, in initialize_subcommand
self.subapp.initialize(argv)
File "<decorator-gen-6>", line 2, in initialize
File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
line 229, in initialize
self.migrate_config()
File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
line 155, in migrate_config
migrate()
File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/migrate.py",
line 244, in migrate
ensure_dir_exists(env['jupyter_config'])
File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/utils/__init__.py",
line 11, in ensure_dir_exists
os.makedirs(path, mode=mode)
File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
[Previous line repeated 12 more times]
File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 223, in makedirs
mkdir(name, mode)

On Wed, 19 May 2021 at 22:23, Sharlatan Hellseher <sharlatanus@gmail.com> wrote:
Toggle quote (67 lines)
>
> Hi,
>
> I've checked the r-irkernel and it's coping existing kernelspec ,
> which is not useful in this case.
>
> As Guillaume mentioned we could tweak it before installation phase by
> using cl-jupyter:install, so here is my draft:
>
> --8<---------------cut here---------------start------------->8---
> (arguments
> `(#:phases
> (modify-phases %standard-phases
> (add-before 'install 'generate-kernelspec
> (lambda* (#:key outputs #:allow-other-keys)
> (let* ((out (assoc-ref outputs "out"))
> (kernelspec (string-append out
> "/share/cl-jupyter/kernelspec")))
> (mkdir-p kernelspec)
> (invoke "sbcl"
> "--eval" "\"(require :asdf)\""
> "--eval" "\"(require :common-lisp-jupyter)\""
> "--eval"
> (string-append
> "\"(cl-jupyter:install"
> ":bin-path" (string-append
> (assoc-ref %build-inputs "sbcl")
> "/bin/sbcl")
> ":prefix" out ")\"")
> "--eval" "\"(exit)\""))
> #t))
> (add-after 'install 'install-kernelspec
> (lambda* (#:key outputs #:allow-other-keys)
> (let ((out (assoc-ref outputs "out"))
> (kernelspec (string-append out
> "/share/cl-jupyter/kernelspec")))
> (invoke "jupyter" "kernelspec" "install"
> "--name" "cl-jupyter"
> "--prefix" out
> kernelspec)
> #t))))))
> --8<---------------cut here---------------end--------------->8---
>
> But there could be a potential blocking issue with :prefix key
>
> https://github.com/yitzchak/common-lisp-jupyter/issues/78
>
> On Tue, 18 May 2021 at 16:58, Guillaume Le Vaillant <glv@posteo.net> wrote:
> >
> > Hi Jack,
> >
> > I guess it will be easier to just add a phase writing the "kernel.json"
> > file in the right place. In this build phase, to know if the package is
> > being built for SBCL or ECL, the '(%lisp-type)' function that will
> > return "sbcl" or "ecl" can be used. There's an example in the
> > sbcl-trivial-backtrace package.
>
>
>
> --
>
> … ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
> ????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
> ?????????? ?? ? ????????? ??????? ????? ? ?????????????????.



--

… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
?????????? ?? ? ????????? ??????? ????? ? ?????????????????.
From cbc17c767069e8fa5175f2f0f30fc961a2be319a Mon Sep 17 00:00:00 2001
From: Sharlatan Hellseher <sharlatanus@gmail.com>
Date: Mon, 24 May 2021 22:18:28 +0100
Subject: [PATCH] gnu: common-lisp-jupyter: Format kernelspec

---
gnu/packages/lisp-xyz.scm | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/lisp-xyz.scm b/gnu/packages/lisp-xyz.scm
index 4a1e9064d5..91370c62c4 100644
--- a/gnu/packages/lisp-xyz.scm
+++ b/gnu/packages/lisp-xyz.scm
@@ -14897,6 +14897,41 @@ and @code{doseq*}.")
(sha256
(base32 "0si69xfzi769dprwfy7gp1x3bl7lxz6d4n98sa26w9r41wvay5ja"))))
(build-system asdf-build-system/sbcl)
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'create-asdf-configuration 'generate-kernelspec
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (kernelspec (string-append out "/share/cl-jupyter/kernelspec")))
+ (mkdir-p kernelspec)
+ (copy-file (string-append "res/" (%lisp-type) "/logo-64x64.png")
+ (string-append kernelspec "/logo-64x64.png"))
+ (with-output-to-file (string-append kernelspec "/kernel.json")
+ (lambda ()
+ ;; FIXME: Use of guile-json would beneficial here
+ (format #t "{ \"argv\": [ ~s,
+ \"--eval\",
+\"(require :asdf)\",
+ \"--eval\",
+\"(asdf:load-system :common-lisp-jupyter)\",
+ \"--eval\",
+\"(jupyter:run-kernel 'common-lisp-jupyter:kernel)\",
+\"{connection_file}\"
+],
+\"display_name\": \"Common Lisp\",
+\"language\": \"common-lisp\",
+\"interrupt_mode\": \"message\"
+}"
+ (string-append (assoc-ref %build-inputs (%lisp-type)) "/bin/" (%lisp-type))))))))
+ (add-after 'generate-kernelspec 'install-kernelspec
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (kernelspec (string-append out "/share/cl-jupyter/kernelspec")))
+ (invoke "jupyter" "kernelspec" "install"
+ "--name" "cl-jupyter"
+ "--prefix" out
+ kernelspec)))))))
(inputs
`(("alexandria" ,sbcl-alexandria)
("babel" ,sbcl-babel)
--
2.31.1
G
G
Guillaume Le Vaillant wrote on 25 May 2021 17:40
(name . Sharlatan Hellseher)(address . sharlatanus@gmail.com)
877djmddop.fsf@kitej
Sharlatan Hellseher <sharlatanus@gmail.com> skribis:

Toggle quote (25 lines)
> I played with cl-jupyter:install but it heavily depends on Quicklisp
> but what basically does - generates simple JSON with CL implementation
> https://github.com/yitzchak/common-lisp-jupyter/issues/78
>
> First I tried to do the same during build phase by evaluationg
> arbitrary Lisp but could not manged it to work.
> Then after checking the kernel.json I just simply formated it with
> right %lispt-type path and copied with the same evaluation as R
> Jupyter package, but I've got some error during coping into Jupyter
> directory:
>
> --8<---------------cut here---------------start------------->8---
> phase `generate-kernelspec' succeeded after 0.0 seconds
> starting phase `install-kernelspec'
> command "jupyter" "kernelspec" "install" "--name" "cl-jupyter"
> "--prefix" "/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7"
> "/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7/share/cl-jupyter/kernelspec"
> failed with status 127
> builder for `/gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv'
> failed with exit code 1
> build of /gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv
> failed
> [...]
> --8<---------------cut here---------------end--------------->8---

I think this error comes from the fact that the jupyter package is
missing as native input, therefore the 'jupyter ...' command can't work.

It lools like the 'install-kernelspec' phase just copies the kernel you
generated in "share/cl-jupyter/kernelspec/" to
"share/jupyter/kernels/cl-jupyter/". Wouldn't it be easier to generate
the kernel in the final directory directly? Then the
'install-kernelspec' phase would not be necessary.
-----BEGIN PGP SIGNATURE-----

iIUEAREKAC0WIQTLxZxm7Ce5cXlAaz5r6CCK3yH+PwUCYK0adg8cZ2x2QHBvc3Rl
by5uZXQACgkQa+ggit8h/j8A/AD/cDV+OzQBT3LQ8/UDbyg/nd3o9ta4YWYg0TzV
CiwzUSMA/32U1H4BM27HJmBbJL/sZUVpAJMSyCdxiSZJj3UTwOLC
=2ml4
-----END PGP SIGNATURE-----

T
T
Tarn Burton wrote on 8 Jun 2021 16:48
Install Command Line
(address . 46333@debbugs.gnu.org)
CAG-qkSruri0OnNXZmtuti6=J+dmD-VUyqtEukVeK0MjJtCNtVw@mail.gmail.com
The command line that you should be using is

sbcl --non-interacive --eval "(require :asdf)" --eval "(asdf:load-system
:common-lisp-jupyter)" --eval "(clj:install :use-implementation :prefix
\"bla\" :system t)"

The sbcl should not have quicklisp installed so that the installer knows
that this is not a quicklisp install. If the user's sbcl binary is located
in a different place then you may need to add :bin-path


-Tarn Burton/CLJ Author
Attachment: file
T
T
Tarn Burton wrote on 14 Jun 2021 17:21
Typos
(address . 46333@debbugs.gnu.org)
CAG-qkSqtLtuv7buRCRTX9EOr-yycuJiYab2uPVtBMCRfzT-Bjw@mail.gmail.com
Looks like I had some typos in the command.

sbcl --non-interactive --eval "(require :asdf)" --eval
"(asdf:load-system:common-lisp-jupyter)"
--eval "(clj:install :use-implementation t :prefix\"bla\" :system t)"

-Tarn Burton
Attachment: file
?