sbcl-common-lisp-jupyter does not install kernel.json

Open

Details

4 participants

Guillaume Le Vaillant
Jack Hill
Sharlatan Hellseher
Tarn Burton

Owner: unassigned

Submitted by: Jack Hill

Severity: normal

Jack Hill wrote on 5 Feb 2021 20:44

Recipients:(address . bug-guix@gnu.org)

Message-ID:alpine.DEB.2.21.2102051236250.11419@marsh.hcoop.net

Hi Guix,

The sbcl-common-lisp-jupyter package does not install a kernel.json file. 
That's the file that tells Jupyter about the kernel and how to run it, and 
should be installed in /share/jupyter/kernels/<kernel-name>/kernel.json.

sbcl-common-lisp-jupyter doesn't come with a kernel.json file to install, 
but it can generate one with the following command line:

`sbcl --eval '(require "asdf")' --eval '(require :common-lisp-jupyter)' 
--eval '(cl-jupyter:install)' --eval '(exit)'`

(please pardon any awkwardness with the sbcl command line, I'm new to 
Common Lisp, and just wanted to play around with it in Jupyter)

That produces the following kernel.json in $HOME/.local/… (I've pretty 
printed it for clarity here:

```
{
   "interrupt_method": "message",
   "language": "common-lisp",
   "display_name": "Common Lisp",
   "argv": [
     "sbcl",
     "--eval",
     "(ql:quickload :common-lisp-jupyter)",
     "--eval",
     "(jupyter:run-kernel 'common-lisp-jupyter:kernel #\"{connection_file}\")"
   ]
}
```

Unfortunately that won't work out of the box, as we don't have quicklisp, 
but changing it to:

```
{
   "interrupt_method": "message",
   "language": "common-lisp",
   "display_name": "Common Lisp",
   "argv": [
     "sbcl",
     "--eval",
     "(require \"asdf\")",
     "--eval",
     "(require :common-lisp-jupyter)",
     "--eval",
     "(jupyter:run-kernel 'common-lisp-jupyter:kernel #\"{connection_file}\")"
   ]
}
```

allows Jupyter to run the kernel. We would of course need to also 
substitute the full store path for sbcl as well.

Is it worth having sbcl-common-lisp-jupyter generate the kernel.json, and 
then make many changes to it? Perhaps it would be better to just write out 
the correct definition of the file from Guix.

A final note is that the other Common Lisp implementation, like 
ecl-common-lisp-jupyter, also have this problem because they are created 
as transformation of the sbcl package. I'm not sure if the kernel.json is 
portable across the implementation or in general how to best to accomplish 
this change for our Common Lisp packages.

Best,
Jack

Jack Hill wrote on 18 May 2021 18:12

Recipients:

Message-ID:alpine.DEB.2.21.2105181208250.2109@marsh.hcoop.net

Sharlatan,

Thanks for your recent work updated sbcl-common-lisp-jupyter. I was

wondering if you had any thoughts on the best way to install the

kernel.json file [0]. The last time I looked at it, I wasn't sure what the

best solution would be. Do you have any ideas?

[0] https://issues.guix.gnu.org/46333

Best,

Jack

Guillaume Le Vaillant wrote on 18 May 2021 18:58

Recipients:(name . Jack Hill)(address . jackhill@jackhill.us)

Message-ID:87lf8cugh1.fsf@kitej

Hi Jack,

I guess it will be easier to just add a phase writing the "kernel.json"
file in the right place. In this build phase, to know if the package is
being built for SBCL or ECL, the '(%lisp-type)' function that will
return "sbcl" or "ecl" can be used. There's an example in the
sbcl-trivial-backtrace package.

-----BEGIN PGP SIGNATURE-----

iIUEAREKAC0WIQTLxZxm7Ce5cXlAaz5r6CCK3yH+PwUCYKPyGg8cZ2x2QHBvc3Rl

by5uZXQACgkQa+ggit8h/j9StAD/Ujy+UgYSmTfbHUO8Ble1isNgDXq2xI2Jf2tH

hjtISBIA/Rq8fH/FB4QaqzSjKvpSKfGFv7I2ZHdS3cgrrosWGsgE

=rwnn

-----END PGP SIGNATURE-----

Sharlatan Hellseher wrote on 20 May 2021 00:23

Recipients:(name . Guillaume Le Vaillant)(address . glv@posteo.net)

Message-ID:CAO+9K5qb=f+F5dSujm+8FLCjigcTXSyBbwRfLN=fKZ2GsSDjsg@mail.gmail.com

Hi,

I've checked the r-irkernel and it's coping existing kernelspec ,

which is not useful in this case.

As Guillaume mentioned we could tweak it before installation phase by

using cl-jupyter:install, so here is my draft:

Toggle snippet (33 lines)    (arguments
     `(#:phases
       (modify-phases %standard-phases
         (add-before 'install 'generate-kernelspec
           (lambda* (#:key outputs #:allow-other-keys)
             (let* ((out (assoc-ref outputs "out"))
                    (kernelspec (string-append out
"/share/cl-jupyter/kernelspec")))
               (mkdir-p kernelspec)
               (invoke "sbcl"
                       "--eval" "\"(require :asdf)\""
                       "--eval" "\"(require :common-lisp-jupyter)\""
                       "--eval"
                       (string-append
                        "\"(cl-jupyter:install"
                        ":bin-path" (string-append
                                     (assoc-ref %build-inputs "sbcl")
"/bin/sbcl")
                        ":prefix" out ")\"")
                       "--eval" "\"(exit)\""))
             #t))
         (add-after 'install 'install-kernelspec
           (lambda* (#:key outputs #:allow-other-keys)
             (let ((out (assoc-ref outputs "out"))
                   (kernelspec (string-append out
"/share/cl-jupyter/kernelspec")))
               (invoke "jupyter" "kernelspec" "install"
                       "--name" "cl-jupyter"
                       "--prefix" out
                       kernelspec)
               #t))))))

But there could be a potential blocking issue with :prefix key

https://github.com/yitzchak/common-lisp-jupyter/issues/78

On Tue, 18 May 2021 at 16:58, Guillaume Le Vaillant <glv@posteo.net> wrote:

Toggle quote (11 lines)

> Hi Jack,

> I guess it will be easier to just add a phase writing the "kernel.json"

> file in the right place. In this build phase, to know if the package is

> being built for SBCL or ECL, the '(%lisp-type)' function that will

> return "sbcl" or "ecl" can be used. There's an example in the

> sbcl-trivial-backtrace package.

… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????

????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??

?????????? ?? ? ????????? ??????? ????? ? ?????????????????.

Sharlatan Hellseher wrote on 24 May 2021 23:28

Recipients:(name . Guillaume Le Vaillant)(address . glv@posteo.net)

Message-ID:CAO+9K5oz-Cwi_Jh7pkGUKZx3mQTbfxKsqcu0=h-7rMNAGsTtdQ@mail.gmail.com

Hi,

I played with cl-jupyter:install but it heavily depends on Quicklisp

but what basically does - generates simple JSON with CL implementation

https://github.com/yitzchak/common-lisp-jupyter/issues/78

First I tried to do the same during build phase by evaluationg

arbitrary Lisp but could not manged it to work.

Then after checking the kernel.json I just simply formated it with

right %lispt-type path and copied with the same evaluation as R

Jupyter package, but I've got some error during coping into Jupyter

directory:

Toggle snippet (79 lines)phase `generate-kernelspec' succeeded after 0.0 seconds
starting phase `install-kernelspec'
command "jupyter" "kernelspec" "install" "--name" "cl-jupyter"
"--prefix" "/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7"
"/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7/share/cl-jupyter/kernelspec"
failed with status 127
builder for `/gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv'
failed with exit code 1
build of /gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv
failed
View build log at
'/var/log/guix/drvs/az/l65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv.bz2'.
guix build: error: build of
`/gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv'
failed
(guix/linux-gnu)[sharlatan@guxtil ~/Projects/prj/guix-channel]$:
jupyter kernelspec install --name cl-jupyter --prefix
/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7
/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7/share/cl-jupyter/kernelspec
Traceback (most recent call last):
  File "/gnu/store/30ydqwp1xccqzn5s4rcq4clpqzcaz3p1-python-jupyter-client-6.1.12/bin/.jupyter-kernelspec-real",
line 11, in <module>
    load_entry_point('jupyter-client==6.1.12', 'console_scripts',
'jupyter-kernelspec')()
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 663, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-2>", line 2, in initialize
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 297, in initialize
    self.parse_command_line(argv)
  File "<decorator-gen-4>", line 2, in parse_command_line
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 515, in parse_command_line
    return self.initialize_subcommand(subc, subargv)
  File "<decorator-gen-3>", line 2, in initialize_subcommand
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 453, in initialize_subcommand
    self.subapp.initialize(argv)
  File "<decorator-gen-6>", line 2, in initialize
  File "/gnu/store/ih0jscm3ilqn2r33ms8014ap26jhjfqa-python-traitlets-4.3.3/lib/python3.8/site-packages/traitlets/config/application.py",
line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
line 229, in initialize
    self.migrate_config()
  File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/application.py",
line 155, in migrate_config
    migrate()
  File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/migrate.py",
line 244, in migrate
    ensure_dir_exists(env['jupyter_config'])
  File "/gnu/store/yj346rpx2mf1afa6jxl2kh9i335an27y-python-jupyter-core-4.7.1/lib/python3.8/site-packages/jupyter_core/utils/__init__.py",
line 11, in ensure_dir_exists
    os.makedirs(path, mode=mode)
  File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  [Previous line repeated 12 more times]
  File "/gnu/store/hq7qr7nc2j29z3pivm3azfjy6jq3d7nx-python-3.8.2/lib/python3.8/os.py",
line 223, in makedirs
    mkdir(name, mode)

On Wed, 19 May 2021 at 22:23, Sharlatan Hellseher <sharlatanus@gmail.com> wrote:

Toggle quote (67 lines)>
> Hi,
>
> I've checked the r-irkernel and it's coping existing kernelspec ,
> which is not useful in this case.
>
> As Guillaume mentioned we could tweak it before installation phase by
> using cl-jupyter:install, so here is my draft:
>
> --8<---------------cut here---------------start------------->8---
>     (arguments
>      `(#:phases
>        (modify-phases %standard-phases
>          (add-before 'install 'generate-kernelspec
>            (lambda* (#:key outputs #:allow-other-keys)
>              (let* ((out (assoc-ref outputs "out"))
>                     (kernelspec (string-append out
> "/share/cl-jupyter/kernelspec")))
>                (mkdir-p kernelspec)
>                (invoke "sbcl"
>                        "--eval" "\"(require :asdf)\""
>                        "--eval" "\"(require :common-lisp-jupyter)\""
>                        "--eval"
>                        (string-append
>                         "\"(cl-jupyter:install"
>                         ":bin-path" (string-append
>                                      (assoc-ref %build-inputs "sbcl")
> "/bin/sbcl")
>                         ":prefix" out ")\"")
>                        "--eval" "\"(exit)\""))
>              #t))
>          (add-after 'install 'install-kernelspec
>            (lambda* (#:key outputs #:allow-other-keys)
>              (let ((out (assoc-ref outputs "out"))
>                    (kernelspec (string-append out
> "/share/cl-jupyter/kernelspec")))
>                (invoke "jupyter" "kernelspec" "install"
>                        "--name" "cl-jupyter"
>                        "--prefix" out
>                        kernelspec)
>                #t))))))
> --8<---------------cut here---------------end--------------->8---
>
> But there could be a potential blocking issue with :prefix key
>
> https://github.com/yitzchak/common-lisp-jupyter/issues/78
>
> On Tue, 18 May 2021 at 16:58, Guillaume Le Vaillant <glv@posteo.net> wrote:
> >
> > Hi Jack,
> >
> > I guess it will be easier to just add a phase writing the "kernel.json"
> > file in the right place. In this build phase, to know if the package is
> > being built for SBCL or ECL, the '(%lisp-type)' function that will
> > return "sbcl" or "ecl" can be used. There's an example in the
> > sbcl-trivial-backtrace package.
>
>
>
> --
>
> … ??? ????? - ???????????? ?????????????? ?????? ??????? ????????
> ????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??
> ?????????? ?? ? ????????? ??????? ????? ? ?????????????????.

… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????

????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??

?????????? ?? ? ????????? ??????? ????? ? ?????????????????.

From cbc17c767069e8fa5175f2f0f30fc961a2be319a Mon Sep 17 00:00:00 2001

---

gnu/packages/lisp-xyz.scm | 35 +++++++++++++++++++++++++++++++++++

1 file changed, 35 insertions(+)

Toggle diff (48 lines)diff --git a/gnu/packages/lisp-xyz.scm b/gnu/packages/lisp-xyz.scm
index 4a1e9064d5..91370c62c4 100644
--- a/gnu/packages/lisp-xyz.scm
+++ b/gnu/packages/lisp-xyz.scm
@@ -14897,6 +14897,41 @@ and @code{doseq*}.")
          (sha256
           (base32 "0si69xfzi769dprwfy7gp1x3bl7lxz6d4n98sa26w9r41wvay5ja"))))
       (build-system asdf-build-system/sbcl)
+      (arguments
+       `(#:phases
+         (modify-phases %standard-phases
+           (add-after 'create-asdf-configuration 'generate-kernelspec
+             (lambda* (#:key outputs #:allow-other-keys)
+               (let* ((out (assoc-ref outputs "out"))
+                      (kernelspec (string-append out "/share/cl-jupyter/kernelspec")))
+                 (mkdir-p kernelspec)
+                 (copy-file (string-append "res/" (%lisp-type) "/logo-64x64.png")
+                            (string-append kernelspec "/logo-64x64.png"))
+                 (with-output-to-file (string-append kernelspec "/kernel.json")
+                   (lambda ()
+                     ;; FIXME: Use of guile-json would beneficial here
+                     (format #t "{ \"argv\": [ ~s,
+ \"--eval\",
+\"(require :asdf)\",
+ \"--eval\",
+\"(asdf:load-system :common-lisp-jupyter)\",
+ \"--eval\",
+\"(jupyter:run-kernel 'common-lisp-jupyter:kernel)\",
+\"{connection_file}\"
+],
+\"display_name\": \"Common Lisp\",
+\"language\": \"common-lisp\",
+\"interrupt_mode\": \"message\"
+}"
+                             (string-append (assoc-ref %build-inputs (%lisp-type)) "/bin/" (%lisp-type))))))))
+      (add-after 'generate-kernelspec 'install-kernelspec
+        (lambda* (#:key outputs #:allow-other-keys)
+          (let* ((out (assoc-ref outputs "out"))
+                (kernelspec (string-append out "/share/cl-jupyter/kernelspec")))
+            (invoke "jupyter" "kernelspec" "install"
+                    "--name" "cl-jupyter"
+                    "--prefix" out
+                    kernelspec)))))))
       (inputs
        `(("alexandria" ,sbcl-alexandria)
          ("babel" ,sbcl-babel)
-- 
2.31.1

Guillaume Le Vaillant wrote on 25 May 2021 17:40

Recipients:(name . Sharlatan Hellseher)(address . sharlatanus@gmail.com)

Message-ID:877djmddop.fsf@kitej

Sharlatan Hellseher <sharlatanus@gmail.com> skribis:

Toggle quote (25 lines)> I played with cl-jupyter:install but it heavily depends on Quicklisp
> but what basically does - generates simple JSON with CL implementation
> https://github.com/yitzchak/common-lisp-jupyter/issues/78
>
> First I tried to do the same during build phase by evaluationg
> arbitrary Lisp but could not manged it to work.
> Then after checking the kernel.json I just simply formated it with
> right %lispt-type path and copied with the same evaluation as R
> Jupyter package, but I've got some error during coping into Jupyter
> directory:
>
> --8<---------------cut here---------------start------------->8---
> phase `generate-kernelspec' succeeded after 0.0 seconds
> starting phase `install-kernelspec'
> command "jupyter" "kernelspec" "install" "--name" "cl-jupyter"
> "--prefix" "/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7"
> "/gnu/store/xjqxskiqjlzirlg478gnlp7x6w2jcz63-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7/share/cl-jupyter/kernelspec"
> failed with status 127
> builder for `/gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv'
> failed with exit code 1
> build of /gnu/store/azl65q1bl2pv920fmgw6d8k0brsx6hdg-sbcl-common-lisp-jupyter-0.1.0-4.ba9f0e7.drv
> failed
> [...]
> --8<---------------cut here---------------end--------------->8---

I think this error comes from the fact that the jupyter package is
missing as native input, therefore the 'jupyter ...' command can't work.

It lools like the 'install-kernelspec' phase just copies the kernel you
generated in "share/cl-jupyter/kernelspec/" to
"share/jupyter/kernels/cl-jupyter/". Wouldn't it be easier to generate
the kernel in the final directory directly? Then the
'install-kernelspec' phase would not be necessary.

-----BEGIN PGP SIGNATURE-----

iIUEAREKAC0WIQTLxZxm7Ce5cXlAaz5r6CCK3yH+PwUCYK0adg8cZ2x2QHBvc3Rl

by5uZXQACgkQa+ggit8h/j8A/AD/cDV+OzQBT3LQ8/UDbyg/nd3o9ta4YWYg0TzV

CiwzUSMA/32U1H4BM27HJmBbJL/sZUVpAJMSyCdxiSZJj3UTwOLC

=2ml4

-----END PGP SIGNATURE-----

Tarn Burton wrote on 8 Jun 2021 16:48

Install Command Line

Recipients:(address . 46333@debbugs.gnu.org)

Message-ID:CAG-qkSruri0OnNXZmtuti6=J+dmD-VUyqtEukVeK0MjJtCNtVw@mail.gmail.com

The command line that you should be using is

sbcl --non-interacive --eval "(require :asdf)" --eval "(asdf:load-system

:common-lisp-jupyter)" --eval "(clj:install :use-implementation :prefix

\"bla\" :system t)"

The sbcl should not have quicklisp installed so that the installer knows

that this is not a quicklisp install. If the user's sbcl binary is located

in a different place then you may need to add :bin-path

https://github.com/yitzchak/common-lisp-jupyter/blob/master/README.md#installing-via-quicklispasdf

-Tarn Burton/CLJ Author

Attachment: file

Tarn Burton wrote on 14 Jun 2021 17:21

Typos

Recipients:(address . 46333@debbugs.gnu.org)

Message-ID:CAG-qkSqtLtuv7buRCRTX9EOr-yycuJiYab2uPVtBMCRfzT-Bjw@mail.gmail.com

Looks like I had some typos in the command.

sbcl --non-interactive --eval "(require :asdf)" --eval

"(asdf:load-system:common-lisp-jupyter)"

--eval "(clj:install :use-implementation t :prefix\"bla\" :system t)"

-Tarn Burton

Attachment: file

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 46333@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 46333

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date