[PATCH] gnu: web: Add jupyter-service

OpenSubmitted by Jesse.
Details
3 participants
  • Jesse
  • Maxim Cournoyer
  • Xinglu Chen
Owner
unassigned
Severity
normal
J
(address . guix-patches@gnu.org)
565a1b21-aefa-4129-a024-52517fdff9bd@gmail.com
The attached patch adds the jupyter service, which automatically starts
a jupyter notebook server at startup. It also includes documentation for
the Jupyter service.

Go ahead, tell me how awful it is.


-Jesse
From ab82ebef9666f57c53342d042f21303b3babcb4b Mon Sep 17 00:00:00 2001
From: Jesse <gib19014@byui.edu>
Date: Mon, 20 Sep 2021 16:01:22 -0600
Subject: [PATCH] gnu: web: Add jupyter-service

* gnu/services/web.scm (gitile-service-type): New variable.
* doc/guix.texi (Version Control Services): Document it.
---
doc/guix.texi | 52 +++++++++++++++++++
gnu/services/web.scm | 120 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 172 insertions(+)

Toggle diff (219 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 6436e83a7c..0f8aaca413 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -26197,6 +26197,58 @@ The file which should store the logging output of Agate.
 @end table
 @end deftp
 
+@cindex jupyter
+The Jupyter Notebook is a web application that allows you to create
+and share documents that contain live code, equations, visualizations
+and explanatory text.  Uses include: data cleaning and transformation,
+numerical simulation, statistical modeling, machine learning and much
+more.
+
+@deffn (Scheme Variable) jupyter-service-type
+This is the type of the agate service, whose value should be an
+@code{jupyter-service-type} object, as in this example:
+
+@lisp
+(service jupyter-service-type
+ (jupyter-configuration
+  (log-file "/var/log/jupyter.log")
+  (server-config
+   (kernels
+    (list jupyter-guile-kernel)))))
+
+@end lisp
+
+The example above tells the the Jupyter service to provide the
+@code{jupyter-guile-kernel} as an optional kernel.
+@end deffn
+
+@deftp {Data Type} jupyter-configuration
+Data type representing the configuration of Agate.
+
+@table @asis
+@item @code{jupyter-package} (default: @code{jupyter})
+The jupyter package to use.
+
+
+@item @code{kernels} (default: @code{'()}
+A list of kernel packages to use with the jupyter service.
+
+@item @code{root-dir} (default: @file{"/var/lib/jupyter/notebooks"})
+The directory where the jupyter server looks for jupyter notebooks.
+
+@item @code{log-file} (default: @code{#f})
+The location of the log file. If #f is given, no log is produced.
+
+@item @code{shell} (default: @code{(file-append bash "/bin/bash")}
+The location of the shell used when starting a web terminal (if it is configured to allow terminals).
+
+@item @code{server-config} (default: @code{#f})
+A file contianing the jupyter server's config file. If @code{#f} is given, an empty file is produced, and the default settings are used.
+
+@end table
+@end deftp
+
+
 @node Certificate Services
 @subsection Certificate Services
 
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index 6a093cf4e4..2ceaa20864 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -15,6 +15,7 @@
 ;;; Copyright © 2020 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2020 Oleg Pykhalov <go.wigust@gmail.com>
 ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton <brown121407@posteo.ro>
+;;; Copyright © 2020, 2021 Jesse Gibbons <jgibbons2357+guix@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -41,11 +42,13 @@
   #:use-module (gnu system shadow)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages bash)
   #:use-module (gnu packages databases)
   #:use-module (gnu packages web)
   #:use-module (gnu packages patchutils)
   #:use-module (gnu packages php)
   #:use-module (gnu packages python)
+  #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages gnupg)
   #:use-module (gnu packages guile)
   #:use-module (gnu packages logging)
@@ -91,6 +94,8 @@
 
             httpd-service-type
 
+            jupyter-configuration
+
             nginx-configuration
             nginx-configuration?
             nginx-configuration-nginx
@@ -1994,3 +1999,118 @@ root=/srv/gemini
           (service-extension shepherd-root-service-type
                              agate-shepherd-service)))
    (default-value (agate-configuration))))
+
+
+;;; Jupyter configuration
+;;; Jupyter server configuration
+;;; -> includes a list of kernels to allow in the jupyter server
+;;; -> includes a list of configuration options specifically for the jupyter server
+;;; --> These options can be concatenated to a python file
+;;; --> Nested alist
+;;; ---> '((NotebookApp (allow-remote-access #t))) -> "c.NotebookApp.allow_remote_access = True"
+;;; -> Include some settings specifying how to run the daemon
+;;; --> location of log file
+;;; --> If a console should be allowed, the location of the shell to be used.
+;;; --> The package containing the jupyter server itself, default jupyter
+(define-public default-jupyter-config "#")
+
+(define-record-type* <jupyter-configuration> jupyter-configuration
+                     this-jupyter-configuration jupyter-configuration?
+                     (jupyter-package jupyter-configuration-jupyter-package
+                                      (default jupyter))
+                     (kernels jupyter-configuration-kernels
+                              (default '()))
+                     (root-dir jupyter-configuration-root-dir
+                               (default "/var/lib/jupyter/notebooks"))
+                     (log-file jupyter-configuration-log-file
+                               (default #f))
+                     (shell jupyter-configuration-shell
+                            (default (file-append bash "/bin/bash")))
+                     (server-config jupyter-configuration-server-config
+                                    (default #f))); TODO: Make configuration DSL.
+
+
+(define (search-path-string search-path-pair)
+  (string-append (search-path-specification-variable (car search-path-pair))
+                 "="
+                 (cdr search-path-pair)))
+
+;;;TODO: Add actions to list jupyter servers, change passwords, etc.
+(define (jupyter-shepherd-service config)
+  (list (shepherd-service
+          (provision '(jupyter)) ;TODO: Add magic to allow multiple Jupyter servers
+          (requirement '(loopback))
+          (start #~(make-forkexec-constructor
+                     (list "/run/current-system/profile/bin/jupyter"
+                           "notebook"
+                           (string-append "--config="
+                     #$(or (jupyter-configuration-server-config config)
+                           (plain-file "jupyter_notebook_config.py"
+                                       default-jupyter-config))
+                     #$(jupyter-configuration-root-dir config))
+                     #:user "jupyter"
+                     #:group "jupyter"
+                     #:environment-variables ; TODO use search-paths magic instead of hard-coding these things.
+                     (list "GI_TYPELIB_PATH=/run/current-system/profile/lib/girepository-1.0"
+                           "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0"
+                           "GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0"
+                           "HOME=/var/lib/jupyter"
+                           "JUPYTER_PATH=/run/current-system/profile/share/jupyter"
+                           "PATH=/run/current-system/profile/bin:/run/current-system/profile/sbin"
+                           "PYTHONPATH=/run/current-system/profile/lib/python3.8/site-packages"
+                           "R_LIBS_SITE=/run/current-system/profile/site-library/"
+                           "TEXMF=/run/current-system/profile/share/texmf-dist"
+                           "TEXMFCNF=/run/current-system/profile//share/texmf-dist/web2c"
+                           "XDG_DATA_DIRS=/run/current-system/profile/share")
+
+                     #:directory #$(jupyter-configuration-root-dir config)
+                     #:log-file #$(jupyter-configuration-log-file config)))
+          (stop #~(make-kill-destructor))
+          (documentation "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text."))))
+
+(define (jupyter-account config)
+  (list
+    (user-account
+      (name "jupyter")
+      (group "jupyter")
+      (comment "Jupyter Notebook Server")
+      (home-directory "/var/lib/jupyter")
+      (shell (jupyter-configuration-shell config))
+      (system? #t))
+    (user-group
+      (name "jupyter")
+      (system? #t))))
+
+(define (jupyter-profile config)
+  (cons*
+    (jupyter-configuration-jupyter-package config)
+    (jupyter-configuration-kernels config)))
+
+(define (jupyter-activation config)
+  #~(begin
+      (let ((root-dir #$(jupyter-configuration-root-dir config))
+            (pw (getpwnam "jupyter")))
+        (unless (file-exists? root-dir)
+          (mkdir root-dir)
+          (chown root-dir (passwd:uid pw)
+                 (passwd:gid pw))
+          (chmod root-dir #o700)))))
+
+;;; Jupyter service type
+;;; -> Information vital to settinng up the server, like the port and accepted parameters
+;;; -> list of kernels considered permissible.
+;;; -> a shepherd service extension that runs the jupyter server
+;;; --> shepherd service to list the running servers, set the password, etc.
+;;; --> Make a log file only readable by root?
+;;; -> an accounts service extension describing the user that runs the accounts
+;;; --> group "jupyter"
+(define-public jupyter-service-type
+  (service-type
+    (name "jupyter")
+    (extensions (list
+                  (service-extension shepherd-root-service-type jupyter-shepherd-service)
+                  (service-extension account-service-type jupyter-account)
+                  (service-extension activation-service-type jupyter-activation)
+                  (service-extension profile-service-type jupyter-profile)))
+    (description "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text.")
+    (default-value (jupyter-configuration))))
-- 
2.33.0
J
(address . 50708@debbugs.gnu.org)
74f78a5c-3f79-7485-0cf2-ae1fbb1969dd@gmail.com
Attached is an updated patch.
From d264da0811480a8d2acd5e73f58d320e15bfa9f3 Mon Sep 17 00:00:00 2001
From: Jesse <gib19014@byui.edu>
Date: Mon, 20 Sep 2021 16:01:22 -0600
Subject: [PATCH] gnu: web: Add jupyter-service

* gnu/services/web.scm (gitile-service-type): New variable.
* doc/guix.texi (Version Control Services): Document it.
---
doc/guix.texi | 52 +++++++++++++++++++
gnu/services/web.scm | 120 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 172 insertions(+)

Toggle diff (219 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 6436e83a7c..0f8aaca413 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -26197,6 +26197,58 @@ The file which should store the logging output of Agate.
 @end table
 @end deftp
 
+@cindex jupyter
+The Jupyter Notebook is a web application that allows you to create
+and share documents that contain live code, equations, visualizations
+and explanatory text.  Uses include: data cleaning and transformation,
+numerical simulation, statistical modeling, machine learning and much
+more.
+
+@deffn (Scheme Variable) jupyter-service-type
+This is the type of the agate service, whose value should be an
+@code{jupyter-service-type} object, as in this example:
+
+@lisp
+(service jupyter-service-type
+ (jupyter-configuration
+  (log-file "/var/log/jupyter.log")
+  (server-config
+   (kernels
+    (list jupyter-guile-kernel)))))
+
+@end lisp
+
+The example above tells the the Jupyter service to provide the
+@code{jupyter-guile-kernel} as an optional kernel.
+@end deffn
+
+@deftp {Data Type} jupyter-configuration
+Data type representing the configuration of Agate.
+
+@table @asis
+@item @code{jupyter-package} (default: @code{jupyter})
+The jupyter package to use.
+
+
+@item @code{kernels} (default: @code{'()}
+A list of kernel packages to use with the jupyter service.
+
+@item @code{root-dir} (default: @file{"/var/lib/jupyter/notebooks"})
+The directory where the jupyter server looks for jupyter notebooks.
+
+@item @code{log-file} (default: @code{#f})
+The location of the log file. If #f is given, no log is produced.
+
+@item @code{shell} (default: @code{(file-append bash "/bin/bash")}
+The location of the shell used when starting a web terminal (if it is configured to allow terminals).
+
+@item @code{server-config} (default: @code{#f})
+A file contianing the jupyter server's config file. If @code{#f} is given, an empty file is produced, and the default settings are used.
+
+@end table
+@end deftp
+
+
 @node Certificate Services
 @subsection Certificate Services
 
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index 6a093cf4e4..2d2946f7a7 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -15,6 +15,7 @@
 ;;; Copyright © 2020 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2020 Oleg Pykhalov <go.wigust@gmail.com>
 ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton <brown121407@posteo.ro>
+;;; Copyright © 2020, 2021 Jesse Gibbons <jgibbons2357+guix@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -41,11 +42,13 @@
   #:use-module (gnu system shadow)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages bash)
   #:use-module (gnu packages databases)
   #:use-module (gnu packages web)
   #:use-module (gnu packages patchutils)
   #:use-module (gnu packages php)
   #:use-module (gnu packages python)
+  #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages gnupg)
   #:use-module (gnu packages guile)
   #:use-module (gnu packages logging)
@@ -91,6 +94,8 @@
 
             httpd-service-type
 
+            jupyter-configuration
+
             nginx-configuration
             nginx-configuration?
             nginx-configuration-nginx
@@ -1994,3 +1999,118 @@ root=/srv/gemini
           (service-extension shepherd-root-service-type
                              agate-shepherd-service)))
    (default-value (agate-configuration))))
+
+
+;;; Jupyter configuration
+;;; Jupyter server configuration
+;;; -> includes a list of kernels to allow in the jupyter server
+;;; -> includes a list of configuration options specifically for the jupyter server
+;;; --> These options can be concatenated to a python file
+;;; --> Nested alist
+;;; ---> '((NotebookApp (allow-remote-access #t))) -> "c.NotebookApp.allow_remote_access = True"
+;;; -> Include some settings specifying how to run the daemon
+;;; --> location of log file
+;;; --> If a console should be allowed, the location of the shell to be used.
+;;; --> The package containing the jupyter server itself, default jupyter
+(define-public default-jupyter-config "#")
+
+(define-record-type* <jupyter-configuration> jupyter-configuration
+                     this-jupyter-configuration jupyter-configuration?
+                     (jupyter-package jupyter-configuration-jupyter-package
+                                      (default jupyter))
+                     (kernels jupyter-configuration-kernels
+                              (default '()))
+                     (root-dir jupyter-configuration-root-dir
+                               (default "/var/lib/jupyter/notebooks"))
+                     (log-file jupyter-configuration-log-file
+                               (default #f))
+                     (shell jupyter-configuration-shell
+                            (default (file-append bash "/bin/bash")))
+                     (server-config jupyter-configuration-server-config
+                                    (default #f))); TODO: Make configuration DSL.
+
+
+(define (search-path-string search-path-pair)
+  (string-append (search-path-specification-variable (car search-path-pair))
+                 "="
+                 (cdr search-path-pair)))
+
+;;;TODO: Add actions to list jupyter servers, change passwords, etc.
+(define (jupyter-shepherd-service config)
+  (list (shepherd-service
+          (provision '(jupyter)) ;TODO: Add magic to allow multiple Jupyter servers
+          (requirement '(loopback))
+          (start #~(make-forkexec-constructor
+                     (list "/run/current-system/profile/bin/jupyter"
+                           "notebook"
+                           (string-append "--config="
+                     #$(or (jupyter-configuration-server-config config)
+                           (plain-file "jupyter_notebook_config.py"
+                                       default-jupyter-config))
+                     #$(jupyter-configuration-root-dir config)))
+                     #:user "jupyter"
+                     #:group "jupyter"
+                     #:environment-variables ; TODO use search-paths magic instead of hard-coding these things.
+                     (list "GI_TYPELIB_PATH=/run/current-system/profile/lib/girepository-1.0"
+                           "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0"
+                           "GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0"
+                           "HOME=/var/lib/jupyter"
+                           "JUPYTER_PATH=/run/current-system/profile/share/jupyter"
+                           "PATH=/run/current-system/profile/bin:/run/current-system/profile/sbin"
+                           "PYTHONPATH=/run/current-system/profile/lib/python3.8/site-packages"
+                           "R_LIBS_SITE=/run/current-system/profile/site-library/"
+                           "TEXMF=/run/current-system/profile/share/texmf-dist"
+                           "TEXMFCNF=/run/current-system/profile//share/texmf-dist/web2c"
+                           "XDG_DATA_DIRS=/run/current-system/profile/share")
+
+                     #:directory #$(jupyter-configuration-root-dir config)
+                     #:log-file #$(jupyter-configuration-log-file config)))
+          (stop #~(make-kill-destructor))
+          (documentation "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text."))))
+
+(define (jupyter-account config)
+  (list
+    (user-account
+      (name "jupyter")
+      (group "jupyter")
+      (comment "Jupyter Notebook Server")
+      (home-directory "/var/lib/jupyter")
+      (shell (jupyter-configuration-shell config))
+      (system? #t))
+    (user-group
+      (name "jupyter")
+      (system? #t))))
+
+(define (jupyter-profile config)
+  (cons*
+    (jupyter-configuration-jupyter-package config)
+    (jupyter-configuration-kernels config)))
+
+(define (jupyter-activation config)
+  #~(begin
+      (let ((root-dir #$(jupyter-configuration-root-dir config))
+            (pw (getpwnam "jupyter")))
+        (unless (file-exists? root-dir)
+          (mkdir root-dir)
+          (chown root-dir (passwd:uid pw)
+                 (passwd:gid pw))
+          (chmod root-dir #o700)))))
+
+;;; Jupyter service type
+;;; -> Information vital to settinng up the server, like the port and accepted parameters
+;;; -> list of kernels considered permissible.
+;;; -> a shepherd service extension that runs the jupyter server
+;;; --> shepherd service to list the running servers, set the password, etc.
+;;; --> Make a log file only readable by root?
+;;; -> an accounts service extension describing the user that runs the accounts
+;;; --> group "jupyter"
+(define-public jupyter-service-type
+  (service-type
+    (name "jupyter")
+    (extensions (list
+                  (service-extension shepherd-root-service-type jupyter-shepherd-service)
+                  (service-extension account-service-type jupyter-account)
+                  (service-extension activation-service-type jupyter-activation)
+                  (service-extension profile-service-type jupyter-profile)))
+    (description "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text.")
+    (default-value (jupyter-configuration))))
-- 
2.33.0
M
M
Maxim Cournoyer wrote on 23 Sep 2021 04:19
(name . Jesse)(address . jgibbons2357@gmail.com)(address . 50708@debbugs.gnu.org)
87pmt0kp83.fsf_-_@gmail.com
Hello Jesse,

Jesse <jgibbons2357@gmail.com> writes:

Toggle quote (31 lines)
> Attached is an updated patch.
>
> From d264da0811480a8d2acd5e73f58d320e15bfa9f3 Mon Sep 17 00:00:00 2001
> From: Jesse <gib19014@byui.edu>
> Date: Mon, 20 Sep 2021 16:01:22 -0600
> Subject: [PATCH] gnu: web: Add jupyter-service
>
> * gnu/services/web.scm (gitile-service-type): New variable.
> * doc/guix.texi (Version Control Services): Document it.
> ---
> doc/guix.texi | 52 +++++++++++++++++++
> gnu/services/web.scm | 120 +++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 172 insertions(+)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 6436e83a7c..0f8aaca413 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -26197,6 +26197,58 @@ The file which should store the logging output of Agate.
> @end table
> @end deftp
>
> +@cindex jupyter
> +The Jupyter Notebook is a web application that allows you to create
> +and share documents that contain live code, equations, visualizations
> +and explanatory text. Uses include: data cleaning and transformation,
> +numerical simulation, statistical modeling, machine learning and much
> +more.
> +
> +@deffn (Scheme Variable) jupyter-service-type
> +This is the type of the agate service, whose value should be an
^
Agate

I'm a bit confused by this service referring to Agate; seems it'd be
clearer for the neophyte to simply refer to Jupyter in its place, even
if it's technically an Agate server powering it?

Toggle quote (12 lines)
> +@code{jupyter-service-type} object, as in this example:
> +
> +@lisp
> +(service jupyter-service-type
> + (jupyter-configuration
> + (log-file "/var/log/jupyter.log")
> + (server-config
> + (kernels
> + (list jupyter-guile-kernel)))))
> +
> +@end lisp

I'd put the (list jupyter-guile-kernel) on the same line as (kernels
...)

Toggle quote (1 lines)
> +The example above tells the the Jupyter service to provide the
^ typo
Toggle quote (9 lines)
> +@code{jupyter-guile-kernel} as an optional kernel.
> +@end deffn
> +
> +@deftp {Data Type} jupyter-configuration
> +Data type representing the configuration of Agate.
> +
> +@table @asis
> +@item @code{jupyter-package} (default: @code{jupyter})
> +The jupyter package to use.
^
Jupyter

I'd capitalize Jupyter in other places as well where it's not decorated
as a command or code, since it's a proper name.

Toggle quote (3 lines)
> +
> +@item @code{kernels} (default: @code{'()}
> +A list of kernel packages to use with the jupyter service.
^ here
Toggle quote (3 lines)
> +
> +@item @code{root-dir} (default: @file{"/var/lib/jupyter/notebooks"})
> +The directory where the jupyter server looks for jupyter notebooks.
^ and here
Toggle quote (8 lines)
> +@item @code{log-file} (default: @code{#f})
> +The location of the log file. If #f is given, no log is produced.
>
> +@item @code{shell} (default: @code{(file-append bash "/bin/bash")}
> +The location of the shell used when starting a web terminal (if it is configured to allow terminals).
> +
> +@item @code{server-config} (default: @code{#f})
> +A file contianing the jupyter server's config file. If @code{#f} is given, an empty file is produced,
^ containing

Toggle quote (63 lines)
> and the default settings are used.
> +
> +@end table
> +@end deftp
> +
> +
> @node Certificate Services
> @subsection Certificate Services
>
> diff --git a/gnu/services/web.scm b/gnu/services/web.scm
> index 6a093cf4e4..2d2946f7a7 100644
> --- a/gnu/services/web.scm
> +++ b/gnu/services/web.scm
> @@ -15,6 +15,7 @@
> ;;; Copyright © 2020 Arun Isaac <arunisaac@systemreboot.net>
> ;;; Copyright © 2020 Oleg Pykhalov <go.wigust@gmail.com>
> ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton <brown121407@posteo.ro>
> +;;; Copyright © 2020, 2021 Jesse Gibbons <jgibbons2357+guix@gmail.com>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -41,11 +42,13 @@
> #:use-module (gnu system shadow)
> #:use-module (gnu packages admin)
> #:use-module (gnu packages base)
> + #:use-module (gnu packages bash)
> #:use-module (gnu packages databases)
> #:use-module (gnu packages web)
> #:use-module (gnu packages patchutils)
> #:use-module (gnu packages php)
> #:use-module (gnu packages python)
> + #:use-module (gnu packages python-xyz)
> #:use-module (gnu packages gnupg)
> #:use-module (gnu packages guile)
> #:use-module (gnu packages logging)
> @@ -91,6 +94,8 @@
>
> httpd-service-type
>
> + jupyter-configuration
> +
> nginx-configuration
> nginx-configuration?
> nginx-configuration-nginx
> @@ -1994,3 +1999,118 @@ root=/srv/gemini
> (service-extension shepherd-root-service-type
> agate-shepherd-service)))
> (default-value (agate-configuration))))
> +
> +
> +;;; Jupyter configuration
> +;;; Jupyter server configuration
> +;;; -> includes a list of kernels to allow in the jupyter server
> +;;; -> includes a list of configuration options specifically for the jupyter server
> +;;; --> These options can be concatenated to a python file
> +;;; --> Nested alist
> +;;; ---> '((NotebookApp (allow-remote-access #t))) -> "c.NotebookApp.allow_remote_access = True"
> +;;; -> Include some settings specifying how to run the daemon
> +;;; --> location of log file
> +;;; --> If a console should be allowed, the location of the shell to be used.
> +;;; --> The package containing the jupyter server itself, default
> jupyter

What is the meaning of the arrows in the above comment? It seems to me
that they were meant to be used during development, not in the final
result? Otherwise, the types of the fields could be self-documented and
controlled by the use of the 'define-configuration' procedure and
friends from the (gnu services configuration) module.

Toggle quote (49 lines)
> +(define-public default-jupyter-config "#")
> +
> +(define-record-type* <jupyter-configuration> jupyter-configuration
> + this-jupyter-configuration jupyter-configuration?
> + (jupyter-package jupyter-configuration-jupyter-package
> + (default jupyter))
> + (kernels jupyter-configuration-kernels
> + (default '()))
> + (root-dir jupyter-configuration-root-dir
> + (default "/var/lib/jupyter/notebooks"))
> + (log-file jupyter-configuration-log-file
> + (default #f))
> + (shell jupyter-configuration-shell
> + (default (file-append bash "/bin/bash")))
> + (server-config jupyter-configuration-server-config
> + (default #f))); TODO: Make configuration DSL.
> +
> +
> +(define (search-path-string search-path-pair)
> + (string-append (search-path-specification-variable (car search-path-pair))
> + "="
> + (cdr search-path-pair)))
> +
> +;;;TODO: Add actions to list jupyter servers, change passwords, etc.
> +(define (jupyter-shepherd-service config)
> + (list (shepherd-service
> + (provision '(jupyter)) ;TODO: Add magic to allow multiple Jupyter servers
> + (requirement '(loopback))
> + (start #~(make-forkexec-constructor
> + (list "/run/current-system/profile/bin/jupyter"
> + "notebook"
> + (string-append "--config="
> + #$(or (jupyter-configuration-server-config config)
> + (plain-file "jupyter_notebook_config.py"
> + default-jupyter-config))
> + #$(jupyter-configuration-root-dir config)))
> + #:user "jupyter"
> + #:group "jupyter"
> + #:environment-variables ; TODO use search-paths magic instead of hard-coding these things.
> + (list "GI_TYPELIB_PATH=/run/current-system/profile/lib/girepository-1.0"
> + "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0"
> + "GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0"
> + "HOME=/var/lib/jupyter"
> + "JUPYTER_PATH=/run/current-system/profile/share/jupyter"
> + "PATH=/run/current-system/profile/bin:/run/current-system/profile/sbin"
> + "PYTHONPATH=/run/current-system/profile/lib/python3.8/site-packages"
> + "R_LIBS_SITE=/run/current-system/profile/site-library/"
> + "TEXMF=/run/current-system/profile/share/texmf-dist"
> + "TEXMFCNF=/run/current-system/profile//share/texmf-dist/web2c"
^
extra slash
Toggle quote (3 lines)
> + "XDG_DATA_DIRS=/run/current-system/profile/share")
> +

Hmm. Would it work if search paths were set on the 'jupyter' package
instead? The above is too fragile, with hard coded versions baked in.

Toggle quote (4 lines)
> + #:directory #$(jupyter-configuration-root-dir config)
> + #:log-file #$(jupyter-configuration-log-file config)))
> + (stop #~(make-kill-destructor))
> + (documentation "Runs a Jupyter Notebook server. A Jupyter Notebook is
^ Use two
spaces after period.

Toggle quote (39 lines)
>a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text."))))
> +
> +(define (jupyter-account config)
> + (list
> + (user-account
> + (name "jupyter")
> + (group "jupyter")
> + (comment "Jupyter Notebook Server")
> + (home-directory "/var/lib/jupyter")
> + (shell (jupyter-configuration-shell config))
> + (system? #t))
> + (user-group
> + (name "jupyter")
> + (system? #t))))
> +
> +(define (jupyter-profile config)
> + (cons*
> + (jupyter-configuration-jupyter-package config)
> + (jupyter-configuration-kernels config)))
> +
> +(define (jupyter-activation config)
> + #~(begin
> + (let ((root-dir #$(jupyter-configuration-root-dir config))
> + (pw (getpwnam "jupyter")))
> + (unless (file-exists? root-dir)
> + (mkdir root-dir)
> + (chown root-dir (passwd:uid pw)
> + (passwd:gid pw))
> + (chmod root-dir #o700)))))
> +
> +;;; Jupyter service type
> +;;; -> Information vital to settinng up the server, like the port and accepted parameters
> +;;; -> list of kernels considered permissible.
> +;;; -> a shepherd service extension that runs the jupyter server
> +;;; --> shepherd service to list the running servers, set the password, etc.
> +;;; --> Make a log file only readable by root?
> +;;; -> an accounts service extension describing the user that runs the accounts
> +;;; --> group "jupyter"

Same comment as earlier.

Toggle quote (11 lines)
> +(define-public jupyter-service-type
> + (service-type
> + (name "jupyter")
> + (extensions (list
> + (service-extension shepherd-root-service-type jupyter-shepherd-service)
> + (service-extension account-service-type jupyter-account)
> + (service-extension activation-service-type jupyter-activation)
> + (service-extension profile-service-type jupyter-profile)))
> + (description "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text.")
> + (default-value (jupyter-configuration))))

I haven't tried it, but it looks promising. If we can find a better
solution than the hard coded version values in the environment variables
list, I think it'd be in good shape for inclusion, although a system
test would be a nice to have.

Thank you!

Maxim
J
(address . 50708@debbugs.gnu.org)
cc9534d6-a448-9c30-e851-8dae9ff0eec6@gmail.com
Attached is yet another updated patch fixing a mistake in the added
documentation.

On 9/21/21 7:26 PM, Jesse wrote:
Toggle quote (1 lines)
> Attached is an updated patch.
From d264da0811480a8d2acd5e73f58d320e15bfa9f3 Mon Sep 17 00:00:00 2001
From: Jesse <gib19014@byui.edu>
Date: Mon, 20 Sep 2021 16:01:22 -0600
Subject: [PATCH] gnu: web: Add jupyter-service

* gnu/services/web.scm (gitile-service-type): New variable.
* doc/guix.texi (Version Control Services): Document it.
---
doc/guix.texi | 52 +++++++++++++++++++
gnu/services/web.scm | 120 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 172 insertions(+)

Toggle diff (219 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 6436e83a7c..0f8aaca413 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -26197,6 +26197,58 @@ The file which should store the logging output of Agate.
 @end table
 @end deftp
 
+@cindex jupyter
+The Jupyter Notebook is a web application that allows you to create
+and share documents that contain live code, equations, visualizations
+and explanatory text.  Uses include: data cleaning and transformation,
+numerical simulation, statistical modeling, machine learning and much
+more.
+
+@deffn (Scheme Variable) jupyter-service-type
+This is the type of the agate service, whose value should be an
+@code{jupyter-service-type} object, as in this example:
+
+@lisp
+(service jupyter-service-type
+ (jupyter-configuration
+  (log-file "/var/log/jupyter.log")
+  (server-config
+   (kernels
+    (list jupyter-guile-kernel)))))
+
+@end lisp
+
+The example above tells the the Jupyter service to provide the
+@code{jupyter-guile-kernel} as an optional kernel.
+@end deffn
+
+@deftp {Data Type} jupyter-configuration
+Data type representing the configuration of Agate.
+
+@table @asis
+@item @code{jupyter-package} (default: @code{jupyter})
+The jupyter package to use.
+
+
+@item @code{kernels} (default: @code{'()}
+A list of kernel packages to use with the jupyter service.
+
+@item @code{root-dir} (default: @file{"/var/lib/jupyter/notebooks"})
+The directory where the jupyter server looks for jupyter notebooks.
+
+@item @code{log-file} (default: @code{#f})
+The location of the log file. If #f is given, no log is produced.
+
+@item @code{shell} (default: @code{(file-append bash "/bin/bash")}
+The location of the shell used when starting a web terminal (if it is configured to allow terminals).
+
+@item @code{server-config} (default: @code{#f})
+A file contianing the jupyter server's config file. If @code{#f} is given, an empty file is produced, and the default settings are used.
+
+@end table
+@end deftp
+
+
 @node Certificate Services
 @subsection Certificate Services
 
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index 6a093cf4e4..2d2946f7a7 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -15,6 +15,7 @@
 ;;; Copyright © 2020 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2020 Oleg Pykhalov <go.wigust@gmail.com>
 ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton <brown121407@posteo.ro>
+;;; Copyright © 2020, 2021 Jesse Gibbons <jgibbons2357+guix@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -41,11 +42,13 @@
   #:use-module (gnu system shadow)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages bash)
   #:use-module (gnu packages databases)
   #:use-module (gnu packages web)
   #:use-module (gnu packages patchutils)
   #:use-module (gnu packages php)
   #:use-module (gnu packages python)
+  #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages gnupg)
   #:use-module (gnu packages guile)
   #:use-module (gnu packages logging)
@@ -91,6 +94,8 @@
 
             httpd-service-type
 
+            jupyter-configuration
+
             nginx-configuration
             nginx-configuration?
             nginx-configuration-nginx
@@ -1994,3 +1999,118 @@ root=/srv/gemini
           (service-extension shepherd-root-service-type
                              agate-shepherd-service)))
    (default-value (agate-configuration))))
+
+
+;;; Jupyter configuration
+;;; Jupyter server configuration
+;;; -> includes a list of kernels to allow in the jupyter server
+;;; -> includes a list of configuration options specifically for the jupyter server
+;;; --> These options can be concatenated to a python file
+;;; --> Nested alist
+;;; ---> '((NotebookApp (allow-remote-access #t))) -> "c.NotebookApp.allow_remote_access = True"
+;;; -> Include some settings specifying how to run the daemon
+;;; --> location of log file
+;;; --> If a console should be allowed, the location of the shell to be used.
+;;; --> The package containing the jupyter server itself, default jupyter
+(define-public default-jupyter-config "#")
+
+(define-record-type* <jupyter-configuration> jupyter-configuration
+                     this-jupyter-configuration jupyter-configuration?
+                     (jupyter-package jupyter-configuration-jupyter-package
+                                      (default jupyter))
+                     (kernels jupyter-configuration-kernels
+                              (default '()))
+                     (root-dir jupyter-configuration-root-dir
+                               (default "/var/lib/jupyter/notebooks"))
+                     (log-file jupyter-configuration-log-file
+                               (default #f))
+                     (shell jupyter-configuration-shell
+                            (default (file-append bash "/bin/bash")))
+                     (server-config jupyter-configuration-server-config
+                                    (default #f))); TODO: Make configuration DSL.
+
+
+(define (search-path-string search-path-pair)
+  (string-append (search-path-specification-variable (car search-path-pair))
+                 "="
+                 (cdr search-path-pair)))
+
+;;;TODO: Add actions to list jupyter servers, change passwords, etc.
+(define (jupyter-shepherd-service config)
+  (list (shepherd-service
+          (provision '(jupyter)) ;TODO: Add magic to allow multiple Jupyter servers
+          (requirement '(loopback))
+          (start #~(make-forkexec-constructor
+                     (list "/run/current-system/profile/bin/jupyter"
+                           "notebook"
+                           (string-append "--config="
+                     #$(or (jupyter-configuration-server-config config)
+                           (plain-file "jupyter_notebook_config.py"
+                                       default-jupyter-config))
+                     #$(jupyter-configuration-root-dir config)))
+                     #:user "jupyter"
+                     #:group "jupyter"
+                     #:environment-variables ; TODO use search-paths magic instead of hard-coding these things.
+                     (list "GI_TYPELIB_PATH=/run/current-system/profile/lib/girepository-1.0"
+                           "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0"
+                           "GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0"
+                           "HOME=/var/lib/jupyter"
+                           "JUPYTER_PATH=/run/current-system/profile/share/jupyter"
+                           "PATH=/run/current-system/profile/bin:/run/current-system/profile/sbin"
+                           "PYTHONPATH=/run/current-system/profile/lib/python3.8/site-packages"
+                           "R_LIBS_SITE=/run/current-system/profile/site-library/"
+                           "TEXMF=/run/current-system/profile/share/texmf-dist"
+                           "TEXMFCNF=/run/current-system/profile//share/texmf-dist/web2c"
+                           "XDG_DATA_DIRS=/run/current-system/profile/share")
+
+                     #:directory #$(jupyter-configuration-root-dir config)
+                     #:log-file #$(jupyter-configuration-log-file config)))
+          (stop #~(make-kill-destructor))
+          (documentation "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text."))))
+
+(define (jupyter-account config)
+  (list
+    (user-account
+      (name "jupyter")
+      (group "jupyter")
+      (comment "Jupyter Notebook Server")
+      (home-directory "/var/lib/jupyter")
+      (shell (jupyter-configuration-shell config))
+      (system? #t))
+    (user-group
+      (name "jupyter")
+      (system? #t))))
+
+(define (jupyter-profile config)
+  (cons*
+    (jupyter-configuration-jupyter-package config)
+    (jupyter-configuration-kernels config)))
+
+(define (jupyter-activation config)
+  #~(begin
+      (let ((root-dir #$(jupyter-configuration-root-dir config))
+            (pw (getpwnam "jupyter")))
+        (unless (file-exists? root-dir)
+          (mkdir root-dir)
+          (chown root-dir (passwd:uid pw)
+                 (passwd:gid pw))
+          (chmod root-dir #o700)))))
+
+;;; Jupyter service type
+;;; -> Information vital to settinng up the server, like the port and accepted parameters
+;;; -> list of kernels considered permissible.
+;;; -> a shepherd service extension that runs the jupyter server
+;;; --> shepherd service to list the running servers, set the password, etc.
+;;; --> Make a log file only readable by root?
+;;; -> an accounts service extension describing the user that runs the accounts
+;;; --> group "jupyter"
+(define-public jupyter-service-type
+  (service-type
+    (name "jupyter")
+    (extensions (list
+                  (service-extension shepherd-root-service-type jupyter-shepherd-service)
+                  (service-extension account-service-type jupyter-account)
+                  (service-extension activation-service-type jupyter-activation)
+                  (service-extension profile-service-type jupyter-profile)))
+    (description "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text.")
+    (default-value (jupyter-configuration))))
-- 
2.33.0
J
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 50708@debbugs.gnu.org)
23294845-1ac9-b2ef-c056-89f6ee51435f@gmail.com
Attached is a patch that takes into account your feedback.

On 9/22/21 8:19 PM, Maxim Cournoyer wrote:
Toggle quote (3 lines)
> Hello Jesse,
>
> Jesse <jgibbons2357@gmail.com> writes:
...
Toggle quote (28 lines)
> + #:environment-variables ; TODO use search-paths magic instead of hard-coding these things.
> + (list "GI_TYPELIB_PATH=/run/current-system/profile/lib/girepository-1.0"
> + "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0"
> + "GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0"
> + "HOME=/var/lib/jupyter"
> + "JUPYTER_PATH=/run/current-system/profile/share/jupyter"
> + "PATH=/run/current-system/profile/bin:/run/current-system/profile/sbin"
> + "PYTHONPATH=/run/current-system/profile/lib/python3.8/site-packages"
> + "R_LIBS_SITE=/run/current-system/profile/site-library/"
> + "TEXMF=/run/current-system/profile/share/texmf-dist"
> + "TEXMFCNF=/run/current-system/profile//share/texmf-dist/web2c"
>
>> + "XDG_DATA_DIRS=/run/current-system/profile/share")
>> +
> Hmm. Would it work if search paths were set on the 'jupyter' package
> instead? The above is too fragile, with hard coded versions baked in.
>
> ...

> I haven't tried it, but it looks promising. If we can find a better
> solution than the hard coded version values in the environment variables
> list, I think it'd be in good shape for inclusion, although a system
> test would be a nice to have.
>
> Thank you!
>
> Maxim

The weird thing about this particular service is it is more useful with
libraries Jupyter and the kernels are not dependent on. This part is
leftover from when I wrote it several months ago for a home server. (I
actually used this service to work on notebooks using the Guile kernel
working with sqlite and to try the other kernels.) I found that it
couldn't find the packages I need and the web shell was broken, so I
settled for hard-coding these variables until I could find time to fix them.

I included a TODO with an idea I have not yet implemented: use the
search-paths to give Jupyter the environment variables it needs the same
way a profile sets up its search paths. I thought it might be a good
idea to associate the service with its own profile for security reasons.
This would also fix a bug where the Jupyter Notebook icon appears on the
desktop but fails to start a new Jupyter server.

Another idea I had is the service could be isolated in its own container
with the network exposed and the home directory shared, but I have no
idea how that would work or if it would even add more security. However,
this could potentially allow multiple Jupyter services to run
simultaneously on different ports. However, one problem is the resulting
server might be unusable because the login information would be in a log
file hidden in a container.

I'll work on the search-paths idea and send an update when I'm done. I'm
also open to other suggestions on how to fix the hard-coded values. I
don't think this will be ready to merge until that problem is fixed, but
I've attached the patch anyway for curious users who want to try it out.
I would appreciate help implementing the container idea if it's feasible.

-Jesse
From f96e41a7222b51cb4ff6e16657de2e87fea78472 Mon Sep 17 00:00:00 2001
From: Jesse <gib19014@byui.edu>
Date: Mon, 20 Sep 2021 16:01:22 -0600
Subject: [PATCH] gnu: web: Add jupyter-service

* gnu/services/web.scm (gitile-service-type): New variable.
* doc/guix.texi (Version Control Services): Document it.
---
doc/guix.texi | 52 +++++++++++++++++++++
gnu/services/web.scm | 105 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 157 insertions(+)

Toggle diff (204 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 6436e83a7c..b92246cc8a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -26197,6 +26197,58 @@ The file which should store the logging output of Agate.
 @end table
 @end deftp
 
+@subsubheading Jupyter
+@cindex jupyter
+The Jupyter Notebook is a web application that allows you to create
+and share documents that contain live code, equations, visualizations
+and explanatory text.  Uses include: data cleaning and transformation,
+numerical simulation, statistical modeling, machine learning and much
+more.
+
+@deffn (Scheme Variable) jupyter-service-type
+This is the type of the Jupyter service, whose value should be an
+@code{jupyter-service-type} object, as in this example:
+
+@lisp
+(service jupyter-service-type
+ (jupyter-configuration
+  (log-file "/var/log/jupyter.log")
+  (server-config
+   (kernels (list jupyter-guile-kernel)))))
+
+@end lisp
+
+The example above tells the Jupyter service to provide the
+@code{jupyter-guile-kernel} as an optional kernel.
+@end deffn
+
+@deftp {Data Type} jupyter-configuration
+Data type representing the configuration of Agate.
+
+@table @asis
+@item @code{jupyter-package} (default: @code{jupyter})
+The Jupyter package to use.
+
+
+@item @code{kernels} (default: @code{'()}
+A list of kernel packages to use with the Jupyter service.
+
+@item @code{root-dir} (default: @file{"/var/lib/jupyter/notebooks"})
+The directory where the Jupyter server looks for Jupyter notebooks.
+
+@item @code{log-file} (default: @code{#f})
+The location of the log file. If #f is given, no log is produced.
+
+@item @code{shell} (default: @code{(file-append bash "/bin/bash")}
+The location of the shell used when starting a web terminal (if it is configured to allow terminals).
+
+@item @code{server-config} (default: @code{#f})
+A file containing the Jupyter server's config file. If @code{#f} is given, an empty file is produced, and the default settings are used.
+
+@end table
+@end deftp
+
+
 @node Certificate Services
 @subsection Certificate Services
 
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index 6a093cf4e4..76700da412 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -15,6 +15,7 @@
 ;;; Copyright © 2020 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2020 Oleg Pykhalov <go.wigust@gmail.com>
 ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton <brown121407@posteo.ro>
+;;; Copyright © 2020, 2021 Jesse Gibbons <jgibbons2357+guix@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -41,11 +42,13 @@
   #:use-module (gnu system shadow)
   #:use-module (gnu packages admin)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages bash)
   #:use-module (gnu packages databases)
   #:use-module (gnu packages web)
   #:use-module (gnu packages patchutils)
   #:use-module (gnu packages php)
   #:use-module (gnu packages python)
+  #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages gnupg)
   #:use-module (gnu packages guile)
   #:use-module (gnu packages logging)
@@ -91,6 +94,8 @@
 
             httpd-service-type
 
+            jupyter-configuration
+
             nginx-configuration
             nginx-configuration?
             nginx-configuration-nginx
@@ -1994,3 +1999,103 @@ root=/srv/gemini
           (service-extension shepherd-root-service-type
                              agate-shepherd-service)))
    (default-value (agate-configuration))))
+
+
+;;;
+;;; Jupyter
+;;;
+
+(define-public default-jupyter-config "#")
+
+(define-record-type* <jupyter-configuration> jupyter-configuration
+                     this-jupyter-configuration jupyter-configuration?
+                     (jupyter-package jupyter-configuration-jupyter-package
+                                      (default jupyter))
+                     (kernels jupyter-configuration-kernels
+                              (default '()))
+                     (root-dir jupyter-configuration-root-dir
+                               (default "/var/lib/jupyter/notebooks"))
+                     (log-file jupyter-configuration-log-file
+                               (default #f))
+                     (shell jupyter-configuration-shell
+                            (default (file-append bash "/bin/bash")))
+                     (server-config jupyter-configuration-server-config
+                                    (default #f))); TODO: Make configuration DSL.
+
+
+(define (search-path-string search-path-pair)
+  (string-append (search-path-specification-variable (car search-path-pair))
+                 "="
+                 (cdr search-path-pair)))
+
+;;;TODO: Add actions to list jupyter servers, change passwords, etc.
+(define (jupyter-shepherd-service config)
+  (list (shepherd-service
+          (provision '(jupyter)) ;TODO: Add magic to allow multiple Jupyter servers
+          (requirement '(loopback))
+          (start #~(make-forkexec-constructor
+                     (list "/run/current-system/profile/bin/jupyter"
+                           "notebook"
+                           (string-append "--config="
+                     #$(or (jupyter-configuration-server-config config)
+                           (plain-file "jupyter_notebook_config.py"
+                                       default-jupyter-config))
+                     #$(jupyter-configuration-root-dir config)))
+                     #:user "jupyter"
+                     #:group "jupyter"
+                     #:environment-variables ; TODO use search-paths magic instead of hard-coding these things.
+                     (list "GI_TYPELIB_PATH=/run/current-system/profile/lib/girepository-1.0"
+                           "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0"
+                           "GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0"
+                           "HOME=/var/lib/jupyter"
+                           "JUPYTER_PATH=/run/current-system/profile/share/jupyter"
+                           "PATH=/run/current-system/profile/bin:/run/current-system/profile/sbin"
+                           "PYTHONPATH=/run/current-system/profile/lib/python3.8/site-packages"
+                           "R_LIBS_SITE=/run/current-system/profile/site-library/"
+                           "TEXMF=/run/current-system/profile/share/texmf-dist"
+                           "TEXMFCNF=/run/current-system/profile/share/texmf-dist/web2c"
+                           "XDG_DATA_DIRS=/run/current-system/profile/share")
+
+                     #:directory #$(jupyter-configuration-root-dir config)
+                     #:log-file #$(jupyter-configuration-log-file config)))
+          (stop #~(make-kill-destructor))
+          (documentation "Runs a Jupyter Notebook server.  A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text."))))
+
+(define (jupyter-account config)
+  (list
+    (user-account
+      (name "jupyter")
+      (group "jupyter")
+      (comment "Jupyter Notebook Server")
+      (home-directory "/var/lib/jupyter")
+      (shell (jupyter-configuration-shell config))
+      (system? #t))
+    (user-group
+      (name "jupyter")
+      (system? #t))))
+
+(define (jupyter-profile config)
+  (cons*
+    (jupyter-configuration-jupyter-package config)
+    (jupyter-configuration-kernels config)))
+
+(define (jupyter-activation config)
+  #~(begin
+      (let ((root-dir #$(jupyter-configuration-root-dir config))
+            (pw (getpwnam "jupyter")))
+        (unless (file-exists? root-dir)
+          (mkdir root-dir)
+          (chown root-dir (passwd:uid pw)
+                 (passwd:gid pw))
+          (chmod root-dir #o700)))))
+
+(define-public jupyter-service-type
+  (service-type
+    (name "jupyter")
+    (extensions (list
+                  (service-extension shepherd-root-service-type jupyter-shepherd-service)
+                  (service-extension account-service-type jupyter-account)
+                  (service-extension activation-service-type jupyter-activation)
+                  (service-extension profile-service-type jupyter-profile)))
+    (description "Runs a Jupyter Notebook server.  A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text.")
+    (default-value (jupyter-configuration))))
-- 
2.33.0
X
X
Xinglu Chen wrote on 28 Sep 2021 21:17
Re: [bug#50708] [PATCH] gnu: web: Add jupyter-service
87bl4c33y4.fsf@yoctocell.xyz
On Wed, Sep 22 2021, Jesse wrote:

Toggle quote (17 lines)
> Attached is yet another updated patch fixing a mistake in the added
> documentation.
>
> On 9/21/21 7:26 PM, Jesse wrote:
>> Attached is an updated patch.
> From d264da0811480a8d2acd5e73f58d320e15bfa9f3 Mon Sep 17 00:00:00 2001
> From: Jesse <gib19014@byui.edu>
> Date: Mon, 20 Sep 2021 16:01:22 -0600
> Subject: [PATCH] gnu: web: Add jupyter-service
>
> * gnu/services/web.scm (gitile-service-type): New variable.
> * doc/guix.texi (Version Control Services): Document it.
> ---
> doc/guix.texi | 52 +++++++++++++++++++
> gnu/services/web.scm | 120 +++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 172 insertions(+)

It would also be good to have a test for the service. You can look at
at (gnu tests ...) modules for some inspiration.

Toggle quote (18 lines)
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 6436e83a7c..0f8aaca413 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -26197,6 +26197,58 @@ The file which should store the logging output of Agate.
> @end table
> @end deftp
>
> +@cindex jupyter
> +The Jupyter Notebook is a web application that allows you to create
> +and share documents that contain live code, equations, visualizations
> +and explanatory text. Uses include: data cleaning and transformation,
> +numerical simulation, statistical modeling, machine learning and much
> +more.
> +
> +@deffn (Scheme Variable) jupyter-service-type
> +This is the type of the agate service, whose value should be an

Agate service? The Jupyter service doesn’t seem to use Agate anywhere.

Toggle quote (12 lines)
> +@code{jupyter-service-type} object, as in this example:
> +
> +@lisp
> +(service jupyter-service-type
> + (jupyter-configuration
> + (log-file "/var/log/jupyter.log")
> + (server-config
> + (kernels
> + (list jupyter-guile-kernel)))))
> +
> +@end lisp

The indentation looks incorrect. Also the trailing newline isn’t
necessary. Something like

Toggle snippet (8 lines)
(service jupyter-service-type
(jupyter-configuration
(log-file "/var/log/jupyter.log")
(server-config
(kernels
(list jupyter-guile-kernel)))))

Toggle quote (7 lines)
> +The example above tells the the Jupyter service to provide the
> +@code{jupyter-guile-kernel} as an optional kernel.
> +@end deffn
> +
> +@deftp {Data Type} jupyter-configuration
> +Data type representing the configuration of Agate.

s/Agate/Jupyter/ ?

Toggle quote (4 lines)
> +@table @asis
> +@item @code{jupyter-package} (default: @code{jupyter})
> +The jupyter package to use.

“jupyter” should be capitalized.

Toggle quote (10 lines)
> +
> +
> +@item @code{kernels} (default: @code{'()}
> +A list of kernel packages to use with the jupyter service.
> +
> +@item @code{root-dir} (default: @file{"/var/lib/jupyter/notebooks"})
> +The directory where the jupyter server looks for jupyter notebooks.
> +
> +@item @code{log-file} (default: @code{#f})
> +The location of the log file. If #f is given, no log is produced.
^
There should be two spaces after a period.

Toggle quote (3 lines)
> +@item @code{shell} (default: @code{(file-append bash "/bin/bash")}
> +The location of the shell used when starting a web terminal (if it is configured to allow terminals).

Line should be kept at <=80 chars.

Toggle quote (3 lines)
> +@item @code{server-config} (default: @code{#f})
> +A file contianing the jupyter server's config file. If @code{#f} is given, an empty file is produced, and the default settings are used.

Likewise (<=80 line length). What exactly does “A file” mean, do you
mean “file-like objects” (see “8.10 G-Expressions” in the manual)?

The example config doesn’t seem to provide anything “A file” either

Toggle snippet (8 lines)
(service jupyter-service-type
(jupyter-configuration
(log-file "/var/log/jupyter.log")
(server-config
(kernels
(list jupyter-guile-kernel)))))

‘kernels’ doesn’t seem to be a procedure or macro, but rather a field in
the ‘jupyter-configuration’ record, so the example needs to be fixed as
well.

Toggle quote (61 lines)
> +
> +@end table
> +@end deftp
> +
> +
> @node Certificate Services
> @subsection Certificate Services
>
> diff --git a/gnu/services/web.scm b/gnu/services/web.scm
> index 6a093cf4e4..2d2946f7a7 100644
> --- a/gnu/services/web.scm
> +++ b/gnu/services/web.scm
> @@ -15,6 +15,7 @@
> ;;; Copyright © 2020 Arun Isaac <arunisaac@systemreboot.net>
> ;;; Copyright © 2020 Oleg Pykhalov <go.wigust@gmail.com>
> ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton <brown121407@posteo.ro>
> +;;; Copyright © 2020, 2021 Jesse Gibbons <jgibbons2357+guix@gmail.com>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -41,11 +42,13 @@
> #:use-module (gnu system shadow)
> #:use-module (gnu packages admin)
> #:use-module (gnu packages base)
> + #:use-module (gnu packages bash)
> #:use-module (gnu packages databases)
> #:use-module (gnu packages web)
> #:use-module (gnu packages patchutils)
> #:use-module (gnu packages php)
> #:use-module (gnu packages python)
> + #:use-module (gnu packages python-xyz)
> #:use-module (gnu packages gnupg)
> #:use-module (gnu packages guile)
> #:use-module (gnu packages logging)
> @@ -91,6 +94,8 @@
>
> httpd-service-type
>
> + jupyter-configuration
> +
> nginx-configuration
> nginx-configuration?
> nginx-configuration-nginx
> @@ -1994,3 +1999,118 @@ root=/srv/gemini
> (service-extension shepherd-root-service-type
> agate-shepherd-service)))
> (default-value (agate-configuration))))
> +
> +
> +;;; Jupyter configuration
> +;;; Jupyter server configuration
> +;;; -> includes a list of kernels to allow in the jupyter server
> +;;; -> includes a list of configuration options specifically for the jupyter server
> +;;; --> These options can be concatenated to a python file
> +;;; --> Nested alist
> +;;; ---> '((NotebookApp (allow-remote-access #t))) -> "c.NotebookApp.allow_remote_access = True"
> +;;; -> Include some settings specifying how to run the daemon
> +;;; --> location of log file
> +;;; --> If a console should be allowed, the location of the shell to be used.
> +;;; --> The package containing the jupyter server itself, default jupyter

What do these arrows mean? Also, keep lines <=80 chars wide.

Toggle quote (2 lines)
> +(define-public default-jupyter-config "#")

Does the default config have to contain a “#”, can’t it just be an empty
string?

Toggle quote (12 lines)
> +
> +(define-record-type* <jupyter-configuration> jupyter-configuration
> + this-jupyter-configuration jupyter-configuration?
> + (jupyter-package jupyter-configuration-jupyter-package
> + (default jupyter))
> + (kernels jupyter-configuration-kernels
> + (default '()))
> + (root-dir jupyter-configuration-root-dir
> + (default "/var/lib/jupyter/notebooks"))
> + (log-file jupyter-configuration-log-file
> + (default #f))

I think it would be good to have a log file by default.

Toggle quote (11 lines)
> + (shell jupyter-configuration-shell
> + (default (file-append bash "/bin/bash")))
> + (server-config jupyter-configuration-server-config
> + (default #f))); TODO: Make configuration DSL.
> +
> +
> +(define (search-path-string search-path-pair)
> + (string-append (search-path-specification-variable (car search-path-pair))
> + "="
> + (cdr search-path-pair)))

Prefer ‘match’ over ‘car’ and ‘cdr’.

Toggle quote (15 lines)
> +
> +;;;TODO: Add actions to list jupyter servers, change passwords, etc.
> +(define (jupyter-shepherd-service config)
> + (list (shepherd-service
> + (provision '(jupyter)) ;TODO: Add magic to allow multiple Jupyter servers
> + (requirement '(loopback))
> + (start #~(make-forkexec-constructor
> + (list "/run/current-system/profile/bin/jupyter"
> + "notebook"
> + (string-append "--config="
> + #$(or (jupyter-configuration-server-config config)
> + (plain-file "jupyter_notebook_config.py"
> + default-jupyter-config))
> + #$(jupyter-configuration-root-dir config)))

Prefer ‘match’ instead of using accessor procedures. Something like

Toggle snippet (5 lines)
(match config
(($ <jupyter-configuration> _ _ root-dir log-file _ server-config)
(list (shepherd-service ...))))

Toggle quote (20 lines)
> + #:user "jupyter"
> + #:group "jupyter"
> + #:environment-variables ; TODO use search-paths magic instead of hard-coding these things.
> + (list "GI_TYPELIB_PATH=/run/current-system/profile/lib/girepository-1.0"
> + "GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/3.0/site-ccache:/run/current-system/profile/share/guile/site/3.0"
> + "GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/3.0"
> + "HOME=/var/lib/jupyter"
> + "JUPYTER_PATH=/run/current-system/profile/share/jupyter"
> + "PATH=/run/current-system/profile/bin:/run/current-system/profile/sbin"
> + "PYTHONPATH=/run/current-system/profile/lib/python3.8/site-packages"
> + "R_LIBS_SITE=/run/current-system/profile/site-library/"
> + "TEXMF=/run/current-system/profile/share/texmf-dist"
> + "TEXMFCNF=/run/current-system/profile//share/texmf-dist/web2c"
> + "XDG_DATA_DIRS=/run/current-system/profile/share")
> +
> + #:directory #$(jupyter-configuration-root-dir config)
> + #:log-file #$(jupyter-configuration-log-file config)))
> + (stop #~(make-kill-destructor))
> + (documentation "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text."))))

Watch the line length! :-)

Toggle quote (19 lines)
> +
> +(define (jupyter-account config)
> + (list
> + (user-account
> + (name "jupyter")
> + (group "jupyter")
> + (comment "Jupyter Notebook Server")
> + (home-directory "/var/lib/jupyter")
> + (shell (jupyter-configuration-shell config))
> + (system? #t))
> + (user-group
> + (name "jupyter")
> + (system? #t))))
> +
> +(define (jupyter-profile config)
> + (cons*
> + (jupyter-configuration-jupyter-package config)
> + (jupyter-configuration-kernels config)))

Prefer ‘match’ instead of using accessor procedures.

Toggle quote (29 lines)
> +
> +(define (jupyter-activation config)
> + #~(begin
> + (let ((root-dir #$(jupyter-configuration-root-dir config))
> + (pw (getpwnam "jupyter")))
> + (unless (file-exists? root-dir)
> + (mkdir root-dir)
> + (chown root-dir (passwd:uid pw)
> + (passwd:gid pw))
> + (chmod root-dir #o700)))))
> +
> +;;; Jupyter service type
> +;;; -> Information vital to settinng up the server, like the port and accepted parameters
> +;;; -> list of kernels considered permissible.
> +;;; -> a shepherd service extension that runs the jupyter server
> +;;; --> shepherd service to list the running servers, set the password, etc.
> +;;; --> Make a log file only readable by root?
> +;;; -> an accounts service extension describing the user that runs the accounts
> +;;; --> group "jupyter"
> +(define-public jupyter-service-type
> + (service-type
> + (name "jupyter")
> + (extensions (list
> + (service-extension shepherd-root-service-type jupyter-shepherd-service)
> + (service-extension account-service-type jupyter-account)
> + (service-extension activation-service-type jupyter-activation)
> + (service-extension profile-service-type jupyter-profile)))
> + (description "Runs a Jupyter Notebook server. A Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations, and explanatory text.")

Keep lines <=80 chars, and use double-spacing.

Toggle quote (3 lines)
> + (default-value (jupyter-configuration))))
> --
> 2.33.0
-----BEGIN PGP SIGNATURE-----

iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmFTakMVHHB1YmxpY0B5
b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5ZeEQALpe1vcjYaUg+tpPsIRIcrgG6AGX
uy44G76f6iodJ61ORImldfIQ7fJWpBRWtFecPOhfuhgFYdoNaGf7poNSF4mRYtgp
0ZPVcQbgOqcxxFy7RASsREtDiSqDdWQwoptEkjryAP3L5hxc7G9rLCVIFKV8WqLj
NHQwSV4JLhik003j2awu1SvGbYHQlANwQt9uC3Yh18vmrBEzucqrB1d7iIyevR8e
PGAz7Z6TYCRf5x3JDYK4CGq4bsgpPHjjyiioAkQ5Pgb8OMIeHkb0zWIt10o1Cv1p
aVinhx3Naf/w1emaad3TL7rG42aN4aPDlLCykLDUwJqlgLJ0hz2BAE4CfhCA2Sl/
uudwNmN14O8slEK8pU/HgzgAXxuYG8iLgx7/zY6RkKxqLmfKLl5N8zx9uE1iImLE
8maGetj9/x8yMfbnlZHMBYT0RQsUEbo/srMGfFNcBQlzoF1lKABIVqkRnBpH+NMt
ogdolWuSqdpLIdKnNGdxoCbk/bXH1YLrX+aHJZz3qLrPbodtm97WR2KnV5/bf04Z
1ns96yMUPX2pcu//3fkU2yrYjBPNL/DOo96jbD88BlmRxIq+rWS1k0PuKD0otjYE
lKjKrSSKWwFoAwyKJkdde7vttHVFFtpqSc3fbeCS1hPoZR5w/RqOSKwaikrabCw2
2LSBikYlfHUKb6Cv
=+0pq
-----END PGP SIGNATURE-----

?