[PATCH 0/1] Add 'eval/container'

DoneSubmitted by Ludovic Courtès.
Details
2 participants
  • Thompson, David
  • Ludovic Courtès
Owner
unassigned
Severity
normal
L
L
Ludovic Courtès wrote on 15 Jul 2019 16:21
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20190715142126.14612-1-ludo@gnu.org
Hello Guix!
This adds ‘eval/container’, which can be used to implement things thatare almost derivation (pure computational processes), but not quite:processes that produce side effects, that need to access the daemon,or that need to talk over the network.
It doesn’t have any users currently. Guix-Jupyter-Kernel will probablyuse it (to spawn proxied kernels in isolated environments), and I thinkRicardo had a use case for it in GWL too.
What do people think?
I wonder if we should target ‘run-in-container’ instead of‘call-with-container’, or maybe both. It’s also a bit troublingthat ‘eval/container’ returns an exit status instead of the evaluationresult, but I think it has to be this way, more or less.
Ludo’.
Ludovic Courtès (1): linux-container: Add 'eval/container'.
gnu/system/linux-container.scm | 49 ++++++++++++++++++++++++++++++++- tests/containers.scm | 50 ++++++++++++++++++++++++++++++++++ 2 files changed, 98 insertions(+), 1 deletion(-)
-- 2.22.0
L
L
Ludovic Courtès wrote on 15 Jul 2019 16:25
[PATCH 1/1] linux-container: Add 'eval/container'.
(address . 36668@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20190715142536.14793-1-ludo@gnu.org
* gnu/system/linux-container.scm (eval/container): New procedure.* tests/containers.scm ("eval/container, exit status")("eval/container, writable user mapping"): New tests.--- gnu/system/linux-container.scm | 49 ++++++++++++++++++++++++++++++++- tests/containers.scm | 50 ++++++++++++++++++++++++++++++++++ 2 files changed, 98 insertions(+), 1 deletion(-)
Toggle diff (133 lines)diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scmindex 61248c62b9..6273cee3d3 100644--- a/gnu/system/linux-container.scm+++ b/gnu/system/linux-container.scm@@ -35,7 +35,8 @@ #:use-module (gnu system file-systems) #:export (system-container containerized-operating-system- container-script))+ container-script+ eval/container)) (define* (container-essential-services os #:key shared-network?) "Return a list of essential services corresponding to OS, a@@ -205,3 +206,49 @@ that will be shared with the host system." %namespaces))))) (gexp->script "run-container" script)))++(define* (eval/container exp+ #:key+ (mappings '())+ (namespaces %namespaces))+ "Evaluate EXP, a gexp, in a new process executing in separate namespaces as+listed in NAMESPACES. Add MAPPINGS, a list of <file-system-mapping>, to the+set of directories visible in the process's mount namespace. Return the+process' exit status as a monadic value.++This is useful to implement processes that, unlike derivations, are not+entirely pure and need to access the outside world or to perform side+effects."+ (mlet %store-monad ((lowered (lower-gexp exp)))+ (define inputs+ (cons (lowered-gexp-guile lowered)+ (lowered-gexp-inputs lowered)))++ (define items+ (append (append-map derivation-input-output-paths inputs)+ (lowered-gexp-sources lowered)))++ (mbegin %store-monad+ (built-derivations inputs)+ (mlet %store-monad ((closure ((store-lift requisites) items)))+ (return (call-with-container (map file-system-mapping->bind-mount+ (append (map (lambda (item)+ (file-system-mapping+ (source item)+ (target source)))+ closure)+ mappings))+ (lambda ()+ (apply execl+ (string-append (derivation-input-output-path+ (lowered-gexp-guile lowered))+ "/bin/guile")+ "guile"+ (append (map (lambda (directory) `("-L" ,directory))+ (lowered-gexp-load-path lowered))+ (map (lambda (directory) `("-C" ,directory))+ (lowered-gexp-load-compiled-path+ lowered))+ (list "-c"+ (object->string+ (lowered-gexp-sexp lowered))))))))))))diff --git a/tests/containers.scm b/tests/containers.scmindex 37408f380d..c6c738f234 100644--- a/tests/containers.scm+++ b/tests/containers.scm@@ -21,7 +21,15 @@ #:use-module (guix utils) #:use-module (guix build syscalls) #:use-module (gnu build linux-container)+ #:use-module ((gnu system linux-container)+ #:select (eval/container)) #:use-module (gnu system file-systems)+ #:use-module (guix store)+ #:use-module (guix monads)+ #:use-module (guix gexp)+ #:use-module (guix derivations)+ #:use-module (guix tests)+ #:use-module (srfi srfi-1) #:use-module (srfi srfi-64) #:use-module (ice-9 match)) @@ -219,4 +227,46 @@ (lambda () (* 6 7)))) +(skip-if-unsupported)+(test-equal "eval/container, exit status"+ 42+ (let* ((store (open-connection-for-tests))+ (status (run-with-store store+ (eval/container #~(exit 42)))))+ (close-connection store)+ (status:exit-val status)))++(skip-if-unsupported)+(test-assert "eval/container, writable user mapping"+ (call-with-temporary-directory+ (lambda (directory)+ (define store+ (open-connection-for-tests))+ (define result+ (string-append directory "/r"))+ (define requisites*+ (store-lift requisites))++ (call-with-output-file result (const #t))+ (run-with-store store+ (mlet %store-monad ((status (eval/container+ #~(begin+ (use-modules (ice-9 ftw))+ (call-with-output-file "/result"+ (lambda (port)+ (write (scandir #$(%store-prefix))+ port))))+ #:mappings+ (list (file-system-mapping+ (source result)+ (target "/result")+ (writable? #t)))))+ (reqs (requisites*+ (list (derivation->output-path+ (%guile-for-build))))))+ (close-connection store)+ (return (and (zero? (pk 'status status))+ (lset= string=? (cons* "." ".." (map basename reqs))+ (pk (call-with-input-file result read))))))))))+ (test-end)-- 2.22.0
T
T
Thompson, David wrote on 15 Jul 2019 17:22
Re: [bug#36668] [PATCH 0/1] Add 'eval/container'
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 36668@debbugs.gnu.org)
CAJ=RwfYmHdmJkpWUyyx1mgw7P77U1OPhjvJ8CVFnHwp8_WgRtw@mail.gmail.com
Hi Ludo,
On Mon, Jul 15, 2019 at 10:22 AM Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (14 lines)>> Hello Guix!>> This adds ‘eval/container’, which can be used to implement things that> are almost derivation (pure computational processes), but not quite:> processes that produce side effects, that need to access the daemon,> or that need to talk over the network.>> It doesn’t have any users currently. Guix-Jupyter-Kernel will probably> use it (to spawn proxied kernels in isolated environments), and I think> Ricardo had a use case for it in GWL too.>> What do people think?
This is great. Love to see 'call-with-container' used for new things.
Toggle quote (3 lines)> I wonder if we should target ‘run-in-container’ instead of> ‘call-with-container’, or maybe both.
I am behind the times. What is special about 'run-in-container'?
Toggle quote (4 lines)> It’s also a bit troubling> that ‘eval/container’ returns an exit status instead of the evaluation> result, but I think it has to be this way, more or less.
I haven't looked at your code, but have you considered supportingreturn values that can be serialized via 'write' and then using 'read'on the host side? (Hmm, I wonder how exceptions could be passed fromcontainer to host.)
Anyway, nice work!
- Dave
L
L
Ludovic Courtès wrote on 15 Jul 2019 17:51
(name . Thompson, David)(address . dthompson2@worcester.edu)(address . 36668@debbugs.gnu.org)
87wogjl2iu.fsf@gnu.org
Hello!
"Thompson, David" <dthompson2@worcester.edu> skribis:
Toggle quote (2 lines)> On Mon, Jul 15, 2019 at 10:22 AM Ludovic Courtès <ludo@gnu.org> wrote:
[...]
Toggle quote (5 lines)>> I wonder if we should target ‘run-in-container’ instead of>> ‘call-with-container’, or maybe both.>> I am behind the times. What is special about 'run-in-container'?
I actually meant ‘run-container’, which is the lower-level procedurethat ‘call-with-container’ invokes: it returns the PID of the processthat has been created.
Toggle quote (9 lines)>> It’s also a bit troubling>> that ‘eval/container’ returns an exit status instead of the evaluation>> result, but I think it has to be this way, more or less.>> I haven't looked at your code, but have you considered supporting> return values that can be serialized via 'write' and then using 'read'> on the host side? (Hmm, I wonder how exceptions could be passed from> container to host.)
I did that in ‘container-excursion*’ a while back, but it’s notgenerally applicable (there needs to be a read syntax for what’s sent),and I think it might be better to build it on top of a more primitiveprocedure like this ‘eval/container’.
Whether we need something like this will depend on use cases I guess…
Thanks for your feedback!
Ludo’.
L
L
Ludovic Courtès wrote on 19 Jul 2019 11:55
(name . Thompson, David)(address . dthompson2@worcester.edu)(address . 36668-done@debbugs.gnu.org)
87d0i6e4by.fsf@gnu.org
Pushed! Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send email to 36668@debbugs.gnu.org