OCI provisioning service

  • Open
  • quality assurance status badge
Details
One participant
  • paul
Owner
unassigned
Submitted by
paul
Severity
normal
P
P
paul wrote 5 days ago
(address . guix-patches@gnu.org)
2f43e635-508c-407a-8309-06e75d492d89@autistici.org
Hi Guix,

for a while now, we have been able to run Docker/OCI images as Shepherd
services with the oci-container-service-type. This was useful especially
to run software that is not packaged yet (or sometimes not packageable
at all, due to JS and other bootstrapping issues). It allows to declare
a list of oci-container-configuration records which would map to Docker
backed Shepherd services:

(service oci-container-service-type
(list
(oci-container-configuration
(image "prom/prometheus")
(network "host")
(ports
'(("9000" . "9000")
("9090" . "9090"))))
(oci-container-configuration
(image "grafana/grafana:10.0.1")
(network "host")
(volumes
'("/var/lib/grafana:/var/lib/grafana")))))

This allows us to have containerized but apparently native services
running on the Guix System. The downside is that unless some isolation
mechanism are disabled, or unless the running services are very simple
and don't have to interact with the world outside them, we lose some of
the nice virtualization features of containers. Above all is the need to
have all containers connected to the host network to be able to interact
with other containers. This is effectively the default behavior of most
if not all Guix native services but with the downside that usually OCI
images lack all the provenance information that is typical of Guix
packages, hence are less trustable than a native Guix package so in some
cases users do prefer to have them running in an isolated environment.

Another shortcoming of the oci-container-service-type is that it only
supports Docker as an OCI runtime which must have a running daemon with
full root privileges to be able to execute containers. Since some weeks
now, with the help of subids/subgids and unprivileged namespaces, we are
able to run completely rootless containers with the
rootless-podman-service-type.

This patch implements a generalization of the
oci-container-service-type, which consequently is made deprecated.  The
oci-service-type, in addition to all the features from the
oci-container-service-type, can now provision OCI networks and volumes:

(service iptables-service-type)
(service rootless-podman-service-type)
(service oci-service-type
(oci-configuration
(runtime 'podman)))
(simple-service 'oci-provisioning
oci-service-type
(oci-extension
(volumes
(list
(oci-network-configuration (name "grafana"))))
(networks
(list
(oci-network-configuration (name "monitoring"))))
(containers
(list
(oci-container-configuration
(image "docker.io/grafana/grafana:10.1.5")
(network "host")
(volumes
`(,( . "/opt/bitnami/grafana/conf/grafana.ini")
("grafana" . "/var/lib/grafana"))))))))

Please mind that this is only an example and it probably won't work on
your system without some more thought. The oci-service-type currently
only handles OCI objects creation, the user is supposed to handle state
once the objects are provsioned. It currently supports two different OCI
runtimes: Docker and rootless Podman.  Both runtimes are tested to make
sure provisioned containers can connect to each other through
provisioned networks and can read/write data with provisioned volumes.
Compared to the oci-container-service-type I added some utility Shepherd
actions that are supposed to help with debugging. The above
configuration would yield the following

$ sudo herd command-line podman-networks
/run/current-system/profile/bin/podman network create monitoring

$ sudo herd command-line podman-volumes
/run/current-system/profile/bin/podman volume create grafana

$ sudo herd doc podman-grafana list-actions

command-line:
Prints podman-grafana's OCI runtime command line invokation.

pull:
Pull podman-grafana's image (docker.io/grafana/grafana:10.1.5).

$ sudo herd command-line podman-grafana
/run/current-system/profile/bin/podman run --rm --name podman-grafana --network host -v grafana:/var/lib/grafana -v /gnu/store/yqfvhvf8j4008ykr52zh2dmc1d2mjxih-grafana.ini:/opt/bitnami/grafana/conf/grafana.ini docker.io/bitnami/grafana:10.1.5


At last the Scheme API is thought to facilitate the implementation of a
Guix Home service in the future (you can find an untested version of
that at [0]). I tested my changes with:

guix shell -D guix -CPW -- make check-system TESTS="oci-service-docker oci-container oci-service-rootless-podman docker docker-system rootless-podman"


Please let me know your thoughts about this!

Thank you for all your work,

giacomo


[0]:
Attachment: file
G
G
Giacomo Leidi wrote 5 days ago
[PATCH 1/4] services: rootless-podman: Use login shell.
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
89778533f32ad1388f03414e884fff10f74ef379.1738792951.git.goodoldpaul@autistici.org
This commit allows for having PATH set when changing the owner of
/sys/fs/group.

* gnu/services/containers.scm (crgroups-fs-owner): Use login shell.

Change-Id: I9510c637a5332325e05ca5ebc9dfd4de32685c50
---
gnu/services/containers.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (17 lines)
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index 19d35ccbcb6..dc66ac4f967 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -140,7 +140,7 @@ (define (cgroups-fs-owner-entrypoint config)
(rootless-podman-configuration-group-name config))
(program-file "cgroups2-fs-owner-entrypoint"
#~(system*
- (string-append #+bash-minimal "/bin/bash") "-c"
+ (string-append #+bash-minimal "/bin/bash") "-l" "-c"
(string-append "echo Setting /sys/fs/cgroup "
"group ownership to " #$group " && chown -v "
"root:" #$group " /sys/fs/cgroup && "

base-commit: 5a897c5c95a81278b044c18d962d3bd83131ba06
--
2.48.1
G
G
Giacomo Leidi wrote 5 days ago
[PATCH 4/4] tests: Use oci-image in container tests.
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
d6b02a87a3827bec004de6a253c6f160a9f33578.1738792951.git.goodoldpaul@autistici.org
This patch replaces boilerplate in container related tests with
oci-image plumbing from (gnu services containers).

* gnu/services/containers.scm: Export lower-oci-image.
* gnu/tests/containers.scm (%oci-tarball): New variable;
(run-rootless-podman-test): use %oci-tarball;
(build-tarball&run-rootless-podman-test): drop procedure.
* gnu/tests/docker.scm (%docker-tarball): New variable;
(build-tarball&run-docker-test): use %docker-tarball;
(%docker-system-tarball): New variable;
(build-tarball&run-docker-system-test): new procedure.

Change-Id: Iad6f0704aee188d89464c83722dea0bb7adb084a
---
gnu/services/containers.scm | 2 +
gnu/tests/containers.scm | 80 ++++++++++++++----------------
gnu/tests/docker.scm | 98 ++++++++++++++++++++-----------------
3 files changed, 93 insertions(+), 87 deletions(-)

Toggle diff (272 lines)
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index c45f79c4ed1..e15dbc6a21c 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -74,6 +74,8 @@ (define-module (gnu services containers)
oci-image-system
oci-image-grafts?
+ lower-oci-image
+
oci-container-configuration
oci-container-configuration?
oci-container-configuration-fields
diff --git a/gnu/tests/containers.scm b/gnu/tests/containers.scm
index 719647c298e..024000bca14 100644
--- a/gnu/tests/containers.scm
+++ b/gnu/tests/containers.scm
@@ -69,13 +69,47 @@ (define %rootless-podman-os
(supplementary-groups '("wheel" "netdev" "cgroup"
"audio" "video")))))))
-(define (run-rootless-podman-test oci-tarball)
+(define %oci-tarball
+ (lower-oci-image
+ "guile-guest"
+ (oci-image
+ (repository "guile-guest")
+ (value
+ (packages->manifest
+ (list
+ guile-3.0 guile-json-3
+ (package
+ (name "guest-script")
+ (version "0")
+ (source #f)
+ (build-system trivial-build-system)
+ (arguments `(#:guile ,guile-3.0
+ #:builder
+ (let ((out (assoc-ref %outputs "out")))
+ (mkdir out)
+ (call-with-output-file (string-append out "/a.scm")
+ (lambda (port)
+ (display "(display \"hello world\n\")" port)))
+ #t)))
+ (synopsis "Display hello world using Guile")
+ (description "This package displays the text \"hello world\" on the
+standard output device and then enters a new line.")
+ (home-page #f)
+ (license license:public-domain)))))
+ (pack-options
+ '(#:entry-point "bin/guile"
+ #:localstatedir? #t
+ #:extra-options (#:image-tag "guile-guest")
+ #:symlinks (("/bin/Guile" -> "bin/guile")
+ ("aa.scm" -> "a.scm")))))))
+
+(define (run-rootless-podman-test)
(define os
(marionette-operating-system
(operating-system-with-gc-roots
%rootless-podman-os
- (list oci-tarball))
+ (list %oci-tarball))
#:imported-modules '((gnu services herd)
(guix combinators))))
@@ -254,7 +288,7 @@ (define (run-rootless-podman-test oci-tarball)
(let* ((loaded (slurp ,(string-append #$podman
"/bin/podman")
"load" "-i"
- ,#$oci-tarball))
+ ,#$%oci-tarball))
(repository&tag "localhost/guile-guest:latest")
(response1 (slurp
,(string-append #$podman "/bin/podman")
@@ -307,49 +341,11 @@ (define (run-rootless-podman-test oci-tarball)
(gexp->derivation "rootless-podman-test" test))
-(define (build-tarball&run-rootless-podman-test)
- (mlet* %store-monad
- ((_ (set-grafting #f))
- (guile (set-guile-for-build (default-guile)))
- (guest-script-package ->
- (package
- (name "guest-script")
- (version "0")
- (source #f)
- (build-system trivial-build-system)
- (arguments `(#:guile ,guile-3.0
- #:builder
- (let ((out (assoc-ref %outputs "out")))
- (mkdir out)
- (call-with-output-file (string-append out "/a.scm")
- (lambda (port)
- (display "(display \"hello world\n\")" port)))
- #t)))
- (synopsis "Display hello world using Guile")
- (description "This package displays the text \"hello world\" on the
-standard output device and then enters a new line.")
- (home-page #f)
- (license license:public-domain)))
- (profile (profile-derivation (packages->manifest
- (list guile-3.0 guile-json-3
- guest-script-package))
- #:hooks '()
- #:locales? #f))
- (tarball (pack:docker-image
- "docker-pack" profile
- #:symlinks '(("/bin/Guile" -> "bin/guile")
- ("aa.scm" -> "a.scm"))
- #:extra-options
- '(#:image-tag "guile-guest")
- #:entry-point "bin/guile"
- #:localstatedir? #t)))
- (run-rootless-podman-test tarball)))
-
(define %test-rootless-podman
(system-test
(name "rootless-podman")
(description "Test rootless Podman service.")
- (value (build-tarball&run-rootless-podman-test))))
+ (value (run-rootless-podman-test))))
(define %guile-oci-image
diff --git a/gnu/tests/docker.scm b/gnu/tests/docker.scm
index 5dcf05a17e3..d969b28a68f 100644
--- a/gnu/tests/docker.scm
+++ b/gnu/tests/docker.scm
@@ -26,6 +26,7 @@ (define-module (gnu tests docker)
#:use-module (gnu system image)
#:use-module (gnu system vm)
#:use-module (gnu services)
+ #:use-module (gnu services containers)
#:use-module (gnu services dbus)
#:use-module (gnu services networking)
#:use-module (gnu services docker)
@@ -57,6 +58,40 @@ (define %docker-os
(service containerd-service-type)
(service docker-service-type)))
+(define %docker-tarball
+ (lower-oci-image
+ "guile-guest"
+ (oci-image
+ (repository "guile-guest")
+ (value
+ (packages->manifest
+ (list
+ guile-3.0 guile-json-3
+ (package
+ (name "guest-script")
+ (version "0")
+ (source #f)
+ (build-system trivial-build-system)
+ (arguments `(#:guile ,guile-3.0
+ #:builder
+ (let ((out (assoc-ref %outputs "out")))
+ (mkdir out)
+ (call-with-output-file (string-append out "/a.scm")
+ (lambda (port)
+ (display "(display \"hello world\n\")" port)))
+ #t)))
+ (synopsis "Display hello world using Guile")
+ (description "This package displays the text \"hello world\" on the
+standard output device and then enters a new line.")
+ (home-page #f)
+ (license license:public-domain)))))
+ (pack-options
+ '(#:entry-point "bin/guile"
+ #:localstatedir? #t
+ #:extra-options (#:image-tag "guile-guest")
+ #:symlinks (("/bin/Guile" -> "bin/guile")
+ ("aa.scm" -> "a.scm")))))))
+
(define (run-docker-test docker-tarball)
"Load DOCKER-TARBALL as Docker image and run it in a Docker container,
inside %DOCKER-OS."
@@ -173,40 +208,7 @@ (define (run-docker-test docker-tarball)
(gexp->derivation "docker-test" test))
(define (build-tarball&run-docker-test)
- (mlet* %store-monad
- ((_ (set-grafting #f))
- (guile (set-guile-for-build (default-guile)))
- (guest-script-package ->
- (package
- (name "guest-script")
- (version "0")
- (source #f)
- (build-system trivial-build-system)
- (arguments `(#:guile ,guile-3.0
- #:builder
- (let ((out (assoc-ref %outputs "out")))
- (mkdir out)
- (call-with-output-file (string-append out "/a.scm")
- (lambda (port)
- (display "(display \"hello world\n\")" port)))
- #t)))
- (synopsis "Display hello world using Guile")
- (description "This package displays the text \"hello world\" on the
-standard output device and then enters a new line.")
- (home-page #f)
- (license license:public-domain)))
- (profile (profile-derivation (packages->manifest
- (list guile-3.0 guile-json-3
- guest-script-package))
- #:hooks '()
- #:locales? #f))
- (tarball (pack:docker-image
- "docker-pack" profile
- #:symlinks '(("/bin/Guile" -> "bin/guile")
- ("aa.scm" -> "a.scm"))
- #:entry-point "bin/guile"
- #:localstatedir? #t)))
- (run-docker-test tarball)))
+ (run-docker-test %docker-tarball))
(define %test-docker
(system-test
@@ -215,8 +217,20 @@ (define %test-docker
(value (build-tarball&run-docker-test))))
+(define %docker-system-tarball
+ (lower-oci-image
+ "guix-system-guest"
+ (oci-image
+ (repository "guix-system-guest")
+ (value
+ (operating-system
+ (inherit (simple-operating-system))
+ ;; Use locales for a single libc to
+ ;; reduce space requirements.
+ (locale-libcs (list glibc)))))))
+
(define (run-docker-system-test tarball)
- "Load DOCKER-TARBALL as Docker image and run it in a Docker container,
+ "Load TARBALL as Docker image and run it in a Docker container,
inside %DOCKER-OS."
(define os
(marionette-operating-system
@@ -333,21 +347,15 @@ (define (run-docker-system-test tarball)
(gexp->derivation "docker-system-test" test))
+(define (build-tarball&run-docker-system-test)
+ (run-docker-system-test %docker-system-tarball))
+
(define %test-docker-system
(system-test
(name "docker-system")
(description "Run a system image as produced by @command{guix system
docker-image} inside Docker.")
- (value (with-monad %store-monad
- (>>= (lower-object
- (system-image (os->image
- (operating-system
- (inherit (simple-operating-system))
- ;; Use locales for a single libc to
- ;; reduce space requirements.
- (locale-libcs (list glibc)))
- #:type docker-image-type)))
- run-docker-system-test)))))
+ (value (build-tarball&run-docker-system-test))))
(define %oci-os
--
2.48.1
G
G
Giacomo Leidi wrote 5 days ago
[PATCH 3/4] services: Add oci-service-type.
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
c3c8730fa100fd8b577c4aa9ed642254421920da.1738792951.git.goodoldpaul@autistici.org
This patch implements a generalization of the
oci-container-service-type, which consequently is made deprecated. The
oci-service-type, in addition to all the features from the
oci-container-service-type, can now provision OCI networks and volumes.
It only handles OCI objects creation, the user is supposed to handle
state once the objects are provsioned.

It currently supports two different OCI runtimes: Docker and rootless
Podman. Both runtimes are tested to make sure provisioned containers
can connect to each other through provisioned networks and can
read/write data with provisioned volumes.

At last the Scheme API is thought to facilitate the implementation of a
Guix Home service in the future.

* gnu/services/containers.scm (%oci-supported-runtimes): New variable;
(oci-runtime-cli): new variable;
(oci-runtime-name): new variable;
(oci-network-configuration): new variable;
(oci-volume-configuration): new variable;
(oci-configuration): new variable;
(oci-extension): new variable;
(oci-networks-shepherd-name): new variable;
(oci-service-type): new variable;
(oci-state->shepherd-services): new variable.
* doc/guix.texi: Document it.
* gnu/tests/containers.scm: Test it.
* gnu/services/docker.scm: Deprecate the oci-container-service-type.

Change-Id: I656b3db85832e42d53072fcbfb91d1226f39ef38
---
doc/guix.texi | 295 +++++++---
gnu/services/containers.scm | 1038 +++++++++++++++++++++++++++++++----
gnu/services/docker.scm | 37 +-
gnu/tests/containers.scm | 956 +++++++++++++++++++++++++++++++-
4 files changed, 2130 insertions(+), 196 deletions(-)

Toggle diff (460 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index bb5f29277fb..ff3e77a1d00 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41778,59 +41778,155 @@ Miscellaneous Services
@cindex OCI-backed, Shepherd services
@subsubheading OCI backed services
-Should you wish to manage your Docker containers with the same consistent
-interface you use for your other Shepherd services,
-@var{oci-container-service-type} is the tool to use: given an
-@acronym{Open Container Initiative, OCI} container image, it will run it in a
+Should you wish to manage your @acronym{Open Container Initiative, OCI} containers
+with the same consistent interface you use for your other Shepherd services,
+@var{oci-service-type} is the tool to use: given an
+OCI container image, it will run it in a
Shepherd service. One example where this is useful: it lets you run services
-that are available as Docker/OCI images but not yet packaged for Guix.
+that are available as OCI images but not yet packaged for Guix.
-@defvar oci-container-service-type
+@defvar oci-service-type
-This is a thin wrapper around Docker's CLI that executes OCI images backed
+This is a thin wrapper around Docker's or Podman's CLI that executes OCI images backed
processes as Shepherd Services.
@lisp
-(service oci-container-service-type
- (list
- (oci-container-configuration
- (network "host")
- (image
- (oci-image
- (repository "guile")
- (tag "3")
- (value (specifications->manifest '("guile")))
- (pack-options '(#:symlinks (("/bin/guile" -> "bin/guile"))
- #:max-layers 2))))
- (entrypoint "/bin/guile")
- (command
- '("-c" "(display \"hello!\n\")")))
- (oci-container-configuration
- (image "prom/prometheus")
- (ports
- '(("9000" . "9000")
- ("9090" . "9090"))))
- (oci-container-configuration
- (image "grafana/grafana:10.0.1")
- (network "host")
- (volumes
- '("/var/lib/grafana:/var/lib/grafana")))))
+(simple-service 'oci-provisioning
+ oci-service-type
+ (oci-extension
+ (networks
+ (list
+ (oci-network-configuration (name "monitoring"))))
+ (containers
+ (list
+ (oci-container-configuration
+ (network "monitoring")
+ (image
+ (oci-image
+ (repository "guile")
+ (tag "3")
+ (value (specifications->manifest '("guile")))
+ (pack-options '(#:symlinks (("/bin/guile" -> "bin/guile"))
+ #:max-layers 2))))
+ (entrypoint "/bin/guile")
+ (command
+ '("-c" "(display \"hello!\n\")")))
+ (oci-container-configuration
+ (image "prom/prometheus")
+ (network "host")
+ (ports
+ '(("9000" . "9000")
+ ("9090" . "9090"))))
+ (oci-container-configuration
+ (image "grafana/grafana:10.0.1")
+ (network "host")
+ (volumes
+ '("/var/lib/grafana:/var/lib/grafana")))))))
@end lisp
In this example three different Shepherd services are going to be added to the
system. Each @code{oci-container-configuration} record translates to a
-@code{docker run} invocation and its fields directly map to options. You can
-refer to the
-@url{https://docs.docker.com/engine/reference/commandline/run,upstream}
-documentation for the semantics of each value. If the images are not found,
-they will be
-@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The
+@command{docker run} or @command{podman run} invocation and its fields directly
+map to options. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html,Podman}
+upstream documentation for semantics of each value. If the images are not found,
+they will be pulled. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/pull/,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-pull.1.html,Podman}
+upstream documentation for semantics. The
services with @code{(network "host")} are going to be attached to the
host network and are supposed to behave like native processes with regard to
networking.
@end defvar
+@c %start of fragment
+
+@deftp {Data Type} oci-configuration
+Available @code{oci-configuration} fields are:
+
+@table @asis
+@item @code{runtime} (default: @code{'docker}) (type: symbol)
+The OCI runtime to use to run commands. It can be either @code{'docker} or
+@code{'podman}.
+
+@item @code{runtime-cli} (type: maybe-package)
+The OCI runtime command line to be installed in the system profile and used
+to provision OCI resources. When unset it will default to @code{docker-cli}
+package for the @code{'docker} runtime or to @code{podman} package for the
+@code{'podman} runtime.
+
+@item @code{user} (default: @code{"oci-container"}) (type: string)
+The user name under whose authority OCI commands will be run.
+
+@item @code{group} (default: @code{"docker"}) (type: string)
+The group name under whose authority OCI commands will be run. When
+using the @code{'podman} OCI runtime, this field will be ignored and the
+default group of the user configured in the @code{user} field will be used.
+
+@item @code{subuids-range} (type: maybe-subid-range)
+An optional @code{subid-range} record allocating subuids for the user from
+the @code{user} field. When unset, with the rootless Podman OCI runtime, it
+defaults to @code{(subid-range (name "oci-container"))}.
+
+@item @code{subgids-range} (type: maybe-subid-range)
+An optional @code{subid-range} record allocating subgids for the user from
+the @code{user} field. When unset, with the rootless Podman OCI runtime, it
+defaults to @code{(subid-range (name "oci-container"))}.
+
+@item @code{containers} (default: @code{'()}) (type: list-of-oci-containers)
+The list of @code{oci-container-configuration} records representing the
+containers to provision. Most users are supposed not to use this field and use
+the @code{oci-extension} record instead.
+
+@item @code{networks} (default: @code{'()}) (type: list-of-oci-networks)
+The list of @code{oci-network-configuration} records representing the
+containers to provision. Most users are supposed not to use this field and use
+the @code{oci-extension} record instead.
+
+@item @code{volumes} (default: @code{'()}) (type: list-of-oci-volumes)
+The list of @code{oci-volumes-configuration} records representing the
+containers to provision. Most users are supposed not to use this field and use
+the @code{oci-extension} record instead.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+When true, additional output will be printed, allowing to better follow the
+flow of execution.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} oci-extension
+Available @code{oci-extension} fields are:
+
+@table @asis
+@item @code{containers} (default: @code{'()}) (type: list-of-oci-containers)
+The list of @code{oci-container-configuration} records representing the
+containers to provision.
+
+@item @code{networks} (default: @code{'()}) (type: list-of-oci-networks)
+The list of @code{oci-network-configuration} records representing the
+containers to provision.
+
+@item @code{volumes} (default: @code{'()}) (type: list-of-oci-volumes)
+The list of @code{oci-volumes-configuration} records representing the
+containers to provision.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+
@c %start of fragment
@deftp {Data Type} oci-container-configuration
@@ -41850,16 +41946,16 @@ Miscellaneous Services
Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
@item @code{host-environment} (default: @code{'()}) (type: list)
-Set environment variables in the host environment where @command{docker
-run} is invoked. This is especially useful to pass secrets from the
-host to the container without having them on the @command{docker run}'s
-command line: by setting the @code{MYSQL_PASSWORD} on the host and by passing
+Set environment variables in the host environment where @command{docker run}
+or @command{podman run} are invoked. This is especially useful to pass secrets
+from the host to the container without having them on the OCI runtime command line,
+for example: by setting the @code{MYSQL_PASSWORD} on the host and by passing
@code{--env MYSQL_PASSWORD} through the @code{extra-arguments} field, it is
possible to securely set values in the container environment. This field's
value can be a list of pairs or strings, even mixed:
@lisp
-(list '("LANGUAGE\" . "eo:ca:eu")
+(list '("LANGUAGE" . "eo:ca:eu")
"JAVA_HOME=/opt/java")
@end lisp
@@ -41867,22 +41963,24 @@ Miscellaneous Services
directly to @code{make-forkexec-constructor}.
@item @code{environment} (default: @code{'()}) (type: list)
-Set environment variables. This can be a list of pairs or strings, even mixed:
+Set environment variables inside the container. This can be a list of pairs
+or strings, even mixed:
@lisp
(list '("LANGUAGE" . "eo:ca:eu")
"JAVA_HOME=/opt/java")
@end lisp
-Pair members can be strings, gexps or file-like objects.
-Strings are passed directly to the Docker CLI. You can refer to the
-@uref{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
-documentation for semantics.
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the OCI runtime CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#env-e-env,Podman}
+upstream documentation for semantics.
@item @code{image} (type: string-or-oci-image)
The image used to build the container. It can be a string or an
-@code{oci-image} record. Strings are resolved by the Docker Engine, and
-follow the usual format
+@code{oci-image} record. Strings are resolved by the OCI runtime,
+and follow the usual format
@code{myregistry.local:5000/testing/test-image:tag}.
@item @code{provision} (default: @code{""}) (type: string)
@@ -41910,7 +42008,7 @@ Miscellaneous Services
by the service.
@item @code{network} (default: @code{""}) (type: string)
-Set a Docker network for the spawned container.
+Set an OCI network for the spawned container.
@item @code{ports} (default: @code{'()}) (type: list)
Set the port or port ranges to expose from the spawned container. This can be a
@@ -41921,10 +42019,11 @@ Miscellaneous Services
"10443:443")
@end lisp
-Pair members can be strings, gexps or file-like objects.
-Strings are passed directly to the Docker CLI. You can refer to the
-@uref{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
-documentation for semantics.
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the OCI runtime CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#publish-p-ip-hostport-containerport-protocol,Podman}
+upstream documentation for semantics.
@item @code{volumes} (default: @code{'()}) (type: list)
Set volume mappings for the spawned container. This can be a
@@ -41935,25 +42034,95 @@ Miscellaneous Services
"/gnu/store:/gnu/store")
@end lisp
-Pair members can be strings, gexps or file-like objects.
-Strings are passed directly to the Docker CLI. You can refer to the
-@uref{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
-documentation for semantics.
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the OCI runtime CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#volume-v-source-volume-host-dir-container-dir-options,Podman}
+upstream documentation for semantics.
@item @code{container-user} (default: @code{""}) (type: string)
Set the current user inside the spawned container. You can refer to the
-@url{https://docs.docker.com/engine/reference/run/#user,upstream}
-documentation for semantics.
+@url{https://docs.docker.com/engine/reference/run/#user,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#user-u-user-group,Podman}
+upstream documentation for semantics.
@item @code{workdir} (default: @code{""}) (type: string)
Set the current working directory for the spawned Shepherd service.
You can refer to the
-@url{https://docs.docker.com/engine/reference/run/#workdir,upstream}
-documentation for semantics.
+@url{https://docs.docker.com/engine/reference/run/#workdir,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#workdir-w-dir,Podman}
+upstream documentation for semantics.
+
+@item @code{extra-arguments} (default: @code{'()}) (type: list)
+A list of strings, gexps or file-like objects that will be directly passed
+to the @command{docker run} or @command{podman run} invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} oci-network-configuration
+Available @code{oci-network-configuration} fields are:
+
+@table @asis
+@item @code{name} (type: string)
+The name of the OCI network to provision.
+
+@item @code{driver} (type: maybe-string)
+The driver to manage the network.
+
+@item @code{gateway} (type: maybe-string)
+IPv4 or IPv6 gateway for the subnet.
+
+@item @code{internal?} (default: @code{#f}) (type: boolean)
+Restrict external access to the network
+
+@item @code{ip-range} (type: maybe-string)
+Allocate container ip from a sub-range in CIDR format.
+
+@item @code{ipam-driver} (type: maybe-string)
+IP Address Management Driver.
+
+@item @code{ipv6?} (default: @code{#f}) (type: boolean)
+Enable IPv6 networking.
+
+@item @code{subnet} (type: maybe-string)
+Subnet in CIDR format that represents a network segment.
+
+@item @code{labels} (default: @code{'()}) (type: list)
+The list of labels that will be used to tag the current volume.
+
+@item @code{extra-arguments} (default: @code{'()}) (type: list)
+A list of strings, gexps or file-like objects that will be directly
+passed to the runtime invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} oci-volume-configuration
+Available @code{oci-volume-configuration} fields are:
+
+@table @asis
+@item @code{name} (type: string)
+The name of the OCI volume to provision.
+
+@item @code{labels} (default: @code{'()}) (type: list)
+The list of labels that will be used to tag the current volume.
@item @code{extra-arguments} (default: @code{'()}) (type: list)
A list of strings, gexps or file-like objects that will be directly
-passed to the @command{docker run} invocation.
+passed to the runtime invokation.
@end table
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index 50bf6c05549..c45f79c4ed1 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -41,6 +41,7 @@ (define-module (gnu services containers)
#:use-module ((guix scripts pack) #:prefix pack:)
#:use-module (guix store)
#:use-module (srfi srfi-1)
+ #:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (rootless-podman-configuration
rootless-podman-configuration?
@@ -96,8 +97,68 @@ (define-module (gnu services containers)
oci-container-configuration-workdir
oci-container-configuration-extra-arguments
+ list-of-oci-containers?
+ list-of-oci-networks?
+ list-of-oci-volumes?
+
+ %oci-supported-runtimes
+ oci-sanitize-runtime
+ oci-runtime-system-requirement
+ oci-runtime-cli
+ oci-runtime-system-cli
+ oci-runtime-name
+ oci-runtime-group
+
+ oci-network-configuration
+ oci-network-configuration?
+ oci-network-configuration-fields
+ oci-network-configuration-name
+ oci-network-configuration-driver
+ oci-network-configuration-gateway
+ oci-network-configuration-internal?
+ oci-network-configuration-ip-range
+ oci-network-configuration-ipam-driver
+ oci-network-configuration-ipv6?
+ oci-network-configuration-subnet
+ oci-network-configuration-labels
+ oci-network-configuration-extra-arguments
+
+ oci-volume-configuration
+ oci-volume-configuration?
+ oci-volume-configuration-fields
+ oci-volume-configuration-name
+ oci-volume-configuration-labels
+ oci-volume-configuration-extra-arguments
+
+ oci-configuration
+ oci-configuration?
+ oci-configuration-fields
+ oci-configuration-runtime
+ oci-configuration-runtime-cli
+ oci-configuration-user
+ oci-configuration-group
+ oci-configuration-containers
+ oci-configuration-networks
+ oci-configuration-volumes
+ oci-configuration-verbose?
+
+ oci-extension
+ oci-extension?
+ oci-extension-fields
+ oci-extension-containers
+ oci-extension-networks
+ oci-extension-volumes
+
+ oci-networks-shepherd-name
+ oci-volumes-shepherd-name
+
oci-container-shepherd-service
- %oci-container-accounts))
+ oci-service-type
+ oci-service-accounts
+ oci-service-profile
+ oci-service-subids
+ oci-state->shepherd-services
+ oci-configuration->shepherd-servi
This message was truncated. Download the full message here.
G
G
Giacomo Leidi wrote 5 days ago
[PATCH 2/4] services: oci-container-configuration: Move to (gnu services containers).
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
92f19ba6c0842885735dfce653eef535360085f4.1738792951.git.goodoldpaul@autistici.org
This patch moves the oci-container-configuration and related
configuration records to (gnu services containers).
Public symbols are still exported for backwards
compatibility but since the oci-container-service-type will be
deprecated in favor of the more general oci-service-type, everything is
moved outside of the docker related module.

* gnu/services/docker.scm: Move everything related to oci-container-configuration
to...
* gnu/services/containers.scm: ...here.scm.
* gnu/tests/docker.scm: Simplify %test-oci-container test case.

Change-Id: Iae599dd5cc7442eb632f0c1b3b12f6b928397ae7
---
gnu/services/containers.scm | 549 +++++++++++++++++++++++++++++++++-
gnu/services/docker.scm | 577 +++---------------------------------
gnu/tests/docker.scm | 99 +++----
3 files changed, 625 insertions(+), 600 deletions(-)

Toggle diff (534 lines)
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index dc66ac4f967..50bf6c05549 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;; Copyright © 2024, 2025 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -17,19 +17,31 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (gnu services containers)
+ #:use-module (gnu image)
+ #:use-module (gnu packages admin)
#:use-module (gnu packages bash)
#:use-module (gnu packages containers)
+ #:use-module (gnu packages docker)
#:use-module (gnu packages file-systems)
#:use-module (gnu services)
#:use-module (gnu services base)
#:use-module (gnu services configuration)
#:use-module (gnu services shepherd)
+ #:use-module (gnu system)
#:use-module (gnu system accounts)
+ #:use-module (gnu system image)
#:use-module (gnu system shadow)
#:use-module (gnu system pam)
+ #:use-module (guix diagnostics)
#:use-module (guix gexp)
+ #:use-module (guix i18n)
+ #:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix profiles)
+ #:use-module ((guix scripts pack) #:prefix pack:)
+ #:use-module (guix store)
#:use-module (srfi srfi-1)
+ #:use-module (ice-9 match)
#:export (rootless-podman-configuration
rootless-podman-configuration?
rootless-podman-configuration-fields
@@ -48,7 +60,44 @@ (define-module (gnu services containers)
rootless-podman-shepherd-services
rootless-podman-service-etc
- rootless-podman-service-type))
+ rootless-podman-service-type
+
+ oci-image
+ oci-image?
+ oci-image-fields
+ oci-image-repository
+ oci-image-tag
+ oci-image-value
+ oci-image-pack-options
+ oci-image-target
+ oci-image-system
+ oci-image-grafts?
+
+ oci-container-configuration
+ oci-container-configuration?
+ oci-container-configuration-fields
+ oci-container-configuration-user
+ oci-container-configuration-group
+ oci-container-configuration-command
+ oci-container-configuration-entrypoint
+ oci-container-configuration-host-environment
+ oci-container-configuration-environment
+ oci-container-configuration-image
+ oci-container-configuration-provision
+ oci-container-configuration-requirement
+ oci-container-configuration-log-file
+ oci-container-configuration-auto-start?
+ oci-container-configuration-respawn?
+ oci-container-configuration-shepherd-actions
+ oci-container-configuration-network
+ oci-container-configuration-ports
+ oci-container-configuration-volumes
+ oci-container-configuration-container-user
+ oci-container-configuration-workdir
+ oci-container-configuration-extra-arguments
+
+ oci-container-shepherd-service
+ %oci-container-accounts))
(define (gexp-or-string? value)
(or (gexp? value)
@@ -188,7 +237,7 @@ (define (rootless-podman-cgroups-limits-service config)
rootless-podman-shared-root-fs))
(one-shot? #t)
(documentation
- "Allow setting cgroups limits: cpu, cpuset, memory and
+ "Allow setting cgroups limits: cpu, cpuset, io, memory and
pids.")
(start
#~(make-forkexec-constructor
@@ -242,3 +291,497 @@ (define rootless-podman-service-type
(default-value (rootless-podman-configuration))
(description
"This service configures rootless @code{podman} on the Guix System.")))
+
+
+;;;
+;;; OCI container.
+;;;
+
+(define (oci-sanitize-pair pair delimiter)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (match pair
+ (((? valid? key) . (? valid? value))
+ #~(string-append #$key #$delimiter #$value))
+ (_
+ (raise
+ (formatted-message
+ (G_ "pair members must contain only strings, gexps or file-like objects
+but ~a was found")
+ pair)))))
+
+(define (oci-sanitize-mixed-list name value delimiter)
+ (map
+ (lambda (el)
+ (cond ((string? el) el)
+ ((pair? el) (oci-sanitize-pair el delimiter))
+ (else
+ (raise
+ (formatted-message
+ (G_ "~a members must be either a string or a pair but ~a was
+found!")
+ name el)))))
+ value))
+
+(define (oci-sanitize-host-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "host-environment" value "="))
+
+(define (oci-sanitize-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "environment" value "="))
+
+(define (oci-sanitize-ports value)
+ ;; Expected spec format:
+ ;; '(("8088" . "80") "2022:22")
+ (oci-sanitize-mixed-list "ports" value ":"))
+
+(define (oci-sanitize-volumes value)
+ ;; Expected spec format:
+ ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java")
+ (oci-sanitize-mixed-list "volumes" value ":"))
+
+(define (oci-sanitize-shepherd-actions value)
+ (map
+ (lambda (el)
+ (if (shepherd-action? el)
+ el
+ (raise
+ (formatted-message
+ (G_ "shepherd-actions may only be shepherd-action records
+but ~a was found") el))))
+ value))
+
+(define (oci-sanitize-extra-arguments value)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (map
+ (lambda (el)
+ (if (valid? el)
+ el
+ (raise
+ (formatted-message
+ (G_ "extra arguments may only be strings, gexps or file-like objects
+but ~a was found") el))))
+ value))
+
+(define (oci-image-reference image)
+ (if (string? image)
+ image
+ (string-append (oci-image-repository image)
+ ":" (oci-image-tag image))))
+
+(define (oci-lowerable-image? image)
+ (or (manifest? image)
+ (operating-system? image)
+ (gexp? image)
+ (file-like? image)))
+
+(define (string-or-oci-image? image)
+ (or (string? image)
+ (oci-image? image)))
+
+(define list-of-symbols?
+ (list-of symbol?))
+
+(define-maybe/no-serialization string)
+
+(define-configuration/no-serialization oci-image
+ (repository
+ (string)
+ "A string like @code{myregistry.local:5000/testing/test-image} that names
+the OCI image.")
+ (tag
+ (string "latest")
+ "A string representing the OCI image tag. Defaults to @code{latest}.")
+ (value
+ (oci-lowerable-image)
+ "A @code{manifest} or @code{operating-system} record that will be lowered
+into an OCI compatible tarball. Otherwise this field's value can be a gexp
+or a file-like object that evaluates to an OCI compatible tarball.")
+ (pack-options
+ (list '())
+ "An optional set of keyword arguments that will be passed to the
+@code{docker-image} procedure from @code{guix scripts pack}. They can be used
+to replicate @command{guix pack} behavior:
+
+@lisp
+(oci-image
+ (repository \"guile\")
+ (tag \"3\")
+ (manifest (specifications->manifest '(\"guile\")))
+ (pack-options
+ '(#:symlinks ((\"/bin/guile\" -> \"bin/guile\"))
+ #:max-layers 2)))
+@end lisp
+
+If the @code{value} field is an @code{operating-system} record, this field's
+value will be ignored.")
+ (system
+ (maybe-string)
+ "Attempt to build for a given system, e.g. \"i686-linux\"")
+ (target
+ (maybe-string)
+ "Attempt to cross-build for a given triple, e.g. \"aarch64-linux-gnu\"")
+ (grafts?
+ (boolean #f)
+ "Whether to allow grafting or not in the pack build."))
+
+(define-configuration/no-serialization oci-container-configuration
+ (user
+ (string "oci-container")
+ "The user under whose authority docker commands will be run.")
+ (group
+ (string "docker")
+ "The group under whose authority docker commands will be run.")
+ (command
+ (list-of-strings '())
+ "Overwrite the default command (@code{CMD}) of the image.")
+ (entrypoint
+ (maybe-string)
+ "Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.")
+ (host-environment
+ (list '())
+ "Set environment variables in the host environment where @command{docker run}
+is invoked. This is especially useful to pass secrets from the host to the
+container without having them on the @command{docker run}'s command line: by
+setting the @code{MYSQL_PASSWORD} on the host and by passing
+@code{--env MYSQL_PASSWORD} through the @code{extra-arguments} field, it is
+possible to securely set values in the container environment. This field's
+value can be a list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"LANGUAGE\" . \"eo:ca:eu\")
+ \"JAVA_HOME=/opt/java\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to @code{make-forkexec-constructor}."
+ (sanitizer oci-sanitize-host-environment))
+ (environment
+ (list '())
+ "Set environment variables inside the container. This can be a list of pairs
+or strings, even mixed:
+
+@lisp
+(list '(\"LANGUAGE\" . \"eo:ca:eu\")
+ \"JAVA_HOME=/opt/java\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-environment))
+ (image
+ (string-or-oci-image)
+ "The image used to build the container. It can be a string or an
+@code{oci-image} record. Strings are resolved by the Docker
+Engine, and follow the usual format
+@code{myregistry.local:5000/testing/test-image:tag}.")
+ (provision
+ (maybe-string)
+ "Set the name of the provisioned Shepherd service.")
+ (requirement
+ (list-of-symbols '())
+ "Set additional Shepherd services dependencies to the provisioned Shepherd
+service.")
+ (log-file
+ (maybe-string)
+ "When @code{log-file} is set, it names the file to which the service’s
+standard output and standard error are redirected. @code{log-file} is created
+if it does not exist, otherwise it is appended to.")
+ (auto-start?
+ (boolean #t)
+ "Whether this service should be started automatically by the Shepherd. If it
+is @code{#f} the service has to be started manually with @command{herd start}.")
+ (respawn?
+ (boolean #f)
+ "Whether to restart the service when it stops, for instance when the
+underlying process dies.")
+ (shepherd-actions
+ (list '())
+ "This is a list of @code{shepherd-action} records defining actions supported
+by the service."
+ (sanitizer oci-sanitize-shepherd-actions))
+ (network
+ (maybe-string)
+ "Set a Docker network for the spawned container.")
+ (ports
+ (list '())
+ "Set the port or port ranges to expose from the spawned container. This can
+be a list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"8080\" . \"80\")
+ \"10443:443\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-ports))
+ (volumes
+ (list '())
+ "Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"/root/data/grafana\" . \"/var/lib/grafana\")
+ \"/gnu/store:/gnu/store\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-volumes))
+ (container-user
+ (maybe-string)
+ "Set the current user inside the spawned container. You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#user,upstream}
+documentation for semantics.")
+ (workdir
+ (maybe-string)
+ "Set the current working for the spawned Shepherd service.
+You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#workdir,upstream}
+documentation for semantics.")
+ (extra-arguments
+ (list '())
+ "A list of strings, gexps or file-like objects that will be directly passed
+to the @command{docker run} invokation."
+ (sanitizer oci-sanitize-extra-arguments)))
+
+(define oci-container-configuration->options
+ (lambda (config)
+ (let ((entrypoint
+ (oci-container-configuration-entrypoint config))
+ (network
+ (oci-container-configuration-network config))
+ (user
+ (oci-container-configuration-container-user config))
+ (workdir
+ (oci-container-configuration-workdir config)))
+ (apply append
+ (filter (compose not unspecified?)
+ `(,(if (maybe-value-set? entrypoint)
+ `("--entrypoint" ,entrypoint)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "--env" spec))
+ (oci-container-configuration-environment config))
+ ,(if (maybe-value-set? network)
+ `("--network" ,network)
+ '())
+ ,(if (maybe-value-set? user)
+ `("--user" ,user)
+ '())
+ ,(if (maybe-value-set? workdir)
+ `("--workdir" ,workdir)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "-p" spec))
+ (oci-container-configuration-ports config))
+ ,(append-map
+ (lambda (spec)
+ (list "-v" spec))
+ (oci-container-configuration-volumes config))))))))
+
+(define* (get-keyword-value args keyword #:key (default #f))
+ (let ((kv (memq keyword args)))
+ (if (and kv (>= (length kv) 2))
+ (cadr kv)
+ default)))
+
+(define (lower-operating-system os target system)
+ (mlet* %store-monad
+ ((tarball
+ (lower-object
+ (system-image (os->image os #:type docker-image-type))
+ system
+ #:target target)))
+ (return tarball)))
+
+(define (lower-manifest name image target system)
+ (define value (oci-image-value image))
+ (define options (oci-image-pack-options image))
+ (define image-reference
+ (oci-image-reference image))
+ (define image-tag
+ (let* ((extra-options
+ (get-keyword-value options #:extra-options))
+ (image-tag-option
+ (and extra-options
+ (get-keyword-value extra-options #:image-tag))))
+ (if image-tag-option
+ '()
+ `(#:extra-options (#:image-tag ,image-reference)))))
+
+ (mlet* %store-monad
+ ((_ (set-grafting
+ (oci-image-grafts? image)))
+ (guile (set-guile-for-build (default-guile)))
+ (profile
+ (profile-derivation value
+ #:target target
+ #:system system
+ #:hooks '()
+ #:locales? #f))
+ (tarball (apply pack:docker-image
+ `(,name ,profile
+ ,@options
+ ,@image-tag
+ #:localstatedir? #t))))
+ (return tarball)))
+
+(define (lower-oci-image name image)
+ (define value (oci-image-value image))
+ (define image-target (oci-image-target image))
+ (define image-system (oci-image-system image))
+ (define target
+ (if (maybe-value-set? image-target)
+ image-target
+ (%current-target-system)))
+ (define system
+ (if (maybe-value-set? image-system)
+ image-system
+ (%current-system)))
+ (with-store store
+ (run-with-store store
+ (match value
+ ((? manifest? value)
+ (lower-manifest name image target system))
+ ((? operating-system? value)
+ (lower-operating-system value target system))
+ ((or (? gexp? value)
+ (? file-like? value))
+ value)
+ (_
+ (raise
+ (formatted-message
+ (G_ "oci-image value must contain only manifest,
+operating-system, gexp or file-like records but ~a was found")
+ value))))
+ #:target target
+ #:system system)))
+
+(define (%oci-image-loader name image tag)
+ (let ((docker (file-append docker-cli "/bin/docker"))
+ (tarball (lower-oci-image name image)))
+ (with-imported-modules '((guix build utils))
+ (program-file (format #f "~a-image-loader" name)
+ #~(begin
+ (use-modules (guix build utils)
+ (ice-9 popen)
+ (ice-9 rdelim))
+
+ (format #t "Loading image for ~a from ~a...~%" #$name #$tarball)
+ (define line
+ (read-line
+ (open-input-pipe
+ (string-append #$docker " load -i " #$tarball))))
+
+ (unless (or (eof-object? line)
+ (string-null? line))
+ (format #t "~a~%" line)
+ (let ((repository&tag
+ (string-drop line
+ (string-length
+ "Loaded image: "))))
+
+ (invoke #$docker "tag" repository&tag #$tag)
+ (format #t "Tagged ~a with ~a...~%" #$tarball #$tag))))))))
+
+(define (oci-container-shepherd-service config)
+ (define (guess-name name image)
+ (if (maybe-value-set? name)
+ name
+ (string-append "docker-"
+ (basename
+ (if (string? image)
+ (first (string-split image #\:))
+ (oci-image-repository image))))))
+
+ (let* ((docker (file-append docker-cli "/bin/docker"))
+ (actions (oci-container-configuration-shepherd-actions config))
+ (auto-start?
+ (oci-container-configuration-auto-start? config))
+ (user (oci-container-configuration-user config))
+ (group (oci-container-configuration-group config))
+ (host-environment
+ (oci-container-configuration-host-environment config))
+ (command (oci-container-configuration-command config))
+ (log-file (oci-container-configuration-log-file config))
+ (provision (oci-container-configuration-provi
This message was truncated. Download the full message here.
P
P
paul wrote 25 hours ago
Re: OCI provisioning service
(address . 76081@debbugs.gnu.org)
c685875a-0b74-4ada-9a04-c387b8db646e@autistici.org
Hi,

I'm about to send a v2. v2 compared to the first revision features:


- it actually compiles all the times :) (rev 1 referenced oci-image too
early for it to be working and generated a compile time error, if you
recompiled it sometimes went away so I thought it was a problem of my
setup. CI caught this)
- it allows more values to be overridden by eventual users of the Scheme API
- it allows passing extra arguments directly after each podman or docker
invokation, allowing for example for overriding podman --root and
similar options.

All of these tests should pass:

guix shell -D guix -CPW -- make check-system TESTS="oci-container
oci-service-rootless-podman docker docker-system rootless-podman
oci-service-docker"


Thank you for your work,

giacomo
G
G
Giacomo Leidi wrote 25 hours ago
[PATCH v2 1/4] services: rootless-podman: Use login shell.
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
89778533f32ad1388f03414e884fff10f74ef379.1739128504.git.goodoldpaul@autistici.org
This commit allows for having PATH set when changing the owner of
/sys/fs/group.

* gnu/services/containers.scm (crgroups-fs-owner): Use login shell.

Change-Id: I9510c637a5332325e05ca5ebc9dfd4de32685c50
---
gnu/services/containers.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (17 lines)
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index 19d35ccbcb6..dc66ac4f967 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -140,7 +140,7 @@ (define (cgroups-fs-owner-entrypoint config)
(rootless-podman-configuration-group-name config))
(program-file "cgroups2-fs-owner-entrypoint"
#~(system*
- (string-append #+bash-minimal "/bin/bash") "-c"
+ (string-append #+bash-minimal "/bin/bash") "-l" "-c"
(string-append "echo Setting /sys/fs/cgroup "
"group ownership to " #$group " && chown -v "
"root:" #$group " /sys/fs/cgroup && "

base-commit: 5a897c5c95a81278b044c18d962d3bd83131ba06
--
2.48.1
G
G
Giacomo Leidi wrote 25 hours ago
[PATCH v2 3/4] services: Add oci-service-type.
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
c94fe584d2ba0071f64e456e708c1447ab094cd7.1739128504.git.goodoldpaul@autistici.org
This patch implements a generalization of the
oci-container-service-type, which consequently is made deprecated. The
oci-service-type, in addition to all the features from the
oci-container-service-type, can now provision OCI networks and volumes.
It only handles OCI objects creation, the user is supposed to handle
state once the objects are provsioned.

It currently supports two different OCI runtimes: Docker and rootless
Podman. Both runtimes are tested to make sure provisioned containers
can connect to each other through provisioned networks and can
read/write data with provisioned volumes.

At last the Scheme API is thought to facilitate the implementation of a
Guix Home service in the future.

* gnu/services/containers.scm (%oci-supported-runtimes): New variable;
(oci-runtime-cli): new variable;
(oci-runtime-name): new variable;
(oci-network-configuration): new variable;
(oci-volume-configuration): new variable;
(oci-configuration): new variable;
(oci-extension): new variable;
(oci-networks-shepherd-name): new variable;
(oci-service-type): new variable;
(oci-state->shepherd-services): new variable.
* doc/guix.texi: Document it.
* gnu/tests/containers.scm: Test it.
* gnu/services/docker.scm: Deprecate the oci-container-service-type.

Change-Id: I656b3db85832e42d53072fcbfb91d1226f39ef38
---
doc/guix.texi | 299 ++++++++--
gnu/services/containers.scm | 1086 +++++++++++++++++++++++++++++++----
gnu/services/docker.scm | 37 +-
gnu/tests/containers.scm | 999 +++++++++++++++++++++++++++++++-
4 files changed, 2225 insertions(+), 196 deletions(-)

Toggle diff (456 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index bb5f29277fb..8f66b35297b 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41778,59 +41778,159 @@ Miscellaneous Services
@cindex OCI-backed, Shepherd services
@subsubheading OCI backed services
-Should you wish to manage your Docker containers with the same consistent
-interface you use for your other Shepherd services,
-@var{oci-container-service-type} is the tool to use: given an
-@acronym{Open Container Initiative, OCI} container image, it will run it in a
+Should you wish to manage your @acronym{Open Container Initiative, OCI} containers
+with the same consistent interface you use for your other Shepherd services,
+@var{oci-service-type} is the tool to use: given an
+OCI container image, it will run it in a
Shepherd service. One example where this is useful: it lets you run services
-that are available as Docker/OCI images but not yet packaged for Guix.
+that are available as OCI images but not yet packaged for Guix.
-@defvar oci-container-service-type
+@defvar oci-service-type
-This is a thin wrapper around Docker's CLI that executes OCI images backed
+This is a thin wrapper around Docker's or Podman's CLI that executes OCI images backed
processes as Shepherd Services.
@lisp
-(service oci-container-service-type
- (list
- (oci-container-configuration
- (network "host")
- (image
- (oci-image
- (repository "guile")
- (tag "3")
- (value (specifications->manifest '("guile")))
- (pack-options '(#:symlinks (("/bin/guile" -> "bin/guile"))
- #:max-layers 2))))
- (entrypoint "/bin/guile")
- (command
- '("-c" "(display \"hello!\n\")")))
- (oci-container-configuration
- (image "prom/prometheus")
- (ports
- '(("9000" . "9000")
- ("9090" . "9090"))))
- (oci-container-configuration
- (image "grafana/grafana:10.0.1")
- (network "host")
- (volumes
- '("/var/lib/grafana:/var/lib/grafana")))))
+(simple-service 'oci-provisioning
+ oci-service-type
+ (oci-extension
+ (networks
+ (list
+ (oci-network-configuration (name "monitoring"))))
+ (containers
+ (list
+ (oci-container-configuration
+ (network "monitoring")
+ (image
+ (oci-image
+ (repository "guile")
+ (tag "3")
+ (value (specifications->manifest '("guile")))
+ (pack-options '(#:symlinks (("/bin/guile" -> "bin/guile"))
+ #:max-layers 2))))
+ (entrypoint "/bin/guile")
+ (command
+ '("-c" "(display \"hello!\n\")")))
+ (oci-container-configuration
+ (image "prom/prometheus")
+ (network "host")
+ (ports
+ '(("9000" . "9000")
+ ("9090" . "9090"))))
+ (oci-container-configuration
+ (image "grafana/grafana:10.0.1")
+ (network "host")
+ (volumes
+ '("/var/lib/grafana:/var/lib/grafana")))))))
@end lisp
In this example three different Shepherd services are going to be added to the
system. Each @code{oci-container-configuration} record translates to a
-@code{docker run} invocation and its fields directly map to options. You can
-refer to the
-@url{https://docs.docker.com/engine/reference/commandline/run,upstream}
-documentation for the semantics of each value. If the images are not found,
-they will be
-@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The
+@command{docker run} or @command{podman run} invocation and its fields directly
+map to options. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html,Podman}
+upstream documentation for semantics of each value. If the images are not found,
+they will be pulled. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/pull/,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-pull.1.html,Podman}
+upstream documentation for semantics. The
services with @code{(network "host")} are going to be attached to the
host network and are supposed to behave like native processes with regard to
networking.
@end defvar
+@c %start of fragment
+
+@deftp {Data Type} oci-configuration
+Available @code{oci-configuration} fields are:
+
+@table @asis
+@item @code{runtime} (default: @code{'docker}) (type: symbol)
+The OCI runtime to use to run commands. It can be either @code{'docker} or
+@code{'podman}.
+
+@item @code{runtime-cli} (type: maybe-package)
+The OCI runtime command line to be installed in the system profile and used
+to provision OCI resources. When unset it will default to @code{docker-cli}
+package for the @code{'docker} runtime or to @code{podman} package for the
+@code{'podman} runtime.
+
+@item @code{runtime-extra-arguments} (default: @code{'()}) (type: list)
+A list of strings, gexps or file-like objects that will be placed
+after each @command{docker} or @command{podman} invokation.
+
+@item @code{user} (default: @code{"oci-container"}) (type: string)
+The user name under whose authority OCI commands will be run.
+
+@item @code{group} (default: @code{"docker"}) (type: string)
+The group name under whose authority OCI commands will be run. When
+using the @code{'podman} OCI runtime, this field will be ignored and the
+default group of the user configured in the @code{user} field will be used.
+
+@item @code{subuids-range} (type: maybe-subid-range)
+An optional @code{subid-range} record allocating subuids for the user from
+the @code{user} field. When unset, with the rootless Podman OCI runtime, it
+defaults to @code{(subid-range (name "oci-container"))}.
+
+@item @code{subgids-range} (type: maybe-subid-range)
+An optional @code{subid-range} record allocating subgids for the user from
+the @code{user} field. When unset, with the rootless Podman OCI runtime, it
+defaults to @code{(subid-range (name "oci-container"))}.
+
+@item @code{containers} (default: @code{'()}) (type: list-of-oci-containers)
+The list of @code{oci-container-configuration} records representing the
+containers to provision. Most users are supposed not to use this field and use
+the @code{oci-extension} record instead.
+
+@item @code{networks} (default: @code{'()}) (type: list-of-oci-networks)
+The list of @code{oci-network-configuration} records representing the
+containers to provision. Most users are supposed not to use this field and use
+the @code{oci-extension} record instead.
+
+@item @code{volumes} (default: @code{'()}) (type: list-of-oci-volumes)
+The list of @code{oci-volumes-configuration} records representing the
+containers to provision. Most users are supposed not to use this field and use
+the @code{oci-extension} record instead.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+When true, additional output will be printed, allowing to better follow the
+flow of execution.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} oci-extension
+Available @code{oci-extension} fields are:
+
+@table @asis
+@item @code{containers} (default: @code{'()}) (type: list-of-oci-containers)
+The list of @code{oci-container-configuration} records representing the
+containers to provision.
+
+@item @code{networks} (default: @code{'()}) (type: list-of-oci-networks)
+The list of @code{oci-network-configuration} records representing the
+containers to provision.
+
+@item @code{volumes} (default: @code{'()}) (type: list-of-oci-volumes)
+The list of @code{oci-volumes-configuration} records representing the
+containers to provision.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+
@c %start of fragment
@deftp {Data Type} oci-container-configuration
@@ -41850,16 +41950,16 @@ Miscellaneous Services
Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
@item @code{host-environment} (default: @code{'()}) (type: list)
-Set environment variables in the host environment where @command{docker
-run} is invoked. This is especially useful to pass secrets from the
-host to the container without having them on the @command{docker run}'s
-command line: by setting the @code{MYSQL_PASSWORD} on the host and by passing
+Set environment variables in the host environment where @command{docker run}
+or @command{podman run} are invoked. This is especially useful to pass secrets
+from the host to the container without having them on the OCI runtime command line,
+for example: by setting the @code{MYSQL_PASSWORD} on the host and by passing
@code{--env MYSQL_PASSWORD} through the @code{extra-arguments} field, it is
possible to securely set values in the container environment. This field's
value can be a list of pairs or strings, even mixed:
@lisp
-(list '("LANGUAGE\" . "eo:ca:eu")
+(list '("LANGUAGE" . "eo:ca:eu")
"JAVA_HOME=/opt/java")
@end lisp
@@ -41867,22 +41967,24 @@ Miscellaneous Services
directly to @code{make-forkexec-constructor}.
@item @code{environment} (default: @code{'()}) (type: list)
-Set environment variables. This can be a list of pairs or strings, even mixed:
+Set environment variables inside the container. This can be a list of pairs
+or strings, even mixed:
@lisp
(list '("LANGUAGE" . "eo:ca:eu")
"JAVA_HOME=/opt/java")
@end lisp
-Pair members can be strings, gexps or file-like objects.
-Strings are passed directly to the Docker CLI. You can refer to the
-@uref{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
-documentation for semantics.
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the OCI runtime CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#env-e-env,Podman}
+upstream documentation for semantics.
@item @code{image} (type: string-or-oci-image)
The image used to build the container. It can be a string or an
-@code{oci-image} record. Strings are resolved by the Docker Engine, and
-follow the usual format
+@code{oci-image} record. Strings are resolved by the OCI runtime,
+and follow the usual format
@code{myregistry.local:5000/testing/test-image:tag}.
@item @code{provision} (default: @code{""}) (type: string)
@@ -41910,7 +42012,7 @@ Miscellaneous Services
by the service.
@item @code{network} (default: @code{""}) (type: string)
-Set a Docker network for the spawned container.
+Set an OCI network for the spawned container.
@item @code{ports} (default: @code{'()}) (type: list)
Set the port or port ranges to expose from the spawned container. This can be a
@@ -41921,10 +42023,11 @@ Miscellaneous Services
"10443:443")
@end lisp
-Pair members can be strings, gexps or file-like objects.
-Strings are passed directly to the Docker CLI. You can refer to the
-@uref{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
-documentation for semantics.
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the OCI runtime CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#publish-p-ip-hostport-containerport-protocol,Podman}
+upstream documentation for semantics.
@item @code{volumes} (default: @code{'()}) (type: list)
Set volume mappings for the spawned container. This can be a
@@ -41935,25 +42038,95 @@ Miscellaneous Services
"/gnu/store:/gnu/store")
@end lisp
-Pair members can be strings, gexps or file-like objects.
-Strings are passed directly to the Docker CLI. You can refer to the
-@uref{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
-documentation for semantics.
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the OCI runtime CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#volume-v-source-volume-host-dir-container-dir-options,Podman}
+upstream documentation for semantics.
@item @code{container-user} (default: @code{""}) (type: string)
Set the current user inside the spawned container. You can refer to the
-@url{https://docs.docker.com/engine/reference/run/#user,upstream}
-documentation for semantics.
+@url{https://docs.docker.com/engine/reference/run/#user,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#user-u-user-group,Podman}
+upstream documentation for semantics.
@item @code{workdir} (default: @code{""}) (type: string)
Set the current working directory for the spawned Shepherd service.
You can refer to the
-@url{https://docs.docker.com/engine/reference/run/#workdir,upstream}
-documentation for semantics.
+@url{https://docs.docker.com/engine/reference/run/#workdir,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#workdir-w-dir,Podman}
+upstream documentation for semantics.
+
+@item @code{extra-arguments} (default: @code{'()}) (type: list)
+A list of strings, gexps or file-like objects that will be directly passed
+to the @command{docker run} or @command{podman run} invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} oci-network-configuration
+Available @code{oci-network-configuration} fields are:
+
+@table @asis
+@item @code{name} (type: string)
+The name of the OCI network to provision.
+
+@item @code{driver} (type: maybe-string)
+The driver to manage the network.
+
+@item @code{gateway} (type: maybe-string)
+IPv4 or IPv6 gateway for the subnet.
+
+@item @code{internal?} (default: @code{#f}) (type: boolean)
+Restrict external access to the network
+
+@item @code{ip-range} (type: maybe-string)
+Allocate container ip from a sub-range in CIDR format.
+
+@item @code{ipam-driver} (type: maybe-string)
+IP Address Management Driver.
+
+@item @code{ipv6?} (default: @code{#f}) (type: boolean)
+Enable IPv6 networking.
+
+@item @code{subnet} (type: maybe-string)
+Subnet in CIDR format that represents a network segment.
+
+@item @code{labels} (default: @code{'()}) (type: list)
+The list of labels that will be used to tag the current volume.
+
+@item @code{extra-arguments} (default: @code{'()}) (type: list)
+A list of strings, gexps or file-like objects that will be directly
+passed to the runtime invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} oci-volume-configuration
+Available @code{oci-volume-configuration} fields are:
+
+@table @asis
+@item @code{name} (type: string)
+The name of the OCI volume to provision.
+
+@item @code{labels} (default: @code{'()}) (type: list)
+The list of labels that will be used to tag the current volume.
@item @code{extra-arguments} (default: @code{'()}) (type: list)
A list of strings, gexps or file-like objects that will be directly
-passed to the @command{docker run} invocation.
+passed to the runtime invokation.
@end table
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index 50bf6c05549..5c50c99eaf6 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -41,6 +41,7 @@ (define-module (gnu services containers)
#:use-module ((guix scripts pack) #:prefix pack:)
#:use-module (guix store)
#:use-module (srfi srfi-1)
+ #:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (rootless-podman-configuration
rootless-podman-configuration?
@@ -96,8 +97,72 @@ (define-module (gnu services containers)
oci-container-configuration-workdir
oci-container-configuration-extra-arguments
+ list-of-oci-containers?
+ list-of-oci-networks?
+ list-of-oci-volumes?
+
+ %oci-supported-runtimes
+ oci-sanitize-runtime
+ oci-runtime-system-environment
+ oci-runtime-system-extra-arguments
+ oci-runtime-system-group
+ oci-runtime-system-requirement
+ oci-runtime-cli
+ oci-runtime-system-cli
+ oci-runtime-name
+ oci-runtime-group
+
+ oci-network-configuration
+ oci-network-configuration?
+ oci-network-configuration-fields
+ oci-network-configuration-name
+ oci-network-configuration-driver
+ oci-network-configuration-gateway
+ oci-network-configuration-internal?
+ oci-network-configuration-ip-range
+ oci-network-configuration-ipam-driver
+ oci-network-configuration-ipv6?
+ oci-network-configuration-subnet
+ oci-network-configuration-labels
+ oci-network-configuration-extra-arguments
+
+ oci-volume-configuration
+ oci-volume-configuration?
+ oci-volume-configuration-fields
+ oci-volume-configuration-name
+ oci-volume-configuration-labels
+ oci-volume-configuration-extra-arguments
+
+ oci-configuration
+ oci-configuration?
+ oci-configuration-fields
+ oci-configuration-runtime
+ oci-configuration-runtime-cli
+ oci-configuration-runtime-extra-arguments
+ oci-configuration-user
+ oci-configuration-group
+ oci-configuration-containers
+ oci-configuration-networks
+ oci-configuration-volumes
+ oci-configuration-verbose?
+
+ oci-extension
+ oci-extension?
+ oci-extension-fields
+ oci-extension-containers
+ oci-extension-networks
+ oci-extension-vol
This message was truncated. Download the full message here.
G
G
Giacomo Leidi wrote 25 hours ago
[PATCH v2 4/4] tests: Use lower-oci-image-state in container tests.
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
10254ceb4ef758730d46c5c30798833f9f592833.1739128504.git.goodoldpaul@autistici.org
This patch replaces boilerplate in container related tests with
oci-image plumbing from (gnu services containers).

* gnu/services/containers.scm: Export lower-oci-image-state.
* gnu/tests/containers.scm (%oci-tarball): New variable;
(run-rootless-podman-test): use %oci-tarball;
(build-tarball&run-rootless-podman-test): drop procedure.
* gnu/tests/docker.scm (%docker-tarball): New variable;
(build-tarball&run-docker-test): use %docker-tarball;
(%docker-system-tarball): New variable;
(build-tarball&run-docker-system-test): new procedure.

Change-Id: Iad6f0704aee188d89464c83722dea0bb7adb084a
---
gnu/services/containers.scm | 35 ++++++++-----
gnu/tests/containers.scm | 80 ++++++++++++++---------------
gnu/tests/docker.scm | 100 ++++++++++++++++++++----------------
3 files changed, 114 insertions(+), 101 deletions(-)

Toggle diff (342 lines)
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index 5c50c99eaf6..2808afe7f08 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -74,6 +74,8 @@ (define-module (gnu services containers)
oci-image-system
oci-image-grafts?
+ lower-oci-image-state
+
oci-container-configuration
oci-container-configuration?
oci-container-configuration-fields
@@ -996,12 +998,9 @@ (define (lower-operating-system os target system)
#:target target)))
(return tarball)))
-(define (lower-manifest name image target system)
- "Lower IMAGE, a manifest record, into a tarball containing an OCI image."
- (define value (oci-image-value image))
- (define options (oci-image-pack-options image))
- (define image-reference
- (oci-image-reference image))
+(define (lower-manifest name value options image-reference
+ target system grafts?)
+ "Lower VALUE, a manifest record, into a tarball containing an OCI image."
(define image-tag
(let* ((extra-options
(get-keyword-value options #:extra-options))
@@ -1013,8 +1012,7 @@ (define (lower-manifest name image target system)
`(#:extra-options (#:image-tag ,image-reference)))))
(mlet* %store-monad
- ((_ (set-grafting
- (oci-image-grafts? image)))
+ ((_ (set-grafting grafts?))
(guile (set-guile-for-build (default-guile)))
(profile
(profile-derivation value
@@ -1029,11 +1027,8 @@ (define (lower-manifest name image target system)
#:localstatedir? #t))))
(return tarball)))
-(define (lower-oci-image name image)
- "Lower IMAGE, a oci-image record, into a tarball containing an OCI image."
- (define value (oci-image-value image))
- (define image-target (oci-image-target image))
- (define image-system (oci-image-system image))
+(define (lower-oci-image-state name value options reference
+ image-target image-system grafts?)
(define target
(if (maybe-value-set? image-target)
image-target
@@ -1046,7 +1041,8 @@ (define (lower-oci-image name image)
(run-with-store store
(match value
((? manifest? value)
- (lower-manifest name image target system))
+ (lower-manifest name value options reference
+ target system grafts?))
((? operating-system? value)
(lower-operating-system value target system))
((or (? gexp? value)
@@ -1061,6 +1057,17 @@ (define (lower-oci-image name image)
#:target target
#:system system)))
+(define (lower-oci-image name image)
+ "Lower IMAGE, a oci-image record, into a tarball containing an OCI image."
+ (lower-oci-image-state
+ name
+ (oci-image-value image)
+ (oci-image-pack-options image)
+ (oci-image-reference image)
+ (oci-image-target image)
+ (oci-image-system image)
+ (oci-image-grafts? image)))
+
(define* (oci-image-loader runtime-cli name image tag #:key (verbose? #f))
"Return a file-like object that, once lowered, will evaluate to a program able
to load IMAGE through RUNTIME-CLI and to tag it with TAG afterwards."
diff --git a/gnu/tests/containers.scm b/gnu/tests/containers.scm
index bac1f47bd34..1fcf6be7ace 100644
--- a/gnu/tests/containers.scm
+++ b/gnu/tests/containers.scm
@@ -69,13 +69,47 @@ (define %rootless-podman-os
(supplementary-groups '("wheel" "netdev" "cgroup"
"audio" "video")))))))
-(define (run-rootless-podman-test oci-tarball)
+(define %oci-tarball
+ (lower-oci-image-state
+ "guile-guest"
+ (packages->manifest
+ (list
+ guile-3.0 guile-json-3
+ (package
+ (name "guest-script")
+ (version "0")
+ (source #f)
+ (build-system trivial-build-system)
+ (arguments `(#:guile ,guile-3.0
+ #:builder
+ (let ((out (assoc-ref %outputs "out")))
+ (mkdir out)
+ (call-with-output-file (string-append out "/a.scm")
+ (lambda (port)
+ (display "(display \"hello world\n\")" port)))
+ #t)))
+ (synopsis "Display hello world using Guile")
+ (description "This package displays the text \"hello world\" on the
+standard output device and then enters a new line.")
+ (home-page #f)
+ (license license:public-domain))))
+ '(#:entry-point "bin/guile"
+ #:localstatedir? #t
+ #:extra-options (#:image-tag "guile-guest")
+ #:symlinks (("/bin/Guile" -> "bin/guile")
+ ("aa.scm" -> "a.scm")))
+ "guile-guest"
+ (%current-target-system)
+ (%current-system)
+ #f))
+
+(define (run-rootless-podman-test)
(define os
(marionette-operating-system
(operating-system-with-gc-roots
%rootless-podman-os
- (list oci-tarball))
+ (list %oci-tarball))
#:imported-modules '((gnu services herd)
(guix combinators))))
@@ -254,7 +288,7 @@ (define (run-rootless-podman-test oci-tarball)
(let* ((loaded (slurp ,(string-append #$podman
"/bin/podman")
"load" "-i"
- ,#$oci-tarball))
+ ,#$%oci-tarball))
(repository&tag "localhost/guile-guest:latest")
(response1 (slurp
,(string-append #$podman "/bin/podman")
@@ -307,49 +341,11 @@ (define (run-rootless-podman-test oci-tarball)
(gexp->derivation "rootless-podman-test" test))
-(define (build-tarball&run-rootless-podman-test)
- (mlet* %store-monad
- ((_ (set-grafting #f))
- (guile (set-guile-for-build (default-guile)))
- (guest-script-package ->
- (package
- (name "guest-script")
- (version "0")
- (source #f)
- (build-system trivial-build-system)
- (arguments `(#:guile ,guile-3.0
- #:builder
- (let ((out (assoc-ref %outputs "out")))
- (mkdir out)
- (call-with-output-file (string-append out "/a.scm")
- (lambda (port)
- (display "(display \"hello world\n\")" port)))
- #t)))
- (synopsis "Display hello world using Guile")
- (description "This package displays the text \"hello world\" on the
-standard output device and then enters a new line.")
- (home-page #f)
- (license license:public-domain)))
- (profile (profile-derivation (packages->manifest
- (list guile-3.0 guile-json-3
- guest-script-package))
- #:hooks '()
- #:locales? #f))
- (tarball (pack:docker-image
- "docker-pack" profile
- #:symlinks '(("/bin/Guile" -> "bin/guile")
- ("aa.scm" -> "a.scm"))
- #:extra-options
- '(#:image-tag "guile-guest")
- #:entry-point "bin/guile"
- #:localstatedir? #t)))
- (run-rootless-podman-test tarball)))
-
(define %test-rootless-podman
(system-test
(name "rootless-podman")
(description "Test rootless Podman service.")
- (value (build-tarball&run-rootless-podman-test))))
+ (value (run-rootless-podman-test))))
(define %oci-rootless-podman-os
diff --git a/gnu/tests/docker.scm b/gnu/tests/docker.scm
index 5dcf05a17e3..07edd9d5341 100644
--- a/gnu/tests/docker.scm
+++ b/gnu/tests/docker.scm
@@ -26,6 +26,7 @@ (define-module (gnu tests docker)
#:use-module (gnu system image)
#:use-module (gnu system vm)
#:use-module (gnu services)
+ #:use-module (gnu services containers)
#:use-module (gnu services dbus)
#:use-module (gnu services networking)
#:use-module (gnu services docker)
@@ -57,6 +58,40 @@ (define %docker-os
(service containerd-service-type)
(service docker-service-type)))
+(define %docker-tarball
+ (lower-oci-image-state
+ "guile-guest"
+ (packages->manifest
+ (list
+ guile-3.0 guile-json-3
+ (package
+ (name "guest-script")
+ (version "0")
+ (source #f)
+ (build-system trivial-build-system)
+ (arguments `(#:guile ,guile-3.0
+ #:builder
+ (let ((out (assoc-ref %outputs "out")))
+ (mkdir out)
+ (call-with-output-file (string-append out "/a.scm")
+ (lambda (port)
+ (display "(display \"hello world\n\")" port)))
+ #t)))
+ (synopsis "Display hello world using Guile")
+ (description "This package displays the text \"hello world\" on the
+standard output device and then enters a new line.")
+ (home-page #f)
+ (license license:public-domain))))
+ '(#:entry-point "bin/guile"
+ #:localstatedir? #t
+ #:extra-options (#:image-tag "guile-guest")
+ #:symlinks (("/bin/Guile" -> "bin/guile")
+ ("aa.scm" -> "a.scm")))
+ "guile-guest"
+ (%current-target-system)
+ (%current-system)
+ #f))
+
(define (run-docker-test docker-tarball)
"Load DOCKER-TARBALL as Docker image and run it in a Docker container,
inside %DOCKER-OS."
@@ -173,40 +208,7 @@ (define (run-docker-test docker-tarball)
(gexp->derivation "docker-test" test))
(define (build-tarball&run-docker-test)
- (mlet* %store-monad
- ((_ (set-grafting #f))
- (guile (set-guile-for-build (default-guile)))
- (guest-script-package ->
- (package
- (name "guest-script")
- (version "0")
- (source #f)
- (build-system trivial-build-system)
- (arguments `(#:guile ,guile-3.0
- #:builder
- (let ((out (assoc-ref %outputs "out")))
- (mkdir out)
- (call-with-output-file (string-append out "/a.scm")
- (lambda (port)
- (display "(display \"hello world\n\")" port)))
- #t)))
- (synopsis "Display hello world using Guile")
- (description "This package displays the text \"hello world\" on the
-standard output device and then enters a new line.")
- (home-page #f)
- (license license:public-domain)))
- (profile (profile-derivation (packages->manifest
- (list guile-3.0 guile-json-3
- guest-script-package))
- #:hooks '()
- #:locales? #f))
- (tarball (pack:docker-image
- "docker-pack" profile
- #:symlinks '(("/bin/Guile" -> "bin/guile")
- ("aa.scm" -> "a.scm"))
- #:entry-point "bin/guile"
- #:localstatedir? #t)))
- (run-docker-test tarball)))
+ (run-docker-test %docker-tarball))
(define %test-docker
(system-test
@@ -215,8 +217,22 @@ (define %test-docker
(value (build-tarball&run-docker-test))))
+(define %docker-system-tarball
+ (lower-oci-image-state
+ "guix-system-guest"
+ (operating-system
+ (inherit (simple-operating-system))
+ ;; Use locales for a single libc to
+ ;; reduce space requirements.
+ (locale-libcs (list glibc)))
+ '()
+ "guix-system-guest"
+ (%current-target-system)
+ (%current-system)
+ #f))
+
(define (run-docker-system-test tarball)
- "Load DOCKER-TARBALL as Docker image and run it in a Docker container,
+ "Load TARBALL as Docker image and run it in a Docker container,
inside %DOCKER-OS."
(define os
(marionette-operating-system
@@ -333,21 +349,15 @@ (define (run-docker-system-test tarball)
(gexp->derivation "docker-system-test" test))
+(define (build-tarball&run-docker-system-test)
+ (run-docker-system-test %docker-system-tarball))
+
(define %test-docker-system
(system-test
(name "docker-system")
(description "Run a system image as produced by @command{guix system
docker-image} inside Docker.")
- (value (with-monad %store-monad
- (>>= (lower-object
- (system-image (os->image
- (operating-system
- (inherit (simple-operating-system))
- ;; Use locales for a single libc to
- ;; reduce space requirements.
- (locale-libcs (list glibc)))
- #:type docker-image-type)))
- run-docker-system-test)))))
+ (value (build-tarball&run-docker-system-test))))
(define %oci-os
--
2.48.1
G
G
Giacomo Leidi wrote 25 hours ago
[PATCH v2 2/4] services: oci-container-configuration: Move to (gnu services containers).
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
92f19ba6c0842885735dfce653eef535360085f4.1739128504.git.goodoldpaul@autistici.org
This patch moves the oci-container-configuration and related
configuration records to (gnu services containers).
Public symbols are still exported for backwards
compatibility but since the oci-container-service-type will be
deprecated in favor of the more general oci-service-type, everything is
moved outside of the docker related module.

* gnu/services/docker.scm: Move everything related to oci-container-configuration
to...
* gnu/services/containers.scm: ...here.scm.
* gnu/tests/docker.scm: Simplify %test-oci-container test case.

Change-Id: Iae599dd5cc7442eb632f0c1b3b12f6b928397ae7
---
gnu/services/containers.scm | 549 +++++++++++++++++++++++++++++++++-
gnu/services/docker.scm | 577 +++---------------------------------
gnu/tests/docker.scm | 99 +++----
3 files changed, 625 insertions(+), 600 deletions(-)

Toggle diff (534 lines)
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index dc66ac4f967..50bf6c05549 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;; Copyright © 2024, 2025 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -17,19 +17,31 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (gnu services containers)
+ #:use-module (gnu image)
+ #:use-module (gnu packages admin)
#:use-module (gnu packages bash)
#:use-module (gnu packages containers)
+ #:use-module (gnu packages docker)
#:use-module (gnu packages file-systems)
#:use-module (gnu services)
#:use-module (gnu services base)
#:use-module (gnu services configuration)
#:use-module (gnu services shepherd)
+ #:use-module (gnu system)
#:use-module (gnu system accounts)
+ #:use-module (gnu system image)
#:use-module (gnu system shadow)
#:use-module (gnu system pam)
+ #:use-module (guix diagnostics)
#:use-module (guix gexp)
+ #:use-module (guix i18n)
+ #:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix profiles)
+ #:use-module ((guix scripts pack) #:prefix pack:)
+ #:use-module (guix store)
#:use-module (srfi srfi-1)
+ #:use-module (ice-9 match)
#:export (rootless-podman-configuration
rootless-podman-configuration?
rootless-podman-configuration-fields
@@ -48,7 +60,44 @@ (define-module (gnu services containers)
rootless-podman-shepherd-services
rootless-podman-service-etc
- rootless-podman-service-type))
+ rootless-podman-service-type
+
+ oci-image
+ oci-image?
+ oci-image-fields
+ oci-image-repository
+ oci-image-tag
+ oci-image-value
+ oci-image-pack-options
+ oci-image-target
+ oci-image-system
+ oci-image-grafts?
+
+ oci-container-configuration
+ oci-container-configuration?
+ oci-container-configuration-fields
+ oci-container-configuration-user
+ oci-container-configuration-group
+ oci-container-configuration-command
+ oci-container-configuration-entrypoint
+ oci-container-configuration-host-environment
+ oci-container-configuration-environment
+ oci-container-configuration-image
+ oci-container-configuration-provision
+ oci-container-configuration-requirement
+ oci-container-configuration-log-file
+ oci-container-configuration-auto-start?
+ oci-container-configuration-respawn?
+ oci-container-configuration-shepherd-actions
+ oci-container-configuration-network
+ oci-container-configuration-ports
+ oci-container-configuration-volumes
+ oci-container-configuration-container-user
+ oci-container-configuration-workdir
+ oci-container-configuration-extra-arguments
+
+ oci-container-shepherd-service
+ %oci-container-accounts))
(define (gexp-or-string? value)
(or (gexp? value)
@@ -188,7 +237,7 @@ (define (rootless-podman-cgroups-limits-service config)
rootless-podman-shared-root-fs))
(one-shot? #t)
(documentation
- "Allow setting cgroups limits: cpu, cpuset, memory and
+ "Allow setting cgroups limits: cpu, cpuset, io, memory and
pids.")
(start
#~(make-forkexec-constructor
@@ -242,3 +291,497 @@ (define rootless-podman-service-type
(default-value (rootless-podman-configuration))
(description
"This service configures rootless @code{podman} on the Guix System.")))
+
+
+;;;
+;;; OCI container.
+;;;
+
+(define (oci-sanitize-pair pair delimiter)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (match pair
+ (((? valid? key) . (? valid? value))
+ #~(string-append #$key #$delimiter #$value))
+ (_
+ (raise
+ (formatted-message
+ (G_ "pair members must contain only strings, gexps or file-like objects
+but ~a was found")
+ pair)))))
+
+(define (oci-sanitize-mixed-list name value delimiter)
+ (map
+ (lambda (el)
+ (cond ((string? el) el)
+ ((pair? el) (oci-sanitize-pair el delimiter))
+ (else
+ (raise
+ (formatted-message
+ (G_ "~a members must be either a string or a pair but ~a was
+found!")
+ name el)))))
+ value))
+
+(define (oci-sanitize-host-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "host-environment" value "="))
+
+(define (oci-sanitize-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "environment" value "="))
+
+(define (oci-sanitize-ports value)
+ ;; Expected spec format:
+ ;; '(("8088" . "80") "2022:22")
+ (oci-sanitize-mixed-list "ports" value ":"))
+
+(define (oci-sanitize-volumes value)
+ ;; Expected spec format:
+ ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java")
+ (oci-sanitize-mixed-list "volumes" value ":"))
+
+(define (oci-sanitize-shepherd-actions value)
+ (map
+ (lambda (el)
+ (if (shepherd-action? el)
+ el
+ (raise
+ (formatted-message
+ (G_ "shepherd-actions may only be shepherd-action records
+but ~a was found") el))))
+ value))
+
+(define (oci-sanitize-extra-arguments value)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (map
+ (lambda (el)
+ (if (valid? el)
+ el
+ (raise
+ (formatted-message
+ (G_ "extra arguments may only be strings, gexps or file-like objects
+but ~a was found") el))))
+ value))
+
+(define (oci-image-reference image)
+ (if (string? image)
+ image
+ (string-append (oci-image-repository image)
+ ":" (oci-image-tag image))))
+
+(define (oci-lowerable-image? image)
+ (or (manifest? image)
+ (operating-system? image)
+ (gexp? image)
+ (file-like? image)))
+
+(define (string-or-oci-image? image)
+ (or (string? image)
+ (oci-image? image)))
+
+(define list-of-symbols?
+ (list-of symbol?))
+
+(define-maybe/no-serialization string)
+
+(define-configuration/no-serialization oci-image
+ (repository
+ (string)
+ "A string like @code{myregistry.local:5000/testing/test-image} that names
+the OCI image.")
+ (tag
+ (string "latest")
+ "A string representing the OCI image tag. Defaults to @code{latest}.")
+ (value
+ (oci-lowerable-image)
+ "A @code{manifest} or @code{operating-system} record that will be lowered
+into an OCI compatible tarball. Otherwise this field's value can be a gexp
+or a file-like object that evaluates to an OCI compatible tarball.")
+ (pack-options
+ (list '())
+ "An optional set of keyword arguments that will be passed to the
+@code{docker-image} procedure from @code{guix scripts pack}. They can be used
+to replicate @command{guix pack} behavior:
+
+@lisp
+(oci-image
+ (repository \"guile\")
+ (tag \"3\")
+ (manifest (specifications->manifest '(\"guile\")))
+ (pack-options
+ '(#:symlinks ((\"/bin/guile\" -> \"bin/guile\"))
+ #:max-layers 2)))
+@end lisp
+
+If the @code{value} field is an @code{operating-system} record, this field's
+value will be ignored.")
+ (system
+ (maybe-string)
+ "Attempt to build for a given system, e.g. \"i686-linux\"")
+ (target
+ (maybe-string)
+ "Attempt to cross-build for a given triple, e.g. \"aarch64-linux-gnu\"")
+ (grafts?
+ (boolean #f)
+ "Whether to allow grafting or not in the pack build."))
+
+(define-configuration/no-serialization oci-container-configuration
+ (user
+ (string "oci-container")
+ "The user under whose authority docker commands will be run.")
+ (group
+ (string "docker")
+ "The group under whose authority docker commands will be run.")
+ (command
+ (list-of-strings '())
+ "Overwrite the default command (@code{CMD}) of the image.")
+ (entrypoint
+ (maybe-string)
+ "Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.")
+ (host-environment
+ (list '())
+ "Set environment variables in the host environment where @command{docker run}
+is invoked. This is especially useful to pass secrets from the host to the
+container without having them on the @command{docker run}'s command line: by
+setting the @code{MYSQL_PASSWORD} on the host and by passing
+@code{--env MYSQL_PASSWORD} through the @code{extra-arguments} field, it is
+possible to securely set values in the container environment. This field's
+value can be a list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"LANGUAGE\" . \"eo:ca:eu\")
+ \"JAVA_HOME=/opt/java\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to @code{make-forkexec-constructor}."
+ (sanitizer oci-sanitize-host-environment))
+ (environment
+ (list '())
+ "Set environment variables inside the container. This can be a list of pairs
+or strings, even mixed:
+
+@lisp
+(list '(\"LANGUAGE\" . \"eo:ca:eu\")
+ \"JAVA_HOME=/opt/java\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-environment))
+ (image
+ (string-or-oci-image)
+ "The image used to build the container. It can be a string or an
+@code{oci-image} record. Strings are resolved by the Docker
+Engine, and follow the usual format
+@code{myregistry.local:5000/testing/test-image:tag}.")
+ (provision
+ (maybe-string)
+ "Set the name of the provisioned Shepherd service.")
+ (requirement
+ (list-of-symbols '())
+ "Set additional Shepherd services dependencies to the provisioned Shepherd
+service.")
+ (log-file
+ (maybe-string)
+ "When @code{log-file} is set, it names the file to which the service’s
+standard output and standard error are redirected. @code{log-file} is created
+if it does not exist, otherwise it is appended to.")
+ (auto-start?
+ (boolean #t)
+ "Whether this service should be started automatically by the Shepherd. If it
+is @code{#f} the service has to be started manually with @command{herd start}.")
+ (respawn?
+ (boolean #f)
+ "Whether to restart the service when it stops, for instance when the
+underlying process dies.")
+ (shepherd-actions
+ (list '())
+ "This is a list of @code{shepherd-action} records defining actions supported
+by the service."
+ (sanitizer oci-sanitize-shepherd-actions))
+ (network
+ (maybe-string)
+ "Set a Docker network for the spawned container.")
+ (ports
+ (list '())
+ "Set the port or port ranges to expose from the spawned container. This can
+be a list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"8080\" . \"80\")
+ \"10443:443\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-ports))
+ (volumes
+ (list '())
+ "Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"/root/data/grafana\" . \"/var/lib/grafana\")
+ \"/gnu/store:/gnu/store\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-volumes))
+ (container-user
+ (maybe-string)
+ "Set the current user inside the spawned container. You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#user,upstream}
+documentation for semantics.")
+ (workdir
+ (maybe-string)
+ "Set the current working for the spawned Shepherd service.
+You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#workdir,upstream}
+documentation for semantics.")
+ (extra-arguments
+ (list '())
+ "A list of strings, gexps or file-like objects that will be directly passed
+to the @command{docker run} invokation."
+ (sanitizer oci-sanitize-extra-arguments)))
+
+(define oci-container-configuration->options
+ (lambda (config)
+ (let ((entrypoint
+ (oci-container-configuration-entrypoint config))
+ (network
+ (oci-container-configuration-network config))
+ (user
+ (oci-container-configuration-container-user config))
+ (workdir
+ (oci-container-configuration-workdir config)))
+ (apply append
+ (filter (compose not unspecified?)
+ `(,(if (maybe-value-set? entrypoint)
+ `("--entrypoint" ,entrypoint)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "--env" spec))
+ (oci-container-configuration-environment config))
+ ,(if (maybe-value-set? network)
+ `("--network" ,network)
+ '())
+ ,(if (maybe-value-set? user)
+ `("--user" ,user)
+ '())
+ ,(if (maybe-value-set? workdir)
+ `("--workdir" ,workdir)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "-p" spec))
+ (oci-container-configuration-ports config))
+ ,(append-map
+ (lambda (spec)
+ (list "-v" spec))
+ (oci-container-configuration-volumes config))))))))
+
+(define* (get-keyword-value args keyword #:key (default #f))
+ (let ((kv (memq keyword args)))
+ (if (and kv (>= (length kv) 2))
+ (cadr kv)
+ default)))
+
+(define (lower-operating-system os target system)
+ (mlet* %store-monad
+ ((tarball
+ (lower-object
+ (system-image (os->image os #:type docker-image-type))
+ system
+ #:target target)))
+ (return tarball)))
+
+(define (lower-manifest name image target system)
+ (define value (oci-image-value image))
+ (define options (oci-image-pack-options image))
+ (define image-reference
+ (oci-image-reference image))
+ (define image-tag
+ (let* ((extra-options
+ (get-keyword-value options #:extra-options))
+ (image-tag-option
+ (and extra-options
+ (get-keyword-value extra-options #:image-tag))))
+ (if image-tag-option
+ '()
+ `(#:extra-options (#:image-tag ,image-reference)))))
+
+ (mlet* %store-monad
+ ((_ (set-grafting
+ (oci-image-grafts? image)))
+ (guile (set-guile-for-build (default-guile)))
+ (profile
+ (profile-derivation value
+ #:target target
+ #:system system
+ #:hooks '()
+ #:locales? #f))
+ (tarball (apply pack:docker-image
+ `(,name ,profile
+ ,@options
+ ,@image-tag
+ #:localstatedir? #t))))
+ (return tarball)))
+
+(define (lower-oci-image name image)
+ (define value (oci-image-value image))
+ (define image-target (oci-image-target image))
+ (define image-system (oci-image-system image))
+ (define target
+ (if (maybe-value-set? image-target)
+ image-target
+ (%current-target-system)))
+ (define system
+ (if (maybe-value-set? image-system)
+ image-system
+ (%current-system)))
+ (with-store store
+ (run-with-store store
+ (match value
+ ((? manifest? value)
+ (lower-manifest name image target system))
+ ((? operating-system? value)
+ (lower-operating-system value target system))
+ ((or (? gexp? value)
+ (? file-like? value))
+ value)
+ (_
+ (raise
+ (formatted-message
+ (G_ "oci-image value must contain only manifest,
+operating-system, gexp or file-like records but ~a was found")
+ value))))
+ #:target target
+ #:system system)))
+
+(define (%oci-image-loader name image tag)
+ (let ((docker (file-append docker-cli "/bin/docker"))
+ (tarball (lower-oci-image name image)))
+ (with-imported-modules '((guix build utils))
+ (program-file (format #f "~a-image-loader" name)
+ #~(begin
+ (use-modules (guix build utils)
+ (ice-9 popen)
+ (ice-9 rdelim))
+
+ (format #t "Loading image for ~a from ~a...~%" #$name #$tarball)
+ (define line
+ (read-line
+ (open-input-pipe
+ (string-append #$docker " load -i " #$tarball))))
+
+ (unless (or (eof-object? line)
+ (string-null? line))
+ (format #t "~a~%" line)
+ (let ((repository&tag
+ (string-drop line
+ (string-length
+ "Loaded image: "))))
+
+ (invoke #$docker "tag" repository&tag #$tag)
+ (format #t "Tagged ~a with ~a...~%" #$tarball #$tag))))))))
+
+(define (oci-container-shepherd-service config)
+ (define (guess-name name image)
+ (if (maybe-value-set? name)
+ name
+ (string-append "docker-"
+ (basename
+ (if (string? image)
+ (first (string-split image #\:))
+ (oci-image-repository image))))))
+
+ (let* ((docker (file-append docker-cli "/bin/docker"))
+ (actions (oci-container-configuration-shepherd-actions config))
+ (auto-start?
+ (oci-container-configuration-auto-start? config))
+ (user (oci-container-configuration-user config))
+ (group (oci-container-configuration-group config))
+ (host-environment
+ (oci-container-configuration-host-environment config))
+ (command (oci-container-configuration-command config))
+ (log-file (oci-container-configuration-log-file config))
+ (provision (oci-container-configuration-provi
This message was truncated. Download the full message here.
P
P
paul wrote 24 hours ago
Re: OCI provisioning service
(address . 76081@debbugs.gnu.org)
e467b1c6-a54b-400b-97a6-1a7b4082686b@autistici.org
I'm sending a v3 fixing a bug in the merge algorithm for volumes and
networks.

On 2/9/25 20:14, paul wrote:
Toggle quote (26 lines)
> Hi,
>
> I'm about to send a v2. v2 compared to the first revision features:
>
>
> - it actually compiles all the times :) (rev 1 referenced oci-image
> too early for it to be working and generated a compile time error, if
> you recompiled it sometimes went away so I thought it was a problem of
> my setup. CI caught this)
> - it allows more values to be overridden by eventual users of the
> Scheme API
> - it allows passing extra arguments directly after each podman or
> docker invokation, allowing for example for overriding podman --root
> and similar options.
>
> All of these tests should pass:
>
> guix shell -D guix -CPW -- make check-system TESTS="oci-container
> oci-service-rootless-podman docker docker-system rootless-podman
> oci-service-docker"
>
>
> Thank you for your work,
>
> giacomo
>
G
G
Giacomo Leidi wrote 24 hours ago
[PATCH v3 1/4] services: rootless-podman: Use login shell.
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
89778533f32ad1388f03414e884fff10f74ef379.1739133548.git.goodoldpaul@autistici.org
This commit allows for having PATH set when changing the owner of
/sys/fs/group.

* gnu/services/containers.scm (crgroups-fs-owner): Use login shell.

Change-Id: I9510c637a5332325e05ca5ebc9dfd4de32685c50
---
gnu/services/containers.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (17 lines)
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index 19d35ccbcb6..dc66ac4f967 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -140,7 +140,7 @@ (define (cgroups-fs-owner-entrypoint config)
(rootless-podman-configuration-group-name config))
(program-file "cgroups2-fs-owner-entrypoint"
#~(system*
- (string-append #+bash-minimal "/bin/bash") "-c"
+ (string-append #+bash-minimal "/bin/bash") "-l" "-c"
(string-append "echo Setting /sys/fs/cgroup "
"group ownership to " #$group " && chown -v "
"root:" #$group " /sys/fs/cgroup && "

base-commit: 5a897c5c95a81278b044c18d962d3bd83131ba06
--
2.48.1
G
G
Giacomo Leidi wrote 24 hours ago
[PATCH v3 3/4] services: Add oci-service-type.
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
5fa06184f4690fd44c553972dad855feaf7491f5.1739133548.git.goodoldpaul@autistici.org
This patch implements a generalization of the
oci-container-service-type, which consequently is made deprecated. The
oci-service-type, in addition to all the features from the
oci-container-service-type, can now provision OCI networks and volumes.
It only handles OCI objects creation, the user is supposed to handle
state once the objects are provsioned.

It currently supports two different OCI runtimes: Docker and rootless
Podman. Both runtimes are tested to make sure provisioned containers
can connect to each other through provisioned networks and can
read/write data with provisioned volumes.

At last the Scheme API is thought to facilitate the implementation of a
Guix Home service in the future.

* gnu/services/containers.scm (%oci-supported-runtimes): New variable;
(oci-runtime-cli): new variable;
(oci-runtime-name): new variable;
(oci-network-configuration): new variable;
(oci-volume-configuration): new variable;
(oci-configuration): new variable;
(oci-extension): new variable;
(oci-networks-shepherd-name): new variable;
(oci-service-type): new variable;
(oci-state->shepherd-services): new variable.
* doc/guix.texi: Document it.
* gnu/tests/containers.scm: Test it.
* gnu/services/docker.scm: Deprecate the oci-container-service-type.

Change-Id: I656b3db85832e42d53072fcbfb91d1226f39ef38
---
doc/guix.texi | 299 ++++++++--
gnu/services/containers.scm | 1086 +++++++++++++++++++++++++++++++----
gnu/services/docker.scm | 37 +-
gnu/tests/containers.scm | 999 +++++++++++++++++++++++++++++++-
4 files changed, 2225 insertions(+), 196 deletions(-)

Toggle diff (456 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index bb5f29277fb..8f66b35297b 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41778,59 +41778,159 @@ Miscellaneous Services
@cindex OCI-backed, Shepherd services
@subsubheading OCI backed services
-Should you wish to manage your Docker containers with the same consistent
-interface you use for your other Shepherd services,
-@var{oci-container-service-type} is the tool to use: given an
-@acronym{Open Container Initiative, OCI} container image, it will run it in a
+Should you wish to manage your @acronym{Open Container Initiative, OCI} containers
+with the same consistent interface you use for your other Shepherd services,
+@var{oci-service-type} is the tool to use: given an
+OCI container image, it will run it in a
Shepherd service. One example where this is useful: it lets you run services
-that are available as Docker/OCI images but not yet packaged for Guix.
+that are available as OCI images but not yet packaged for Guix.
-@defvar oci-container-service-type
+@defvar oci-service-type
-This is a thin wrapper around Docker's CLI that executes OCI images backed
+This is a thin wrapper around Docker's or Podman's CLI that executes OCI images backed
processes as Shepherd Services.
@lisp
-(service oci-container-service-type
- (list
- (oci-container-configuration
- (network "host")
- (image
- (oci-image
- (repository "guile")
- (tag "3")
- (value (specifications->manifest '("guile")))
- (pack-options '(#:symlinks (("/bin/guile" -> "bin/guile"))
- #:max-layers 2))))
- (entrypoint "/bin/guile")
- (command
- '("-c" "(display \"hello!\n\")")))
- (oci-container-configuration
- (image "prom/prometheus")
- (ports
- '(("9000" . "9000")
- ("9090" . "9090"))))
- (oci-container-configuration
- (image "grafana/grafana:10.0.1")
- (network "host")
- (volumes
- '("/var/lib/grafana:/var/lib/grafana")))))
+(simple-service 'oci-provisioning
+ oci-service-type
+ (oci-extension
+ (networks
+ (list
+ (oci-network-configuration (name "monitoring"))))
+ (containers
+ (list
+ (oci-container-configuration
+ (network "monitoring")
+ (image
+ (oci-image
+ (repository "guile")
+ (tag "3")
+ (value (specifications->manifest '("guile")))
+ (pack-options '(#:symlinks (("/bin/guile" -> "bin/guile"))
+ #:max-layers 2))))
+ (entrypoint "/bin/guile")
+ (command
+ '("-c" "(display \"hello!\n\")")))
+ (oci-container-configuration
+ (image "prom/prometheus")
+ (network "host")
+ (ports
+ '(("9000" . "9000")
+ ("9090" . "9090"))))
+ (oci-container-configuration
+ (image "grafana/grafana:10.0.1")
+ (network "host")
+ (volumes
+ '("/var/lib/grafana:/var/lib/grafana")))))))
@end lisp
In this example three different Shepherd services are going to be added to the
system. Each @code{oci-container-configuration} record translates to a
-@code{docker run} invocation and its fields directly map to options. You can
-refer to the
-@url{https://docs.docker.com/engine/reference/commandline/run,upstream}
-documentation for the semantics of each value. If the images are not found,
-they will be
-@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The
+@command{docker run} or @command{podman run} invocation and its fields directly
+map to options. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html,Podman}
+upstream documentation for semantics of each value. If the images are not found,
+they will be pulled. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/pull/,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-pull.1.html,Podman}
+upstream documentation for semantics. The
services with @code{(network "host")} are going to be attached to the
host network and are supposed to behave like native processes with regard to
networking.
@end defvar
+@c %start of fragment
+
+@deftp {Data Type} oci-configuration
+Available @code{oci-configuration} fields are:
+
+@table @asis
+@item @code{runtime} (default: @code{'docker}) (type: symbol)
+The OCI runtime to use to run commands. It can be either @code{'docker} or
+@code{'podman}.
+
+@item @code{runtime-cli} (type: maybe-package)
+The OCI runtime command line to be installed in the system profile and used
+to provision OCI resources. When unset it will default to @code{docker-cli}
+package for the @code{'docker} runtime or to @code{podman} package for the
+@code{'podman} runtime.
+
+@item @code{runtime-extra-arguments} (default: @code{'()}) (type: list)
+A list of strings, gexps or file-like objects that will be placed
+after each @command{docker} or @command{podman} invokation.
+
+@item @code{user} (default: @code{"oci-container"}) (type: string)
+The user name under whose authority OCI commands will be run.
+
+@item @code{group} (default: @code{"docker"}) (type: string)
+The group name under whose authority OCI commands will be run. When
+using the @code{'podman} OCI runtime, this field will be ignored and the
+default group of the user configured in the @code{user} field will be used.
+
+@item @code{subuids-range} (type: maybe-subid-range)
+An optional @code{subid-range} record allocating subuids for the user from
+the @code{user} field. When unset, with the rootless Podman OCI runtime, it
+defaults to @code{(subid-range (name "oci-container"))}.
+
+@item @code{subgids-range} (type: maybe-subid-range)
+An optional @code{subid-range} record allocating subgids for the user from
+the @code{user} field. When unset, with the rootless Podman OCI runtime, it
+defaults to @code{(subid-range (name "oci-container"))}.
+
+@item @code{containers} (default: @code{'()}) (type: list-of-oci-containers)
+The list of @code{oci-container-configuration} records representing the
+containers to provision. Most users are supposed not to use this field and use
+the @code{oci-extension} record instead.
+
+@item @code{networks} (default: @code{'()}) (type: list-of-oci-networks)
+The list of @code{oci-network-configuration} records representing the
+containers to provision. Most users are supposed not to use this field and use
+the @code{oci-extension} record instead.
+
+@item @code{volumes} (default: @code{'()}) (type: list-of-oci-volumes)
+The list of @code{oci-volumes-configuration} records representing the
+containers to provision. Most users are supposed not to use this field and use
+the @code{oci-extension} record instead.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+When true, additional output will be printed, allowing to better follow the
+flow of execution.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} oci-extension
+Available @code{oci-extension} fields are:
+
+@table @asis
+@item @code{containers} (default: @code{'()}) (type: list-of-oci-containers)
+The list of @code{oci-container-configuration} records representing the
+containers to provision.
+
+@item @code{networks} (default: @code{'()}) (type: list-of-oci-networks)
+The list of @code{oci-network-configuration} records representing the
+containers to provision.
+
+@item @code{volumes} (default: @code{'()}) (type: list-of-oci-volumes)
+The list of @code{oci-volumes-configuration} records representing the
+containers to provision.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+
@c %start of fragment
@deftp {Data Type} oci-container-configuration
@@ -41850,16 +41950,16 @@ Miscellaneous Services
Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.
@item @code{host-environment} (default: @code{'()}) (type: list)
-Set environment variables in the host environment where @command{docker
-run} is invoked. This is especially useful to pass secrets from the
-host to the container without having them on the @command{docker run}'s
-command line: by setting the @code{MYSQL_PASSWORD} on the host and by passing
+Set environment variables in the host environment where @command{docker run}
+or @command{podman run} are invoked. This is especially useful to pass secrets
+from the host to the container without having them on the OCI runtime command line,
+for example: by setting the @code{MYSQL_PASSWORD} on the host and by passing
@code{--env MYSQL_PASSWORD} through the @code{extra-arguments} field, it is
possible to securely set values in the container environment. This field's
value can be a list of pairs or strings, even mixed:
@lisp
-(list '("LANGUAGE\" . "eo:ca:eu")
+(list '("LANGUAGE" . "eo:ca:eu")
"JAVA_HOME=/opt/java")
@end lisp
@@ -41867,22 +41967,24 @@ Miscellaneous Services
directly to @code{make-forkexec-constructor}.
@item @code{environment} (default: @code{'()}) (type: list)
-Set environment variables. This can be a list of pairs or strings, even mixed:
+Set environment variables inside the container. This can be a list of pairs
+or strings, even mixed:
@lisp
(list '("LANGUAGE" . "eo:ca:eu")
"JAVA_HOME=/opt/java")
@end lisp
-Pair members can be strings, gexps or file-like objects.
-Strings are passed directly to the Docker CLI. You can refer to the
-@uref{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
-documentation for semantics.
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the OCI runtime CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#env-e-env,Podman}
+upstream documentation for semantics.
@item @code{image} (type: string-or-oci-image)
The image used to build the container. It can be a string or an
-@code{oci-image} record. Strings are resolved by the Docker Engine, and
-follow the usual format
+@code{oci-image} record. Strings are resolved by the OCI runtime,
+and follow the usual format
@code{myregistry.local:5000/testing/test-image:tag}.
@item @code{provision} (default: @code{""}) (type: string)
@@ -41910,7 +42012,7 @@ Miscellaneous Services
by the service.
@item @code{network} (default: @code{""}) (type: string)
-Set a Docker network for the spawned container.
+Set an OCI network for the spawned container.
@item @code{ports} (default: @code{'()}) (type: list)
Set the port or port ranges to expose from the spawned container. This can be a
@@ -41921,10 +42023,11 @@ Miscellaneous Services
"10443:443")
@end lisp
-Pair members can be strings, gexps or file-like objects.
-Strings are passed directly to the Docker CLI. You can refer to the
-@uref{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
-documentation for semantics.
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the OCI runtime CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#publish-p-ip-hostport-containerport-protocol,Podman}
+upstream documentation for semantics.
@item @code{volumes} (default: @code{'()}) (type: list)
Set volume mappings for the spawned container. This can be a
@@ -41935,25 +42038,95 @@ Miscellaneous Services
"/gnu/store:/gnu/store")
@end lisp
-Pair members can be strings, gexps or file-like objects.
-Strings are passed directly to the Docker CLI. You can refer to the
-@uref{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
-documentation for semantics.
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the OCI runtime CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#volume-v-source-volume-host-dir-container-dir-options,Podman}
+upstream documentation for semantics.
@item @code{container-user} (default: @code{""}) (type: string)
Set the current user inside the spawned container. You can refer to the
-@url{https://docs.docker.com/engine/reference/run/#user,upstream}
-documentation for semantics.
+@url{https://docs.docker.com/engine/reference/run/#user,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#user-u-user-group,Podman}
+upstream documentation for semantics.
@item @code{workdir} (default: @code{""}) (type: string)
Set the current working directory for the spawned Shepherd service.
You can refer to the
-@url{https://docs.docker.com/engine/reference/run/#workdir,upstream}
-documentation for semantics.
+@url{https://docs.docker.com/engine/reference/run/#workdir,Docker}
+or @url{https://docs.podman.io/en/stable/markdown/podman-run.1.html#workdir-w-dir,Podman}
+upstream documentation for semantics.
+
+@item @code{extra-arguments} (default: @code{'()}) (type: list)
+A list of strings, gexps or file-like objects that will be directly passed
+to the @command{docker run} or @command{podman run} invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} oci-network-configuration
+Available @code{oci-network-configuration} fields are:
+
+@table @asis
+@item @code{name} (type: string)
+The name of the OCI network to provision.
+
+@item @code{driver} (type: maybe-string)
+The driver to manage the network.
+
+@item @code{gateway} (type: maybe-string)
+IPv4 or IPv6 gateway for the subnet.
+
+@item @code{internal?} (default: @code{#f}) (type: boolean)
+Restrict external access to the network
+
+@item @code{ip-range} (type: maybe-string)
+Allocate container ip from a sub-range in CIDR format.
+
+@item @code{ipam-driver} (type: maybe-string)
+IP Address Management Driver.
+
+@item @code{ipv6?} (default: @code{#f}) (type: boolean)
+Enable IPv6 networking.
+
+@item @code{subnet} (type: maybe-string)
+Subnet in CIDR format that represents a network segment.
+
+@item @code{labels} (default: @code{'()}) (type: list)
+The list of labels that will be used to tag the current volume.
+
+@item @code{extra-arguments} (default: @code{'()}) (type: list)
+A list of strings, gexps or file-like objects that will be directly
+passed to the runtime invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} oci-volume-configuration
+Available @code{oci-volume-configuration} fields are:
+
+@table @asis
+@item @code{name} (type: string)
+The name of the OCI volume to provision.
+
+@item @code{labels} (default: @code{'()}) (type: list)
+The list of labels that will be used to tag the current volume.
@item @code{extra-arguments} (default: @code{'()}) (type: list)
A list of strings, gexps or file-like objects that will be directly
-passed to the @command{docker run} invocation.
+passed to the runtime invokation.
@end table
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index 50bf6c05549..12c509c07ab 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -41,6 +41,7 @@ (define-module (gnu services containers)
#:use-module ((guix scripts pack) #:prefix pack:)
#:use-module (guix store)
#:use-module (srfi srfi-1)
+ #:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (rootless-podman-configuration
rootless-podman-configuration?
@@ -96,8 +97,72 @@ (define-module (gnu services containers)
oci-container-configuration-workdir
oci-container-configuration-extra-arguments
+ list-of-oci-containers?
+ list-of-oci-networks?
+ list-of-oci-volumes?
+
+ %oci-supported-runtimes
+ oci-sanitize-runtime
+ oci-runtime-system-environment
+ oci-runtime-system-extra-arguments
+ oci-runtime-system-group
+ oci-runtime-system-requirement
+ oci-runtime-cli
+ oci-runtime-system-cli
+ oci-runtime-name
+ oci-runtime-group
+
+ oci-network-configuration
+ oci-network-configuration?
+ oci-network-configuration-fields
+ oci-network-configuration-name
+ oci-network-configuration-driver
+ oci-network-configuration-gateway
+ oci-network-configuration-internal?
+ oci-network-configuration-ip-range
+ oci-network-configuration-ipam-driver
+ oci-network-configuration-ipv6?
+ oci-network-configuration-subnet
+ oci-network-configuration-labels
+ oci-network-configuration-extra-arguments
+
+ oci-volume-configuration
+ oci-volume-configuration?
+ oci-volume-configuration-fields
+ oci-volume-configuration-name
+ oci-volume-configuration-labels
+ oci-volume-configuration-extra-arguments
+
+ oci-configuration
+ oci-configuration?
+ oci-configuration-fields
+ oci-configuration-runtime
+ oci-configuration-runtime-cli
+ oci-configuration-runtime-extra-arguments
+ oci-configuration-user
+ oci-configuration-group
+ oci-configuration-containers
+ oci-configuration-networks
+ oci-configuration-volumes
+ oci-configuration-verbose?
+
+ oci-extension
+ oci-extension?
+ oci-extension-fields
+ oci-extension-containers
+ oci-extension-networks
+ oci-extension-vol
This message was truncated. Download the full message here.
G
G
Giacomo Leidi wrote 24 hours ago
[PATCH v3 4/4] tests: Use lower-oci-image-state in container tests.
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
de9951c72836295e9eae310c2e19b372982915b3.1739133548.git.goodoldpaul@autistici.org
This patch replaces boilerplate in container related tests with
oci-image plumbing from (gnu services containers).

* gnu/services/containers.scm: Export lower-oci-image-state.
* gnu/tests/containers.scm (%oci-tarball): New variable;
(run-rootless-podman-test): use %oci-tarball;
(build-tarball&run-rootless-podman-test): drop procedure.
* gnu/tests/docker.scm (%docker-tarball): New variable;
(build-tarball&run-docker-test): use %docker-tarball;
(%docker-system-tarball): New variable;
(build-tarball&run-docker-system-test): new procedure.

Change-Id: Iad6f0704aee188d89464c83722dea0bb7adb084a
---
gnu/services/containers.scm | 35 ++++++++-----
gnu/tests/containers.scm | 80 ++++++++++++++---------------
gnu/tests/docker.scm | 100 ++++++++++++++++++++----------------
3 files changed, 114 insertions(+), 101 deletions(-)

Toggle diff (342 lines)
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index 12c509c07ab..5979b26e6c9 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -74,6 +74,8 @@ (define-module (gnu services containers)
oci-image-system
oci-image-grafts?
+ lower-oci-image-state
+
oci-container-configuration
oci-container-configuration?
oci-container-configuration-fields
@@ -996,12 +998,9 @@ (define (lower-operating-system os target system)
#:target target)))
(return tarball)))
-(define (lower-manifest name image target system)
- "Lower IMAGE, a manifest record, into a tarball containing an OCI image."
- (define value (oci-image-value image))
- (define options (oci-image-pack-options image))
- (define image-reference
- (oci-image-reference image))
+(define (lower-manifest name value options image-reference
+ target system grafts?)
+ "Lower VALUE, a manifest record, into a tarball containing an OCI image."
(define image-tag
(let* ((extra-options
(get-keyword-value options #:extra-options))
@@ -1013,8 +1012,7 @@ (define (lower-manifest name image target system)
`(#:extra-options (#:image-tag ,image-reference)))))
(mlet* %store-monad
- ((_ (set-grafting
- (oci-image-grafts? image)))
+ ((_ (set-grafting grafts?))
(guile (set-guile-for-build (default-guile)))
(profile
(profile-derivation value
@@ -1029,11 +1027,8 @@ (define (lower-manifest name image target system)
#:localstatedir? #t))))
(return tarball)))
-(define (lower-oci-image name image)
- "Lower IMAGE, a oci-image record, into a tarball containing an OCI image."
- (define value (oci-image-value image))
- (define image-target (oci-image-target image))
- (define image-system (oci-image-system image))
+(define (lower-oci-image-state name value options reference
+ image-target image-system grafts?)
(define target
(if (maybe-value-set? image-target)
image-target
@@ -1046,7 +1041,8 @@ (define (lower-oci-image name image)
(run-with-store store
(match value
((? manifest? value)
- (lower-manifest name image target system))
+ (lower-manifest name value options reference
+ target system grafts?))
((? operating-system? value)
(lower-operating-system value target system))
((or (? gexp? value)
@@ -1061,6 +1057,17 @@ (define (lower-oci-image name image)
#:target target
#:system system)))
+(define (lower-oci-image name image)
+ "Lower IMAGE, a oci-image record, into a tarball containing an OCI image."
+ (lower-oci-image-state
+ name
+ (oci-image-value image)
+ (oci-image-pack-options image)
+ (oci-image-reference image)
+ (oci-image-target image)
+ (oci-image-system image)
+ (oci-image-grafts? image)))
+
(define* (oci-image-loader runtime-cli name image tag #:key (verbose? #f))
"Return a file-like object that, once lowered, will evaluate to a program able
to load IMAGE through RUNTIME-CLI and to tag it with TAG afterwards."
diff --git a/gnu/tests/containers.scm b/gnu/tests/containers.scm
index bac1f47bd34..1fcf6be7ace 100644
--- a/gnu/tests/containers.scm
+++ b/gnu/tests/containers.scm
@@ -69,13 +69,47 @@ (define %rootless-podman-os
(supplementary-groups '("wheel" "netdev" "cgroup"
"audio" "video")))))))
-(define (run-rootless-podman-test oci-tarball)
+(define %oci-tarball
+ (lower-oci-image-state
+ "guile-guest"
+ (packages->manifest
+ (list
+ guile-3.0 guile-json-3
+ (package
+ (name "guest-script")
+ (version "0")
+ (source #f)
+ (build-system trivial-build-system)
+ (arguments `(#:guile ,guile-3.0
+ #:builder
+ (let ((out (assoc-ref %outputs "out")))
+ (mkdir out)
+ (call-with-output-file (string-append out "/a.scm")
+ (lambda (port)
+ (display "(display \"hello world\n\")" port)))
+ #t)))
+ (synopsis "Display hello world using Guile")
+ (description "This package displays the text \"hello world\" on the
+standard output device and then enters a new line.")
+ (home-page #f)
+ (license license:public-domain))))
+ '(#:entry-point "bin/guile"
+ #:localstatedir? #t
+ #:extra-options (#:image-tag "guile-guest")
+ #:symlinks (("/bin/Guile" -> "bin/guile")
+ ("aa.scm" -> "a.scm")))
+ "guile-guest"
+ (%current-target-system)
+ (%current-system)
+ #f))
+
+(define (run-rootless-podman-test)
(define os
(marionette-operating-system
(operating-system-with-gc-roots
%rootless-podman-os
- (list oci-tarball))
+ (list %oci-tarball))
#:imported-modules '((gnu services herd)
(guix combinators))))
@@ -254,7 +288,7 @@ (define (run-rootless-podman-test oci-tarball)
(let* ((loaded (slurp ,(string-append #$podman
"/bin/podman")
"load" "-i"
- ,#$oci-tarball))
+ ,#$%oci-tarball))
(repository&tag "localhost/guile-guest:latest")
(response1 (slurp
,(string-append #$podman "/bin/podman")
@@ -307,49 +341,11 @@ (define (run-rootless-podman-test oci-tarball)
(gexp->derivation "rootless-podman-test" test))
-(define (build-tarball&run-rootless-podman-test)
- (mlet* %store-monad
- ((_ (set-grafting #f))
- (guile (set-guile-for-build (default-guile)))
- (guest-script-package ->
- (package
- (name "guest-script")
- (version "0")
- (source #f)
- (build-system trivial-build-system)
- (arguments `(#:guile ,guile-3.0
- #:builder
- (let ((out (assoc-ref %outputs "out")))
- (mkdir out)
- (call-with-output-file (string-append out "/a.scm")
- (lambda (port)
- (display "(display \"hello world\n\")" port)))
- #t)))
- (synopsis "Display hello world using Guile")
- (description "This package displays the text \"hello world\" on the
-standard output device and then enters a new line.")
- (home-page #f)
- (license license:public-domain)))
- (profile (profile-derivation (packages->manifest
- (list guile-3.0 guile-json-3
- guest-script-package))
- #:hooks '()
- #:locales? #f))
- (tarball (pack:docker-image
- "docker-pack" profile
- #:symlinks '(("/bin/Guile" -> "bin/guile")
- ("aa.scm" -> "a.scm"))
- #:extra-options
- '(#:image-tag "guile-guest")
- #:entry-point "bin/guile"
- #:localstatedir? #t)))
- (run-rootless-podman-test tarball)))
-
(define %test-rootless-podman
(system-test
(name "rootless-podman")
(description "Test rootless Podman service.")
- (value (build-tarball&run-rootless-podman-test))))
+ (value (run-rootless-podman-test))))
(define %oci-rootless-podman-os
diff --git a/gnu/tests/docker.scm b/gnu/tests/docker.scm
index 5dcf05a17e3..07edd9d5341 100644
--- a/gnu/tests/docker.scm
+++ b/gnu/tests/docker.scm
@@ -26,6 +26,7 @@ (define-module (gnu tests docker)
#:use-module (gnu system image)
#:use-module (gnu system vm)
#:use-module (gnu services)
+ #:use-module (gnu services containers)
#:use-module (gnu services dbus)
#:use-module (gnu services networking)
#:use-module (gnu services docker)
@@ -57,6 +58,40 @@ (define %docker-os
(service containerd-service-type)
(service docker-service-type)))
+(define %docker-tarball
+ (lower-oci-image-state
+ "guile-guest"
+ (packages->manifest
+ (list
+ guile-3.0 guile-json-3
+ (package
+ (name "guest-script")
+ (version "0")
+ (source #f)
+ (build-system trivial-build-system)
+ (arguments `(#:guile ,guile-3.0
+ #:builder
+ (let ((out (assoc-ref %outputs "out")))
+ (mkdir out)
+ (call-with-output-file (string-append out "/a.scm")
+ (lambda (port)
+ (display "(display \"hello world\n\")" port)))
+ #t)))
+ (synopsis "Display hello world using Guile")
+ (description "This package displays the text \"hello world\" on the
+standard output device and then enters a new line.")
+ (home-page #f)
+ (license license:public-domain))))
+ '(#:entry-point "bin/guile"
+ #:localstatedir? #t
+ #:extra-options (#:image-tag "guile-guest")
+ #:symlinks (("/bin/Guile" -> "bin/guile")
+ ("aa.scm" -> "a.scm")))
+ "guile-guest"
+ (%current-target-system)
+ (%current-system)
+ #f))
+
(define (run-docker-test docker-tarball)
"Load DOCKER-TARBALL as Docker image and run it in a Docker container,
inside %DOCKER-OS."
@@ -173,40 +208,7 @@ (define (run-docker-test docker-tarball)
(gexp->derivation "docker-test" test))
(define (build-tarball&run-docker-test)
- (mlet* %store-monad
- ((_ (set-grafting #f))
- (guile (set-guile-for-build (default-guile)))
- (guest-script-package ->
- (package
- (name "guest-script")
- (version "0")
- (source #f)
- (build-system trivial-build-system)
- (arguments `(#:guile ,guile-3.0
- #:builder
- (let ((out (assoc-ref %outputs "out")))
- (mkdir out)
- (call-with-output-file (string-append out "/a.scm")
- (lambda (port)
- (display "(display \"hello world\n\")" port)))
- #t)))
- (synopsis "Display hello world using Guile")
- (description "This package displays the text \"hello world\" on the
-standard output device and then enters a new line.")
- (home-page #f)
- (license license:public-domain)))
- (profile (profile-derivation (packages->manifest
- (list guile-3.0 guile-json-3
- guest-script-package))
- #:hooks '()
- #:locales? #f))
- (tarball (pack:docker-image
- "docker-pack" profile
- #:symlinks '(("/bin/Guile" -> "bin/guile")
- ("aa.scm" -> "a.scm"))
- #:entry-point "bin/guile"
- #:localstatedir? #t)))
- (run-docker-test tarball)))
+ (run-docker-test %docker-tarball))
(define %test-docker
(system-test
@@ -215,8 +217,22 @@ (define %test-docker
(value (build-tarball&run-docker-test))))
+(define %docker-system-tarball
+ (lower-oci-image-state
+ "guix-system-guest"
+ (operating-system
+ (inherit (simple-operating-system))
+ ;; Use locales for a single libc to
+ ;; reduce space requirements.
+ (locale-libcs (list glibc)))
+ '()
+ "guix-system-guest"
+ (%current-target-system)
+ (%current-system)
+ #f))
+
(define (run-docker-system-test tarball)
- "Load DOCKER-TARBALL as Docker image and run it in a Docker container,
+ "Load TARBALL as Docker image and run it in a Docker container,
inside %DOCKER-OS."
(define os
(marionette-operating-system
@@ -333,21 +349,15 @@ (define (run-docker-system-test tarball)
(gexp->derivation "docker-system-test" test))
+(define (build-tarball&run-docker-system-test)
+ (run-docker-system-test %docker-system-tarball))
+
(define %test-docker-system
(system-test
(name "docker-system")
(description "Run a system image as produced by @command{guix system
docker-image} inside Docker.")
- (value (with-monad %store-monad
- (>>= (lower-object
- (system-image (os->image
- (operating-system
- (inherit (simple-operating-system))
- ;; Use locales for a single libc to
- ;; reduce space requirements.
- (locale-libcs (list glibc)))
- #:type docker-image-type)))
- run-docker-system-test)))))
+ (value (build-tarball&run-docker-system-test))))
(define %oci-os
--
2.48.1
G
G
Giacomo Leidi wrote 24 hours ago
[PATCH v3 2/4] services: oci-container-configuration: Move to (gnu services containers).
(address . 76081@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
92f19ba6c0842885735dfce653eef535360085f4.1739133548.git.goodoldpaul@autistici.org
This patch moves the oci-container-configuration and related
configuration records to (gnu services containers).
Public symbols are still exported for backwards
compatibility but since the oci-container-service-type will be
deprecated in favor of the more general oci-service-type, everything is
moved outside of the docker related module.

* gnu/services/docker.scm: Move everything related to oci-container-configuration
to...
* gnu/services/containers.scm: ...here.scm.
* gnu/tests/docker.scm: Simplify %test-oci-container test case.

Change-Id: Iae599dd5cc7442eb632f0c1b3b12f6b928397ae7
---
gnu/services/containers.scm | 549 +++++++++++++++++++++++++++++++++-
gnu/services/docker.scm | 577 +++---------------------------------
gnu/tests/docker.scm | 99 +++----
3 files changed, 625 insertions(+), 600 deletions(-)

Toggle diff (534 lines)
diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index dc66ac4f967..50bf6c05549 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;; Copyright © 2024, 2025 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -17,19 +17,31 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (gnu services containers)
+ #:use-module (gnu image)
+ #:use-module (gnu packages admin)
#:use-module (gnu packages bash)
#:use-module (gnu packages containers)
+ #:use-module (gnu packages docker)
#:use-module (gnu packages file-systems)
#:use-module (gnu services)
#:use-module (gnu services base)
#:use-module (gnu services configuration)
#:use-module (gnu services shepherd)
+ #:use-module (gnu system)
#:use-module (gnu system accounts)
+ #:use-module (gnu system image)
#:use-module (gnu system shadow)
#:use-module (gnu system pam)
+ #:use-module (guix diagnostics)
#:use-module (guix gexp)
+ #:use-module (guix i18n)
+ #:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix profiles)
+ #:use-module ((guix scripts pack) #:prefix pack:)
+ #:use-module (guix store)
#:use-module (srfi srfi-1)
+ #:use-module (ice-9 match)
#:export (rootless-podman-configuration
rootless-podman-configuration?
rootless-podman-configuration-fields
@@ -48,7 +60,44 @@ (define-module (gnu services containers)
rootless-podman-shepherd-services
rootless-podman-service-etc
- rootless-podman-service-type))
+ rootless-podman-service-type
+
+ oci-image
+ oci-image?
+ oci-image-fields
+ oci-image-repository
+ oci-image-tag
+ oci-image-value
+ oci-image-pack-options
+ oci-image-target
+ oci-image-system
+ oci-image-grafts?
+
+ oci-container-configuration
+ oci-container-configuration?
+ oci-container-configuration-fields
+ oci-container-configuration-user
+ oci-container-configuration-group
+ oci-container-configuration-command
+ oci-container-configuration-entrypoint
+ oci-container-configuration-host-environment
+ oci-container-configuration-environment
+ oci-container-configuration-image
+ oci-container-configuration-provision
+ oci-container-configuration-requirement
+ oci-container-configuration-log-file
+ oci-container-configuration-auto-start?
+ oci-container-configuration-respawn?
+ oci-container-configuration-shepherd-actions
+ oci-container-configuration-network
+ oci-container-configuration-ports
+ oci-container-configuration-volumes
+ oci-container-configuration-container-user
+ oci-container-configuration-workdir
+ oci-container-configuration-extra-arguments
+
+ oci-container-shepherd-service
+ %oci-container-accounts))
(define (gexp-or-string? value)
(or (gexp? value)
@@ -188,7 +237,7 @@ (define (rootless-podman-cgroups-limits-service config)
rootless-podman-shared-root-fs))
(one-shot? #t)
(documentation
- "Allow setting cgroups limits: cpu, cpuset, memory and
+ "Allow setting cgroups limits: cpu, cpuset, io, memory and
pids.")
(start
#~(make-forkexec-constructor
@@ -242,3 +291,497 @@ (define rootless-podman-service-type
(default-value (rootless-podman-configuration))
(description
"This service configures rootless @code{podman} on the Guix System.")))
+
+
+;;;
+;;; OCI container.
+;;;
+
+(define (oci-sanitize-pair pair delimiter)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (match pair
+ (((? valid? key) . (? valid? value))
+ #~(string-append #$key #$delimiter #$value))
+ (_
+ (raise
+ (formatted-message
+ (G_ "pair members must contain only strings, gexps or file-like objects
+but ~a was found")
+ pair)))))
+
+(define (oci-sanitize-mixed-list name value delimiter)
+ (map
+ (lambda (el)
+ (cond ((string? el) el)
+ ((pair? el) (oci-sanitize-pair el delimiter))
+ (else
+ (raise
+ (formatted-message
+ (G_ "~a members must be either a string or a pair but ~a was
+found!")
+ name el)))))
+ value))
+
+(define (oci-sanitize-host-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "host-environment" value "="))
+
+(define (oci-sanitize-environment value)
+ ;; Expected spec format:
+ ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java")
+ (oci-sanitize-mixed-list "environment" value "="))
+
+(define (oci-sanitize-ports value)
+ ;; Expected spec format:
+ ;; '(("8088" . "80") "2022:22")
+ (oci-sanitize-mixed-list "ports" value ":"))
+
+(define (oci-sanitize-volumes value)
+ ;; Expected spec format:
+ ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java")
+ (oci-sanitize-mixed-list "volumes" value ":"))
+
+(define (oci-sanitize-shepherd-actions value)
+ (map
+ (lambda (el)
+ (if (shepherd-action? el)
+ el
+ (raise
+ (formatted-message
+ (G_ "shepherd-actions may only be shepherd-action records
+but ~a was found") el))))
+ value))
+
+(define (oci-sanitize-extra-arguments value)
+ (define (valid? member)
+ (or (string? member)
+ (gexp? member)
+ (file-like? member)))
+ (map
+ (lambda (el)
+ (if (valid? el)
+ el
+ (raise
+ (formatted-message
+ (G_ "extra arguments may only be strings, gexps or file-like objects
+but ~a was found") el))))
+ value))
+
+(define (oci-image-reference image)
+ (if (string? image)
+ image
+ (string-append (oci-image-repository image)
+ ":" (oci-image-tag image))))
+
+(define (oci-lowerable-image? image)
+ (or (manifest? image)
+ (operating-system? image)
+ (gexp? image)
+ (file-like? image)))
+
+(define (string-or-oci-image? image)
+ (or (string? image)
+ (oci-image? image)))
+
+(define list-of-symbols?
+ (list-of symbol?))
+
+(define-maybe/no-serialization string)
+
+(define-configuration/no-serialization oci-image
+ (repository
+ (string)
+ "A string like @code{myregistry.local:5000/testing/test-image} that names
+the OCI image.")
+ (tag
+ (string "latest")
+ "A string representing the OCI image tag. Defaults to @code{latest}.")
+ (value
+ (oci-lowerable-image)
+ "A @code{manifest} or @code{operating-system} record that will be lowered
+into an OCI compatible tarball. Otherwise this field's value can be a gexp
+or a file-like object that evaluates to an OCI compatible tarball.")
+ (pack-options
+ (list '())
+ "An optional set of keyword arguments that will be passed to the
+@code{docker-image} procedure from @code{guix scripts pack}. They can be used
+to replicate @command{guix pack} behavior:
+
+@lisp
+(oci-image
+ (repository \"guile\")
+ (tag \"3\")
+ (manifest (specifications->manifest '(\"guile\")))
+ (pack-options
+ '(#:symlinks ((\"/bin/guile\" -> \"bin/guile\"))
+ #:max-layers 2)))
+@end lisp
+
+If the @code{value} field is an @code{operating-system} record, this field's
+value will be ignored.")
+ (system
+ (maybe-string)
+ "Attempt to build for a given system, e.g. \"i686-linux\"")
+ (target
+ (maybe-string)
+ "Attempt to cross-build for a given triple, e.g. \"aarch64-linux-gnu\"")
+ (grafts?
+ (boolean #f)
+ "Whether to allow grafting or not in the pack build."))
+
+(define-configuration/no-serialization oci-container-configuration
+ (user
+ (string "oci-container")
+ "The user under whose authority docker commands will be run.")
+ (group
+ (string "docker")
+ "The group under whose authority docker commands will be run.")
+ (command
+ (list-of-strings '())
+ "Overwrite the default command (@code{CMD}) of the image.")
+ (entrypoint
+ (maybe-string)
+ "Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.")
+ (host-environment
+ (list '())
+ "Set environment variables in the host environment where @command{docker run}
+is invoked. This is especially useful to pass secrets from the host to the
+container without having them on the @command{docker run}'s command line: by
+setting the @code{MYSQL_PASSWORD} on the host and by passing
+@code{--env MYSQL_PASSWORD} through the @code{extra-arguments} field, it is
+possible to securely set values in the container environment. This field's
+value can be a list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"LANGUAGE\" . \"eo:ca:eu\")
+ \"JAVA_HOME=/opt/java\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to @code{make-forkexec-constructor}."
+ (sanitizer oci-sanitize-host-environment))
+ (environment
+ (list '())
+ "Set environment variables inside the container. This can be a list of pairs
+or strings, even mixed:
+
+@lisp
+(list '(\"LANGUAGE\" . \"eo:ca:eu\")
+ \"JAVA_HOME=/opt/java\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-environment))
+ (image
+ (string-or-oci-image)
+ "The image used to build the container. It can be a string or an
+@code{oci-image} record. Strings are resolved by the Docker
+Engine, and follow the usual format
+@code{myregistry.local:5000/testing/test-image:tag}.")
+ (provision
+ (maybe-string)
+ "Set the name of the provisioned Shepherd service.")
+ (requirement
+ (list-of-symbols '())
+ "Set additional Shepherd services dependencies to the provisioned Shepherd
+service.")
+ (log-file
+ (maybe-string)
+ "When @code{log-file} is set, it names the file to which the service’s
+standard output and standard error are redirected. @code{log-file} is created
+if it does not exist, otherwise it is appended to.")
+ (auto-start?
+ (boolean #t)
+ "Whether this service should be started automatically by the Shepherd. If it
+is @code{#f} the service has to be started manually with @command{herd start}.")
+ (respawn?
+ (boolean #f)
+ "Whether to restart the service when it stops, for instance when the
+underlying process dies.")
+ (shepherd-actions
+ (list '())
+ "This is a list of @code{shepherd-action} records defining actions supported
+by the service."
+ (sanitizer oci-sanitize-shepherd-actions))
+ (network
+ (maybe-string)
+ "Set a Docker network for the spawned container.")
+ (ports
+ (list '())
+ "Set the port or port ranges to expose from the spawned container. This can
+be a list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"8080\" . \"80\")
+ \"10443:443\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-ports))
+ (volumes
+ (list '())
+ "Set volume mappings for the spawned container. This can be a
+list of pairs or strings, even mixed:
+
+@lisp
+(list '(\"/root/data/grafana\" . \"/var/lib/grafana\")
+ \"/gnu/store:/gnu/store\")
+@end lisp
+
+Pair members can be strings, gexps or file-like objects. Strings are passed
+directly to the Docker CLI. You can refer to the
+@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream}
+documentation for semantics."
+ (sanitizer oci-sanitize-volumes))
+ (container-user
+ (maybe-string)
+ "Set the current user inside the spawned container. You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#user,upstream}
+documentation for semantics.")
+ (workdir
+ (maybe-string)
+ "Set the current working for the spawned Shepherd service.
+You can refer to the
+@url{https://docs.docker.com/engine/reference/run/#workdir,upstream}
+documentation for semantics.")
+ (extra-arguments
+ (list '())
+ "A list of strings, gexps or file-like objects that will be directly passed
+to the @command{docker run} invokation."
+ (sanitizer oci-sanitize-extra-arguments)))
+
+(define oci-container-configuration->options
+ (lambda (config)
+ (let ((entrypoint
+ (oci-container-configuration-entrypoint config))
+ (network
+ (oci-container-configuration-network config))
+ (user
+ (oci-container-configuration-container-user config))
+ (workdir
+ (oci-container-configuration-workdir config)))
+ (apply append
+ (filter (compose not unspecified?)
+ `(,(if (maybe-value-set? entrypoint)
+ `("--entrypoint" ,entrypoint)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "--env" spec))
+ (oci-container-configuration-environment config))
+ ,(if (maybe-value-set? network)
+ `("--network" ,network)
+ '())
+ ,(if (maybe-value-set? user)
+ `("--user" ,user)
+ '())
+ ,(if (maybe-value-set? workdir)
+ `("--workdir" ,workdir)
+ '())
+ ,(append-map
+ (lambda (spec)
+ (list "-p" spec))
+ (oci-container-configuration-ports config))
+ ,(append-map
+ (lambda (spec)
+ (list "-v" spec))
+ (oci-container-configuration-volumes config))))))))
+
+(define* (get-keyword-value args keyword #:key (default #f))
+ (let ((kv (memq keyword args)))
+ (if (and kv (>= (length kv) 2))
+ (cadr kv)
+ default)))
+
+(define (lower-operating-system os target system)
+ (mlet* %store-monad
+ ((tarball
+ (lower-object
+ (system-image (os->image os #:type docker-image-type))
+ system
+ #:target target)))
+ (return tarball)))
+
+(define (lower-manifest name image target system)
+ (define value (oci-image-value image))
+ (define options (oci-image-pack-options image))
+ (define image-reference
+ (oci-image-reference image))
+ (define image-tag
+ (let* ((extra-options
+ (get-keyword-value options #:extra-options))
+ (image-tag-option
+ (and extra-options
+ (get-keyword-value extra-options #:image-tag))))
+ (if image-tag-option
+ '()
+ `(#:extra-options (#:image-tag ,image-reference)))))
+
+ (mlet* %store-monad
+ ((_ (set-grafting
+ (oci-image-grafts? image)))
+ (guile (set-guile-for-build (default-guile)))
+ (profile
+ (profile-derivation value
+ #:target target
+ #:system system
+ #:hooks '()
+ #:locales? #f))
+ (tarball (apply pack:docker-image
+ `(,name ,profile
+ ,@options
+ ,@image-tag
+ #:localstatedir? #t))))
+ (return tarball)))
+
+(define (lower-oci-image name image)
+ (define value (oci-image-value image))
+ (define image-target (oci-image-target image))
+ (define image-system (oci-image-system image))
+ (define target
+ (if (maybe-value-set? image-target)
+ image-target
+ (%current-target-system)))
+ (define system
+ (if (maybe-value-set? image-system)
+ image-system
+ (%current-system)))
+ (with-store store
+ (run-with-store store
+ (match value
+ ((? manifest? value)
+ (lower-manifest name image target system))
+ ((? operating-system? value)
+ (lower-operating-system value target system))
+ ((or (? gexp? value)
+ (? file-like? value))
+ value)
+ (_
+ (raise
+ (formatted-message
+ (G_ "oci-image value must contain only manifest,
+operating-system, gexp or file-like records but ~a was found")
+ value))))
+ #:target target
+ #:system system)))
+
+(define (%oci-image-loader name image tag)
+ (let ((docker (file-append docker-cli "/bin/docker"))
+ (tarball (lower-oci-image name image)))
+ (with-imported-modules '((guix build utils))
+ (program-file (format #f "~a-image-loader" name)
+ #~(begin
+ (use-modules (guix build utils)
+ (ice-9 popen)
+ (ice-9 rdelim))
+
+ (format #t "Loading image for ~a from ~a...~%" #$name #$tarball)
+ (define line
+ (read-line
+ (open-input-pipe
+ (string-append #$docker " load -i " #$tarball))))
+
+ (unless (or (eof-object? line)
+ (string-null? line))
+ (format #t "~a~%" line)
+ (let ((repository&tag
+ (string-drop line
+ (string-length
+ "Loaded image: "))))
+
+ (invoke #$docker "tag" repository&tag #$tag)
+ (format #t "Tagged ~a with ~a...~%" #$tarball #$tag))))))))
+
+(define (oci-container-shepherd-service config)
+ (define (guess-name name image)
+ (if (maybe-value-set? name)
+ name
+ (string-append "docker-"
+ (basename
+ (if (string? image)
+ (first (string-split image #\:))
+ (oci-image-repository image))))))
+
+ (let* ((docker (file-append docker-cli "/bin/docker"))
+ (actions (oci-container-configuration-shepherd-actions config))
+ (auto-start?
+ (oci-container-configuration-auto-start? config))
+ (user (oci-container-configuration-user config))
+ (group (oci-container-configuration-group config))
+ (host-environment
+ (oci-container-configuration-host-environment config))
+ (command (oci-container-configuration-command config))
+ (log-file (oci-container-configuration-log-file config))
+ (provision (oci-container-configuration-provi
This message was truncated. Download the full message here.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 76081@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 76081
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch