[PATCH 0/2] Add docker.

DoneSubmitted by Danny Milosavljevic.
Details
4 participants
  • Danny Milosavljevic
  • Leo Famulari
  • Ludovic Courtès
  • Meiyo Peng
Owner
unassigned
Severity
normal
D
D
Danny Milosavljevic wrote on 28 Dec 2018 11:13
(address . guix-patches@gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181228101311.30793-1-dannym@scratchpost.org
Danny Milosavljevic (2):
gnu: Add docker-engine.
gnu: Add docker-cli.

gnu/packages/docker.scm | 144 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 144 insertions(+)
D
D
Danny Milosavljevic wrote on 28 Dec 2018 11:17
[PATCH 1/2] gnu: Add docker-engine.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181228101751.676-1-dannym@scratchpost.org
* gnu/packages/docker.scm (%docker-version): New variable.
(docker-engine): New variable. Export it.
---
gnu/packages/docker.scm | 83 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 83 insertions(+)

Toggle diff (104 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..3d1a90fc7 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,12 +23,18 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
+(define %docker-version "18.09.0")
+
 (define-public python-docker-py
   (package
     (name "python-docker-py")
@@ -142,3 +148,80 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             ;(substitute* "hack/make/install-binary"
+             ; (("/binary-daemon") "/dynbinary-daemon"))
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("lvm2" ,lvm2)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 28 Dec 2018 11:17
[PATCH 2/2] gnu: Add docker-cli.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181228101751.676-2-dannym@scratchpost.org
* gnu/packages/docker.scm (docker-cli): New variable. Export it.
---
gnu/packages/docker.scm | 61 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 61 insertions(+)

Toggle diff (78 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 3d1a90fc7..caf70cbc9 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -24,8 +24,10 @@
   #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
@@ -225,3 +227,62 @@ management, secret management, configuration management, networking,
 provisioning etc.")
     (home-page "https://mobyproject.org/")
     (license license:asl2.0)))
+
+(define-public docker-cli
+  (package
+    (name "docker-cli")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/cli.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1ivisys20kphvbqlazc3bsg7pk0ykj9gjx5d4yg439x4n13jxwvb"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/docker/cli"
+       ;; TODO: Tests require a running Docker daemon.
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'setup-environment-2
+           (lambda _
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             (symlink "src/github.com/docker/cli/scripts" "./scripts")
+             (symlink "src/github.com/docker/cli/docker.Makefile" "./docker.Makefile")
+             #t))
+         (replace 'build
+           (lambda _
+             (invoke "./scripts/build/dynbinary")))
+         (replace 'check
+           (lambda* (#:key make-flags tests? #:allow-other-keys)
+             (setenv "PATH" (string-append (getcwd) "/build:" (getenv "PATH")))
+             (if tests?
+                 ;; Use the newly-built docker client for the tests.
+                 (with-directory-excursion "src/github.com/docker/cli"
+                   ;; TODO: Run test-e2e as well?
+                   (apply invoke "make" "-f" "docker.Makefile" "test-unit"
+                          (or make-flags '())))
+                 #t)))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (chdir "build")
+               (install-file (readlink "docker") out-bin)
+               (install-file "docker" out-bin)
+               #t))))))
+    (native-inputs
+     `(("go" ,go)
+       ("libltdl" ,libltdl)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Command line interface to Docker")
+    (description "This package provides a command line interface to Docker.")
+    (home-page "http://www.docker.com/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 29 Dec 2018 02:32
[PATCH v2 0/3] Add docker.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181229013245.12853-1-dannym@scratchpost.org
Danny Milosavljevic (3):
gnu: Add containerd.
gnu: Add docker-engine.
services: Add docker.

gnu/local.mk | 1 +
gnu/packages/docker.scm | 201 +++++++++++++++++++++++++++++++++++++++-
gnu/services/docker.scm | 90 ++++++++++++++++++
3 files changed, 291 insertions(+), 1 deletion(-)
create mode 100644 gnu/services/docker.scm
D
D
Danny Milosavljevic wrote on 29 Dec 2018 02:32
[PATCH v2 1/3] gnu: Add containerd.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181229013245.12853-2-dannym@scratchpost.org
* gnu/packages/docker.scm (containerd): New variable.
---
gnu/packages/docker.scm | 49 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)

Toggle diff (67 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..f4e676a9f 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,9 +23,13 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
@@ -142,3 +146,48 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public containerd
+  (package
+    (name "containerd")
+    (version "1.2.1")
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/containerd/containerd.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "16zn6p1ky3yrgn53z8h9wza53ch91fj47wj5xgz6w4c57j30f66p"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/containerd/containerd"
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'chdir
+           (lambda _
+             (chdir "src/github.com/containerd/containerd")
+             #t))
+         (replace 'build
+           (lambda* (#:key (make-flags '()) #:allow-other-keys)
+             (apply invoke "make" make-flags)))
+         (replace 'install
+           (lambda* (#:key outputs (make-flags '()) #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (apply invoke "make" (string-append "DESTDIR=" out) "install"
+                      make-flags)))))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("libseccomp" ,libseccomp)))
+    (native-inputs
+     `(("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Container runtime")
+    (description "This package provides the container daemon for Docker.
+It includes image transfer and storage, container execution and supervision,
+network attachments.")
+    (home-page "http://containerd.io/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 29 Dec 2018 02:32
[PATCH v2 2/3] gnu: Add docker-engine.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181229013245.12853-3-dannym@scratchpost.org
* gnu/packages/docker.scm (docker-engine): New variable.
(%docker-version): New variable.
---
gnu/packages/docker.scm | 152 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 151 insertions(+), 1 deletion(-)

Toggle diff (175 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index f4e676a9f..3ca2fadfd 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,15 +23,20 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
-  #:use-module (gnu packages python-web))
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages virtualization))
+
+(define %docker-version "18.09.0")
 
 (define-public python-docker-py
   (package
@@ -191,3 +196,148 @@ It includes image transfer and storage, container execution and supervision,
 network attachments.")
     (home-page "http://containerd.io/")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;(substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+             ; (("") ""))
+             (substitute* "builder/builder-next/executor_unix.go"
+              (("CommandCandidates:.*runc.*")
+               (string-append "CommandCandidates: []string{\""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"},\n")))
+             (substitute* "vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand = .*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "daemon/daemon_unix.go"
+              (("DefaultShimBinary = .*")
+               (string-append "DefaultShimBinary = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n"))
+              (("DefaultRuntimeBinary = .*")
+               (string-append "DefaultRuntimeBinary = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+              (("var defaultCommandCandidates = .*")
+               (string-append "var defaultCommandCandidates = []string{\""
+                              (assoc-ref inputs "runc") "/sbin/runc\"}")))
+             (substitute* (filter (lambda (name)
+                                    (not (string-contains name "test")))
+                                  (find-files "\\.go$"))
+              (("\"ps\"")
+               (string-append "\"" (assoc-ref inputs "procps") "/bin/ps\""))
+              ; TODO: zfs ?
+;getPlatformContainerdDaemonOpts ./cmd/dockerd/daemon_unix.go
+; TODO --init-path for docker-init
+; ./cmd/dockerd/config_unix.go InitPath
+;./daemon/config/config.go DefaultInitBinary
+              (("exec\\.LookPath\\(\"mkfs\\.xfs\"\\)")
+               (string-append "\"" (assoc-ref inputs "xfsprogs")
+                              "/bin/mkfs.xfs\""))
+              (("exec\\.LookPath\\(\"lvmdiskscan\"\\)")
+               (string-append "\"" (assoc-ref inputs "lvm2")
+                              "/sbin/lvmdiskscan\""))
+              (("exec\\.LookPath\\(\"pvdisplay\"\\)")
+               (string-append "\"" (assoc-ref inputs "lvm2")
+                              "/sbin/pvdisplay\""))
+              (("exec\\.LookPath\\(\"blkid\"\\)")
+               (string-append "\"" (assoc-ref inputs "util-linux")
+                              "/sbin/blkid\""))
+              (("exec\\.LookPath\\(\"unpigz\"\\)")
+               (string-append "\"" (assoc-ref inputs "pigz")
+                              "/bin/unpigz\""))
+              (("exec\\.LookPath\\(\"iptables\"\\)")
+               (string-append "\"" (assoc-ref inputs "iptables")
+                              "/sbin/iptables\""))
+              (("exec\\.LookPath\\(\"ip\"\\)")
+               (string-append "\"" (assoc-ref inputs "iproute2")
+                              "/sbin/ip\""))
+              ;(("LookPath") "Guix_doesnt_want_LookPath")
+             )
+             #t))
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             ;(substitute* "hack/make/install-binary"
+             ; (("/binary-daemon") "/dynbinary-daemon"))
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("containerd" ,containerd) ; for containerd-shim
+       ("runc" ,runc)
+       ("iproute2" ,iproute)
+       ("iptables" ,iptables)
+       ("pigz" ,pigz)
+       ("procps" ,procps)
+       ("util-linux" ,util-linux)
+       ("lvm2" ,lvm2)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 29 Dec 2018 02:32
[PATCH v2 3/3] services: Add docker.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181229013245.12853-4-dannym@scratchpost.org
* gnu/services/docker.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
---
gnu/local.mk | 1 +
gnu/services/docker.scm | 90 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 91 insertions(+)
create mode 100644 gnu/services/docker.scm

Toggle diff (108 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 925d955a6..f6c91dcc7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -482,6 +482,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/desktop.scm			\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
+  %D%/services/docker.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/kerberos.scm			\
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
new file mode 100644
index 000000000..e592185f8
--- /dev/null
+++ b/gnu/services/docker.scm
@@ -0,0 +1,90 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Danny Milosavljevic <dannym@scratchpost.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services docker)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services base)
+  #:use-module (gnu services dbus)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages docker)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+
+  #:export (docker-configuration
+            docker-service-type))
+
+(define-configuration docker-configuration
+  (docker
+   (package docker-engine)
+   "Docker daemon package.")
+  (containerd
+   (package containerd)
+   "containerd package."))
+
+(define %docker-accounts
+  (list (user-group (name "docker") (system? #t))))
+
+(define (%containerd-activation config)
+  (let ((state-dir "/var/lib/containerd"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+(define (%docker-activation config)
+  (%containerd-activation config)
+  (let ((state-dir "/var/lib/docker"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+;; TODO: Refactor out into its own module?  How to depend on it then?
+(define (containerd-shepherd-service config)
+  (let* ((package (docker-configuration-containerd config)))
+    (shepherd-service
+           (documentation "containerd daemon.")
+           (provision '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$package "/bin/containerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define (docker-shepherd-service config)
+  (let* ((docker (docker-configuration-docker config)))
+    (shepherd-service
+           (documentation "Docker daemon.")
+           (provision '(dockerd))
+           (requirement '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$docker "/bin/dockerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define docker-service-type
+  (service-type (name 'docker)
+		(extensions
+                 (list
+                  (service-extension activation-service-type
+                                     %docker-activation)
+                  (service-extension shepherd-root-service-type
+                                     (lambda args
+                                       (list (apply containerd-shepherd-service args)
+                                             (apply docker-shepherd-service args))))
+                  (service-extension account-service-type
+                                     (const %docker-accounts))))
+                (default-value (docker-configuration))))
D
D
Danny Milosavljevic wrote on 29 Dec 2018 02:39
[PATCH v3 0/4] Add docker.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181229013906.17705-1-dannym@scratchpost.org
Includes docker-cli now.

Danny Milosavljevic (4):
gnu: Add containerd.
gnu: Add docker-engine.
services: Add docker.
gnu: Add docker-cli.

gnu/local.mk | 1 +
gnu/packages/docker.scm | 264 +++++++++++++++++++++++++++++++++++++++-
gnu/services/docker.scm | 90 ++++++++++++++
3 files changed, 354 insertions(+), 1 deletion(-)
create mode 100644 gnu/services/docker.scm
D
D
Danny Milosavljevic wrote on 29 Dec 2018 02:39
[PATCH v3 1/4] gnu: Add containerd.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181229013906.17705-2-dannym@scratchpost.org
* gnu/packages/docker.scm (containerd): New variable.
---
gnu/packages/docker.scm | 49 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)

Toggle diff (67 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..f4e676a9f 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,9 +23,13 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
@@ -142,3 +146,48 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public containerd
+  (package
+    (name "containerd")
+    (version "1.2.1")
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/containerd/containerd.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "16zn6p1ky3yrgn53z8h9wza53ch91fj47wj5xgz6w4c57j30f66p"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/containerd/containerd"
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'chdir
+           (lambda _
+             (chdir "src/github.com/containerd/containerd")
+             #t))
+         (replace 'build
+           (lambda* (#:key (make-flags '()) #:allow-other-keys)
+             (apply invoke "make" make-flags)))
+         (replace 'install
+           (lambda* (#:key outputs (make-flags '()) #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (apply invoke "make" (string-append "DESTDIR=" out) "install"
+                      make-flags)))))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("libseccomp" ,libseccomp)))
+    (native-inputs
+     `(("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Container runtime")
+    (description "This package provides the container daemon for Docker.
+It includes image transfer and storage, container execution and supervision,
+network attachments.")
+    (home-page "http://containerd.io/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 29 Dec 2018 02:39
[PATCH v3 2/4] gnu: Add docker-engine.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181229013906.17705-3-dannym@scratchpost.org
* gnu/packages/docker.scm (docker-engine): New variable.
(%docker-version): New variable.
---
gnu/packages/docker.scm | 152 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 151 insertions(+), 1 deletion(-)

Toggle diff (175 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index f4e676a9f..3ca2fadfd 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,15 +23,20 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
-  #:use-module (gnu packages python-web))
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages virtualization))
+
+(define %docker-version "18.09.0")
 
 (define-public python-docker-py
   (package
@@ -191,3 +196,148 @@ It includes image transfer and storage, container execution and supervision,
 network attachments.")
     (home-page "http://containerd.io/")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;(substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+             ; (("") ""))
+             (substitute* "builder/builder-next/executor_unix.go"
+              (("CommandCandidates:.*runc.*")
+               (string-append "CommandCandidates: []string{\""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"},\n")))
+             (substitute* "vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand = .*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "daemon/daemon_unix.go"
+              (("DefaultShimBinary = .*")
+               (string-append "DefaultShimBinary = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n"))
+              (("DefaultRuntimeBinary = .*")
+               (string-append "DefaultRuntimeBinary = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+              (("var defaultCommandCandidates = .*")
+               (string-append "var defaultCommandCandidates = []string{\""
+                              (assoc-ref inputs "runc") "/sbin/runc\"}")))
+             (substitute* (filter (lambda (name)
+                                    (not (string-contains name "test")))
+                                  (find-files "\\.go$"))
+              (("\"ps\"")
+               (string-append "\"" (assoc-ref inputs "procps") "/bin/ps\""))
+              ; TODO: zfs ?
+;getPlatformContainerdDaemonOpts ./cmd/dockerd/daemon_unix.go
+; TODO --init-path for docker-init
+; ./cmd/dockerd/config_unix.go InitPath
+;./daemon/config/config.go DefaultInitBinary
+              (("exec\\.LookPath\\(\"mkfs\\.xfs\"\\)")
+               (string-append "\"" (assoc-ref inputs "xfsprogs")
+                              "/bin/mkfs.xfs\""))
+              (("exec\\.LookPath\\(\"lvmdiskscan\"\\)")
+               (string-append "\"" (assoc-ref inputs "lvm2")
+                              "/sbin/lvmdiskscan\""))
+              (("exec\\.LookPath\\(\"pvdisplay\"\\)")
+               (string-append "\"" (assoc-ref inputs "lvm2")
+                              "/sbin/pvdisplay\""))
+              (("exec\\.LookPath\\(\"blkid\"\\)")
+               (string-append "\"" (assoc-ref inputs "util-linux")
+                              "/sbin/blkid\""))
+              (("exec\\.LookPath\\(\"unpigz\"\\)")
+               (string-append "\"" (assoc-ref inputs "pigz")
+                              "/bin/unpigz\""))
+              (("exec\\.LookPath\\(\"iptables\"\\)")
+               (string-append "\"" (assoc-ref inputs "iptables")
+                              "/sbin/iptables\""))
+              (("exec\\.LookPath\\(\"ip\"\\)")
+               (string-append "\"" (assoc-ref inputs "iproute2")
+                              "/sbin/ip\""))
+              ;(("LookPath") "Guix_doesnt_want_LookPath")
+             )
+             #t))
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             ;(substitute* "hack/make/install-binary"
+             ; (("/binary-daemon") "/dynbinary-daemon"))
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("containerd" ,containerd) ; for containerd-shim
+       ("runc" ,runc)
+       ("iproute2" ,iproute)
+       ("iptables" ,iptables)
+       ("pigz" ,pigz)
+       ("procps" ,procps)
+       ("util-linux" ,util-linux)
+       ("lvm2" ,lvm2)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 29 Dec 2018 02:39
[PATCH v3 3/4] services: Add docker.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181229013906.17705-4-dannym@scratchpost.org
* gnu/services/docker.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
---
gnu/local.mk | 1 +
gnu/services/docker.scm | 90 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 91 insertions(+)
create mode 100644 gnu/services/docker.scm

Toggle diff (108 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 925d955a6..f6c91dcc7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -482,6 +482,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/desktop.scm			\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
+  %D%/services/docker.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/kerberos.scm			\
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
new file mode 100644
index 000000000..e592185f8
--- /dev/null
+++ b/gnu/services/docker.scm
@@ -0,0 +1,90 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Danny Milosavljevic <dannym@scratchpost.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services docker)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services base)
+  #:use-module (gnu services dbus)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages docker)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+
+  #:export (docker-configuration
+            docker-service-type))
+
+(define-configuration docker-configuration
+  (docker
+   (package docker-engine)
+   "Docker daemon package.")
+  (containerd
+   (package containerd)
+   "containerd package."))
+
+(define %docker-accounts
+  (list (user-group (name "docker") (system? #t))))
+
+(define (%containerd-activation config)
+  (let ((state-dir "/var/lib/containerd"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+(define (%docker-activation config)
+  (%containerd-activation config)
+  (let ((state-dir "/var/lib/docker"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+;; TODO: Refactor out into its own module?  How to depend on it then?
+(define (containerd-shepherd-service config)
+  (let* ((package (docker-configuration-containerd config)))
+    (shepherd-service
+           (documentation "containerd daemon.")
+           (provision '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$package "/bin/containerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define (docker-shepherd-service config)
+  (let* ((docker (docker-configuration-docker config)))
+    (shepherd-service
+           (documentation "Docker daemon.")
+           (provision '(dockerd))
+           (requirement '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$docker "/bin/dockerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define docker-service-type
+  (service-type (name 'docker)
+		(extensions
+                 (list
+                  (service-extension activation-service-type
+                                     %docker-activation)
+                  (service-extension shepherd-root-service-type
+                                     (lambda args
+                                       (list (apply containerd-shepherd-service args)
+                                             (apply docker-shepherd-service args))))
+                  (service-extension account-service-type
+                                     (const %docker-accounts))))
+                (default-value (docker-configuration))))
D
D
Danny Milosavljevic wrote on 29 Dec 2018 02:39
[PATCH v3 4/4] gnu: Add docker-cli.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181229013906.17705-5-dannym@scratchpost.org
* gnu/packages/docker.scm (docker-cli): New variable.
---
gnu/packages/docker.scm | 63 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)

Toggle diff (78 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 3ca2fadfd..cbf84aecf 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -27,6 +27,7 @@
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
@@ -341,3 +342,65 @@ management, secret management, configuration management, networking,
 provisioning etc.")
     (home-page "https://mobyproject.org/")
     (license license:asl2.0)))
+
+(define-public docker-cli
+  (package
+    (name "docker-cli")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/cli.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1ivisys20kphvbqlazc3bsg7pk0ykj9gjx5d4yg439x4n13jxwvb"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/docker/cli"
+       ;; TODO: Tests require a running Docker daemon.
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'setup-environment-2
+           (lambda _
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+
+             ;; Make build reproducible.
+             (setenv "BUILDTIME" "1970-01-01 00:00:01.000000000+00:00")
+             (symlink "src/github.com/docker/cli/scripts" "./scripts")
+             (symlink "src/github.com/docker/cli/docker.Makefile" "./docker.Makefile")
+             #t))
+         (replace 'build
+           (lambda _
+             (invoke "./scripts/build/dynbinary")))
+         (replace 'check
+           (lambda* (#:key make-flags tests? #:allow-other-keys)
+             (setenv "PATH" (string-append (getcwd) "/build:" (getenv "PATH")))
+             (if tests?
+                 ;; Use the newly-built docker client for the tests.
+                 (with-directory-excursion "src/github.com/docker/cli"
+                   ;; TODO: Run test-e2e as well?
+                   (apply invoke "make" "-f" "docker.Makefile" "test-unit"
+                          (or make-flags '())))
+                 #t)))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (chdir "build")
+               (install-file (readlink "docker") out-bin)
+               (install-file "docker" out-bin)
+               #t))))))
+    (native-inputs
+     `(("go" ,go)
+       ("libltdl" ,libltdl)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Command line interface to Docker")
+    (description "This package provides a command line interface to Docker.")
+    (home-page "http://www.docker.com/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 30 Dec 2018 10:50
Re: [PATCH v3 3/4] services: Add docker.
(address . 33893@debbugs.gnu.org)
20181230105012.75f6b1a0@scratchpost.org
Better with this additional patch:

Toggle diff (16 lines)
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
index e592185f8..19d7e598f 100644
--- a/gnu/services/docker.scm
+++ b/gnu/services/docker.scm
@@ -72,7 +72,10 @@
            (provision '(dockerd))
            (requirement '(containerd))
            (start #~(make-forkexec-constructor
-                     (list (string-append #$docker "/bin/dockerd"))))
+                     (list (string-append #$docker "/bin/dockerd")
+                           "-p" "/var/run/docker.pid")
+                     #:pid-file "/var/run/docker.pid"
+                     #:log-file "/var/log/docker.log"))
            (stop #~(make-kill-destructor)))))
 
 (define docker-service-type
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlwolNQACgkQ5xo1VCww
uqWwWwgAosFFGb4Ou3eqQIeNIX+RNTccw5nOC2MakIATWmRKIkdMXHUaGNE/S8vQ
ES/Hnbz8ZIsUiA6UhGweHup1ltaCwyz7/d0g1oqhq7sUXS7TP/ioVPGca6vLD4Nl
hk9GRyauop8GeTdM5aMKfeJAZv/pbDuNuGWaQcyeUtX1BZDgjTpOk8WJOYQnec2C
c6IJ2oCuy37Ee9dRbPGYmhRpPeDc9wt575tL+QB1MAdXdqTc9N/c4F1K14OJWzLG
HeYtwtAYgIGboXsZrGHdYXF5XHSb0SJb9J1+stFEHqhlk0nv2QiTv36duxRWdhId
6Iye+3Ux1Sryjx+CnNC4kAP4rpr7LQ==
=Lg4Z
-----END PGP SIGNATURE-----


D
D
Danny Milosavljevic wrote on 30 Dec 2018 13:17
[PATCH v4 0/4] Add docker.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230121754.775-1-dannym@scratchpost.org
Now with macro.

Danny Milosavljevic (4):
gnu: Add containerd.
gnu: Add docker-engine.
services: Add docker.
gnu: Add docker-cli.

doc/guix.texi | 10 ++
gnu/local.mk | 1 +
gnu/packages/docker.scm | 261 +++++++++++++++++++++++++++++++++++++++-
gnu/services/docker.scm | 93 ++++++++++++++
4 files changed, 364 insertions(+), 1 deletion(-)
create mode 100644 gnu/services/docker.scm
D
D
Danny Milosavljevic wrote on 30 Dec 2018 13:17
[PATCH v4 1/4] gnu: Add containerd.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230121754.775-2-dannym@scratchpost.org
* gnu/packages/docker.scm (containerd): New variable.
---
gnu/packages/docker.scm | 49 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)

Toggle diff (67 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..f4e676a9f 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,9 +23,13 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
@@ -142,3 +146,48 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public containerd
+  (package
+    (name "containerd")
+    (version "1.2.1")
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/containerd/containerd.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "16zn6p1ky3yrgn53z8h9wza53ch91fj47wj5xgz6w4c57j30f66p"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/containerd/containerd"
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'chdir
+           (lambda _
+             (chdir "src/github.com/containerd/containerd")
+             #t))
+         (replace 'build
+           (lambda* (#:key (make-flags '()) #:allow-other-keys)
+             (apply invoke "make" make-flags)))
+         (replace 'install
+           (lambda* (#:key outputs (make-flags '()) #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (apply invoke "make" (string-append "DESTDIR=" out) "install"
+                      make-flags)))))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("libseccomp" ,libseccomp)))
+    (native-inputs
+     `(("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Container runtime")
+    (description "This package provides the container daemon for Docker.
+It includes image transfer and storage, container execution and supervision,
+network attachments.")
+    (home-page "http://containerd.io/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 30 Dec 2018 13:17
[PATCH v4 2/4] gnu: Add docker-engine.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230121754.775-3-dannym@scratchpost.org
* gnu/packages/docker.scm (docker-engine): New variable.
(%docker-version): New variable.
---
gnu/packages/docker.scm | 149 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 148 insertions(+), 1 deletion(-)

Toggle diff (172 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index f4e676a9f..3b6f00834 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,15 +23,20 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
-  #:use-module (gnu packages python-web))
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages virtualization))
+
+(define %docker-version "18.09.0")
 
 (define-public python-docker-py
   (package
@@ -191,3 +196,145 @@ It includes image transfer and storage, container execution and supervision,
 network attachments.")
     (home-page "http://containerd.io/")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;(substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+             ; (("") ""))
+             (substitute* "builder/builder-next/executor_unix.go"
+              (("CommandCandidates:.*runc.*")
+               (string-append "CommandCandidates: []string{\""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"},\n")))
+             (substitute* "vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand = .*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "daemon/daemon_unix.go"
+              (("DefaultShimBinary = .*")
+               (string-append "DefaultShimBinary = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n"))
+              (("DefaultRuntimeBinary = .*")
+               (string-append "DefaultRuntimeBinary = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+              (("var defaultCommandCandidates = .*")
+               (string-append "var defaultCommandCandidates = []string{\""
+                              (assoc-ref inputs "runc") "/sbin/runc\"}")))
+             (let ((source-files (filter (lambda (name)
+                                    (not (string-contains name "test")))
+                                  (find-files "." "\\.go$"))))
+               (let-syntax ((substitute-LookPath
+                             (lambda (x)
+                               (syntax-case x ()
+                                 ((substitute-LookPath source-text package
+                                                       relative-path)
+                                  #`(substitute* source-files
+                                     ((#,(string-append "exec\\.LookPath\\(\""
+                                                        (syntax->datum
+                                                         #'source-text)
+                                                        "\")"))
+                                      (string-append "\""
+                                                     (assoc-ref inputs package)
+                                                     relative-path
+                                                     "\", error(nil)"))))))))
+                 (substitute-LookPath "ps" "procps" "/bin/ps")
+                 (substitute-LookPath "mkfs.xfs" "xfsprogs" "/bin/mkfs.xfs")
+                 (substitute-LookPath "lvmdiskscan" "lvm2" "/sbin/lvmdiskscan")
+                 (substitute-LookPath "pvdisplay" "lvm2" "/sbin/pvdisplay")
+                 (substitute-LookPath "blkid" "util-linux" "/sbin/blkid")
+                 (substitute-LookPath "unpigz" "pigz" "/bin/unpigz")
+                 (substitute-LookPath "iptables" "iptables" "/sbin/iptables")
+                 (substitute-LookPath "ip" "iproute2" "/sbin/ip")
+                 ; TODO: zfs ?
+; TODO: getPlatformContainerdDaemonOpts ./cmd/dockerd/daemon_unix.go
+; TODO: --init-path for docker-init [./cmd/dockerd/config_unix.go InitPath];
+; ./daemon/config/config.go DefaultInitBinary
+              ;(("LookPath") "Guix_doesnt_want_LookPath")
+             ))
+             #t))
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("containerd" ,containerd) ; for containerd-shim
+       ("runc" ,runc)
+       ("iproute2" ,iproute)
+       ("iptables" ,iptables)
+       ("pigz" ,pigz)
+       ("procps" ,procps)
+       ("util-linux" ,util-linux)
+       ("lvm2" ,lvm2)
+       ("xfsprogs" ,xfsprogs)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 30 Dec 2018 13:17
[PATCH v4 3/4] services: Add docker.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230121754.775-4-dannym@scratchpost.org
* gnu/services/docker.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
* doc/guix.texi (Miscellaneous Services): Document the service.
---
doc/guix.texi | 10 +++++
gnu/local.mk | 1 +
gnu/services/docker.scm | 93 +++++++++++++++++++++++++++++++++++++++++
3 files changed, 104 insertions(+)
create mode 100644 gnu/services/docker.scm

Toggle diff (132 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index fcb5b8c08..b129b1bd1 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -22115,6 +22115,16 @@ The following is an example @code{dicod-service} configuration.
                     %dicod-database:gcide))))
 @end example
 
+@cindex docker
+@subsubheading Docker Service
+
+The @code{(gnu services docker)} module provides the following service.
+
+@defvr {Scheme Variable} docker-service-type
+
+This is a service that runs @url{http://www.docker.com,Docker}, a daemon that
+provides container functionality.
+
 @node Setuid Programs
 @subsection Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index 925d955a6..f6c91dcc7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -482,6 +482,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/desktop.scm			\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
+  %D%/services/docker.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/kerberos.scm			\
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
new file mode 100644
index 000000000..19d7e598f
--- /dev/null
+++ b/gnu/services/docker.scm
@@ -0,0 +1,93 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Danny Milosavljevic <dannym@scratchpost.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services docker)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services base)
+  #:use-module (gnu services dbus)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages docker)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+
+  #:export (docker-configuration
+            docker-service-type))
+
+(define-configuration docker-configuration
+  (docker
+   (package docker-engine)
+   "Docker daemon package.")
+  (containerd
+   (package containerd)
+   "containerd package."))
+
+(define %docker-accounts
+  (list (user-group (name "docker") (system? #t))))
+
+(define (%containerd-activation config)
+  (let ((state-dir "/var/lib/containerd"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+(define (%docker-activation config)
+  (%containerd-activation config)
+  (let ((state-dir "/var/lib/docker"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+;; TODO: Refactor out into its own module?  How to depend on it then?
+(define (containerd-shepherd-service config)
+  (let* ((package (docker-configuration-containerd config)))
+    (shepherd-service
+           (documentation "containerd daemon.")
+           (provision '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$package "/bin/containerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define (docker-shepherd-service config)
+  (let* ((docker (docker-configuration-docker config)))
+    (shepherd-service
+           (documentation "Docker daemon.")
+           (provision '(dockerd))
+           (requirement '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$docker "/bin/dockerd")
+                           "-p" "/var/run/docker.pid")
+                     #:pid-file "/var/run/docker.pid"
+                     #:log-file "/var/log/docker.log"))
+           (stop #~(make-kill-destructor)))))
+
+(define docker-service-type
+  (service-type (name 'docker)
+		(extensions
+                 (list
+                  (service-extension activation-service-type
+                                     %docker-activation)
+                  (service-extension shepherd-root-service-type
+                                     (lambda args
+                                       (list (apply containerd-shepherd-service args)
+                                             (apply docker-shepherd-service args))))
+                  (service-extension account-service-type
+                                     (const %docker-accounts))))
+                (default-value (docker-configuration))))
D
D
Danny Milosavljevic wrote on 30 Dec 2018 13:17
[PATCH v4 4/4] gnu: Add docker-cli.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230121754.775-5-dannym@scratchpost.org
* gnu/packages/docker.scm (docker-cli): New variable.
---
gnu/packages/docker.scm | 63 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)

Toggle diff (78 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 3b6f00834..81e79c42d 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -27,6 +27,7 @@
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
@@ -338,3 +339,65 @@ management, secret management, configuration management, networking,
 provisioning etc.")
     (home-page "https://mobyproject.org/")
     (license license:asl2.0)))
+
+(define-public docker-cli
+  (package
+    (name "docker-cli")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/cli.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1ivisys20kphvbqlazc3bsg7pk0ykj9gjx5d4yg439x4n13jxwvb"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/docker/cli"
+       ;; TODO: Tests require a running Docker daemon.
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'setup-environment-2
+           (lambda _
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+
+             ;; Make build reproducible.
+             (setenv "BUILDTIME" "1970-01-01 00:00:01.000000000+00:00")
+             (symlink "src/github.com/docker/cli/scripts" "./scripts")
+             (symlink "src/github.com/docker/cli/docker.Makefile" "./docker.Makefile")
+             #t))
+         (replace 'build
+           (lambda _
+             (invoke "./scripts/build/dynbinary")))
+         (replace 'check
+           (lambda* (#:key make-flags tests? #:allow-other-keys)
+             (setenv "PATH" (string-append (getcwd) "/build:" (getenv "PATH")))
+             (if tests?
+                 ;; Use the newly-built docker client for the tests.
+                 (with-directory-excursion "src/github.com/docker/cli"
+                   ;; TODO: Run test-e2e as well?
+                   (apply invoke "make" "-f" "docker.Makefile" "test-unit"
+                          (or make-flags '())))
+                 #t)))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (chdir "build")
+               (install-file (readlink "docker") out-bin)
+               (install-file "docker" out-bin)
+               #t))))))
+    (native-inputs
+     `(("go" ,go)
+       ("libltdl" ,libltdl)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Command line interface to Docker")
+    (description "This package provides a command line interface to Docker.")
+    (home-page "http://www.docker.com/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 31 Dec 2018 00:38
[PATCH v5 0/4] Add docker.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230233903.23426-1-dannym@scratchpost.org
Danny Milosavljevic (4):
gnu: Add containerd.
gnu: Add docker-engine.
services: Add docker.
gnu: Add docker-cli.

doc/guix.texi | 10 ++
gnu/local.mk | 1 +
gnu/packages/docker.scm | 299 +++++++++++++++++++++++++++++++++++++++-
gnu/services/docker.scm | 93 +++++++++++++
4 files changed, 402 insertions(+), 1 deletion(-)
create mode 100644 gnu/services/docker.scm
D
D
Danny Milosavljevic wrote on 31 Dec 2018 00:39
[PATCH v5 1/4] gnu: Add containerd.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230233903.23426-2-dannym@scratchpost.org
* gnu/packages/docker.scm (containerd): New variable.
---
gnu/packages/docker.scm | 68 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 68 insertions(+)

Toggle diff (86 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..877800042 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,9 +23,13 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
@@ -142,3 +146,67 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public containerd
+  (package
+    (name "containerd")
+    (version "1.2.1")
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/containerd/containerd.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "16zn6p1ky3yrgn53z8h9wza53ch91fj47wj5xgz6w4c57j30f66p"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/containerd/containerd"
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'chdir
+           (lambda _
+             (chdir "src/github.com/containerd/containerd")
+             #t))
+         (add-after 'chdir 'patch-paths
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             ;; TODO: Patch "socat", "unpigz".
+             (substitute* "./runtime/v1/linux/runtime.go"
+              (("defaultRuntime[ \t]*=.*")
+               (string-append "defaultRuntime = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n"))
+              (("defaultShim[ \t]*=.*")
+               (string-append "defaultShim = \""
+                              (assoc-ref outputs "out")
+                              "/bin/containerd-shim\"\n")))
+            (substitute* "./vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand[ \t]*=.*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             #t))
+         (replace 'build
+           (lambda* (#:key (make-flags '()) #:allow-other-keys)
+             (apply invoke "make" make-flags)))
+         (replace 'install
+           (lambda* (#:key outputs (make-flags '()) #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (apply invoke "make" (string-append "DESTDIR=" out) "install"
+                      make-flags)))))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("libseccomp" ,libseccomp)
+       ("runc" ,runc)))
+    (native-inputs
+     `(("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Container runtime")
+    (description "This package provides the container daemon for Docker.
+It includes image transfer and storage, container execution and supervision,
+network attachments.")
+    (home-page "http://containerd.io/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 31 Dec 2018 00:39
[PATCH v5 2/4] gnu: Add docker-engine.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230233903.23426-3-dannym@scratchpost.org
* gnu/packages/docker.scm (docker-engine): New variable.
(%docker-version): New variable.
---
gnu/packages/docker.scm | 168 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 167 insertions(+), 1 deletion(-)

Toggle diff (191 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 877800042..a3510529a 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,15 +23,20 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
-  #:use-module (gnu packages python-web))
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages virtualization))
+
+(define %docker-version "18.09.0")
 
 (define-public python-docker-py
   (package
@@ -210,3 +215,164 @@ It includes image transfer and storage, container execution and supervision,
 network attachments.")
     (home-page "http://containerd.io/")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;(substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+             ; (("") ""))
+             (substitute* "builder/builder-next/executor_unix.go"
+              (("CommandCandidates:.*runc.*")
+               (string-append "CommandCandidates: []string{\""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"},\n")))
+             (substitute* "vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand = .*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "vendor/github.com/containerd/containerd/runtime/v1/linux/runtime.go"
+              (("defaultRuntime[ \t]*=.*")
+               (string-append "defaultRuntime = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n"))
+              (("defaultShim[ \t]*=.*")
+               (string-append "defaultShim = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n")))
+             (substitute* "daemon/daemon_unix.go"
+              (("DefaultShimBinary = .*")
+               (string-append "DefaultShimBinary = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n"))
+              (("DefaultRuntimeBinary = .*")
+               (string-append "DefaultRuntimeBinary = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n"))
+              (("DefaultRuntimeName = .*")
+               (string-append "DefaultRuntimeName = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "daemon/config/config.go"
+              (("StockRuntimeName = .*")
+               (string-append "StockRuntimeName = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             ; TODO DefaultInitBinary
+
+             (substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+              (("var defaultCommandCandidates = .*")
+               (string-append "var defaultCommandCandidates = []string{\""
+                              (assoc-ref inputs "runc") "/sbin/runc\"}")))
+             (let ((source-files (filter (lambda (name)
+                                    (not (string-contains name "test")))
+                                  (find-files "." "\\.go$"))))
+               (let-syntax ((substitute-LookPath
+                             (lambda (x)
+                               (syntax-case x ()
+                                 ((substitute-LookPath source-text package
+                                                       relative-path)
+                                  #`(substitute* source-files
+                                     ((#,(string-append "exec\\.LookPath\\(\""
+                                                        (syntax->datum
+                                                         #'source-text)
+                                                        "\")"))
+                                      (string-append "\""
+                                                     (assoc-ref inputs package)
+                                                     relative-path
+                                                     "\", error(nil)"))))))))
+                 (substitute-LookPath "ps" "procps" "/bin/ps")
+                 (substitute-LookPath "mkfs.xfs" "xfsprogs" "/bin/mkfs.xfs")
+                 (substitute-LookPath "lvmdiskscan" "lvm2" "/sbin/lvmdiskscan")
+                 (substitute-LookPath "pvdisplay" "lvm2" "/sbin/pvdisplay")
+                 (substitute-LookPath "blkid" "util-linux" "/sbin/blkid")
+                 (substitute-LookPath "unpigz" "pigz" "/bin/unpigz")
+                 (substitute-LookPath "iptables" "iptables" "/sbin/iptables")
+                 (substitute-LookPath "ip" "iproute2" "/sbin/ip")
+                 ; TODO: zfs ?
+; TODO: getPlatformContainerdDaemonOpts ./cmd/dockerd/daemon_unix.go
+; TODO: --init-path for docker-init [./cmd/dockerd/config_unix.go InitPath];
+              ;(("LookPath") "Guix_doesnt_want_LookPath")
+             ))
+             #t))
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("containerd" ,containerd) ; for containerd-shim
+       ("runc" ,runc)
+       ("iproute2" ,iproute)
+       ("iptables" ,iptables)
+       ("pigz" ,pigz)
+       ("procps" ,procps)
+       ("util-linux" ,util-linux)
+       ("lvm2" ,lvm2)
+       ("xfsprogs" ,xfsprogs)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))
D
D
Danny Milosavljevic wrote on 31 Dec 2018 00:39
[PATCH v5 3/4] services: Add docker.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230233903.23426-4-dannym@scratchpost.org
* gnu/services/docker.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
* doc/guix.texi (Miscellaneous Services): Document the service.
---
doc/guix.texi | 10 +++++
gnu/local.mk | 1 +
gnu/services/docker.scm | 93 +++++++++++++++++++++++++++++++++++++++++
3 files changed, 104 insertions(+)
create mode 100644 gnu/services/docker.scm

Toggle diff (132 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index fcb5b8c08..b129b1bd1 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -22115,6 +22115,16 @@ The following is an example @code{dicod-service} configuration.
                     %dicod-database:gcide))))
 @end example
 
+@cindex docker
+@subsubheading Docker Service
+
+The @code{(gnu services docker)} module provides the following service.
+
+@defvr {Scheme Variable} docker-service-type
+
+This is a service that runs @url{http://www.docker.com,Docker}, a daemon that
+provides container functionality.
+
 @node Setuid Programs
 @subsection Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index 925d955a6..f6c91dcc7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -482,6 +482,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/desktop.scm			\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
+  %D%/services/docker.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/kerberos.scm			\
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
new file mode 100644
index 000000000..19d7e598f
--- /dev/null
+++ b/gnu/services/docker.scm
@@ -0,0 +1,93 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Danny Milosavljevic <dannym@scratchpost.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services docker)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services base)
+  #:use-module (gnu services dbus)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages docker)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+
+  #:export (docker-configuration
+            docker-service-type))
+
+(define-configuration docker-configuration
+  (docker
+   (package docker-engine)
+   "Docker daemon package.")
+  (containerd
+   (package containerd)
+   "containerd package."))
+
+(define %docker-accounts
+  (list (user-group (name "docker") (system? #t))))
+
+(define (%containerd-activation config)
+  (let ((state-dir "/var/lib/containerd"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+(define (%docker-activation config)
+  (%containerd-activation config)
+  (let ((state-dir "/var/lib/docker"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+;; TODO: Refactor out into its own module?  How to depend on it then?
+(define (containerd-shepherd-service config)
+  (let* ((package (docker-configuration-containerd config)))
+    (shepherd-service
+           (documentation "containerd daemon.")
+           (provision '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$package "/bin/containerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define (docker-shepherd-service config)
+  (let* ((docker (docker-configuration-docker config)))
+    (shepherd-service
+           (documentation "Docker daemon.")
+           (provision '(dockerd))
+           (requirement '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$docker "/bin/dockerd")
+                           "-p" "/var/run/docker.pid")
+                     #:pid-file "/var/run/docker.pid"
+                     #:log-file "/var/log/docker.log"))
+           (stop #~(make-kill-destructor)))))
+
+(define docker-service-type
+  (service-type (name 'docker)
+		(extensions
+                 (list
+                  (service-extension activation-service-type
+                                     %docker-activation)
+                  (service-extension shepherd-root-service-type
+                                     (lambda args
+                                       (list (apply containerd-shepherd-service args)
+                                             (apply docker-shepherd-service args))))
+                  (service-extension account-service-type
+                                     (const %docker-accounts))))
+                (default-value (docker-configuration))))
D
D
Danny Milosavljevic wrote on 31 Dec 2018 00:39
[PATCH v5 4/4] gnu: Add docker-cli.
(address . 33893@debbugs.gnu.org)(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20181230233903.23426-5-dannym@scratchpost.org
* gnu/packages/docker.scm (docker-cli): New variable.
---
gnu/packages/docker.scm | 63 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)

Toggle diff (78 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index a3510529a..19b4d504f 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -27,6 +27,7 @@
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
@@ -376,3 +377,65 @@ management, secret management, configuration management, networking,
 provisioning etc.")
     (home-page "https://mobyproject.org/")
     (license license:asl2.0)))
+
+(define-public docker-cli
+  (package
+    (name "docker-cli")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/cli.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1ivisys20kphvbqlazc3bsg7pk0ykj9gjx5d4yg439x4n13jxwvb"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/docker/cli"
+       ;; TODO: Tests require a running Docker daemon.
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'setup-environment-2
+           (lambda _
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+
+             ;; Make build reproducible.
+             (setenv "BUILDTIME" "1970-01-01 00:00:01.000000000+00:00")
+             (symlink "src/github.com/docker/cli/scripts" "./scripts")
+             (symlink "src/github.com/docker/cli/docker.Makefile" "./docker.Makefile")
+             #t))
+         (replace 'build
+           (lambda _
+             (invoke "./scripts/build/dynbinary")))
+         (replace 'check
+           (lambda* (#:key make-flags tests? #:allow-other-keys)
+             (setenv "PATH" (string-append (getcwd) "/build:" (getenv "PATH")))
+             (if tests?
+                 ;; Use the newly-built docker client for the tests.
+                 (with-directory-excursion "src/github.com/docker/cli"
+                   ;; TODO: Run test-e2e as well?
+                   (apply invoke "make" "-f" "docker.Makefile" "test-unit"
+                          (or make-flags '())))
+                 #t)))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (chdir "build")
+               (install-file (readlink "docker") out-bin)
+               (install-file "docker" out-bin)
+               #t))))))
+    (native-inputs
+     `(("go" ,go)
+       ("libltdl" ,libltdl)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Command line interface to Docker")
+    (description "This package provides a command line interface to Docker.")
+    (home-page "http://www.docker.com/")
+    (license license:asl2.0)))
L
L
Ludovic Courtès wrote on 6 Jan 2019 21:14
Re: [bug#33893] [PATCH v5 1/4] gnu: Add containerd.
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(address . 33893@debbugs.gnu.org)
87tvily24c.fsf@gnu.org
Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

Toggle quote (2 lines)
> * gnu/packages/docker.scm (containerd): New variable.

LGTM, thanks!

Ludo’.
L
L
Ludovic Courtès wrote on 6 Jan 2019 21:20
Re: [bug#33893] [PATCH v5 2/4] gnu: Add docker-engine.
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(address . 33893@debbugs.gnu.org)
87pnt9y1v0.fsf@gnu.org
Danny Milosavljevic <dannym@scratchpost.org> skribis:

Toggle quote (3 lines)
> * gnu/packages/docker.scm (docker-engine): New variable.
> (%docker-version): New variable.

[...]

Toggle quote (2 lines)
> + ;(("LookPath") "Guix_doesnt_want_LookPath")

No longer needed?

Toggle quote (5 lines)
> + (replace 'configure
> + (lambda _
> + (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
> + (setenv "AUTO_GOPATH" "1")

Could you add a comment saying what AUTO_GOPATH does?

Toggle quote (5 lines)
> + (replace 'build
> + (lambda _
> + ;(invoke "hack/make.sh" "binary")
> + ; FIXME: bash -c 'hack/validate/default && hack/make.sh'

It’s not clear to me what should be fixed; perhaps a leftover?

Toggle quote (7 lines)
> + (replace 'check
> + (lambda _
> + ; FIXME: Those don't find any of the go packages
> + ; needed. Probably GOPATH/GOROOT related.
> + ;(invoke "hack/test/unit")
> + #t))

That’s potentially problematic. :-) Any idea how difficult it would be
to run these tests?

Toggle quote (11 lines)
> + (replace 'install
> + (lambda* (#:key outputs #:allow-other-keys)
> + (let* ((out (assoc-ref outputs "out"))
> + (out-bin (string-append out "/bin")))
> + (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
> + (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
> + ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
> + ; TODO: KEEPBUNDLE=1
> + ;./source/bundles/dynbinary-daemon/dockerd
> + ;(invoke "hack/make.sh" "install-binary")

Comments can be removed?

Otherwise LGTM, thanks!

Ludo’.
L
L
Ludovic Courtès wrote on 6 Jan 2019 21:31
Re: [bug#33893] [PATCH v5 3/4] services: Add docker.
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(address . 33893@debbugs.gnu.org)
87ftu5y1dn.fsf@gnu.org
Danny Milosavljevic <dannym@scratchpost.org> skribis:

Toggle quote (4 lines)
> * gnu/services/docker.scm: New file.
> * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
> * doc/guix.texi (Miscellaneous Services): Document the service.

Nice!

Toggle quote (2 lines)
> +@cindex docker

“Docker” with a capital.

Toggle quote (10 lines)
> +@subsubheading Docker Service
> +
> +The @code{(gnu services docker)} module provides the following service.
> +
> +@defvr {Scheme Variable} docker-service-type
> +
> +This is a service that runs @url{http://www.docker.com,Docker}, a daemon that
> +provides container functionality.
> +

We’re missing “@end defvr” I guess.

I think we shouldn’t propagate the narrative that Docker = container.
So what about something like:

This is the type of the service that runs @url{…, Docker}, a daemon
that can execute application bundles (sometimes referred to as
``containers'') in isolated environments.

?

Also could you document ‘docker-configuration’ as well?


[...]

Toggle quote (10 lines)
> +;; TODO: Refactor out into its own module? How to depend on it then?
> +(define (containerd-shepherd-service config)
> + (let* ((package (docker-configuration-containerd config)))
> + (shepherd-service
> + (documentation "containerd daemon.")
> + (provision '(containerd))
> + (start #~(make-forkexec-constructor
> + (list (string-append #$package "/bin/containerd"))))
> + (stop #~(make-kill-destructor)))))

I suppose there could be a separate ‘containerd-service-type’ if it’s
useful; if it’s not, it’s OK to keep it this way.

As for the dependency, users would have to add both docker and
containerd to their service list, or docker-service-type could extend
containerd-service-type, which would ensure containerd-service-type is
automatically instantiated if it’s not already in the user’s service
list.

Toggle quote (11 lines)
> +(define docker-service-type
> + (service-type (name 'docker)
> + (extensions
> + (list
> + (service-extension activation-service-type
> + %docker-activation)
> + (service-extension shepherd-root-service-type
> + (lambda args
> + (list (apply containerd-shepherd-service args)
> + (apply docker-shepherd-service args))))

You can make the above (lambda (config) …) instead of (lambda (args) …).

Toggle quote (4 lines)
> + (service-extension account-service-type
> + (const %docker-accounts))))
> + (default-value (docker-configuration))))

Please add a ‘description’ field here, and please remove tabs from the
file. :-)

Could you consider adding a system test for docker/containerd? Perhaps
we could go as far as using ‘docker-image’ in (guix scripts pack) to
generate an image and make sure ‘docker load’ works, but maybe that’s
too much work.

Thank you,
Ludo’.
L
L
Ludovic Courtès wrote on 6 Jan 2019 21:33
Re: [bug#33893] [PATCH v5 4/4] gnu: Add docker-cli.
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(address . 33893@debbugs.gnu.org)
87bm4ty19z.fsf@gnu.org
Danny Milosavljevic <dannym@scratchpost.org> skribis:

Toggle quote (2 lines)
> * gnu/packages/docker.scm (docker-cli): New variable.

[...]

Toggle quote (5 lines)
> + (arguments
> + `(#:import-path "github.com/docker/cli"
> + ;; TODO: Tests require a running Docker daemon.
> + #:tests? #f

I suppose we cannot run the daemon in the build environment, can we?

Or is it possible to use some of the tests?

Toggle quote (4 lines)
> + (native-inputs
> + `(("go" ,go)
> + ("libltdl" ,libltdl)

Shouldn’t libltdl be an input?

Otherwise LGTM, thanks!

Ludo’.
D
D
Danny Milosavljevic wrote on 7 Jan 2019 19:44
Re: [bug#33893] [PATCH v5 2/4] gnu: Add docker-engine.
(name . Ludovic Courtès)(address . ludo@gnu.org)
20190107194455.69823f82@scratchpost.org
Hi Ludo,

On Sun, 06 Jan 2019 21:20:35 +0100
Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (11 lines)
> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>
> > * gnu/packages/docker.scm (docker-engine): New variable.
> > (%docker-version): New variable.
>
> [...]
>
> > + ;(("LookPath") "Guix_doesnt_want_LookPath")
>
> No longer needed?

It was meant as a detector in order to make compilation fail when, in future
versions, docker wants to invok new stuff that we didn't patch yet.
Should we do that?

Toggle quote (7 lines)
> > + (replace 'configure
> > + (lambda _
> > + (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
> > + (setenv "AUTO_GOPATH" "1")
>
> Could you add a comment saying what AUTO_GOPATH does?

Yes, I'll add one.

Toggle quote (7 lines)
> > + (replace 'build
> > + (lambda _
> > + ;(invoke "hack/make.sh" "binary")
> > + ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
>
> It’s not clear to me what should be fixed; perhaps a leftover?

Yeah, I meant to check what hack/validate/default does and it seems to do
developer-specific tests (commit message formatted the right way etc), so
I guess we can just not invoke it.

Toggle quote (10 lines)
> > + (replace 'check
> > + (lambda _
> > + ; FIXME: Those don't find any of the go packages
> > + ; needed. Probably GOPATH/GOROOT related.
> > + ;(invoke "hack/test/unit")
> > + #t))
>
> That’s potentially problematic. :-) Any idea how difficult it would be
> to run these tests?

Go has peculiar ideas of how the directory layout is supposed to be set up.
I could probably figure it out - but if someone with more Go knowledge could
step forward it would be much faster.

Toggle quote (13 lines)
> > + (replace 'install
> > + (lambda* (#:key outputs #:allow-other-keys)
> > + (let* ((out (assoc-ref outputs "out"))
> > + (out-bin (string-append out "/bin")))
> > + (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
> > + (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
> > + ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
> > + ; TODO: KEEPBUNDLE=1
> > + ;./source/bundles/dynbinary-daemon/dockerd
> > + ;(invoke "hack/make.sh" "install-binary")
>
> Comments can be removed?

Yeah.

Thanks!
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlwznicACgkQ5xo1VCww
uqU13Qf/X56xpbhPZf+HKBrRgQgt5SI1yC7uMV3fpZERiApWi2xKq7Jask1JX25/
qENdd01e/7H8h/BbtiBRV7mFNtqMcHC7wmm3ntIRp/Mk4FwCU97oZ0jJlBvqhOxq
92/Lywb3m4OaVh+1P0OEmAU+jluER8i9zx4Gj9L8zl9XYnp3im5ZaZTdpwDvQzyF
TM3eKc268RFHBiwi8djpOXLm7g7TKbW7voD78sRZDpFwIvv3fCl9uhpGCfMWjhPL
jUfRmh52hxnyg41cJmBseIRzwE0vQRcdxVfx9rm0AFFZbmYqpq8tWx5odLl1Yz6y
FyOTRyyhbXrV2OpdCBr3LpdW0aixlA==
=MQVW
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 8 Jan 2019 09:42
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87va2z1qxl.fsf@gnu.org
Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

Toggle quote (18 lines)
> On Sun, 06 Jan 2019 21:20:35 +0100
> Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>>
>> > * gnu/packages/docker.scm (docker-engine): New variable.
>> > (%docker-version): New variable.
>>
>> [...]
>>
>> > + ;(("LookPath") "Guix_doesnt_want_LookPath")
>>
>> No longer needed?
>
> It was meant as a detector in order to make compilation fail when, in future
> versions, docker wants to invok new stuff that we didn't patch yet.
> Should we do that?

I see, it sounds like a good idea. Also add a comment explaining the
rationale.

Toggle quote (11 lines)
>> > + (replace 'build
>> > + (lambda _
>> > + ;(invoke "hack/make.sh" "binary")
>> > + ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
>>
>> It’s not clear to me what should be fixed; perhaps a leftover?
>
> Yeah, I meant to check what hack/validate/default does and it seems to do
> developer-specific tests (commit message formatted the right way etc), so
> I guess we can just not invoke it.

OK.

Toggle quote (14 lines)
>> > + (replace 'check
>> > + (lambda _
>> > + ; FIXME: Those don't find any of the go packages
>> > + ; needed. Probably GOPATH/GOROOT related.
>> > + ;(invoke "hack/test/unit")
>> > + #t))
>>
>> That’s potentially problematic. :-) Any idea how difficult it would be
>> to run these tests?
>
> Go has peculiar ideas of how the directory layout is supposed to be set up.
> I could probably figure it out - but if someone with more Go knowledge could
> step forward it would be much faster.

I see Leo is Cc’d so we’ll see. :-)

Thank you,
Ludo’.
D
D
Danny Milosavljevic wrote on 10 Jan 2019 03:22
(address . 33893@debbugs.gnu.org)
20190110032210.1242af1f@scratchpost.org
Hi Ludo,
Hi Leo,

On Tue, 08 Jan 2019 09:42:14 +0100
Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (6 lines)
> > Go has peculiar ideas of how the directory layout is supposed to be set up.
> > I could probably figure it out - but if someone with more Go knowledge could
> > step forward it would be much faster.
>
> I see Leo is Cc’d so we’ll see. :-)

Nevermind, I've fixed it and learned something in the process:

Linux doesn't actually know the current working directory as a string.
It only knows the inode, so if you call getcwd, what libc actually does is
it opendirs "..", then finds the entry with the same inode number as
the current directory, and then returns the name of that entry.

Now, gopath uses symlinks to set up their preferred directory hierarchy
in such a way:

ln -s ../../../.. .gopath/src/github.com/docker/docker

Now if you chdir into ".gopath/src/github.com/docker/docker" and then Go later
does getcwd, it will appear as if the chdir did not succeed (because it will
just use the old working directory because it has the same inode).

So Go was erroring out because the directory structure there was *still* wrong.

Solution: Set environment variable PWD to the correct name of the directory.

I've pushed this patchset to master.

I'll try to add a system test next - let's see.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlw2rFIACgkQ5xo1VCww
uqVkmgf+NN96Dt8+cclYpjUmAzw2gOIfAEAmBoLdHCAWV+FyLIyEKiJ6gdBQqdMF
rkZVtezJhwmfEc34bEEe2VB1CASaL2rN5BVHZYUBIQqrjnj05PULDNAQRbZcXh93
PElhl3a+arHS024WIlirwbvLUV7qTWnNWjvEgCDhitT64hhpVn/FWEzyremlvkuJ
0PUF6bhLzRwvoaSDAZQlRbGWAYV7yoIN2mxWDR2ZyEIDrNQ7JyTbBydWSa4rS7kB
8l0QoLFASDYd1VaPltSHKR3eITgU+rLy3incTcRbIpk8NtFQgkhbRbteCc5WgNnj
lE4xAq4o6DsTtpfrOE3ObTX0GxuPJA==
=vXcU
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 10 Jan 2019 09:50
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87k1jcew0m.fsf@gnu.org
Howdy!

Danny Milosavljevic <dannym@scratchpost.org> skribis:

Toggle quote (16 lines)
> On Tue, 08 Jan 2019 09:42:14 +0100
> Ludovic Courtès <ludo@gnu.org> wrote:
>
>> > Go has peculiar ideas of how the directory layout is supposed to be set up.
>> > I could probably figure it out - but if someone with more Go knowledge could
>> > step forward it would be much faster.
>>
>> I see Leo is Cc’d so we’ll see. :-)
>
> Nevermind, I've fixed it and learned something in the process:
>
> Linux doesn't actually know the current working directory as a string.
> It only knows the inode, so if you call getcwd, what libc actually does is
> it opendirs "..", then finds the entry with the same inode number as
> the current directory, and then returns the name of that entry.

Are you sure? In the Linux port of glibc I see this:

Toggle snippet (11 lines)
char *
__getcwd (char *buf, size_t size)
{
char *path;
char *result;

// […]

retval = INLINE_SYSCALL (getcwd, 2, path, alloc_size);

And indeed, there’s a ‘getcwd’ syscall:

Toggle snippet (5 lines)
$ strace -e getcwd guile -c '(getcwd)'
getcwd("/home/ludo", 100) = 11
+++ exited with 0 +++

Toggle quote (13 lines)
> Now, gopath uses symlinks to set up their preferred directory hierarchy
> in such a way:
>
> ln -s ../../../.. .gopath/src/github.com/docker/docker
>
> Now if you chdir into ".gopath/src/github.com/docker/docker" and then Go later
> does getcwd, it will appear as if the chdir did not succeed (because it will
> just use the old working directory because it has the same inode).
>
> So Go was erroring out because the directory structure there was *still* wrong.
>
> Solution: Set environment variable PWD to the correct name of the directory.

Great that you found a solution.

Thanks for taking the time to address this!

Ludo’.
Closed
D
D
Danny Milosavljevic wrote on 10 Jan 2019 14:15
(name . Ludovic Courtès)(address . ludo@gnu.org)
20190110141542.715a0274@scratchpost.org
Hi Ludo,

On Thu, 10 Jan 2019 09:50:49 +0100
Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (20 lines)
> Howdy!
>
> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>
> > On Tue, 08 Jan 2019 09:42:14 +0100
> > Ludovic Courtès <ludo@gnu.org> wrote:
> >
> >> > Go has peculiar ideas of how the directory layout is supposed to be set up.
> >> > I could probably figure it out - but if someone with more Go knowledge could
> >> > step forward it would be much faster.
> >>
> >> I see Leo is Cc’d so we’ll see. :-)
> >
> > Nevermind, I've fixed it and learned something in the process:
> >
> > Linux doesn't actually know the current working directory as a string.
> > It only knows the inode, so if you call getcwd, what libc actually does is
> > it opendirs "..", then finds the entry with the same inode number as
> > the current directory, and then returns the name of that entry.

According to the POSIX standard ;)

Toggle quote (22 lines)
> Are you sure? In the Linux port of glibc I see this:
>
> --8<---------------cut here---------------start------------->8---
> char *
> __getcwd (char *buf, size_t size)
> {
> char *path;
> char *result;
>
> // […]
>
> retval = INLINE_SYSCALL (getcwd, 2, path, alloc_size);
> --8<---------------cut here---------------end--------------->8---
>
> And indeed, there’s a ‘getcwd’ syscall:
>
> --8<---------------cut here---------------start------------->8---
> $ strace -e getcwd guile -c '(getcwd)'
> getcwd("/home/ludo", 100) = 11
> +++ exited with 0 +++
> --8<---------------cut here---------------end--------------->8---

Huh. I guess it boils down to whether the Linux "process" structure
has the cwd in it as a string or as an inode.

In Linux sources:

static inline void get_fs_pwd(struct fs_struct *fs, struct path *pwd)
{
spin_lock(&fs->lock);
*pwd = fs->pwd;
path_get(pwd);
spin_unlock(&fs->lock);
}

static void get_fs_root_and_pwd_rcu(struct fs_struct *fs, struct path *root,
struct path *pwd)
{
unsigned seq;

do {
seq = read_seqcount_begin(&fs->seq);
*root = fs->root;
*pwd = fs->pwd;
} while (read_seqcount_retry(&fs->seq, seq));
}

struct path {
struct vfsmount *mnt;
struct dentry *dentry;
} __randomize_layout;

SYSCALL_DEFINE2(getcwd, char __user *, buf, unsigned long, size)
{
int error;
struct path pwd, root;
char *page = __getname();

if (!page)
return -ENOMEM;

rcu_read_lock();
get_fs_root_and_pwd_rcu(current->fs, &root, &pwd);

error = -ENOENT;
if (!d_unlinked(pwd.dentry)) {
unsigned long len;
char *cwd = page + PATH_MAX;
int buflen = PATH_MAX;

prepend(&cwd, &buflen, "\0", 1);
error = prepend_path(&pwd, &root, &cwd, &buflen);
rcu_read_unlock();

if (error < 0)
goto out;

/* Unreachable from current root */
if (error > 0) {
error = prepend_unreachable(&cwd, &buflen);
if (error)
goto out;
}

error = -ERANGE;
len = PATH_MAX + page - cwd;
if (len <= size) {
error = len;
if (copy_to_user(buf, cwd, len))
error = -EFAULT;
}
} else {
rcu_read_unlock();
}

out:
__putname(page);
return error;
}

/*
* Replace the fs->{pwdmnt,pwd} with {mnt,dentry}. Put the old values.
* It can block.
*/
void set_fs_pwd(struct fs_struct *fs, const struct path *path)
{
struct path old_pwd;

path_get(path);
spin_lock(&fs->lock);
write_seqcount_begin(&fs->seq);
old_pwd = fs->pwd;
fs->pwd = *path; <----------------- !!!!
write_seqcount_end(&fs->seq);
spin_unlock(&fs->lock);

if (old_pwd.dentry)
path_put(&old_pwd);
}

int ksys_chdir(const char __user *filename)
{
struct path path;
int error;
unsigned int lookup_flags = LOOKUP_FOLLOW | LOOKUP_DIRECTORY;
retry:
error = user_path_at(AT_FDCWD, filename, lookup_flags, &path);
if (error)
goto out;

error = inode_permission(path.dentry->d_inode, MAY_EXEC | MAY_CHDIR);
if (error)
goto dput_and_out;

set_fs_pwd(current->fs, &path); <----------------- !!!

dput_and_out:
path_put(&path);
if (retry_estale(error, lookup_flags)) {
lookup_flags |= LOOKUP_REVAL;
goto retry;
}
out:
return error;
}

SYSCALL_DEFINE1(chdir, const char __user *, filename)
{
return ksys_chdir(filename);
}


SYSCALL_DEFINE1(fchdir, unsigned int, fd)
{
struct fd f = fdget_raw(fd);
int error;

error = -EBADF;
if (!f.file)
goto out;

error = -ENOTDIR;
if (!d_can_lookup(f.file->f_path.dentry))
goto out_putf;

error = inode_permission(file_inode(f.file), MAY_EXEC | MAY_CHDIR);
if (!error)
set_fs_pwd(current->fs, &f.file->f_path);
out_putf:
fdput(f);
out:
return error;
}

Interesting!

Toggle quote (2 lines)
> Thanks for taking the time to address this!

No problem :)
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlw3RX4ACgkQ5xo1VCww
uqW/Ngf8DrCj8kxRvkwVUpBmjoO+ywau0iykiwWhAHuQEcxyD/Rg1QYaw662in/p
r0GCzjKCtxaaFrK7JmDbuwm2/5d7yaI8ABqj95Os+y+IrN47rysSuTa3LmdcbzCt
YJKMoOD87vKEh3yLTvIfHOiQekTWUY+Fgac9W6ZRGCoKwN+Lz9TGPkcmNIGPrYA5
gOFX46Flka0SjPK7REbfguViR2ZLO6i8vyK/w2s4Pll+WShn15iWVzaHOr1PWDo6
o7yrMnqebXnFXdYGGghULHlvz54fRnetEN+pJZUfmenjV3VYyPS/cxpZnXfUNPKE
28B0D94VvFS4ZHjP6Pu0hD0avAHygw==
=SUVE
-----END PGP SIGNATURE-----


Closed
L
L
Leo Famulari wrote on 10 Jan 2019 21:31
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
20190110203107.GC14234@jasmine.lan
On Thu, Jan 10, 2019 at 03:22:10AM +0100, Danny Milosavljevic wrote:
Toggle quote (12 lines)
> Hi Ludo,
> Hi Leo,
>
> On Tue, 08 Jan 2019 09:42:14 +0100
> Ludovic Courtès <ludo@gnu.org> wrote:
>
> > > Go has peculiar ideas of how the directory layout is supposed to be set up.
> > > I could probably figure it out - but if someone with more Go knowledge could
> > > step forward it would be much faster.
> >
> > I see Leo is Cc’d so we’ll see. :-)

Indeed, Go is very particular about this...

Toggle quote (2 lines)
> Nevermind, I've fixed it and learned something in the process:

Okay, good :) Let me know if you have more Go questions.
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEsFFZSPHn08G5gDigJkb6MLrKfwgFAlw3q4oACgkQJkb6MLrK
fwjkrhAA2KCK5RfxEfqHVXmGS/ux42HqzAqTALSIL16j7T77zq71vnufpipyQheK
yJ0RltAbmIAUkK7e7vNqNWqypZvbNZtvUsqXIIVM/cYp9oPU9mdPV+uJqGEn8oqm
4I8cz1ac6wXefYqmre2UIAulBKPAEPefG6JdZHHDhtW5hqpocaPXIp8u+Fy4jI9R
z/1l3oGn2co8odWGlwuMAmjTn2IvNUwp/aPQ5zXB++IXxwdofQnhYwZaGXeqpUaS
zafsbnNxCU+Dk9iTNH6xXMaHUuui8gucY+iIj6mK5GmkcmDfNRrchIrbP+QYspyo
fzVTAuXiRuxcyO/4shpnJS9KKZzgXxOqmXj4mw7DlEfELxQA7TF0XkJfGP7eynH3
fQahJkn5XhJm2mAUKO7v3DVI/DY/sNdYEw8h/41VgKmz4MPBX45wDzAKz9nsLrLf
Z/Krz1Ox9uUmpudWajVdPQephw0Q+vyarv6+3g41TWx85oge0BPQj0d4ikiABCb7
BwCXC90NU9U2GdfFYMBU8zlE1gnboDoHezcEvFdr387ZMI95CPWeqFMGWJbtRIju
eycKksDNt0687yAqHugzAVZYPUo6Qq4ClV+LlmlmL+UXR7PBpdtdyCivHvO3e0Jc
xODNE+YsdQVg2t8JFC5ombz5bQhFHJvZkGwYKR4OAZmXkbmoxhQ=
=AIAR
-----END PGP SIGNATURE-----


M
M
Meiyo Peng wrote on 14 Jan 2019 04:20
Re: [bug#33893] [PATCH v5 4/4] gnu: Add docker-cli.
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(address . 33893@debbugs.gnu.org)
87tvicylfr.fsf@gmail.com
Hi Danny,

docker-cli provides two identical commands in
"/gnu/store/*docker-cli*/bin/".

#+begin_SRC sh
~ ll /gnu/store/*docker-cli*/bin/*
-r-xr-xr-x 3 root root 64M Jan 1 1970 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker*
-r-xr-xr-x 3 root root 64M Jan 1 1970 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker-linux-amd64*

~ sha256sum /gnu/store/*docker-cli*/bin/*
62bc8199fd11f37129d6e8183865df698f495faf90a86bdbe5ee4891b201cbc8 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker
62bc8199fd11f37129d6e8183865df698f495faf90a86bdbe5ee4891b201cbc8 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker-linux-amd64
#+end_SRC

It wastes 64MB disk space. Can we remove "docker-linux-amd64"?


--
Meiyo Peng
D
D
Danny Milosavljevic wrote on 15 Jan 2019 13:34
(name . Meiyo Peng)(address . meiyo.peng@gmail.com)(address . 33893-done@debbugs.gnu.org)
20190115133431.7fdcf512@scratchpost.org
Hi,

Toggle quote (2 lines)
> It wastes 64MB disk space. Can we remove "docker-linux-amd64"?

Done in commit f3705090965c2470a0ccc2c045edbc5f5fb7bb8d.

Thanks!
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlw901cACgkQ5xo1VCww
uqXrnQf/e9X78rzEdzhUlALgQf8yniKAKq38SPf4f1jPZq5JKJ4mgeNXxAHhoSMQ
YOoGw8TNVIzwfPvn324Dt70Xp3NuBIh1LOVazcXuGkGQF+ksik+/BOvq/oYOj7jf
69kYTot596pUoYQFRZddURWFSparFgATQ3C7wD043EX9Jx2GWeVrNprt1KQR1lng
ocdNLXHGfmag/r19G4P4VZKJyxNNH5XKaClOERt+FTVIPF5QZNtfdgxe6UkJdjry
qBRIJMxkKTk25T6aaCzpzt/Qm4G9+Ez3ZdIv7HSlPSzrQm50Iy2IdyioNdyiUAa6
vTcC00ZzoiwzMutngbwLdDwkUP2Keg==
=z1/Z
-----END PGP SIGNATURE-----


Closed
?
Your comment

This issue is archived.

To comment on this conversation send email to 33893@debbugs.gnu.org