Add docker cli Guix Home service and some docker authentication plugins

  • Open
  • quality assurance status badge
Details
2 participants
  • paul
  • Ludovic Courtès
Owner
unassigned
Submitted by
paul
Severity
normal
P
(address . guix-patches@gnu.org)
d562f46d-01ca-0d41-dbed-b5cf2e4926ad@autistici.org
Hello Guixers,


I'm sending a patchset to:


- add two docker cli authentication plugins for libsecret and pass
respectively

- a Guix Home service to make the docker cli aware of Guix provided
plugins. this could be used for docker compose v2 in the future ( I
actually do use it now with a binary package and it works as far as I
can tell).


Thank you for your work,

giacomo
G
G
Giacomo Leidi wrote on 7 Apr 22:57 +0200
[PATCH 1/3] gnu: Add docker-credential-secretservice.
(address . 70265@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
95e5c6e85e09ead19a43fc37612d62bc91a64048.1712523439.git.goodoldpaul@autistici.org
* gnu/packages/docker.scm (docker-credential-helpers): New variable.
* gnu/packages/gnome.scm (docker-credential-secretservice): New variable.

Change-Id: I6c46d429fa2842969b0fcde58ded72e5b04ee321
---
gnu/packages/docker.scm | 70 ++++++++++++++++++++++++++++++++++++++++-
gnu/packages/gnome.scm | 6 +++-
2 files changed, 74 insertions(+), 2 deletions(-)

Toggle diff (123 lines)
diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 0fe1f2c1c7..31501e50b9 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -8,6 +8,7 @@
;;; Copyright © 2020 Jesse Dowell <jessedowell@gmail.com>
;;; Copyright © 2021, 2022 Oleg Pykhalov <go.wigust@gmail.com>
;;; Copyright © 2022 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -51,7 +52,8 @@ (define-module (gnu packages docker)
#:use-module (gnu packages python-web)
#:use-module (gnu packages python-xyz)
#:use-module (gnu packages version-control)
- #:use-module (gnu packages virtualization))
+ #:use-module (gnu packages virtualization)
+ #:export (docker-credential-helpers))
;; Note - when changing Docker versions it is important to update the versions
;; of several associated packages (docker-libnetwork and go-sctp).
@@ -670,6 +672,72 @@ (define-public docker-cli
(home-page "https://www.docker.com/")
(license license:asl2.0)))
+;; Actual users of this procedure are
+;; docker-credentials-secretservice and docker-credential-pass, they live in
+;; different modules to avoid circular imports.
+(define* (docker-credential-helpers plugin-name #:key (inputs '()))
+ (package
+ (name (string-append "docker-credential-" plugin-name))
+ (version "0.8.1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/docker/docker-credential-helpers")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1kric2yrgypdqncqfrmrh7l7904km5zisygi3fg6zlfkyh6rsm23"))))
+ (build-system go-build-system)
+ (arguments
+ (list
+ #:install-source? #f
+ #:go go-1.19
+ #:unpack-path "github.com/docker/docker-credential-helpers"
+ #:import-path
+ (string-append "github.com/docker/docker-credential-helpers/"
+ plugin-name "/cmd")
+ #:phases
+ #~(modify-phases %standard-phases
+ (replace 'build
+ (lambda* (#:key unpack-path import-path build-flags #:allow-other-keys)
+ (apply invoke "go" "build"
+ "-v"
+ "-x"
+ (string-append "-ldflags=-s -w "
+ "-X github.com/docker/docker-credential-helpers"
+ "/credentials.Version=" #$version " "
+ "-X github.com/docker/docker-credential-helpers"
+ "/credentials.Package=" unpack-path " "
+ "-X github.com/docker/docker-credential-helpers"
+ "/credentials.Name=" #$name)
+ "-o" (string-append "bin/" #$name)
+ `(,@build-flags ,import-path))))
+ (replace 'install
+ (lambda _
+ (let* ((bin
+ (string-append #$output "/bin"))
+ (lib
+ (string-append #$output "/libexec/docker/cli-plugins"))
+ (entrypoint
+ (string-append lib "/" #$name)))
+ (mkdir-p bin)
+ (mkdir-p lib)
+ (copy-file (string-append "bin/" #$name) entrypoint)
+ (symlink entrypoint
+ (string-append bin "/" #$name))))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs inputs)
+ (home-page "https://github.com/docker/docker-credential-helpers")
+ (synopsis "Store Docker login credentials in platform keystores")
+ (description
+ (string-append "docker-credential-helpers is a suite of programs to use native stores to keep
+Docker credentials safe.
+
+This package provides the @code{" name "} plugin."))
+ (license license:expat)))
+
(define-public cqfd
(package
(name "cqfd")
diff --git a/gnu/packages/gnome.scm b/gnu/packages/gnome.scm
index 06256066bc..58b53aba22 100644
--- a/gnu/packages/gnome.scm
+++ b/gnu/packages/gnome.scm
@@ -36,7 +36,7 @@
;;; Copyright © 2019 Danny Milosavljevic <dannym@scratchpost.org>
;;; Copyright © 2019, 2020, 2022 Marius Bakke <marius@gnu.org>
;;; Copyright © 2019 Florian Pelz <pelzflorian@pelzflorian.de>
-;;; Copyright © 2019 Giacomo Leidi <goodoldpaul@autistici.org>
+;;; Copyright © 2019, 2024 Giacomo Leidi <goodoldpaul@autistici.org>
;;; Copyright © 2019 Jelle Licht <jlicht@fsfe.org>
;;; Copyright © 2019 Jonathan Frederickson <jonathan@terracrypt.net>
;;; Copyright © 2019, 2020, 2021, 2022, 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
@@ -5229,6 +5229,10 @@ (define-public libsecret
and other secrets. It communicates with the \"Secret Service\" using DBus.")
(license license:lgpl2.1+)))
+(define-public docker-credential-secretservice
+ (docker-credential-helpers "secretservice"
+ #:inputs (list libsecret)))
+
(define-public five-or-more
(package
(name "five-or-more")

base-commit: 69951a61a1d8f1f2135ea2dc836738be282b97bc
--
2.41.0
G
G
Giacomo Leidi wrote on 7 Apr 22:57 +0200
[PATCH 2/3] gnu: Add docker-credential-pass.
(address . 70265@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
70e8777b04110b3b20c489d91da72ec629ef6617.1712523439.git.goodoldpaul@autistici.org
* gnu/packages/gnome.scm (docker-credential-pass): New variable.

Change-Id: I442ff509aaed8cc3809de27714710abd24f7e8e0
---
gnu/packages/gnome.scm | 4 ++++
1 file changed, 4 insertions(+)

Toggle diff (17 lines)
diff --git a/gnu/packages/gnome.scm b/gnu/packages/gnome.scm
index 58b53aba22..6f5f28e008 100644
--- a/gnu/packages/gnome.scm
+++ b/gnu/packages/gnome.scm
@@ -5229,6 +5229,10 @@ (define-public libsecret
and other secrets. It communicates with the \"Secret Service\" using DBus.")
(license license:lgpl2.1+)))
+(define-public docker-credential-pass
+ (docker-credential-helpers "pass"
+ #:inputs (list password-store)))
+
(define-public docker-credential-secretservice
(docker-credential-helpers "secretservice"
#:inputs (list libsecret)))
--
2.41.0
G
G
Giacomo Leidi wrote on 7 Apr 22:57 +0200
[PATCH 3/3] gnu: Add home-docker-cli-service-type.
(address . 70265@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
814c5c6c8ba59b4069966f46c995e7a8fd0d9b88.1712523439.git.goodoldpaul@autistici.org
* gnu/home/services/containers.scm (home-docker-cli-service-type): New
variable;
* doc/guix.texi: document it;
* gnu/local.mk: add it.

Change-Id: I71e7a2805fe8754511c8e02ee4ae667e34e4aaf6
---
doc/guix.texi | 53 +++++++++++
gnu/home/services/containers.scm | 148 +++++++++++++++++++++++++++++++
gnu/local.mk | 1 +
3 files changed, 202 insertions(+)
create mode 100644 gnu/home/services/containers.scm

Toggle diff (232 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 20f007b1c0..61cc904f9d 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -46500,6 +46500,59 @@ Miscellaneous Home Services
(dicod-configuration @dots{})))
@end lisp
+@subsubheading Container Services
+
+@cindex docker cli service, for Home
+The @code{(gnu home services containers)} module provides the following service:
+
+@defvar home-docker-cli-service-type
+This service allows for configuring the Docker command line interface, for
+example to make it aware of Guix provided plugins.
+@end defvar
+
+For example, you can use it like this to make Docker safely store your registry
+credentials with the system
+@uref{https://wiki.gnome.org/Projects/Libsecret, libsecret} compatible Secret service:
+
+@lisp
+(use-modules (gnu packages docker))
+
+(service home-docker-cli-service-type
+ (docker-cli-configuration
+ (creds-store "secretservice")
+ (cli-plugins
+ (list docker-credential-secretservice))
+ (extra-content ", \"auths\": @{\"https://index.docker.io/v1/\": @{@}@}")))
+@end lisp
+
+
+@c %start of fragment
+
+@deftp {Data Type} docker-cli-configuration
+Available @code{docker-cli-configuration} fields are:
+
+@table @asis
+@item @code{docker-cli} (default: @code{docker-cli}) (type: package)
+The Docker cli package installed to the Home profile.
+
+@item @code{creds-store} (type: maybe-string)
+A native secrets store used to store Docker credentials.
+
+@item @code{cli-plugins} (default: @code{()}) (type: list-of-docker-cli-plugins)
+A list of Docker cli plugin package records that will be configured to
+work with Docker's cli.
+
+@item @code{extra-content} (default: @code{""}) (type: string)
+Additional literal content that will be appended to Docker cli
+config.json.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
@node Invoking guix home
@section Invoking @command{guix home}
diff --git a/gnu/home/services/containers.scm b/gnu/home/services/containers.scm
new file mode 100644
index 0000000000..89d6ad5f39
--- /dev/null
+++ b/gnu/home/services/containers.scm
@@ -0,0 +1,148 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify
+;;; it under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation, either version 3 of the License, or
+;;; (at your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful,
+;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu home services containers)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (gnu packages docker)
+ #:use-module (gnu services configuration)
+ #:use-module (gnu home services)
+ #:use-module (ice-9 match)
+ #:use-module (ice-9 string-fun)
+ #:export (docker-cli-configuration
+ docker-cli-configuration?
+ docker-cli-configuration-fields
+ docker-cli-configuration-docker-cli
+ docker-cli-configuration-creds-store
+ docker-cli-configuration-cli-plugins
+
+ home-docker-cli-service-type
+ home-docker-cli-configuration-file
+ docker-cli-configuration->json-fields))
+
+;; Turn field names, which are Scheme symbols into strings
+;; cli-plugins-extra-dirs -> cliPluginsExtraDirs
+(define (format-name name)
+ (define without-dashes (string-replace-substring (symbol->string name) "-" " "))
+ (define splitted (string-split without-dashes #\space))
+ (string-replace-substring
+ (apply string-append
+ `(,(car splitted)
+ ,@(map string-capitalize (cdr splitted))))
+ " " ""))
+
+(define (serialize-json-value name value)
+ #~(begin
+ (use-modules (ice-9 format))
+ (format #f "\"~a\": ~a" #$(format-name name) #$value)))
+
+(define (serialize-json-list values)
+ #~(string-append "["
+ (string-join
+ (map (lambda (s) (string-append "\"" s "\""))
+ (list #$@values))
+ ", ")
+ "]"))
+
+(define (serialize-string field-name value)
+ (serialize-json-value field-name (string-append "\"" value "\"")))
+
+(define (serialize-maybe-string field-name value)
+ (if (maybe-value-set? value)
+ (serialize-string field-name value)
+ '()))
+
+(define (serialize-list-of-strings field-name value)
+ (serialize-json-value field-name (serialize-json-list value)))
+
+(define (serialize-list-of-docker-cli-plugins value)
+ (serialize-json-value 'cli-plugins-extra-dirs
+ (serialize-json-list
+ (map (lambda (p)
+ (file-append p "/libexec/docker/cli-plugins"))
+ value))))
+
+(define list-of-strings?
+ (list-of string?))
+
+(define list-of-docker-cli-plugins?
+ (list-of package?))
+
+(define-maybe string)
+
+(define-configuration/no-serialization docker-cli-configuration
+ (docker-cli
+ (package docker-cli)
+ "The Docker cli package installed to the Home profile.")
+ (creds-store
+ (maybe-string)
+ "A native secrets store used to store Docker credentials.")
+ (cli-plugins
+ (list-of-docker-cli-plugins '())
+ "A list of Docker cli plugin package records that will be configured to work with Docker's cli.")
+ (extra-content
+ (string "")
+ "Additional literal content that will be appended to Docker cli config.json."))
+
+(define docker-cli-configuration->json-fields
+ (lambda (config)
+ (filter (compose not (lambda (f) (or (null? f) (and (string? f) (string-null? f)))))
+ (map (lambda (f)
+ (let ((field-name (configuration-field-name f))
+ (type (configuration-field-type f))
+ (value ((configuration-field-getter f) config)))
+ (if (not (member field-name '(docker-cli extra-content)))
+ (match type
+ ('string
+ (serialize-string field-name value))
+ ('maybe-string
+ (serialize-maybe-string field-name value))
+ ('list-of-strings
+ (serialize-list-of-strings field-name value))
+ ('list-of-docker-cli-plugins
+ (serialize-list-of-docker-cli-plugins value))
+ (_
+ (raise
+ (formatted-message
+ (G_ "Unknown docker-cli-configuration field type: ~a")
+ type))))
+ '())))
+ docker-cli-configuration-fields))))
+
+(define (home-docker-cli-configuration-file config)
+ `((".docker/config.json"
+ ,(computed-file "docker-cli-config.json"
+ #~(with-output-to-file #$output
+ (lambda _
+ (display
+ (string-append "{"
+ (string-join (list #$@(docker-cli-configuration->json-fields config)) ",")
+ #$(docker-cli-configuration-extra-content config)
+ "}\n"))))))))
+
+(define home-docker-cli-service-type
+ (service-type (name 'docker-cli)
+ (extensions (list (service-extension home-profile-service-type
+ (lambda (config)
+ `(,(docker-cli-configuration-docker-cli config)
+ ,@(docker-cli-configuration-cli-plugins config))))
+ (service-extension home-files-service-type
+ home-docker-cli-configuration-file)))
+ (default-value (docker-cli-configuration))
+ (description
+ "This service install and configures Docker's command line interface.")))
diff --git a/gnu/local.mk b/gnu/local.mk
index 7f1006010b..4960f32a1a 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -95,6 +95,7 @@ GNU_SYSTEM_MODULES = \
%D%/compression.scm \
%D%/home.scm \
%D%/home/services.scm \
+ %D%/home/services/containers.scm \
%D%/home/services/desktop.scm \
%D%/home/services/dict.scm \
%D%/home/services/dotfiles.scm \
--
2.41.0
L
L
Ludovic Courtès wrote on 4 May 18:36 +0200
Re: bug#70265: Add docker cli Guix Home service and some docker authentication plugins
(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
87a5l5ttz5.fsf_-_@gnu.org
Hi,

Giacomo Leidi <goodoldpaul@autistici.org> skribis:

Toggle quote (5 lines)
> * gnu/home/services/containers.scm (home-docker-cli-service-type): New
> variable;
> * doc/guix.texi: document it;
> * gnu/local.mk: add it.

Nice. For the documentation part, the convention is something like:

* doc/guix.texi (Miscellaneous Services)[Container Services]: New
heading.

Toggle quote (2 lines)
> +@cindex docker cli service, for Home

s/docker cli/Docker command-line interface/

Toggle quote (11 lines)
> +The @code{(gnu home services containers)} module provides the following service:
> +
> +@defvar home-docker-cli-service-type
> +This service allows for configuring the Docker command line interface, for
> +example to make it aware of Guix provided plugins.
> +@end defvar
> +
> +For example, you can use it like this to make Docker safely store your registry
> +credentials with the system
> +@uref{https://wiki.gnome.org/Projects/Libsecret, libsecret} compatible Secret service:

Rather:

@uref{https://wiki.gnome.org/Projects/Libsecret,libsecret-compatible}
secret service:

(So that the parenthesized URL appears in the right place in Info and
PDF.)

Toggle quote (10 lines)
> +@lisp
> +(use-modules (gnu packages docker))
> +
> +(service home-docker-cli-service-type
> + (docker-cli-configuration
> + (creds-store "secretservice")
> + (cli-plugins
> + (list docker-credential-secretservice))
> + (extra-content ", \"auths\": @{\"https://index.docker.io/v1/\": @{@}@}")))

Sounds scary: how can I know as a user where that comma is going to be
stuck in the resulting file? Providing partial JSON strings should
rather be avoided IMO.

Also, if that part is necessary, it should be explained.

Toggle quote (9 lines)
> +(define (format-name name)
> + (define without-dashes (string-replace-substring (symbol->string name) "-" " "))
> + (define splitted (string-split without-dashes #\space))
> + (string-replace-substring
> + (apply string-append
> + `(,(car splitted)
> + ,@(map string-capitalize (cdr splitted))))
> + " " ""))

Rather:

(match split ;past participate of “to split”
((head . rest)
(string-concatenate (cons head (map string-capitalize rest)))))

See
for the rationale.

Toggle quote (5 lines)
> +(define docker-cli-configuration->json-fields
> + (lambda (config)
> + (filter (compose not (lambda (f) (or (null? f) (and (string? f) (string-null? f)))))
> + (map (lambda (f)

To improve readability, I’d make it:

(filter-map (lambda (f)
(match f
(() #f)
("" #f)
(_ …)))
docker-cli-configuration-fields)

Toggle quote (10 lines)
> +(define (home-docker-cli-configuration-file config)
> + `((".docker/config.json"
> + ,(computed-file "docker-cli-config.json"
> + #~(with-output-to-file #$output
> + (lambda _
> + (display
> + (string-append "{"
> + (string-join (list #$@(docker-cli-configuration->json-fields config)) ",")
> + #$(docker-cli-configuration-extra-content config)

I think the comma should be automatically added when ‘extra-content’ is
non-empty.

A more general question: do you think this particular example (libsecret
plugin) could be solved in another way without involving Home? (For
example by having a plugin search path in the package.)

Do you have other use cases in mind?

The reason I’m asking is that it feels heavyweight for what looks like
“basic” Docker configuration. But maybe Docker is like this and a Home
service is what it takes to make it more easily configurable (I’m not
really familiar with Docker), in which case I’m all for this patch
series!

Thanks,
Ludo’.
P
(name . Ludovic Courtès)(address . ludo@gnu.org)
db2997e0-10a8-7bc9-a73d-091188121b7d@autistici.org
Hello Ludo’ ,

I think you are raising a fair point. I was also not sure about whether
to upstream this service, even if I have been using it for some months.

On 5/4/24 18:36, Ludovic Courtès wrote:
Toggle quote (5 lines)
> A more general question: do you think this particular example (libsecret
> plugin) could be solved in another way without involving Home? (For
> example by having a plugin search path in the package.)
>
> Do you have other use cases in mind?
My main use case for this is currently is docker compose v2. I packaged
the binary from github in my personal channel [0] and with this service
[1]in my home-environment I'm able to use docker compose (without dash)
commands.
Toggle quote (3 lines)
> The reason I’m asking is that it feels heavyweight for what looks like
> “basic” Docker configuration.

It definitely is and whether to accept this patch imho depends on the
ETA for the optimal solution. Assuming someone packages docker compose
v2 and its dependencies, then as far as I'm aware there are are 3 ways
to make docker-cli aware of plugins on Guix:

1. this service, i.e. listing plugins in docker's configuration file.
the advantage of this solution is that it doesn't require changing the
docker-cli package. the disadvantage is that docker for some reason
always tries to write the following to the config file, even if it is
not needed :


this is why i added it in the documentation, but as you pointed out it
is useless.

2. We make a docker-cli-with-bundled-plugins function or similar that
takes a list of plugin packages and hardcodes their paths into docker's
source code before compilation. This is the approach I took here [2].
This currently requires recompilation every time since docker compose v2
is not in guix.

3. We patch docker [3] to make it aware of plugins with a search path
like DOCKER_CLI_PLUGINS . But I don't think this patch should live in
Guix. It should go into docker mainline to make sure that it is
supported also in future releases. I never tried contributing to docker,
this does not seem a complex change but I wonder why it was never done
until now.

Given all of the above the shortest way to achieve my goal of using
docker compose v2 seemed #1 but I can see how for the Guix project the
best would be #3. What do you think (and everyone CCed as well)? Could
this service be a temporary workaround (after addressing your comments),
while I try to see whether docker mainline could be interested in a patch?

I will address your comments after we reach consensus on how to proceed,
so that I don't make everyone lose more time than I already have. Thank
you all for your work,

giacomo

[0]:
[1]:
[2]:
[3]:
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 70265@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 70265
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch