[PATCH] services: Add restic-backup service.

  • Done
  • quality assurance status badge
Details
3 participants
  • Giacomo Leidi
  • Ludovic Courtès
  • Richard Sent
Owner
unassigned
Submitted by
Giacomo Leidi
Severity
normal
G
G
Giacomo Leidi wrote on 2 Mar 21:51 +0100
(address . guix-patches@gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
3afc07b0f3e6663a9fb64203544bce1659f97364.1709412684.git.goodoldpaul@autistici.org
* gnu/services/backup.scm: New file.
* gnu/local.mk: Add this.
* doc/guix.texi: Document this.

Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66
---
doc/guix.texi | 95 +++++++++++++++++++++++-
gnu/local.mk | 1 +
gnu/services/backup.scm | 160 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 255 insertions(+), 1 deletion(-)
create mode 100644 gnu/services/backup.scm

Toggle diff (295 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 87fe9f803c..4e53d22c5a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -111,7 +111,7 @@
Copyright @copyright{} 2022 John Kehayias@*
Copyright @copyright{} 2022?–?2023 Bruno Victal@*
Copyright @copyright{} 2022 Ivan Vilata-i-Balaguer@*
-Copyright @copyright{} 2023 Giacomo Leidi@*
+Copyright @copyright{} 2023, 2024 Giacomo Leidi@*
Copyright @copyright{} 2022 Antero Mejr@*
Copyright @copyright{} 2023 Karl Hallsby@*
Copyright @copyright{} 2023 Nathaniel Nicandro@*
@@ -41045,6 +41045,99 @@ Miscellaneous Services
@c End of auto-generated fail2ban documentation.
+@cindex Backup
+@subsubheading Backup services
+
+The @code{(gnu services backup)} module offers services for backing up
+file system trees. For now, it provides the @code{restic-backup-service-type}.
+
+To backup a list of file system trees to a pre-initialized, end-to-end
+encrypted, deduplicated data repository, you could so with the
+@code{restic-backup-service-type}. For example with the following
+configuration:
+
+@lisp
+(service restic-backup-service-type
+ (restic-backup-configuration
+ (jobs
+ (list (restic-backup-job
+ (repository "rclone:remote-ftp:backup/restic")
+ (password-file "/root/.restic")
+ ;; Every day at 23.
+ (specification "0 23 * * *")
+ (included '("/root/.restic"
+ "/root/.config/rclone"
+ "/etc/ssh/ssh_host_rsa_key"
+ "/etc/ssh/ssh_host_rsa_key.pub"
+ "/etc/guix/signing-key.pub"
+ "/etc/guix/signing-key.sec")))))))
+@end lisp
+
+Each @code{restic-backup-job} translates to an mcron job which sets the
+@code{RESTIC_PASSWORD} environment variable by reading the first line of
+@code{password-file} and runs @command{restic backup}.
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-configuration
+Available @code{restic-backup-configuration} fields are:
+
+@table @asis
+@item @code{jobs} (default: @code{'()}) (type: list-of-restic-backup-jobs)
+The list of backup jobs for the current system.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-job
+Available @code{restic-backup-job} fields are:
+
+@table @asis
+@item @code{restic} (default: @code{restic}) (type: package)
+The restic package to be used for the current job.
+
+@item @code{user} (default: @code{"root"}) (type: string)
+The user used for running the current job.
+
+@item @code{repository} (type: string)
+The restic repository target of this job.
+
+@item @code{password-file} (type: string)
+The path of a password file, readable by the configured @code{user},
+that will be used to set the @code{RESTIC_PASSWORD} environment variable
+for the current job.
+
+@item @code{specification} (type: gexp-or-string)
+A string or a gexp that will be passed as time specification in the
+mcron job specification (@pxref{Syntax, mcron job specifications,,
+mcron,GNU@tie{}mcron}).
+
+@item @code{included} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings representing filesystem
+paths. These are the paths that will be recursively included in the
+current job.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+Whether to enable verbose output for the current backup job.
+
+@item @code{extra-flags} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings. These will be passed as
+command-line arguments to the current job @command{restic backup}
+invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
@node Setuid Programs
@section Setuid Programs
diff --git a/gnu/local.mk b/gnu/local.mk
index cabd82f532..bf911327f4 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -693,6 +693,7 @@ GNU_SYSTEM_MODULES = \
%D%/services/auditd.scm \
%D%/services/avahi.scm \
%D%/services/base.scm \
+ %D%/services/backup.scm \
%D%/services/certbot.scm \
%D%/services/cgit.scm \
%D%/services/ci.scm \
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
new file mode 100644
index 0000000000..e9172af8c4
--- /dev/null
+++ b/gnu/services/backup.scm
@@ -0,0 +1,160 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services backup)
+ #:use-module (gnu packages backup)
+ #:use-module (gnu services)
+ #:use-module (gnu services configuration)
+ #:use-module (gnu services mcron)
+ #:use-module (guix gexp)
+ #:use-module (guix modules)
+ #:use-module (guix packages)
+ #:use-module (srfi srfi-1)
+ #:export (restic-backup-job
+ restic-backup-job?
+ restic-backup-job-fields
+ restic-backup-job-restic
+ restic-backup-job-user
+ restic-backup-job-repository
+ restic-backup-job-password-file
+ restic-backup-job-included
+ restic-backup-job-verbose?
+ restic-backup-job-extra-flags
+
+ restic-backup-configuration
+ restic-backup-configuration?
+ restic-backup-configuration-fields
+ restic-backup-configuration-jobs
+
+ restic-backup-job-program
+ restic-backup-job->mcron-job
+ restic-backup-service-type))
+
+(define (gexp-or-string? value)
+ (or (gexp? value)
+ (string? value)))
+
+(define (lowerable? value)
+ (or (file-like? value)
+ (gexp-or-string? value)))
+
+(define list-of-lowerables?
+ (list-of lowerable?))
+
+(define-configuration/no-serialization restic-backup-job
+ (restic
+ (package restic)
+ "The restic package to be used for the current job.")
+ (user
+ (string "root")
+ "The user used for running the current job.")
+ (repository
+ (string)
+ "The restic repository target of this job.")
+ (password-file
+ (string)
+ "The path of a password file, readable by the configured @code{user}, that
+will be used to set the @code{RESTIC_PASSWORD} environment variable for the
+current job.")
+ (specification
+ (gexp-or-string)
+ "A string or a gexp that will be passed as time specification in the mcron
+job specification (@pxref{Syntax, mcron job specifications,, mcron,
+GNU@tie{}mcron}).")
+ (included
+ (list-of-lowerables '())
+ "A list of values that are lowered to strings representing filesystem paths.
+These are the paths that will be recursively included in the current job.")
+ (verbose?
+ (boolean #f)
+ "Whether to enable verbose output for the current backup job.")
+ (extra-flags
+ (list-of-lowerables '())
+ "A list of values that are lowered to strings. These will be passed as
+command-line arguments to the current job @command{restic backup} invokation."))
+
+(define list-of-restic-backup-jobs?
+ (list-of restic-backup-job?))
+
+(define-configuration/no-serialization restic-backup-configuration
+ (jobs
+ (list-of-restic-backup-jobs '())
+ "The list of backup jobs for the current system."))
+
+(define (restic-backup-job-program config)
+ (let ((restic
+ (file-append (restic-backup-job-restic config) "/bin/restic"))
+ (repository
+ (restic-backup-job-repository config))
+ (password-file
+ (restic-backup-job-password-file config))
+ (included
+ (restic-backup-job-included config))
+ (extra-flags
+ (restic-backup-job-extra-flags config))
+ (verbose
+ (if (restic-backup-job-verbose? config)
+ '("--verbose")
+ '())))
+ (program-file
+ "restic-backup-job.scm"
+ (with-imported-modules (source-module-closure
+ '((guix build utils)))
+ #~(begin
+ (use-modules (guix build utils)
+ (ice-9 popen)
+ (ice-9 rdelim))
+ (setenv "RESTIC_PASSWORD"
+ (with-input-from-file #$password-file read-line))
+
+ (execlp #$restic #$@verbose
+ "-r" #$repository
+ #$@extra-flags
+ "backup" #$@included))))))
+
+(define (restic-backup-job->mcron-job config)
+ (let ((user
+ (restic-backup-job-user config))
+ (specification
+ (restic-backup-job-specification config))
+ (program
+ (restic-backup-job-program config)))
+ #~(job #$specification
+ #$program
+ #:user #$user)))
+
+(define restic-backup-service-type
+ (service-type (name 'restic-backup)
+ (extensions
+ (list
+ (service-extension mcron-service-type
+ (lambda (config)
+ (map restic-backup-job->mcron-job
+ (restic-backup-configuration-jobs
+ config))))))
+ (compose concatenate)
+ (extend
+ (lambda (config jobs)
+ (restic-backup-configuration
+ (inherit config)
+ (jobs (append (restic-backup-configuration-jobs config)
+ jobs)))))
+ (default-value (restic-backup-configuration))
+ (description
+ "This service configures @code{mcron} jobs for running backups
+with @code{restic}.")))

base-commit: 6f5ea7ac1acb3d1c53baf7620cca66cc87fe5a73
--
2.41.0
L
L
Ludovic Courtès wrote on 29 Mar 23:36 +0100
(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)(address . 69513@debbugs.gnu.org)
87il14fyxg.fsf@gnu.org
Hi,

Giacomo Leidi <goodoldpaul@autistici.org> skribis:

Toggle quote (6 lines)
> * gnu/services/backup.scm: New file.
> * gnu/local.mk: Add this.
> * doc/guix.texi: Document this.
>
> Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66

[...]

Toggle quote (2 lines)
> +@subsubheading Backup services

Please capitalize headings: “Backup Services”.

We should probably move documentation of ‘syncthing-service-type’ here,
even if they live in different modules for now.

Toggle quote (2 lines)
> +The @code{(gnu services backup)} module offers services for backing up
> +file system trees. For now, it provides the @code{restic-backup-service-type}.
^
Nitpick: Please leave two spaces after an end-of-sentence period (for
easier Emacs navigation, readability, and consistency).

Toggle quote (5 lines)
> +To backup a list of file system trees to a pre-initialized, end-to-end
> +encrypted, deduplicated data repository, you could so with the
> +@code{restic-backup-service-type}. For example with the following
> +configuration:

How about:

With @code{restic-backup-service-type}, you can periodically back up
directories and files with @uref{https://restic.net/,Restic}, which
supports end-to-end encryption and deduplication. Consider the
following configuration:

?

Toggle quote (3 lines)
> +Each @code{restic-backup-job} translates to an mcron job which sets the
> +@code{RESTIC_PASSWORD} environment variable by reading the first line of

@env{RESTIC_PASSWORD}

Toggle quote (5 lines)
> +@item @code{specification} (type: gexp-or-string)
> +A string or a gexp that will be passed as time specification in the
> +mcron job specification (@pxref{Syntax, mcron job specifications,,
> +mcron,GNU@tie{}mcron}).

Maybe ‘schedule’ rather than ‘specification’, to clarify what’s being
specified? (That’s the name I chose in <unattended-upgrade-configuration>.)

Toggle quote (5 lines)
> +@item @code{included} (default: @code{'()}) (type: list-of-lowerables)
> +A list of values that are lowered to strings representing filesystem
> +paths. These are the paths that will be recursively included in the
> +current job.

In GNU and Guix, “path” is used to denote “search paths”; in other
cases, we write “file name” or “file”. So I’d suggest something like:

The list of files or directories to be backed up.

The ‘files-to-backup’ (or ‘files’?) may be more descriptive that
‘included’.

Toggle quote (4 lines)
> + (password-file
> + (string)
> + "The path of a password file, readable by the configured @code{user}, that

“Name of the password file”

Toggle quote (2 lines)
> +will be used to set the @code{RESTIC_PASSWORD} environment variable for the

s/@code/@env/

Toggle quote (16 lines)
> + (program-file
> + "restic-backup-job.scm"
> + (with-imported-modules (source-module-closure
> + '((guix build utils)))
> + #~(begin
> + (use-modules (guix build utils)
> + (ice-9 popen)
> + (ice-9 rdelim))
> + (setenv "RESTIC_PASSWORD"
> + (with-input-from-file #$password-file read-line))
> +
> + (execlp #$restic #$@verbose
> + "-r" #$repository
> + #$@extra-flags
> + "backup" #$@included))))))

I believe (guix build utils) is unused, in which case you can remove it.

The ‘execlp’ call lacks argv[0]; it should look like this:

(execlp #$restic #$restic #$@verbose "-r" …)

(Notice that #$restic appears twice.)

Could you send an updated patch?

Thanks,
Ludo’.
P
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 69513@debbugs.gnu.org)
2799d601-6ca3-c042-30ae-5a045c65b026@autistici.org
Hello Ludo',

thank you for your insight, I should have addressed all of your
comments. I'm sending an updated patch.


cheers

giacomo
G
G
Giacomo Leidi wrote on 2 Apr 22:34 +0200
[PATCH v2] services: Add restic-backup service.
(address . 69513@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
bf4da43485bcd3a0a6ab5217760347bbff732077.1712090046.git.goodoldpaul@autistici.org
* gnu/services/backup.scm: New file.
* gnu/local.mk: Add this.
* doc/guix.texi: Document this.

Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66
---
doc/guix.texi | 98 +++++++++++++++++++++++++
gnu/local.mk | 1 +
gnu/services/backup.scm | 158 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 257 insertions(+)
create mode 100644 gnu/services/backup.scm

Toggle diff (289 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 69a904473c..a13efbff7b 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41129,6 +41129,104 @@ Miscellaneous Services
@c End of auto-generated fail2ban documentation.
+@cindex Backup
+@subsubheading Backup Services
+
+The @code{(gnu services backup)} module offers services for backing up
+file system trees. For now, it provides the @code{restic-backup-service-type}.
+
+With @code{restic-backup-service-type}, you can periodically back up
+directories and files with @uref{https://restic.net/, Restic}, which
+supports end-to-end encryption and deduplication. Consider the
+following configuration:
+
+@lisp
+(operating-system
+
+ (packages (list "rclone"))
+
+ (services
+ (list
+ (service restic-backup-service-type
+ (restic-backup-configuration
+ (jobs
+ (list (restic-backup-job
+ (repository "rclone:remote-ftp:backup/restic")
+ (password-file "/root/.restic")
+ ;; Every day at 23.
+ (schedule "0 23 * * *")
+ (files '("/root/.restic"
+ "/root/.config/rclone"
+ "/etc/ssh/ssh_host_rsa_key"
+ "/etc/ssh/ssh_host_rsa_key.pub"
+ "/etc/guix/signing-key.pub"
+ "/etc/guix/signing-key.sec"))))))))))
+@end lisp
+
+Each @code{restic-backup-job} translates to an mcron job which sets the
+@env{RESTIC_PASSWORD} environment variable by reading the first line of
+@code{password-file} and runs @command{restic backup}.
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-configuration
+Available @code{restic-backup-configuration} fields are:
+
+@table @asis
+@item @code{jobs} (default: @code{'()}) (type: list-of-restic-backup-jobs)
+The list of backup jobs for the current system.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-job
+Available @code{restic-backup-job} fields are:
+
+@table @asis
+@item @code{restic} (default: @code{restic}) (type: package)
+The restic package to be used for the current job.
+
+@item @code{user} (default: @code{"root"}) (type: string)
+The user used for running the current job.
+
+@item @code{repository} (type: string)
+The restic repository target of this job.
+
+@item @code{password-file} (type: string)
+Name of the password file, readable by the configured @code{user},
+that will be used to set the @env{RESTIC_PASSWORD} environment variable
+for the current job.
+
+@item @code{schedule} (type: gexp-or-string)
+A string or a gexp that will be passed as time specification in the
+mcron job specification (@pxref{Syntax, mcron job specifications,,
+mcron,GNU@tie{}mcron}).
+
+@item @code{files} (default: @code{'()}) (type: list-of-lowerables)
+The list of files or directories to be backed up. It must be a list of
+values that can be lowered to strings.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+Whether to enable verbose output for the current backup job.
+
+@item @code{extra-flags} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings. These will be passed as
+command-line arguments to the current job @command{restic backup}
+invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
@node Setuid Programs
@section Setuid Programs
diff --git a/gnu/local.mk b/gnu/local.mk
index f2b480bded..be7a968459 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -696,6 +696,7 @@ GNU_SYSTEM_MODULES = \
%D%/services/auditd.scm \
%D%/services/avahi.scm \
%D%/services/base.scm \
+ %D%/services/backup.scm \
%D%/services/certbot.scm \
%D%/services/cgit.scm \
%D%/services/ci.scm \
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
new file mode 100644
index 0000000000..2bd9e2f29a
--- /dev/null
+++ b/gnu/services/backup.scm
@@ -0,0 +1,158 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services backup)
+ #:use-module (gnu packages backup)
+ #:use-module (gnu services)
+ #:use-module (gnu services configuration)
+ #:use-module (gnu services mcron)
+ #:use-module (guix gexp)
+ #:use-module (guix modules)
+ #:use-module (guix packages)
+ #:use-module (srfi srfi-1)
+ #:export (restic-backup-job
+ restic-backup-job?
+ restic-backup-job-fields
+ restic-backup-job-restic
+ restic-backup-job-user
+ restic-backup-job-repository
+ restic-backup-job-password-file
+ restic-backup-job-schedule
+ restic-backup-job-files
+ restic-backup-job-verbose?
+ restic-backup-job-extra-flags
+
+ restic-backup-configuration
+ restic-backup-configuration?
+ restic-backup-configuration-fields
+ restic-backup-configuration-jobs
+
+ restic-backup-job-program
+ restic-backup-job->mcron-job
+ restic-backup-service-type))
+
+(define (gexp-or-string? value)
+ (or (gexp? value)
+ (string? value)))
+
+(define (lowerable? value)
+ (or (file-like? value)
+ (gexp-or-string? value)))
+
+(define list-of-lowerables?
+ (list-of lowerable?))
+
+(define-configuration/no-serialization restic-backup-job
+ (restic
+ (package restic)
+ "The restic package to be used for the current job.")
+ (user
+ (string "root")
+ "The user used for running the current job.")
+ (repository
+ (string)
+ "The restic repository target of this job.")
+ (password-file
+ (string)
+ "Name of the password file, readable by the configured @code{user}, that
+will be used to set the @code{RESTIC_PASSWORD} environment variable for the
+current job.")
+ (schedule
+ (gexp-or-string)
+ "A string or a gexp that will be passed as time specification in the mcron
+job specification (@pxref{Syntax, mcron job specifications,, mcron,
+GNU@tie{}mcron}).")
+ (files
+ (list-of-lowerables '())
+ "The list of files or directories to be backed up. It must be a list of
+values that can be lowered to strings.")
+ (verbose?
+ (boolean #f)
+ "Whether to enable verbose output for the current backup job.")
+ (extra-flags
+ (list-of-lowerables '())
+ "A list of values that are lowered to strings. These will be passed as
+command-line arguments to the current job @command{restic backup} invokation."))
+
+(define list-of-restic-backup-jobs?
+ (list-of restic-backup-job?))
+
+(define-configuration/no-serialization restic-backup-configuration
+ (jobs
+ (list-of-restic-backup-jobs '())
+ "The list of backup jobs for the current system."))
+
+(define (restic-backup-job-program config)
+ (let ((restic
+ (file-append (restic-backup-job-restic config) "/bin/restic"))
+ (repository
+ (restic-backup-job-repository config))
+ (password-file
+ (restic-backup-job-password-file config))
+ (files
+ (restic-backup-job-files config))
+ (extra-flags
+ (restic-backup-job-extra-flags config))
+ (verbose
+ (if (restic-backup-job-verbose? config)
+ '("--verbose")
+ '())))
+ (program-file
+ "restic-backup-job.scm"
+ #~(begin
+ (use-modules (ice-9 popen)
+ (ice-9 rdelim))
+ (setenv "RESTIC_PASSWORD"
+ (with-input-from-file #$password-file read-line))
+
+ (execlp #$restic #$restic #$@verbose
+ "-r" #$repository
+ #$@extra-flags
+ "backup" #$@files)))))
+
+(define (restic-backup-job->mcron-job config)
+ (let ((user
+ (restic-backup-job-user config))
+ (schedule
+ (restic-backup-job-schedule config))
+ (program
+ (restic-backup-job-program config)))
+ #~(job #$schedule
+ #$program
+ #:user #$user)))
+
+(define restic-backup-service-type
+ (service-type (name 'restic-backup)
+ (extensions
+ (list
+ (service-extension mcron-service-type
+ (lambda (config)
+ (map restic-backup-job->mcron-job
+ (restic-backup-configuration-jobs
+ config))))))
+ (compose concatenate)
+ (extend
+ (lambda (config jobs)
+ (restic-backup-configuration
+ (inherit config)
+ (jobs (append (restic-backup-configuration-jobs config)
+ jobs)))))
+ (default-value (restic-backup-configuration))
+ (description
+ "This service configures @code{mcron} jobs for running backups
+with @code{restic}.")))

base-commit: 7af70efd7633b0d70091762cf43ce01a86176e8e
--
2.41.0
P
Re: [bug#69513] [PATCH] services: Add restic-backup service.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 69513@debbugs.gnu.org)
b3095165-9663-2d13-3534-498c094f6e57@autistici.org
Hello Ludo' ,


I'm sending a v3 rebased on current master. In this revision I added the
possibility to manually trigger a backup without waiting for the
scheduled job to run.

Thank you for your work,


giacomo
G
G
Giacomo Leidi wrote on 1 May 23:15 +0200
[PATCH v3] services: Add restic-backup service.
(address . 69513@debbugs.gnu.org)(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)
4d1b9e86fa4c0f70b30d45de5a29ec19b41c0e8f.1714598106.git.goodoldpaul@autistici.org
* gnu/services/backup.scm: New file.
* gnu/local.mk: Add this.
* doc/guix.texi: Document this.

Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66
---
doc/guix.texi | 112 +++++++++++++++++++
gnu/local.mk | 1 +
gnu/services/backup.scm | 236 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 349 insertions(+)
create mode 100644 gnu/services/backup.scm

Toggle diff (381 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 3f5d4e7f0d..1b52c43c34 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -40946,6 +40946,118 @@ Miscellaneous Services
@c End of auto-generated fail2ban documentation.
+@cindex Backup
+@subsubheading Backup Services
+
+The @code{(gnu services backup)} module offers services for backing up
+file system trees. For now, it provides the @code{restic-backup-service-type}.
+
+With @code{restic-backup-service-type}, you can periodically back up
+directories and files with @uref{https://restic.net/, Restic}, which
+supports end-to-end encryption and deduplication. Consider the
+following configuration:
+
+@lisp
+(operating-system
+
+ (packages (list "rclone"))
+
+ (services
+ (list
+ (service restic-backup-service-type
+ (restic-backup-configuration
+ (jobs
+ (list (restic-backup-job
+ (name "remote-ftp")
+ (repository "rclone:remote-ftp:backup/restic")
+ (password-file "/root/.restic")
+ ;; Every day at 23.
+ (schedule "0 23 * * *")
+ (files '("/root/.restic"
+ "/root/.config/rclone"
+ "/etc/ssh/ssh_host_rsa_key"
+ "/etc/ssh/ssh_host_rsa_key.pub"
+ "/etc/guix/signing-key.pub"
+ "/etc/guix/signing-key.sec"))))))))))
+@end lisp
+
+Each @code{restic-backup-job} translates to an mcron job which sets the
+@env{RESTIC_PASSWORD} environment variable by reading the first line of
+@code{password-file} and runs @command{restic backup}.
+
+The @code{restic-backup-service-type} installs as well @code{restic-guix}
+to the system profile, a @code{restic} utility wrapper that allows for easier
+interaction with the Guix configured backup jobs. For example the following
+could be used to instantaneusly trigger a backup for the above shown
+configuration, without waiting for the scheduled job:
+
+@example
+restic-guix backup remote-ftp
+@end example
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-configuration
+Available @code{restic-backup-configuration} fields are:
+
+@table @asis
+@item @code{jobs} (default: @code{'()}) (type: list-of-restic-backup-jobs)
+The list of backup jobs for the current system.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
+@c %start of fragment
+
+@deftp {Data Type} restic-backup-job
+Available @code{restic-backup-job} fields are:
+
+@table @asis
+@item @code{restic} (default: @code{restic}) (type: package)
+The restic package to be used for the current job.
+
+@item @code{user} (default: @code{"root"}) (type: string)
+The user used for running the current job.
+
+@item @code{repository} (type: string)
+The restic repository target of this job.
+
+@item @code{name} (type: string)
+A string denoting a name for this job.
+
+@item @code{password-file} (type: string)
+Name of the password file, readable by the configured @code{user},
+that will be used to set the @env{RESTIC_PASSWORD} environment variable
+for the current job.
+
+@item @code{schedule} (type: gexp-or-string)
+A string or a gexp that will be passed as time specification in the
+mcron job specification (@pxref{Syntax, mcron job specifications,,
+mcron,GNU@tie{}mcron}).
+
+@item @code{files} (default: @code{'()}) (type: list-of-lowerables)
+The list of files or directories to be backed up. It must be a list of
+values that can be lowered to strings.
+
+@item @code{verbose?} (default: @code{#f}) (type: boolean)
+Whether to enable verbose output for the current backup job.
+
+@item @code{extra-flags} (default: @code{'()}) (type: list-of-lowerables)
+A list of values that are lowered to strings. These will be passed as
+command-line arguments to the current job @command{restic backup}
+invokation.
+
+@end table
+
+@end deftp
+
+
+@c %end of fragment
+
@node Setuid Programs
@section Setuid Programs
diff --git a/gnu/local.mk b/gnu/local.mk
index f1dab53f2b..bb17ef182a 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -699,6 +699,7 @@ GNU_SYSTEM_MODULES = \
%D%/services/auditd.scm \
%D%/services/avahi.scm \
%D%/services/base.scm \
+ %D%/services/backup.scm \
%D%/services/certbot.scm \
%D%/services/cgit.scm \
%D%/services/ci.scm \
diff --git a/gnu/services/backup.scm b/gnu/services/backup.scm
new file mode 100644
index 0000000000..555e9fc959
--- /dev/null
+++ b/gnu/services/backup.scm
@@ -0,0 +1,236 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services backup)
+ #:use-module (gnu packages backup)
+ #:use-module (gnu services)
+ #:use-module (gnu services configuration)
+ #:use-module (gnu services mcron)
+ #:use-module (guix build-system copy)
+ #:use-module (guix gexp)
+ #:use-module ((guix licenses)
+ #:prefix license:)
+ #:use-module (guix modules)
+ #:use-module (guix packages)
+ #:use-module (srfi srfi-1)
+ #:export (restic-backup-job
+ restic-backup-job?
+ restic-backup-job-fields
+ restic-backup-job-restic
+ restic-backup-job-user
+ restic-backup-job-name
+ restic-backup-job-repository
+ restic-backup-job-password-file
+ restic-backup-job-schedule
+ restic-backup-job-files
+ restic-backup-job-verbose?
+ restic-backup-job-extra-flags
+
+ restic-backup-configuration
+ restic-backup-configuration?
+ restic-backup-configuration-fields
+ restic-backup-configuration-jobs
+
+ restic-backup-job-program
+ restic-backup-job->mcron-job
+ restic-guix
+ restic-guix-wrapper-package
+ restic-backup-service-profile
+ restic-backup-service-type))
+
+(define (gexp-or-string? value)
+ (or (gexp? value)
+ (string? value)))
+
+(define (lowerable? value)
+ (or (file-like? value)
+ (gexp-or-string? value)))
+
+(define list-of-lowerables?
+ (list-of lowerable?))
+
+(define-configuration/no-serialization restic-backup-job
+ (restic
+ (package restic)
+ "The restic package to be used for the current job.")
+ (user
+ (string "root")
+ "The user used for running the current job.")
+ (name
+ (string)
+ "A string denoting a name for this job.")
+ (repository
+ (string)
+ "The restic repository target of this job.")
+ (password-file
+ (string)
+ "Name of the password file, readable by the configured @code{user}, that
+will be used to set the @code{RESTIC_PASSWORD} environment variable for the
+current job.")
+ (schedule
+ (gexp-or-string)
+ "A string or a gexp that will be passed as time specification in the mcron
+job specification (@pxref{Syntax, mcron job specifications,, mcron,
+GNU@tie{}mcron}).")
+ (files
+ (list-of-lowerables '())
+ "The list of files or directories to be backed up. It must be a list of
+values that can be lowered to strings.")
+ (verbose?
+ (boolean #f)
+ "Whether to enable verbose output for the current backup job.")
+ (extra-flags
+ (list-of-lowerables '())
+ "A list of values that are lowered to strings. These will be passed as
+command-line arguments to the current job @command{restic backup} invokation."))
+
+(define list-of-restic-backup-jobs?
+ (list-of restic-backup-job?))
+
+(define-configuration/no-serialization restic-backup-configuration
+ (jobs
+ (list-of-restic-backup-jobs '())
+ "The list of backup jobs for the current system."))
+
+(define (restic-backup-job-program config)
+ (let ((restic
+ (file-append (restic-backup-job-restic config) "/bin/restic"))
+ (repository
+ (restic-backup-job-repository config))
+ (password-file
+ (restic-backup-job-password-file config))
+ (files
+ (restic-backup-job-files config))
+ (extra-flags
+ (restic-backup-job-extra-flags config))
+ (verbose
+ (if (restic-backup-job-verbose? config)
+ '("--verbose")
+ '())))
+ (program-file
+ "restic-backup-job.scm"
+ #~(begin
+ (use-modules (ice-9 popen)
+ (ice-9 rdelim))
+ (setenv "RESTIC_PASSWORD"
+ (with-input-from-file #$password-file read-line))
+
+ (execlp #$restic #$restic #$@verbose
+ "-r" #$repository
+ #$@extra-flags
+ "backup" #$@files)))))
+
+(define (restic-guix jobs)
+ (program-file
+ "restic-guix"
+ #~(begin
+ (use-modules (ice-9 match)
+ (srfi srfi-1))
+
+ (define names '#$(map restic-backup-job-name jobs))
+ (define programs '#$(map restic-backup-job-program jobs))
+
+ (define (get-program name)
+ (define idx
+ (list-index (lambda (n) (string=? n name)) names))
+ (unless idx
+ (error (string-append "Unknown job name " name "\n\n"
+ "Possible job names are: "
+ (string-join names " "))))
+ (list-ref programs idx))
+
+ (define (backup args)
+ (define name (third args))
+ (define program (get-program name))
+ (execlp program program))
+
+ (define (validate-args args)
+ (when (not (>= (length args) 3))
+ (error (string-append "Usage: " (basename (car args))
+ " backup NAME"))))
+
+ (define (main args)
+ (validate-args args)
+ (define action (second args))
+ (match action
+ ("backup"
+ (backup args))
+ (_
+ (error (string-append "Unknown action: " action)))))
+
+ (main (command-line)))))
+
+(define (restic-backup-job->mcron-job config)
+ (let ((user
+ (restic-backup-job-user config))
+ (schedule
+ (restic-backup-job-schedule config))
+ (name
+ (restic-backup-job-name config)))
+ #~(job #$schedule
+ #$(string-append "restic-guix backup " name)
+ #:user #$user)))
+
+(define (restic-guix-wrapper-package jobs)
+ (package
+ (name "restic-backup-service-wrapper")
+ (version "0.0.0")
+ (source (restic-guix jobs))
+ (build-system copy-build-system)
+ (arguments
+ (list #:install-plan #~'(("./" "/bin"))))
+ (home-page "https://restic.net")
+ (synopsis
+ "Easily interact from the CLI with Guix configured backups")
+ (description
+ "This package provides a simple wrapper around @code{restic}, handled
+by the @code{restic-backup-service-type}. It allows for easily interacting
+with Guix configured backup jobs, for example for manually triggering a backup
+without waiting for the scheduled job to run.")
+ (license license:gpl3+)))
+
+(define restic-backup-service-profile
+ (lambda (config)
+ (define jobs (restic-backup-configuration-jobs config))
+ (if (> (length jobs) 0)
+ (list
+ (restic-guix-wrapper-package jobs))
+ '())))
+
+(define restic-backup-service-type
+ (service-type (name 'restic-backup)
+ (extensions
+ (list
+ (service-extension profile-service-type
+ restic-backup-service-profile)
+ (service-extension mcron-service-type
+ (lambda (config)
+ (map restic-backup-job->mcron-job
+ (restic-backup-configuration-jobs
+ config))))))
+ (compose concatenate)
+ (extend
+ (lambda (config jobs)
+ (restic-backup-configuration
+ (inherit config)
+ (jobs (append (restic-backup-configuration-jobs config)
+ jobs)))))
+ (default-value (restic-backup-configuration))
+ (description
+ "This service configures @code{mcron} jobs for running backups
+with @code{restic}.")))

base-commit: 7d4ae2fca723114fb1df56de33b82177fbc4d0a6
--
2.41.0
L
L
Ludovic Courtès wrote on 25 May 15:23 +0200
(name . Giacomo Leidi)(address . goodoldpaul@autistici.org)(address . 69513-done@debbugs.gnu.org)
87r0dq10uq.fsf@gnu.org
Hello,

Giacomo Leidi <goodoldpaul@autistici.org> skribis:

Toggle quote (6 lines)
> * gnu/services/backup.scm: New file.
> * gnu/local.mk: Add this.
> * doc/guix.texi: Document this.
>
> Change-Id: I9efd5559bb445b484107a7c27c2d0a65ccad1e66

Please consider adding a system test for this: as previously discussed,
we try hard to have tests for every system service.

I’ve applied it with the minor doc changes below.

There might be more work that could be done to ensure the doc is
self-contained. For instance, I merely guessed that ‘rclone’ needed to
be in the system profile so ‘restic’ would use it, and I cannot tell
what those repository URIs should look like. Perhaps this can be solved
with a few more words, examples, and/or links to the upstream doc?

Thank you!

Ludo’.
Toggle diff (31 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index acf35357a60..d2643cf7fd9 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -41102,10 +41102,13 @@ Miscellaneous Services
following configuration:
@lisp
+(use-service-modules backup @dots{}) ;for 'restic-backup-service-type'
+(use-package-modules sync @dots{}) ;for 'rclone'
+
(operating-system
-
- (packages (list "rclone"))
-
+ ;; @dots{}
+ (packages (append (list rclone) ;for use by restic
+ %base-packages))
(services
(list
(service restic-backup-service-type
@@ -41127,7 +41130,8 @@ Miscellaneous Services
Each @code{restic-backup-job} translates to an mcron job which sets the
@env{RESTIC_PASSWORD} environment variable by reading the first line of
-@code{password-file} and runs @command{restic backup}.
+@code{password-file} and runs @command{restic backup}, creating backups
+using rclone of all the files listed in the @code{files} field.
The @code{restic-backup-service-type} installs as well @code{restic-guix}
to the system profile, a @code{restic} utility wrapper that allows for easier
Closed
R
R
Richard Sent wrote on 30 May 21:23 +0200
Adding a couple new features and tests
(address . 69513@debbugs.gnu.org)(address . goodoldpaul@autistici.org)
87y17r85od.fsf@freakingpenguin.com
Hi all,

Toggle quote (3 lines)
> Please consider adding a system test for this: as previously
> discussed, we try hard to have tests for every system service.

I have a few changes I plan on submitting soon regarding the restic
service. FYSA Giacomo, I'm writing system tests as well, although those
tests will obviously be limited to local repositories.

For the curious I'm working on the following changes:

1. Add an init? field to attempt to bootstrap local repository

2. Add support for Restic's password command feature in addition to the
existing password file

3. Either add rclone to the profile or add an extra-packages field to
restic-backup-job so restic-required packages can be located in one
spot.

4. The aforementioned tests

I haven't used restic before so this might be harder than I think it'll
be. So far things seem to be going well.

--
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.
?
Your comment

This issue is archived.

To comment on this conversation send an email to 69513@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 69513
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch