[PATCH 0/4] Even Better ZFS Support on Guix

OpenSubmitted by raid5atemyhomework.
Details
10 participants
  • Danny Milosavljevic
  • 宋文武
  • Léo Le Bouter
  • Ludovic Courtès
  • Mason Loring Bliss
  • Maxime Devos
  • pelzflorian (Florian Pelz)
  • Xinglu Chen
  • raid5atemyhomework
  • zimoun
Owner
unassigned
Severity
normal
Merged with
R
R
raid5atemyhomework wrote on 6 Jan 16:52 +0100
(name . guix-patches@gnu.org)(address . guix-patches@gnu.org)
f9YNSdmHSIAVHWwDRwnh8QVeNjUG4M6YMJMntulZ3s8cYPVCBHrVIirwkuGxpYIduWVNAyi13dFVlqUKs7_QXPDBvbDe_JNNEC_SoNBTprk=@protonmail.com
Hi all,
Sorry for the excessive number of patches.
This patchset contains 4 patches, two that add new features to Guix and do not involve ZFS at all, one bugfix of the ZFS package, and one which makes installing ZFS on Guix easy.
* Patch 1: Create an extensible service type that installs kernel-loadable modules into the kernel. * This allows a ZFS service to add the ZFS module to the kernel.* Patch 2: Make the `file-systems` shepherd service an extensible target, so that other services can add more requirements for `file-systems`. * This allows a ZFS service to make `file-systems` wait for the ZFS automount at startup, which is necessary in any case in order to ensure that ZFS has scanned for ZFS pools installed in the hardware.* Patch 3: Fix the ZFS package definition.* Patch 4: Add a ZFS service type that installs everything needed for basic ZFS usage!
I've tested the ZFS functionality in about a half-dozen VMs already, including checking that the ZFS setup persists across reboots of the VM:
* No ZFS installed.* With ZFS installed but without any ZFS pools.* With ZFS pool that is automounted.* With ZFS pool in legacy mode (needs manual `mount -t zfs poolname /mountpoint`).* Encrypted ZFS dataset (prompts passphrase at boot to mount).* Encrypted ZFS dataseet as `/home` (prompts passphrase at boot to mount, user `/home` directories are created and maintained).
Other bits of ZFS functionality would be *nice* but aren't in this patchset yet (and at least for my purposes are not really necessary, except perhaps the auto-scrubbing, I just need to wrap my head around `mcron`...):
* Auto-scrubbing of ZFS pools to ensure no data corruption.* Auto-snapshotting of ZFS datasets to protect against user error.* `/` on ZFS.* `/boot` on ZFS.* Managing ZFS "legacy" mountpoints on `(file-system ...)` declarations.* Managing ZFS pool setup in the `operating-system` declaration somehow (not sure how to do this).
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 6 Jan 16:54 +0100
[PATCH 1/4] gnu: Allow services to install kernel-loadable modules.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
2yOs7mxigs8_Vx_yi858pxAkoyXmU8IidizK5grcmX4M92O5QK-_Ywo9nrHMHvzLGj4U7B9ysl26eNbiPmzmeOQsuFvp5VjouVptRfwxKng=@protonmail.com
From 4beb73c62995cf236b402dad8e1c36016027c781 Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Tue, 5 Jan 2021 22:27:56 +0800Subject: [PATCH 1/4] gnu: Allow services to install kernel-loadable modules.
* gnu/system.scm (operating-system-directory-base-entries): Remove codeto handle generation of "kernel" and "hurd".(operating-system-default-essential-services): Instantiatekernel-loadable-module-service.(hurd-default-essential-services): Instantiatekernel-loadable-module-service.(package-for-kernel): Move ...* gnu/services.scm: ... to here.(kernel-loadable-module-service-type): New variable.(kernel-loadable-module-service): New procedure.* gnu/tests/linux-modules.scm (run-loadable-kernel-modules-test): Movecode to ...(run-loadable-kernel-modules-test-base): ... new procedure here.(run-loadable-kernel-modules-service-test): New procedure.(%test-loadable-kernel-modules-service-0): New variable.(%test-loadable-kernel-modules-service-1): New variable.(%test-loadable-kernel-modules-service-2): New variable.* doc/guix.texi: Document kernel-loadable-module-service-type.--- doc/guix.texi | 6 +++ gnu/services.scm | 70 ++++++++++++++++++++++++++++++++ gnu/system.scm | 37 +++++------------ gnu/tests/linux-modules.scm | 81 ++++++++++++++++++++++++++++++++----- 4 files changed, 157 insertions(+), 37 deletions(-)
Toggle diff (324 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 0f6e95a65a..78770151e3 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -32409,6 +32409,12 @@ configuration when you use @command{guix system reconfigure}, @command{guix system init}, or @command{guix deploy}. @end defvr
+@defvr {Scheme Variable} kernel-loadable-module-service-type+Type of the service that collects lists of packages containing+kernel-loadable modules, and adds them to the set of kernel-loadable+modules.+@end defvr+ @node Shepherd Services @subsection Shepherd Services
diff --git a/gnu/services.scm b/gnu/services.scmindex 13259dfaee..d7332a46b2 100644--- a/gnu/services.scm+++ b/gnu/services.scm@@ -2,6 +2,7 @@ ;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org> ;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com> ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -33,6 +34,8 @@ #:use-module (guix diagnostics) #:autoload (guix openpgp) (openpgp-format-fingerprint) #:use-module (guix modules)+ #:use-module (guix packages)+ #:use-module (guix utils) #:use-module (gnu packages base) #:use-module (gnu packages bash) #:use-module (gnu packages hurd)@@ -75,6 +78,7 @@ service-back-edges instantiate-missing-services fold-services+ kernel-loadable-module-service
service-error? missing-value-service-error?@@ -106,6 +110,7 @@ profile-service-type firmware-service-type gc-root-service-type+ kernel-loadable-module-service-type
%boot-service %activation-service@@ -864,6 +869,71 @@ as Wifi cards."))) will not be reclaimed by the garbage collector.") (default-value '())))
+;; Configuration for the kernel builder.+(define-record-type* <kernel-builder-configuration> kernel-builder-configuration+ make-kernel-builder-configuration+ kernel-builder-configuration?+ this-kernel-builder-configuration++ (kernel kernel-builder-configuration-kernel (default #f))+ (hurd kernel-builder-configuration-hurd (default #f))+ (modules kernel-builder-configuration-modules (default '())))++(define (package-for-kernel target-kernel module-package)+ "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if+possible (that is if there's a LINUX keyword argument in the build system)."+ (package+ (inherit module-package)+ (arguments+ (substitute-keyword-arguments (package-arguments module-package)+ ((#:linux kernel #f)+ target-kernel)))))++(define (kernel-builder-configuration->system-entry config)+ "Return the kernel and hurd entries of the 'system' directory."+ (mbegin %store-monad+ (let* ((kernel (kernel-builder-configuration-kernel config))+ (hurd (kernel-builder-configuration-hurd config))+ (modules (kernel-builder-configuration-modules config))+ (kernel (if hurd+ kernel+ (profile+ (content (packages->manifest+ (cons kernel+ (map (lambda (module)+ (if (package? module)+ (package-for-kernel kernel module)+ module))+ modules))))+ (hooks (list linux-module-database))))))+ (return `(("kernel" ,kernel)+ ,@(if hurd `(("hurd" ,hurd)) '()))))))++(define (kernel-builder-configuration-add-modules config modules)+ "Constructs a kernel builder configuration that has its modules extended."+ (kernel-builder-configuration+ (inherit config)+ (modules (append (kernel-builder-configuration-modules config) modules))))++(define kernel-loadable-module-service-type+ (service-type (name 'kernel-loadable-modules)+ (extensions+ (list (service-extension system-service-type+ kernel-builder-configuration->system-entry)))+ (compose concatenate)+ (extend kernel-builder-configuration-add-modules)+ (description+ "Register packages containing kernel-loadable modules and adds them+to the system.")))++(define (kernel-loadable-module-service kernel hurd modules)+ "Constructs the service that sets up kernel loadable modules."+ (service kernel-loadable-module-service-type+ (kernel-builder-configuration+ (kernel kernel)+ (hurd hurd)+ (modules modules))))+ ;;; ;;; Service folding.diff --git a/gnu/system.scm b/gnu/system.scmindex c284a18379..5c530f176e 100644--- a/gnu/system.scm+++ b/gnu/system.scm@@ -12,6 +12,7 @@ ;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@gmail.com> ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <jannek@gnu.org> ;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -600,16 +601,6 @@ OS." (file-append (operating-system-kernel os) "/" (system-linux-image-file-name))))
-(define (package-for-kernel target-kernel module-package)- "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if-possible (that is if there's a LINUX keyword argument in the build system)."- (package- (inherit module-package)- (arguments- (substitute-keyword-arguments (package-arguments module-package)- ((#:linux kernel #f)- target-kernel)))))- (define %default-modprobe-blacklist ;; List of kernel modules to blacklist by default. '("usbmouse" ;races with bcm5974, see <https://bugs.gnu.org/35574>@@ -625,26 +616,10 @@ possible (that is if there's a LINUX keyword argument in the build system)." "Return the basic entries of the 'system' directory of OS for use as the value of the SYSTEM-SERVICE-TYPE service." (let* ((locale (operating-system-locale-directory os))- (kernel (operating-system-kernel os)) (hurd (operating-system-hurd os))- (modules (operating-system-kernel-loadable-modules os))- (kernel (if hurd- kernel- (profile- (content (packages->manifest- (cons kernel- (map (lambda (module)- (if (package? module)- (package-for-kernel kernel- module)- module))- modules))))- (hooks (list linux-module-database))))) (initrd (and (not hurd) (operating-system-initrd-file os))) (params (operating-system-boot-parameters-file os)))- `(("kernel" ,kernel)- ,@(if hurd `(("hurd" ,hurd)) '())- ("parameters" ,params)+ `(("parameters" ,params) ,@(if initrd `(("initrd" ,initrd)) '()) ("locale" ,locale)))) ;used by libc
@@ -663,6 +638,10 @@ bookkeeping." (host-name (host-name-service (operating-system-host-name os))) (entries (operating-system-directory-base-entries os))) (cons* (service system-service-type entries)+ (kernel-loadable-module-service+ (operating-system-kernel os)+ (operating-system-hurd os)+ (operating-system-kernel-loadable-modules os)) %boot-service
;; %SHEPHERD-ROOT-SERVICE must come last so that the gexp that@@ -699,6 +678,10 @@ bookkeeping." (define (hurd-default-essential-services os) (let ((entries (operating-system-directory-base-entries os))) (list (service system-service-type entries)+ (kernel-loadable-module-service+ (operating-system-kernel os)+ (operating-system-hurd os)+ (operating-system-kernel-loadable-modules os)) %boot-service %hurd-startup-service %activation-servicediff --git a/gnu/tests/linux-modules.scm b/gnu/tests/linux-modules.scmindex 953b132ef7..9739e4124d 100644--- a/gnu/tests/linux-modules.scm+++ b/gnu/tests/linux-modules.scm@@ -2,6 +2,7 @@ ;;; Copyright © 2019 Jakob L. Kreuze <zerodaysfordays@sdf.org> ;;; Copyright © 2020 Danny Milosavljevic <dannym@scratchpost.org> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -34,7 +35,10 @@ #:use-module (guix utils) #:export (%test-loadable-kernel-modules-0 %test-loadable-kernel-modules-1- %test-loadable-kernel-modules-2))+ %test-loadable-kernel-modules-2+ %test-loadable-kernel-modules-service-0+ %test-loadable-kernel-modules-service-1+ %test-loadable-kernel-modules-service-2))
;;; Commentary: ;;;@@ -66,17 +70,11 @@ that MODULES are actually loaded." (member module modules string=?)) '#$modules))))))
-(define* (run-loadable-kernel-modules-test module-packages module-names)- "Run a test of an OS having MODULE-PACKAGES, and verify that MODULE-NAMES-are loaded in memory."+(define* (run-loadable-kernel-modules-test-base base-os module-names)+ "Run a test of BASE-OS, verifying that MODULE-NAMES are loaded in memory." (define os (marionette-operating-system- (operating-system- (inherit (simple-operating-system))- (services (cons (service kernel-module-loader-service-type module-names)- (operating-system-user-services- (simple-operating-system))))- (kernel-loadable-modules module-packages))+ base-os #:imported-modules '((guix combinators)))) (define vm (virtual-machine os)) (define (test script)@@ -98,6 +96,37 @@ are loaded in memory." (gexp->derivation "loadable-kernel-modules" (test (modules-loaded?-program os module-names))))
+(define* (run-loadable-kernel-modules-test module-packages module-names)+ "Run a test of an OS having MODULE-PACKAGES, and verify that MODULE-NAMES+are loaded in memory."+ (run-loadable-kernel-modules-test-base+ (operating-system+ (inherit (simple-operating-system))+ (services (cons (service kernel-module-loader-service-type module-names)+ (operating-system-user-services+ (simple-operating-system))))+ (kernel-loadable-modules module-packages))+ module-names))++(define* (run-loadable-kernel-modules-service-test module-packages module-names)+ "Run a test of an OS having MODULE-PACKAGES, which are loaded by creating a+service that extends KERNEL-LOADABLE-MODULE-SERVICE-TYPE. Then verify that+MODULE-NAMES are loaded in memory."+ (define module-installing-service-type+ (service-type+ (name 'module-installing-service)+ (extensions (list (service-extension kernel-loadable-module-service-type+ (const module-packages))))+ (default-value #f)))+ (run-loadable-kernel-modules-test-base+ (operating-system+ (inherit (simple-operating-system))+ (services (cons* (service kernel-module-loader-service-type module-names)+ (service module-installing-service-type)+ (operating-system-user-services+ (simple-operating-system)))))+ module-names))+ (define %test-loadable-kernel-modules-0 (system-test (name "loadable-kernel-modules-0")@@ -129,3 +158,35 @@ with two extra modules.") (package-arguments ddcci-driver-linux)))))) '("acpi_call" "ddcci")))))++(define %test-loadable-kernel-modules-service-0+ (system-test+ (name "loadable-kernel-modules-service-0")+ (description "Tests loadable kernel modules extensible service with no+extra modules.")+ (value (run-loadable-kernel-modules-service-test '() '()))))++(define %test-loadable-kernel-modules-service-1+ (system-test+ (name "loadable-kernel-modules-service-1")+ (description "Tests loadable kernel modules extensible service with one+extra module.")+ (value (run-loadable-kernel-modules-service-test+ (list ddcci-driver-linux)+ '("ddcci")))))++(define %test-loadable-kernel-modules-service-2+ (system-test+ (name "loadable-kernel-modules-service-2")+ (description "Tests loadable kernel modules extensible service with two+extra modules.")+ (value (run-loadable-kernel-modules-service-test+ (list acpi-call-linux-module+ (package+ (inherit ddcci-driver-linux)+ (arguments+ `(#:linux #f+ ,@(strip-keyword-arguments '(#:linux)+ (package-arguments+ ddcci-driver-linux))))))+ '("acpi_call" "ddcci")))))
--2.29.2
R
R
raid5atemyhomework wrote on 6 Jan 16:55 +0100
[PATCH 2/4] gnu: Make file-systems target extensible by services.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
Mg_8FIL2l-Zd7f6rM0kLocdknqSlZfYwXsKBY88f0vqvn_pqTW3otQGnYeTcg1SLbUXuzsO0P5-O35udXT5-c-gzr21W0E0XEYTAbAkdlpI=@protonmail.com
From 792a8f8efc95e4fe9a94d42f839ddcfb034b8540 Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Wed, 6 Jan 2021 08:15:54 +0800Subject: [PATCH 2/4] gnu: Make file-systems target extensible by services.
* gnu/services/base.scm (file-system-shepherd-services): Move file-systemsshepherd service to ...(file-systems-target-shepherd-services): ... new procedure here.(file-systems-target-service-type): New variable.(file-system-service-type): Extend file-systems-target service to add eachfile-system as a requirement.* gnu/system.scm (operating-system-default-essential-services): Instantiatefile-systems-target-service-type.(hurd-default-essential-services): Instantiate file-systems-target-service-type.--- gnu/services/base.scm | 37 +++++++++++++++++++++++++++---------- gnu/system.scm | 2 ++ 2 files changed, 29 insertions(+), 10 deletions(-)
Toggle diff (99 lines)diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex 945b546607..13cfb6a8a2 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -13,6 +13,7 @@ ;;; Copyright © 2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> ;;; Copyright © 2020 Florian Pelz <pelzflorian@pelzflorian.de> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -67,6 +68,7 @@ #:export (fstab-service-type root-file-system-service file-system-service-type+ file-systems-target-service-type swap-service host-name-service console-keymap-service@@ -362,18 +364,29 @@ FILE-SYSTEM." (gnu system file-systems) ,@%default-modules)))))))
+(define (file-systems-target-shepherd-services requirements)+ (list+ (shepherd-service+ (provision '(file-systems))+ (requirement (cons* 'root-file-system 'user-file-systems requirements))+ (documentation "Target for all the initially-mounted file systems")+ (start #~(const #t))+ (stop #~(const #t)))))+(define file-systems-target-service-type+ (service-type+ (name 'file-systems)+ (extensions (list (service-extension shepherd-root-service-type+ file-systems-target-shepherd-services)))+ (compose concatenate)+ (extend append)+ ;; Extensions can add new values to this list.+ (default-value '())+ (description "The @code{file-systems} service is the target that is started+when all file systems have been mounted.")))+ (define (file-system-shepherd-services file-systems) "Return the list of Shepherd services for FILE-SYSTEMS." (let* ((file-systems (filter file-system-mount? file-systems)))- (define sink- (shepherd-service- (provision '(file-systems))- (requirement (cons* 'root-file-system 'user-file-systems- (map file-system->shepherd-service-name- file-systems)))- (documentation "Target for all the initially-mounted file systems")- (start #~(const #t))- (stop #~(const #f))))
(define known-mount-points (map file-system-mount-point file-systems))@@ -403,7 +416,7 @@ FILE-SYSTEM." (filter (negate known?) (mount-points))) #f))))
- (cons* sink user-unmount+ (cons* user-unmount (map file-system-shepherd-service file-systems))))
(define (file-system-fstab-entries file-systems)@@ -431,6 +444,10 @@ FILE-SYSTEM." (service-extension fstab-service-type file-system-fstab-entries)
+ ;; Have 'file-systems' depend on each declared file system.+ (service-extension file-systems-target-service-type+ (cut map file-system->shepherd-service-name <>))+ ;; Have 'user-processes' depend on 'file-systems'. (service-extension user-processes-service-type (const '(file-systems)))))diff --git a/gnu/system.scm b/gnu/system.scmindex 5c530f176e..6987641ee8 100644--- a/gnu/system.scm+++ b/gnu/system.scm@@ -667,6 +667,7 @@ bookkeeping." (operating-system-setuid-programs os)) (service profile-service-type (operating-system-packages os))+ (service file-systems-target-service-type) other-fs (append mappings swaps
@@ -691,6 +692,7 @@ bookkeeping." (operating-system-groups os)) (operating-system-skeletons os)) (root-file-system-service)+ (service file-systems-target-service-type) (service file-system-service-type '()) (service fstab-service-type (filter file-system-needed-for-boot?
--2.29.2
R
R
raid5atemyhomework wrote on 6 Jan 16:56 +0100
[PATCH 3/4] gnu: Fix ZFS package.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
acw65x5599Yvgtn3iJ_lyNBHCVnac88F4pw4NdemkLG-SFzy_URDm9Fp01CWjbIkNT-5UjqceOHDNBmZpakN0ifetm1ehaED4z65hHgUKwo=@protonmail.com
From d1c88161f2a32f4c03de57ee9d7582cc6c40f6bc Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Tue, 5 Jan 2021 20:00:50 +0800Subject: [PATCH 3/4] gnu: Fix ZFS package.
* gnu/packages/file-systems.scm (zfs): Correct "util-linux" input, addseparate "util-linux:lib" input.--- gnu/packages/file-systems.scm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Toggle diff (22 lines)diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scmindex 895ad069c5..2c5ad95d63 100644--- a/gnu/packages/file-systems.scm+++ b/gnu/packages/file-systems.scm@@ -6,6 +6,7 @@ ;;; Copyright © 2019, 2020 Efraim Flashner <efraim@flashner.co.il> ;;; Copyright © 2020 Raghav Gururajan <raghavgururajan@disroot.org> ;;; Copyright © 2020 Morgan Smith <Morgan.J.Smith@outlook.com>+;;; Copyright © 2021 raid5atemyhoemwork <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -944,7 +945,8 @@ APFS.") ("openssl" ,openssl) ("python" ,python) ("python-cffi" ,python-cffi)- ("util-linux" ,util-linux "lib")+ ("util-linux" ,util-linux)+ ("util-linux:lib" ,util-linux "lib") ("zlib" ,zlib))) (home-page "https://zfsonlinux.org/") (synopsis "Native ZFS on Linux")
--2.29.2
R
R
raid5atemyhomework wrote on 6 Jan 16:57 +0100
[PATCH 4/4] gnu: Add ZFS service.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
6wemXB-PfHUqbuVr5-XRf0-tY4cKGGtKiUqrZPrIZYXoBw17L3xRuZrGOJQfTo5PKfFNCM8KyRTllidoc7asPE2x98BTiJSPVR7OSjxCuw8=@protonmail.com
From deded39fe88c44a18b0b66bab8c4300aca4c387e Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Wed, 6 Jan 2021 09:24:20 +0800Subject: [PATCH 4/4] gnu: Add ZFS service.
* gnu/services/file-systems.scm: New file.(zfs-service-type): New variable.(zfs-configuration): New variable.(zfs-configuration?): New procedure.* gnu/local.mk: Add gnu/services/file-systems.scm.* doc/guix.texi (ZFS file system): New subsection.--- doc/guix.texi | 107 ++++++++++++++++++++++++++ gnu/local.mk | 2 + gnu/services/file-systems.scm | 136 ++++++++++++++++++++++++++++++++++ 3 files changed, 245 insertions(+) create mode 100644 gnu/services/file-systems.scm
Toggle diff (280 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 78770151e3..62437083cb 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -13885,6 +13885,113 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp
+@node ZFS file system+@subsection ZFS file system++The ZFS file system has a license incompatible with the Linux kernel,+and thus cannot be distributed with the kernel. However, as a user+you have the right to do whatever you want on your own hardware,+including download the ZFS source code, compile it, link it to your+own private copy of Linux, and run it. You can even use the Guix build+system to automate this.++As a large and complex kernel module, ZFS on Linux has to be compiled+with a specific version of the kernel. Often even the latest ZFS+package available cannot be compiled with the latest Linux kernel+package provided by Guix System. Thus, installing the @code{zfs}+package is likely to fail.++Instead, you have to use an older long-term-support Linux kernel.+Do not use @code{linux-libre-lts} as the latest long-term-support+kernel might be too new for the ZFS package; instead, explicitly+select the version number, like @code{linux-libre-5.4}.++Then, you have to modify your system configuration file and use the+selected older kernel, and add the @code{zfs-service-type} service.++@lisp+(use-modules (gnu))+(use-package-modules+ ; @dots{}+ linux)+(use-service-modules+ ; @dots{}+ file-systems)++(define my-kernel linux-libre-5.4)++(operating-system+ (kernel my-kernel)+ ;; @dots{}+ (services+ (cons* (service zfs-service-type+ (zfs-configuration+ (kernel my-kernel)))+ ; @dots{}+ %desktop-services))+ ;; @dots{}+ )+@end lisp++@defvr {Scheme Variable} zfs-service-type+This is the type of the service to compile and install OpenZFS to+your operating system. It loads the ZFS module at startup, imports+pools, mounts automounted ZFS datasets, installs the ZFS command+line tools, and can provide module options for the ZFS module. Its+value must be a @code{zfs-configuration} record (see below).++Here is an example use:++@lisp+(service zfs-service-type+ (zfs-configuration+ (kernel linux-libre-5.4)+ (options '("zfs_arc_max=5000000000"))))+@end lisp+@end defvr++@deftp {Data Type} zfs-configuration+This data type represents the configuration of the ZFS service.+The available fields are:++@table @asis+@item @code{kernel}+The package of the Linux kernel to compile ZFS for. Required.++@item @code{base-zfs} (default: @code{zfs})+The ZFS package to use. It will be modified to use the indicated+kernel.++@item @code{options} (default: @code{'()})+A list of string options to pass as options to the ZFS module.+These will be put in a @file{/etc/modprobe.d/zfs.conf} file,+for example setting this to @code{'("zfs_admin_snapshot=1"+"zfs_trim_extent_bytes_min=0")} will create the following file:++@example+options zfs zfs_admin_snapshot=1 zfs_trim_extent_bytes_min=0+@end example+@end table+@end deftp++Once your system has been configured to include the ZFS service+and you have restarted the system, you can manage ZFS pools and+datasets with @code{zpool} and @code{zfs} commands.++ZFS datasets with an appropriate @code{mountpoint} property will+be automounted at startup after the root file system is started.+Encrypted datasets that are automounted will cause boot to pause+and prompt for the password to be provided on the console.++It's possible to have a ZFS dataset as your @code{/home} by simply+setting the @code{mountpoint} property. However, note that ZFS will+refuse to mount over a non-empty directory, so if your root+filesystem already has a non-empty @code{/home}, you should remove+it and its contents, then restart the system.++Having ZFS as a root filesystem or as @code{/boot} is not supported+yet.+ @node Mapped Devices @section Mapped Devices
diff --git a/gnu/local.mk b/gnu/local.mkindex 53a67e6ffe..49073ac2ac 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -39,6 +39,7 @@ # Copyright © 2020 Martin Becze <mjbecze@riseup.net> # Copyright © 2020 Malte Frank Gerdes <mate.f.gerdes@gmail.com> # Copyright © 2020 Vinicius Monego <monego@posteo.net>+# Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> # # This file is part of GNU Guix. #@@ -601,6 +602,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/dict.scm \ %D%/services/dns.scm \ %D%/services/docker.scm \+ %D%/services/file-systems.scm \ %D%/services/authentication.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scmnew file mode 100644index 0000000000..7a92a2b9d5--- /dev/null+++ b/gnu/services/file-systems.scm@@ -0,0 +1,136 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu services file-systems)+ #:use-module (gnu packages file-systems)+ #:use-module (gnu services)+ #:use-module (gnu services base)+ #:use-module (gnu services shepherd)+ #:use-module (guix gexp)+ #:use-module (guix packages)+ #:use-module (guix records)+ #:export (zfs-service-type+ zfs-configuration+ zfs-configuration?))++(define-record-type* <zfs-configuration>+ zfs-configuration make-zfs-configuration zfs-configuration?+ ; kernel you want to compile the base-zfs module for.+ (kernel zfs-configuration-kernel)+ ; base package that will be compiled for the kernel+ (base-zfs zfs-configuration-base-zfs (default zfs))+ ; list of string options.+ (options zfs-configuration-options (default '())))++(define (make-zfs-package conf)+ (let ((base-zfs (zfs-configuration-base-zfs conf))+ (kernel (zfs-configuration-kernel conf)))+ (package+ (inherit base-zfs)+ (name (string-join (list (package-name base-zfs)+ "for"+ (package-name kernel)+ (package-version kernel)+ "version")+ "-"))+ (arguments (cons* #:linux kernel (package-arguments base-zfs))))))++(define (zfs-loadable-module conf)+ (list (list (make-zfs-package conf) "module")))++(define (zfs-shepherd-services conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool")))+ (list+ (shepherd-service+ (documentation "Scans for ZFS pools and automounts filesystems.")+ (provision '(zfs-scan-automount))+ (requirement '(root-file-system))+ (modules `((srfi srfi-1)+ (srfi srfi-34)+ (srfi srfi-35)+ (rnrs io ports)+ ,@%default-modules))+ (start #~(lambda _+ (and+ ;; You'd think we could've used kernel-module-loader-service-type,+ ;; but the kernel-module-loader shepherd service is dependent on+ ;; file-systems, and file-systems is made dependent on this+ ;; service. And we need the kernel module to be loaded before we+ ;; scan for ZFS pools. So break the dependency loop by just+ ;; loading ZFS module here by ourselves.+ (or (file-exists? "/proc/sys/kernel/modprobe")+ (begin+ (format (current-error-port) "error loading 'zfs' module: ~a~%"+ "Kernel is missing loadable module support.")+ #f))+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error loading 'zfs' module: ~a~%"+ (condition-message c))+ #f))+ (let ((modprobe (call-with-input-file "/proc/sys/kernel/modprobe"+ get-line)))+ (invoke/quiet modprobe "--" "zfs")))++ ; scan for pools and automount contained datasets.+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error importing zpools: ~a~%"+ (condition-message?))+ #f))+ ;; (current-output-port) is typically connected to /dev/klog,+ ;; so redirect it to (current-error-port) so that user can see+ ;; prompts for passphrases on console+ (with-output-to-port (current-error-port)+ (lambda ()+ (invoke #$zpool "import" "-a" "-l")))))))+ (stop #~(const #t))))))++(define (zfs-profile-service conf)+ (list (make-zfs-package conf)))++(define (zfs-etc-service conf)+ (let ((options (zfs-configuration-options conf)))+ (if (null? options)+ '()+ `(("modprobe.d/zfs.conf"+ ,(plain-file "zfs.conf"+ (string-join (cons "options zfs" options) " ")))))))++(define zfs-service-type+ (service-type (name 'zfs)+ (extensions+ (list+ ; install the kernel module+ (service-extension kernel-loadable-module-service-type+ zfs-loadable-module)+ ; load ZFS module, scan ZFS pools, and automount filesystems+ (service-extension shepherd-root-service-type+ zfs-shepherd-services)+ ; make sure automount occurs before file-systems target is reached+ (service-extension file-systems-target-service-type+ (const '(zfs-scan-automount)))+ ; install ZFS management tools+ (service-extension profile-service-type+ zfs-profile-service)+ ; install ZFS module options+ (service-extension etc-service-type+ zfs-etc-service)))+ (description+ "Install ZFS, an advanced filesystem and volume manager.")))
--2.29.2
D
D
Danny Milosavljevic wrote on 6 Jan 20:41 +0100
kernel-module-configuration-service for configuring kernel parameters
20210106204134.38e83db4@scratchpost.org
Hi raid5atemyhomework,Hi everyone,
@raid5atemyhomework, thanks for all the patches!
On Wed, 06 Jan 2021 15:57:19 +0000raid5atemyhomework via Guix-patches via <guix-patches@gnu.org> wrote:
Toggle quote (7 lines)> +@item @code{options} (default: @code{'()})> +A list of string options to pass as options to the ZFS module.
> +These will be put in a @file{/etc/modprobe.d/zfs.conf} file,> +for example setting this to @code{'("zfs_admin_snapshot=1"> +"zfs_trim_extent_bytes_min=0")} will create the following file:
Sure, but it would be better to create a way to configure those module parametersin Guix in a declarative way, first.
Your new kernel-loadable-module-service-type would be a good templateto write a kernel-module-configuration-service that can be extended byother services. The latter should allow users to parametrize the kernelin general via the operating-system form, and also allow services to extendsuch configuration (with merging, and conflict detection).
It would be used:
(1) To declaratively specify the contents of something like /etc/modprobe.d .It shouldn't even be called "/etc/modprobe.d"--it should also be in the storeinstead. This directory is only useful for non-Linux-builtins.
(2) If the "module" is built-in then the kernel command line must get theoptions instead. (in fact, it works as a kernel command line option alsoif it's a loadable module--so not sure we need /etc/modprobe.d at all--atleast at first. But there's probably a maximal length for the kernel commandline that we could exceed if we did that long term)
I know it's annoying that Guix doesn't have this facility already, but thetime to introduce an interface for /etc/modprobe.d and the kernel commandline for builtin modules is before other services introduce their ownad hoc way to create /etc/modprobe.d--like this tries to do here.
See also https://issues.guix.info/issue/42193for an earlier attempt (whichis already very far--but it has a bug somewhere). There's also already akernel profile thing like you wrote in that patchset.(Note that I would prefer there not to be a "LOAD?" in there because itconfuses loading the module (which is usually NOT started by user spacebut by the kernel on its own) and confguring the module (which has to bedone by user space because it's specifying policy, not mechanism))
Also, because the kernel usually loads loadable modules on its own (potentiallyreally early), /etc/modprobe.d has to be preset and known to the modprobeexecutable VERY EARLY (via environment variable MODPROBE_OPTIONS--see gnu/services.scm %modprobe-wrapper).
It is totally possible that some modules in the initrd need options, too(see load-linux-modules-from-directory for where this would need to go).load-linux-module/fd already accepts options and flags--but both are not givenon the call. For this, part of future kernel-module-configuration entries(the ones needed for modules in the initrd) should be copied into the initrd,too.
Then there's the handoff between initrd and main system. It would be badif the kernel tried and succeeded to load a module that is not in the initrdjust before the modprobe.d directory is set up (because it would be loadedwithout passing the options the user configured)--so that needs to be avoided.
@ludo: Could you help here?
Toggle quote (3 lines)> +@example> +options zfs zfs_admin_snapshot=1 zfs_trim_extent_bytes_min=0
Note:
This can be usefully put in a modprobe.d-like directory if zfs is a module, butnot if it's built into the kernel. But it can be put into the kernel commandline in both cases.
But I guess the ZFS Linux kernel module can't be built-in into the kernelanyway.
But that's a special case--in general, it's very much possible to make modulesbuilt-in.
Toggle quote (2 lines)> + ;; You'd think we could've used kernel-module-loader-service-type,
Definitely.
Toggle quote (3 lines)> + ;; but the kernel-module-loader shepherd service is dependent on> + ;; file-systems,
Yes--but why is that dependent on 'file-systems ? Is it because it needs /proc ?Or is it an oversight ? I would prefer to get rid of this dependency and thenuse kernel-module-loader-service-type.
Also, this manual loading of kernel modules is not supposed to be the way todo things in Linux. That a kernel module was compiled as a module isan *implementation detail*--so Linux should (and usually does) automaticallyload kernel modules the first time a device for them is accessed (after all,how would user space know whether something is compiled as a module orbuilt-in--that would be too much to ask).
Linux is not a microkernel, so the kind of modularily modprobe.d suggestsexists does not in fact exist in kernel space--even though Linux does a goodjob faking it: modprobe.d contains:
* "alias": a feature to configure aliases, with wildcards (only one level ofaliases allowed!)* "options" per module (also works for aliases with wildcards! That will be"fun" to map to Guix)* "install" in order to run some custom executable instead of loading themodule.* "remove" in order to run some custom executable instead of unloading themodule.* "blacklist" to ignore specific internal aliases of a module (that doesnot do what one would intuitively think!).
If the file name of the regular file under /etc/modprobe.d is not used foranything, then we can just have one file /gnu/store/*modprobe.d/guix.confin total in there.
Then there are sysctl kernel parameters--but those Guix already exposes viasysctl-service-type. But those should also be made able to be extendedby other services, and merge conflicts should be handled. For example, usersoften set net.ipv4.ip_forward=1 (for example via sysctl).
Thank you for all your effort to make ZFS work nicely in Guix.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/2Em4ACgkQ5xo1VCwwuqX7VQf/bHxCWBZt0FXRb5E0zvb0AZVwMPajkYvOBjfXJC+XcMbgjP3FKNW7VLGKFakRsyWnVX6gtFPxUDImjnCIGcY5LGYuaHxnDR4CTqzDBb06LRAMeJ1YTIyr+j+SZO17TG4hZa+xWLqM6MUpfEwuc1I/b5o9oN39XJJLJAWJOdXMi2ZGwOKY8dpYhaDwzDPoOb3xHj9oDyXyWjOfb/8VK8bF8+L9Fzy3Bjt2WiG58j5jXpkYz9ZQe3zA1hbHzwFmmKqkFbL2gFhTxGWuQrp6x29T60lzBwUcjG58waCQee23TVFR9K/v7fvjrBalkHMS+9/Dc8FBIdzlHAbxj8/74qYkUQ===nYdc-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 7 Jan 01:04 +0100
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
pilwAVCb0NEG83BVTSEqXkLCNEvBBrnb-QyKV_9PoF4xuCjSXKuF-F3_L5lSOke5BRjagVrUoSzJCST1fkigF162f9j76K12jBraNMAj7E4=@protonmail.com
Hello Danny,

Toggle quote (8 lines)> See also https://issues.guix.info/issue/42193for an earlier attempt (which> is already very far--but it has a bug somewhere). There's also already a> kernel profile thing like you wrote in that patchset.> (Note that I would prefer there not to be a "LOAD?" in there because it> confuses loading the module (which is usually NOT started by user space> but by the kernel on its own) and confguring the module (which has to be> done by user space because it's specifying policy, not mechanism))
Looks like that patchset was merged in, so basically I can just depend on that? So the first patch in this patchset would be dropped?
Toggle quote (6 lines)> But I guess the ZFS Linux kernel module can't be built-in into the kernel> anyway.>> But that's a special case--in general, it's very much possible to make modules> built-in.
ZFS *can* be built-in to the kernel, Ubuntu does it. You can't distribute it like that (Ubuntu distributes it like that but presumably they have enough lawyers to muddy the waters so that they can get away with it), but as the documentation in this patch notes: the user has every right to do whatever they want on the machine they own, including build a Linux kernel that has ZFS built-in and run it, they just can't make that version available to somebody else.
So to go whole-hog, we would have a service that replaces the kernel package and inserts kernel module sources in-tree somehow, then compiles Linux-libre on the user's machine. That would probably be a lot more painful to install ZFS with (the user has to recompile the whole kernel at each update of either kernel or ZFS, whereas with a kernel module the user has to recompile just the kernel module), so maybe kernel module is still better overall.

Toggle quote (17 lines)> > - ;; You'd think we could've used kernel-module-loader-service-type,> >> >>> Definitely.>> > - ;; but the kernel-module-loader shepherd service is dependent on> >> >> > - ;; file-systems,> >> >>> Yes--but why is that dependent on 'file-systems ? Is it because it needs /proc ?> Or is it an oversight ? I would prefer to get rid of this dependency and then> use kernel-module-loader-service-type.
Dunno --- one VM I tested, I removed the `zfs-scan-automount` shepherd service from the `file-systems` target, and the VM still wouldn't boot, claiming a stack overflow (the same error which I got when I was still trying to use kernel-module-loader-service-type here). Or maybe I just got confused with which VM was which, testing VMs wasn't a stress-free vacation. I just want ZFS, because MD RAID5 ate my homework, this is getting tiresome...
One thing I notice about `kernel-module-loader-service-type` is that it's not instantiated in essential services, or indeed anywhere in Guix. A few services *do* extend it. But my **very rough** understanding is that if you're going to extend a service, it had better be instantiated *once* in the list of services.
In particular I note that the documentation for `kernel-module-loader-service-type` shows an example where it uses `service` to program the `kernel-module-loader-service-type`, not `simple-service`. This suggests to me that `kernel-module-loader-service-type` is broken because it's not in the list of essential services but is extensible. Maybe. It's designed as an extensible service, but isn't instantiated at default. Maybe that's what really bit me and not the shepherd circular dependency loop? *shrug*

Toggle quote (8 lines)>> Also, this manual loading of kernel modules is not supposed to be the way to> do things in Linux. That a kernel module was compiled as a module is> an implementation detail--so Linux should (and usually does) automatically> load kernel modules the first time a device for them is accessed (after all,> how would user space know whether something is compiled as a module or> built-in--that would be too much to ask).
So how do I get ZFS loaded? Note that the devices it targets are block devices and it needs to scan for block devices that are formatted for ZFS. Do other filesystems have some autoload rule?
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 7 Jan 06:38 +0100
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
7kahfDckMJIQd8o2gH_zQ-G9ORP_P0w-VD44R2OJHNP6rJHss8przoTyObTYxx_UEI5CERTD3OlGORnTCBmZA2VMASWpF2Ea0Hx1ZAbUh-I=@protonmail.com
Hi Danny,
Toggle quote (10 lines)> > See also https://issues.guix.info/issue/42193for an earlier attempt (which> > is already very far--but it has a bug somewhere). There's also already a> > kernel profile thing like you wrote in that patchset.> > (Note that I would prefer there not to be a "LOAD?" in there because it> > confuses loading the module (which is usually NOT started by user space> > but by the kernel on its own) and confguring the module (which has to be> > done by user space because it's specifying policy, not mechanism))>> Looks like that patchset was merged in, so basically I can just depend on that? So the first patch in this patchset would be dropped?
No, sorry, my mistake, only one patch completely unrelated to the actual new service type was merged in.
In any case --- is this objection something that would block this patchset from being added to Guix? I appreciate that we should "do it right" --- but there's also an argument for "keep it simple", and the first patch in this patchset gets us a good part of the way to what is needed.
It seems that my patch is equivalent to the existing WIP 2/6 `kernel-profile-service-type`, so maybe I can just steal that patch for now, write tests specific for it, and get on with running ZFS in production?
Can we at least get patch 3/4 of *this* patchset merged since it's a trivial bugfix? How about patch 2/4, which is not so trivial, but does give some flexibility in case other filesystems want to be as ambitious as ZFS is?
I imagine a later, much more comprehensive `kernel-module-configuration-service-type` can be built on top of `kernel-profile-service-type`, `kernel-module-loader-service-type`, and a later `kernel-module-options-service-type`. Then `zfs-service-type` can be modified to use that `kernel-module-configuration-service-type` instead of rolling its own bits and pieces.
----
On the other hand, if we want to think of `/` on ZFS, then we need a notion of kernel modules that are added to the `initrd` file, rather than to the `kernel` directory.
Diving into `gnu/system/linux-initrd.scm`, I note that we can provide a kernel package and a list of modules to copy from the kernel package to the `initrd` by a `(flat-linux-module-directory linux modules)` function.
I imagine that it would be possible to modify this as follows:
* Have `flat-linux-module-directory` accept a list of packages from which to find module names, not just a single package.* Remove more code from `(operating-system-directory-base-entries os)`, and put the creation of the `"initrd"` file into a service, in much the same way that my patch does (unless there's an existing more-guixy way of putting files into `initrd`?). * Create a service type that gathers packages whose modules are to be added to the `initrd`, and if that list is non-empty, pass it to a new key `#:extra-linux-module-packages` of the `initrd` function. * `raw-initrd` would then pass that field together with the `linux` argument to the modified `flat-linux-module-directory` procedure. * Create another service type that gathers module names to be loaded at `boot` and appends them to the `#:linux-modules` to the `initrd` function. * Modify the `kernel-module-options-service-type` to pass in options via kernel command line always, maybe.
Then ZFS module can get loaded early, at boot, before the switch from `initrd` root to the "real" root.
* Create a service type that gathers additional pre-mount actions, which `raw-initrd` will accept as a new key `#:additional-pre-mount` and append to the `pre-mount` it passes to `boot-system`. * The `zfs-service-type` would then extend this service type to pass in an action to perform a `zpool import -a -N`, which makes ZFS scan for devices containing ZFS pools.* Somehow figure out a static build for the ZFS package, so we can use a statically-linked `zpool` in the above. * The `zfs-service-type` already inherits a ZFS package from the given `base-package`, I imagine it would be possible to make further inheritance which modifies the build to be static. It does require an additional build, though. Maybe an additional `root-on-zfs?` field in `zfs-configuration` can gate this, so that we don't add the static build and extend the boot script if `(not root-on-zfs?)` (i.e. use `kernel-module-configuration-service-type` if `(not root-on-zfs?)`, else use the new put-it-in-the-initrd service types).* Modify the various checks elsewhere so that ZFS poolnames can be passed as the `device` field of `file-system` records. See https://issues.guix.gnu.org/45643#3 * Then the installer can be modified so that if ZFS is included with the installer's profile, it can look at ZFS pools and offer to install to a ZFS pool instead of a `/dev` partition directly, then add `(file-system (device "rootpool") (mountpoint "/") (type "zfs"))` and the appropriate `zfs-service-type` and etc. and now we get `/` on ZFS.
But that can come later, much much later, I just need ZFS, *any* ZFS, before RAID5 eats more of my homework.
Thanksraid5atemyhomework

Toggle quote (43 lines)> > But I guess the ZFS Linux kernel module can't be built-in into the kernel> > anyway.> > But that's a special case--in general, it's very much possible to make modules> > built-in.>> ZFScan be built-in to the kernel, Ubuntu does it. You can't distribute it like that (Ubuntu distributes it like that but presumably they have enough lawyers to muddy the waters so that they can get away with it), but as the documentation in this patch notes: the user has every right to do whatever they want on the machine they own, including build a Linux kernel that has ZFS built-in and run it, they just can't make that version available to somebody else.>> So to go whole-hog, we would have a service that replaces the kernel package and inserts kernel module sources in-tree somehow, then compiles Linux-libre on the user's machine. That would probably be a lot more painful to install ZFS with (the user has to recompile the whole kernel at each update of either kernel or ZFS, whereas with a kernel module the user has to recompile just the kernel module), so maybe kernel module is still better overall.>> > > - ;; You'd think we could've used kernel-module-loader-service-type,> > >> > >> >> > Definitely.> >> > > - ;; but the kernel-module-loader shepherd service is dependent on> > >> > >> > > - ;; file-systems,> > >> > >> >> > Yes--but why is that dependent on 'file-systems ? Is it because it needs /proc ?> > Or is it an oversight ? I would prefer to get rid of this dependency and then> > use kernel-module-loader-service-type.>> Dunno --- one VM I tested, I removed the`zfs-scan-automount` shepherd service from the `file-systems` target, and the VM still wouldn't boot, claiming a stack overflow (the same error which I got when I was still trying to use kernel-module-loader-service-type here). Or maybe I just got confused with which VM was which, testing VMs wasn't a stress-free vacation. I just want ZFS, because MD RAID5 ate my homework, this is getting tiresome...>> One thing I notice about `kernel-module-loader-service-type` is that it's not instantiated in essential services, or indeed anywhere in Guix. A few services do extend it. But my very rough understanding is that if you're going to extend a service, it had better be instantiated once in the list of services.>> In particular I note that the documentation for `kernel-module-loader-service-type` shows an example where it uses `service` to program the `kernel-module-loader-service-type`, not `simple-service`. This suggests to me that `kernel-module-loader-service-type` is broken because it's not in the list of essential services but is extensible. Maybe. It's designed as an extensible service, but isn't instantiated at default. Maybe that's what really bit me and not the shepherd circular dependency loop? shrug>> > Also, this manual loading of kernel modules is not supposed to be the way to> > do things in Linux. That a kernel module was compiled as a module is> > an implementation detail--so Linux should (and usually does) automatically> > load kernel modules the first time a device for them is accessed (after all,> > how would user space know whether something is compiled as a module or> > built-in--that would be too much to ask).>> So how do I get ZFS loaded? Note that the devices it targets are block devices and it needs to scan for block devices that are formatted for ZFS. Do other filesystems have some autoload rule?>> Thanks> raid5atemyhomework
D
D
Danny Milosavljevic wrote on 7 Jan 09:23 +0100
Re: [bug#45692] [PATCH 3/4] gnu: Fix ZFS package.
20210107092331.0b2fc752@scratchpost.org
Pushed this one patch that fixes package "zfs" to guix master ascommit 8d3184c5bcf1a590f59a701f0484ab5cec673188.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/2xQMACgkQ5xo1VCwwuqU2tQgAmIWZQ+ODdXCvUgDlwdVR5olRbLAZeTC7ZIHErRte4VzFAjJyvUkk1mAr0k/o9Y+SFda5TRza7GWHwgfZXiAoSyGgqUyrKH6MO0YKJlAljN9W69LSyigDuSvZSPQxRW8WHhDCJFrKkaW8cgXN2Im/QJ2Pd2kZDav266riE3Jk4JyRpS60VHHfnuE82jNT2UfnJuPK/SgAH8Q3G7z8+wd9iiMWKxpkkapHFGEdPUxJGsupNX8XtnWcCSegWQhRglVn0yY7gFhMODJcfjfqGLoDce6OjOWab4Snnv3Wjzmq3fn1gMUSzv1avIwXKbmSBldqAUk+VjiPrI6Si3ixZjK+xQ===ilmp-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 8 Jan 16:02 +0100
Re: [PATCH 4/4] gnu: Add ZFS service.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
LmlpVtErWRRqKI9LIqHvswFA3ig_rjwIk9u8S96Bp6A9VG_l0bAnQdhIQsxf_fB5CAJxMnoClNXDObuPloDX7JDDOl4ZuFGn6u54FmJ5K2o=@protonmail.com
This new version is dependent on https://issues.guix.info/45723and https://issues.guix.info/45722
New features:
* We can now delay pool importation after particular mapped devices are opened or file systems are mounted. This is useful if you want to make a ZFS pool out of LUKS containers, or mount an encrypted ZFS dataset with the encryption key in another (hopefully also encrypted) filesystem.* We can now put other non-ZFS filesystems inside ZVOLs, block devices provided by a ZFS pool, and have them mounted after ZFS has scanned pools and opened such ZVOLs for use by the ystem.
It's now possible to have a bunch of LUKS containers in a ZPOOL that provides a ZVOL that contains an EXT4 filesystem (though why would you do that, just use builtin ZFS encryption).
This uses my [PATCH 1/4] instead of https://issues.guix.info/42193#4. Should I use the latter instead? What would get ZFS on Guix faster?
I decided to remove ZFS parameter options, because I don't know whether I should wait for https://issues.guix.info/42193and how long that would take.

From d0e095136cd471f083d92416f12ca22b47301f31 Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Wed, 6 Jan 2021 09:24:20 +0800Subject: [PATCH 4/4] gnu: Add service to install ZFS.
* gnu/services/file-systems.scm: New file.(zfs-service-type): New variable.(<zfs-configuration>): New type.(%zfs-zvol-dependency): New variable.* gnu/local.mk: Add gnu/services/file-systems.scm.* gnu/services/base.scm (dependency->shepherd-service-name): Export.* doc/guix.texi (ZFS file system): New subsection.--- doc/guix.texi | 107 +++++++++++++++++++++ gnu/local.mk | 1 + gnu/services/base.scm | 4 +- gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++ 4 files changed, 285 insertions(+), 1 deletion(-) create mode 100644 gnu/services/file-systems.scm
Toggle diff (325 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex a31d355780..714274b8c9 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -13886,6 +13886,113 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp
+@node ZFS file system+@subsection ZFS file system++The ZFS file system has a license incompatible with the Linux kernel,+and thus cannot be distributed with the kernel. However, as a user+you have the right to do whatever you want on your own hardware,+including download the ZFS source code, compile it, link it to your+own private copy of Linux, and run it. You can even use the Guix build+system to automate this.++As a large and complex kernel module, ZFS on Linux has to be compiled+with a specific version of the kernel. Often even the latest ZFS+package available cannot be compiled with the latest Linux kernel+package provided by Guix System. Thus, installing the @code{zfs}+package is likely to fail.++Instead, you have to use an older long-term-support Linux kernel.+Do not use @code{linux-libre-lts} as the latest long-term-support+kernel might be too new for the ZFS package; instead, explicitly+select the version number, like @code{linux-libre-5.4}.++Then, you have to modify your system configuration file and use the+selected older kernel, and add the @code{zfs-service-type} service.++@lisp+(use-modules (gnu))+(use-package-modules+ ; @dots{}+ linux)+(use-service-modules+ ; @dots{}+ file-systems)++(define my-kernel linux-libre-5.4)++(operating-system+ (kernel my-kernel)+ ;; @dots{}+ (services+ (cons* (service zfs-service-type+ (zfs-configuration+ (kernel my-kernel)))+ ; @dots{}+ %desktop-services))+ ;; @dots{}+ )+@end lisp++@defvr {Scheme Variable} zfs-service-type+This is the type of the service to compile and install OpenZFS to+your operating system. It loads the ZFS module at startup, imports+pools, mounts automounted ZFS datasets, installs the ZFS command+line tools, and can provide module options for the ZFS module. Its+value must be a @code{zfs-configuration} record (see below).++Here is an example use:++@lisp+(service zfs-service-type+ (zfs-configuration+ (kernel linux-libre-5.4)+ (options '("zfs_arc_max=5000000000"))))+@end lisp+@end defvr++@deftp {Data Type} zfs-configuration+This data type represents the configuration of the ZFS service.+The available fields are:++@table @asis+@item @code{kernel}+The package of the Linux kernel to compile ZFS for. Required.++@item @code{base-zfs} (default: @code{zfs})+The ZFS package to use. It will be modified to use the indicated+kernel.++@item @code{options} (default: @code{'()})+A list of string options to pass as options to the ZFS module.+These will be put in a @file{/etc/modprobe.d/zfs.conf} file,+for example setting this to @code{'("zfs_admin_snapshot=1"+"zfs_trim_extent_bytes_min=0")} will create the following file:++@example+options zfs zfs_admin_snapshot=1 zfs_trim_extent_bytes_min=0+@end example+@end table+@end deftp++Once your system has been configured to include the ZFS service+and you have restarted the system, you can manage ZFS pools and+datasets with @code{zpool} and @code{zfs} commands.++ZFS datasets with an appropriate @code{mountpoint} property will+be automounted at startup after the root file system is started.+Encrypted datasets that are automounted will cause boot to pause+and prompt for the password to be provided on the console.++It's possible to have a ZFS dataset as your @code{/home} by simply+setting the @code{mountpoint} property. However, note that ZFS will+refuse to mount over a non-empty directory, so if your root+filesystem already has a non-empty @code{/home}, you should remove+it and its contents, then restart the system.++Having ZFS as a root filesystem or as @code{/boot} is not supported+yet.+ @node Mapped Devices @section Mapped Devices
diff --git a/gnu/local.mk b/gnu/local.mkindex 1151d4642e..5aeb45c4c2 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -602,6 +602,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/dict.scm \ %D%/services/dns.scm \ %D%/services/docker.scm \+ %D%/services/file-systems.scm \ %D%/services/authentication.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex 13cfb6a8a2..ef3680583b 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -188,7 +188,9 @@
references-file
- %base-services))+ %base-services++ dependency->shepherd-service-name))
;;; Commentary: ;;;diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scmnew file mode 100644index 0000000000..9061ab9582--- /dev/null+++ b/gnu/services/file-systems.scm@@ -0,0 +1,174 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu services file-systems)+ #:use-module (gnu packages file-systems)+ #:use-module (gnu services)+ #:use-module (gnu services base)+ #:use-module (gnu services linux)+ #:use-module (gnu services shepherd)+ #:use-module (gnu system mapped-devices)+ #:use-module (guix gexp)+ #:use-module (guix packages)+ #:use-module (guix records)+ #:export (zfs-service-type++ zfs-configuration+ zfs-configuration?+ zfs-configuration-kernel+ zfs-configuration-base-zfs+ zfs-configuration-dependencies++ %zfs-zvol-dependency))++(define-record-type* <zfs-configuration>+ zfs-configuration make-zfs-configuration zfs-configuration?+ ; kernel you want to compile the base-zfs module for.+ (kernel zfs-configuration-kernel)+ ; base package that will be compiled for the kernel+ (base-zfs zfs-configuration-base-zfs+ (default zfs))+ ; list of <mapped-device> | <file-system> we should wait for,+ ; before scanning for ZFS pools.+ (dependencies zfs-configuration-dependencies+ (default '())))++;; This is a synthetic and unusable MAPPED-DEVICE; its only use+;; is to be added as a (dependency ...) of some FILE-SYSTEM.+(define %zfs-zvol-dependency+ (mapped-device+ (source '())+ ;; The /* prevents naming conflict with non-ZFS device mappings,+ ;; since it is not a valid name for mapped devices, and also+ ;; implies "all zvols" in terms of globs.+ (targets '("zvol/*"))+ (type #f)))++(define (make-zfs-package conf)+ (let ((base-zfs (zfs-configuration-base-zfs conf))+ (kernel (zfs-configuration-kernel conf)))+ (package+ (inherit base-zfs)+ (name (string-join (list (package-name base-zfs)+ "for"+ (package-name kernel)+ (package-version kernel)+ "version")+ "-"))+ (arguments (cons* #:linux kernel (package-arguments base-zfs))))))++(define (zfs-loadable-module conf)+ (list (list (make-zfs-package conf) "module")))++(define (zfs-shepherd-services conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (zfs (file-append zfs-package "/sbin/zfs"))+ (zvol_wait (file-append zfs-package "/bin/zvol_wait"))+ (scheme-modules `((srfi srfi-1)+ (srfi srfi-34)+ (srfi srfi-35)+ (rnrs io ports)+ ,@%default-modules)))++ (define zfs-scan+ (shepherd-service+ (provision '(zfs-scan))+ (documentation "Scans for ZFS pools.")+ (requirement `(kernel-module-loader+ root-file-system+ ,@(map dependency->shepherd-service-name+ (zfs-configuration-dependencies conf))))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error importing zpools: ~a~%"+ (condition-message c))+ #f))+ ; TODO: optionally use a cachefile, for systems with dozens or+ ; hundreds of devices.+ (invoke/quiet #$zpool "import" "-a" "-N"))))+ (stop #~(const #t))))++ (define device-mapping-zvol/*+ (shepherd-service+ (provision '(device-mapping-zvol/*))+ (documentation "Waits for ZFS ZVOL devices to appear.")+ (requirement '(zfs-scan))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error waiting for zvols: ~a~%"+ (condition-message c))+ #f))+ (invoke/quiet #$zvol_wait))))+ (stop #~(const #t))))++ (define zfs-automount+ (shepherd-service+ (provision '(zfs-automount))+ (documentation "Automounts ZFS datasets.")+ (requirement '(zfs-scan))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error automounting zfs: ~a~$")+ #f))+ ; (current-output-port) is typically connected to /dev/klog,+ ; so redirect it to (current-error-port) so that user can see+ ; prompts for passphrases on console+ (with-output-to-port (current-error-port)+ (lambda ()+ (invoke #$zfs "mount" "-a" "-l"))))))+ (stop #~(lambda _+ ;; make sure we don't keep any ZFS mountpoints busy.+ (chdir "/")+ ;; unmount everything.+ (invoke/quiet #$zfs "unmount" "-a" "-f")))))++ (list zfs-scan+ device-mapping-zvol/*+ zfs-automount)))++(define zfs-service-type+ (service-type (name 'zfs)+ (extensions+ (list+ ; install the kernel module+ (service-extension kernel-loadable-module-service-type+ zfs-loadable-module)+ ; load the kernel module+ (service-extension kernel-module-loader-service-type+ (const '("zfs")))+ ; scan ZFS pools, automount filesystem, wait for zvols.+ (service-extension shepherd-root-service-type+ zfs-shepherd-services)+ ; make sure automount occurs before file-systems target is reached+ (service-extension file-systems-target-service-type+ (const '(zfs-automount)))+ ; install ZFS management tools+ (service-extension profile-service-type+ (compose list make-zfs-package))+ ; install ZFS udev rules+ (service-extension udev-service-type+ (compose list make-zfs-package))))+ (description+ "Install ZFS, an advanced filesystem and volume manager.")))
--2.30.0
R
R
raid5atemyhomework wrote on 8 Jan 17:16 +0100
Re: [PATCH 1/4] gnu: Allow services to install kernel-loadable modules.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
dczqwG2jPjcW8kLCB2rWgE7UxjX4eGFliB97Uug4pxwzaEVw7Wtrwm3S4fO1tRv3AG816TT0pAZWaaZY-3gE5RmAaDV9NLbCKyuW4HeBoZA=@protonmail.com
Is this patch acceptable? https://lists.nongnu.org/archive/html/guix-devel/2021-01/msg00070.htmlI reported that I compared a `guix system build` result of an `operating-system` that used the existing `(kernel-loadable-modules ...)` field, with and without this patch.
The resulting builds resulted in different hashes, but with exactly the same contents in the build --- `diff -r` very quickly reported no differences because it saw that nearly all the symlinks pointed to the same gnu store items.
R
R
raid5atemyhomework wrote on 9 Jan 09:31 +0100
Re: [PATCH 4/4] gnu: Add ZFS service.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
4XtBUBZvLlaRj3Qw8d44cVUm2QWePWKlxR4G7yH_bwXA62SoedCr5VHkch5VkOXfEd5frGaddsTLEEyiVQC0kQmearhfiiveI626gGbVjts=@protonmail.com
New version again. Change: Fix documentation `doc/guix.texi` to remove mentions of `options` and add documentation about the new `dependencies` features.

From dfe9ad7512d348933beb5b42041965ff604422b5 Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Wed, 6 Jan 2021 09:24:20 +0800Subject: [PATCH 1/2] gnu: Add service to install ZFS.
* gnu/services/file-systems.scm: New file.(zfs-service-type): New variable.(<zfs-configuration>): New type.(%zfs-zvol-dependency): New variable.* gnu/local.mk: Add gnu/services/file-systems.scm.* gnu/services/base.scm (dependency->shepherd-service-name): Export.* doc/guix.texi (ZFS file system): New subsection.--- doc/guix.texi | 129 +++++++++++++++++++++++++ gnu/local.mk | 1 + gnu/services/base.scm | 4 +- gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++ 4 files changed, 307 insertions(+), 1 deletion(-) create mode 100644 gnu/services/file-systems.scm
Toggle diff (347 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex a31d355780..7004efe3c4 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -13886,6 +13886,135 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp
+@node ZFS file system+@subsection ZFS file system++The ZFS file system has a license incompatible with the Linux kernel,+and thus cannot be distributed with the kernel. However, as a user+you have the right to do whatever you want on your own hardware,+including download the ZFS source code, compile it, link it to your+own private copy of Linux, and run it. You can even use the Guix build+system to automate this.++As a large and complex kernel module, ZFS on Linux has to be compiled+with a specific version of the kernel. Often even the latest ZFS+package available cannot be compiled with the latest Linux kernel+package provided by Guix System. Thus, installing the @code{zfs}+package is likely to fail.++Instead, you have to use an older long-term-support Linux kernel.+Do not use @code{linux-libre-lts}, since the latest long-term-support+kernel might be too new for the ZFS package; instead, explicitly+select the version number, like @code{linux-libre-5.4}, and upgrade+it manually later when you have verified that the ZFS version+available on Guix can be compiled with a later LTS kernel.++Then, you have to modify your system configuration file and use the+selected older kernel, and add the @code{zfs-service-type} service.++@lisp+(use-modules (gnu))+(use-package-modules+ ; @dots{}+ linux)+(use-service-modules+ ; @dots{}+ file-systems)++(define my-kernel linux-libre-5.4)++(operating-system+ (kernel my-kernel)+ ;; @dots{}+ (services+ (cons* (service zfs-service-type+ (zfs-configuration+ (kernel my-kernel)))+ ; @dots{}+ %desktop-services))+ ;; @dots{}+ )+@end lisp++@defvr {Scheme Variable} zfs-service-type+This is the type of the service to compile and install OpenZFS to+your operating system. It loads the ZFS module at startup, imports+pools, mounts automounted ZFS datasets, and installs the ZFS command+line tools. Its value must be a @code{zfs-configuration} record+(see below).++Here is an example use:++@lisp+(service zfs-service-type+ (zfs-configuration+ (kernel linux-libre-5.4)))+@end lisp+@end defvr++@deftp {Data Type} zfs-configuration+This data type represents the configuration of the ZFS service.+The available fields are:++@table @asis+@item @code{kernel}+The package of the Linux kernel to compile ZFS for. Required. It+@emph{must} be the same kernel you use in your operating system.++@item @code{base-zfs} (default: @code{zfs})+The ZFS package to use. It will be modified to use the indicated+kernel.++@item @code{dependencies} (default: @code{'()})+A list of @code{<file-system>}s or @code{<mapped-device>}s that+must be mounted or opened before ZFS scans for pools to import.+For example, you might have LUKS containers as the leaf VDEVs of+a ZFS pool.+@end table+@end deftp++Once your system has been configured to include the ZFS service+and you have restarted the system, you can manage ZFS pools and+datasets with @code{zpool} and @code{zfs} commands.++ZFS datasets with an appropriate @code{mountpoint} property will+be automounted at startup after the root file system is started.+Encrypted datasets that are automounted will cause boot to pause+and prompt for the password to be provided on the console.++It's possible to have a ZFS dataset as your @code{/home} by simply+setting the @code{mountpoint} ZFS property. However, note that ZFS+will refuse to mount over a non-empty directory, so if your root+filesystem already has a non-empty @code{/home}, you should remove+it and its contents, then restart the system.++ZFS features @dfn{ZVOL}s, which are block devices that ZFS exposes+to the system. You can put any file system inside a ZVOL. In order+to mount such a filesystem at boot, you can declare it as dependent+on @code{%zfs-zvol-dependency}.++@defvr {Scheme Variable} %zfs-zvol-dependency+This is an artificial @code{<mapped-device>} object which tells+the file mounting service to wait for ZFS to provide ZVOLs before+mounting the file system that is dependent on it. For example:++@lisp+(file-system+ (device "/dev/zvol/pool-name/zvol-name")+ (mount-point "/ext4-on-zfs")+ (type "ext4")+ (dependencies (list %zfs-zvol-dependency)))+@end lisp++Do @emph{not} add @code{%zfs-zvol-dependency} to your+@code{mapped-devices} declaration, In addition, only use it as a+@code{file-system} dependency if you instantiate a+@code{zfs-service-type} service in your operating system.+@end defvr++Having ZFS as a root filesystem or as @code{/boot} is not supported+yet.+ @node Mapped Devices @section Mapped Devices
diff --git a/gnu/local.mk b/gnu/local.mkindex 1151d4642e..5aeb45c4c2 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -602,6 +602,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/dict.scm \ %D%/services/dns.scm \ %D%/services/docker.scm \+ %D%/services/file-systems.scm \ %D%/services/authentication.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex 13cfb6a8a2..ef3680583b 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -188,7 +188,9 @@
references-file
- %base-services))+ %base-services++ dependency->shepherd-service-name))
;;; Commentary: ;;;diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scmnew file mode 100644index 0000000000..9061ab9582--- /dev/null+++ b/gnu/services/file-systems.scm@@ -0,0 +1,174 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu services file-systems)+ #:use-module (gnu packages file-systems)+ #:use-module (gnu services)+ #:use-module (gnu services base)+ #:use-module (gnu services linux)+ #:use-module (gnu services shepherd)+ #:use-module (gnu system mapped-devices)+ #:use-module (guix gexp)+ #:use-module (guix packages)+ #:use-module (guix records)+ #:export (zfs-service-type++ zfs-configuration+ zfs-configuration?+ zfs-configuration-kernel+ zfs-configuration-base-zfs+ zfs-configuration-dependencies++ %zfs-zvol-dependency))++(define-record-type* <zfs-configuration>+ zfs-configuration make-zfs-configuration zfs-configuration?+ ; kernel you want to compile the base-zfs module for.+ (kernel zfs-configuration-kernel)+ ; base package that will be compiled for the kernel+ (base-zfs zfs-configuration-base-zfs+ (default zfs))+ ; list of <mapped-device> | <file-system> we should wait for,+ ; before scanning for ZFS pools.+ (dependencies zfs-configuration-dependencies+ (default '())))++;; This is a synthetic and unusable MAPPED-DEVICE; its only use+;; is to be added as a (dependency ...) of some FILE-SYSTEM.+(define %zfs-zvol-dependency+ (mapped-device+ (source '())+ ;; The /* prevents naming conflict with non-ZFS device mappings,+ ;; since it is not a valid name for mapped devices, and also+ ;; implies "all zvols" in terms of globs.+ (targets '("zvol/*"))+ (type #f)))++(define (make-zfs-package conf)+ (let ((base-zfs (zfs-configuration-base-zfs conf))+ (kernel (zfs-configuration-kernel conf)))+ (package+ (inherit base-zfs)+ (name (string-join (list (package-name base-zfs)+ "for"+ (package-name kernel)+ (package-version kernel)+ "version")+ "-"))+ (arguments (cons* #:linux kernel (package-arguments base-zfs))))))++(define (zfs-loadable-module conf)+ (list (list (make-zfs-package conf) "module")))++(define (zfs-shepherd-services conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (zfs (file-append zfs-package "/sbin/zfs"))+ (zvol_wait (file-append zfs-package "/bin/zvol_wait"))+ (scheme-modules `((srfi srfi-1)+ (srfi srfi-34)+ (srfi srfi-35)+ (rnrs io ports)+ ,@%default-modules)))++ (define zfs-scan+ (shepherd-service+ (provision '(zfs-scan))+ (documentation "Scans for ZFS pools.")+ (requirement `(kernel-module-loader+ root-file-system+ ,@(map dependency->shepherd-service-name+ (zfs-configuration-dependencies conf))))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error importing zpools: ~a~%"+ (condition-message c))+ #f))+ ; TODO: optionally use a cachefile, for systems with dozens or+ ; hundreds of devices.+ (invoke/quiet #$zpool "import" "-a" "-N"))))+ (stop #~(const #t))))++ (define device-mapping-zvol/*+ (shepherd-service+ (provision '(device-mapping-zvol/*))+ (documentation "Waits for ZFS ZVOL devices to appear.")+ (requirement '(zfs-scan))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error waiting for zvols: ~a~%"+ (condition-message c))+ #f))+ (invoke/quiet #$zvol_wait))))+ (stop #~(const #t))))++ (define zfs-automount+ (shepherd-service+ (provision '(zfs-automount))+ (documentation "Automounts ZFS datasets.")+ (requirement '(zfs-scan))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error automounting zfs: ~a~$")+ #f))+ ; (current-output-port) is typically connected to /dev/klog,+ ; so redirect it to (current-error-port) so that user can see+ ; prompts for passphrases on console+ (with-output-to-port (current-error-port)+ (lambda ()+ (invoke #$zfs "mount" "-a" "-l"))))))+ (stop #~(lambda _+ ;; make sure we don't keep any ZFS mountpoints busy.+ (chdir "/")+ ;; unmount everything.+ (invoke/quiet #$zfs "unmount" "-a" "-f")))))++ (list zfs-scan+ device-mapping-zvol/*+ zfs-automount)))++(define zfs-service-type+ (service-type (name 'zfs)+ (extensions+ (list+ ; install the kernel module+ (service-extension kernel-loadable-module-service-type+ zfs-loadable-module)+ ; load the kernel module+ (service-extension kernel-module-loader-service-type+ (const '("zfs")))+ ; scan ZFS pools, automount filesystem, wait for zvols.+ (service-extension shepherd-root-service-type+ zfs-shepherd-services)+ ; make sure automount occurs before file-systems target is reached+ (service-extension file-systems-target-service-type+ (const '(zfs-automount)))+ ; install ZFS management tools+ (service-extension profile-service-type+ (compose list make-zfs-package))+ ; install ZFS udev rules+ (service-extension udev-service-type+ (compose list make-zfs-package))))+ (description+ "Install ZFS, an advanced filesystem and volume manager.")))
--2.30.0
宋文武 wrote on 23 Jan 14:05 +0100
Re: [bug#45692] [PATCH 2/4] gnu: Make file-systems target extensible by services.
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
OSZP286MB066439455D4B38ABA5F04337A3BF0@OSZP286MB0664.JPNP286.PROD.OUTLOOK.COM
Hello!
raid5atemyhomework <raid5atemyhomework@protonmail.com> writes:
Toggle quote (5 lines)>>From 792a8f8efc95e4fe9a94d42f839ddcfb034b8540 Mon Sep 17 00:00:00 2001> From: raid5atemyhomework <raid5atemyhomework@protonmail.com>> Date: Wed, 6 Jan 2021 08:15:54 +0800> Subject: [PATCH 2/4] gnu: Make file-systems target extensible by services.
It’s not clear to me what “file-systems target” is, and why we’reextending ‘file-systems-target-service-type’… I think what we want isto extend the ‘file-system-service-type’ with shepherd services’ names,which means some shepherd services that will handle file systemsmounting themself instead of the usual <file-system> objects, fstabentries, mounted by kernel.

So I write with this patch to extend file-system-service-type directlyinstead of introducing a new ‘file-systems-target-service-type’:
From 44ee1e470a2f9d4985af4d51654d9f943caa0f24 Mon Sep 17 00:00:00 2001From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>Date: Sat, 23 Jan 2021 20:39:06 +0800Subject: [PATCH] services: Allow 'file-system-service-type' extensible by service name.MIME-Version: 1.0Content-Type: text/plain; charset=UTF-8Content-Transfer-Encoding: 8bit
* gnu/services/base.scm (file-system-shepherd-services): Add'extra-services-names' paramater.(file-system-service-type): Handle services’ names from extensions.--- gnu/services/base.scm | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)
Toggle diff (53 lines)diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex f6a490f712..7bddef5034 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -364,15 +364,16 @@ FILE-SYSTEM." (gnu system file-systems) ,@%default-modules))))))) -(define (file-system-shepherd-services file-systems)+(define (file-system-shepherd-services file-systems extra-services-names) "Return the list of Shepherd services for FILE-SYSTEMS." (let* ((file-systems (filter file-system-mount? file-systems))) (define sink (shepherd-service (provision '(file-systems))- (requirement (cons* 'root-file-system 'user-file-systems- (map file-system->shepherd-service-name- file-systems)))+ (requirement (append '(root-file-system user-file-systems)+ (map file-system->shepherd-service-name+ file-systems)+ extra-services-names)) (documentation "Target for all the initially-mounted file systems") (start #~(const #t)) (stop #~(const #f))))@@ -429,13 +430,23 @@ FILE-SYSTEM." (service-type (name 'file-systems) (extensions (list (service-extension shepherd-root-service-type- file-system-shepherd-services)+ (lambda (value)+ (file-system-shepherd-services+ (filter file-system? value)+ (filter symbol? value)))) (service-extension fstab-service-type- file-system-fstab-entries)+ (lambda (value)+ (file-system-fstab-entries+ (filter file-system? value)))) ;; Have 'user-processes' depend on 'file-systems'. (service-extension user-processes-service-type (const '(file-systems)))))++ ;; Extensions consist of lists of <file-system> objects or+ ;; shepherd services’ names (symbols). In the latter case,+ ;; the provided shepherd services supposed to mount and+ ;; unmount some file systems themself. (compose concatenate) (extend append) (description-- 2.29.2
What do you think? Thank you!
R
R
raid5atemyhomework wrote on 25 Jan 01:18 +0100
(name . 宋文武)(address . iyzsong@outlook.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
yrjG3wstz9GpyH32kV3JqsEiG-KdxmP9_JHe6ghdMoizrBWpIZhd3hl2VPjFuWagt1WLPXMqSGdMZZWTcvE6J7qxVjdcvq3zTK-Wccd3wEs=@protonmail.com
Hello as well,
That is certainly another possibility, would this be more palatable to Guix?
Thanksraid5atemyhomework
Toggle quote (20 lines)> Hello!>> raid5atemyhomework raid5atemyhomework@protonmail.com writes:>> > > From 792a8f8efc95e4fe9a94d42f839ddcfb034b8540 Mon Sep 17 00:00:00 2001> > > From: raid5atemyhomework raid5atemyhomework@protonmail.com> > > Date: Wed, 6 Jan 2021 08:15:54 +0800> > > Subject: [PATCH 2/4] gnu: Make file-systems target extensible by services.>> It’s not clear to me what “file-systems target” is, and why we’re> extending ‘file-systems-target-service-type’… I think what we want is> to extend the ‘file-system-service-type’ with shepherd services’ names,> which means some shepherd services that will handle file systems> mounting themself instead of the usual <file-system> objects, fstab> entries, mounted by kernel.>> So I write with this patch to extend file-system-service-type directly> instead of introducing a new ‘file-systems-target-service-type’:>> What do you think? Thank you!
From 44ee1e470a2f9d4985af4d51654d9f943caa0f24 Mon Sep 17 00:00:00 2001From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>Date: Sat, 23 Jan 2021 20:39:06 +0800Subject: [PATCH] services: Allow 'file-system-service-type' extensible by service name.MIME-Version: 1.0Content-Type: text/plain; charset=UTF-8Content-Transfer-Encoding: 8bit
* gnu/services/base.scm (file-system-shepherd-services): Add'extra-services-names' paramater.(file-system-service-type): Handle services’ names from extensions.--- gnu/services/base.scm | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)
Toggle diff (53 lines)diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex f6a490f712..7bddef5034 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -364,15 +364,16 @@ FILE-SYSTEM." (gnu system file-systems) ,@%default-modules))))))) -(define (file-system-shepherd-services file-systems)+(define (file-system-shepherd-services file-systems extra-services-names) "Return the list of Shepherd services for FILE-SYSTEMS." (let* ((file-systems (filter file-system-mount? file-systems))) (define sink (shepherd-service (provision '(file-systems))- (requirement (cons* 'root-file-system 'user-file-systems- (map file-system->shepherd-service-name- file-systems)))+ (requirement (append '(root-file-system user-file-systems)+ (map file-system->shepherd-service-name+ file-systems)+ extra-services-names)) (documentation "Target for all the initially-mounted file systems") (start #~(const #t)) (stop #~(const #f))))@@ -429,13 +430,23 @@ FILE-SYSTEM." (service-type (name 'file-systems) (extensions (list (service-extension shepherd-root-service-type- file-system-shepherd-services)+ (lambda (value)+ (file-system-shepherd-services+ (filter file-system? value)+ (filter symbol? value)))) (service-extension fstab-service-type- file-system-fstab-entries)+ (lambda (value)+ (file-system-fstab-entries+ (filter file-system? value)))) ;; Have 'user-processes' depend on 'file-systems'. (service-extension user-processes-service-type (const '(file-systems)))))++ ;; Extensions consist of lists of <file-system> objects or+ ;; shepherd services’ names (symbols). In the latter case,+ ;; the provided shepherd services supposed to mount and+ ;; unmount some file systems themself. (compose concatenate) (extend append) (description-- 2.29.2
D
D
Danny Milosavljevic wrote on 8 Feb 04:31 +0100
Re: [bug#45692] [PATCH 4/4] gnu: Add ZFS service.
(name . raid5atemyhomework via Guix-patches via)(address . guix-patches@gnu.org)
20210208043152.7cdd5900@scratchpost.org
Hello,
is it necessary to manually load the ZFS Linux kernel module? Usually, Linuxshould autoload its drivers when the device files are accessed--module or not.
If that fails, didn't you patch the zfs package so it refers to modprobe anyway?
(I wanna prevent having 543 places in Guix where modprobe is invoked)
If it doesn't autoload, we can totally modprobe it, though.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAmAgsKgACgkQ5xo1VCwwuqWYlAf+LmpUSOE+T8s2fUOta0h6kXCDklfZ7ehCrHa83LfNAuOl0jVQS9OeoabcIN6y9gujcUY345NJI/QvWTqiKoU6deogK2Ut4VDLHfzhiD5H4J7WYGm1Z//ygFEqku7+YL1rbMTo/qe0s40U0MeMNq7V0PEDyg3XvKZmL7GdtYa71sAKRID3RMQ6M5pdGJUUWOlNOs9p+VQOcobReTS1t9NMRhrCHg6tjrVtTrnc7aVQxuEg7y+BSGeNC6MpLA/SHxYcu8w6vomlNyGQL7oY6xNMNc6NihiToWrinkV56bG7x7ssLenJGYsWOCOG5xqXwUy5VmcTHDYADCmx0q2v/4iJvQ===jbFl-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 8 Feb 07:25 +0100
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
yKAqBHv-O0IjcNxi3O1RSMfxY_53Le1QYhA8AS6AIp8JYPpR7eeK3wdWvvBluJhoA13O0iUD904p_DXG78_1XP4mQtE2pKixjvtO7tXSQv0=@protonmail.com
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐On Monday, February 8, 2021 3:31 AM, Danny Milosavljevic <dannym@scratchpost.org> wrote:
Toggle quote (7 lines)> Hello,>> is it necessary to manually load the ZFS Linux kernel module? Usually, Linux> should autoload its drivers when the device files are accessed--module or not.>> If that fails, didn't you patch the zfs package so it refers to modprobe anyway?
It does, for `udev`. Not certain if it would trigger the load, though. Presumably it would, but I have not tested.
A lot of this code was written in 0.8.5, where the ZFS tools will *not* attempt to `modprobe` the module first, they just error out. I think in 2.0.1 upstream added code to load the module if the ZFS tool(s) were invoked while the module was not loaded, which is why additional patch was added by me for the 2.0.1 to patch in `modprobe`. I have not tested if the `modprobe`-ing by the ZFS tools actually *works*, though.
Maybe when I find some amount of motivation I can go test it.
Toggle quote (6 lines)>> (I wanna prevent having 543 places in Guix where modprobe is invoked)>> If it doesn't autoload, we can totally modprobe it, though.

Thanksraid5atemyhomework
L
L
Ludovic Courtès wrote on 10 Feb 15:13 +0100
Re: bug#45692: [PATCH 0/4] Even Better ZFS Support on Guix
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
87zh0c6mf4.fsf_-_@gnu.org
raid5atemyhomework <raid5atemyhomework@protonmail.com> skribis:
Toggle quote (20 lines)> +(define (kernel-builder-configuration->system-entry config)> + "Return the kernel and hurd entries of the 'system' directory."> + (mbegin %store-monad> + (let* ((kernel (kernel-builder-configuration-kernel config))> + (hurd (kernel-builder-configuration-hurd config))> + (modules (kernel-builder-configuration-modules config))> + (kernel (if hurd> + kernel> + (profile> + (content (packages->manifest> + (cons kernel> + (map (lambda (module)> + (if (package? module)> + (package-for-kernel kernel module)> + module))> + modules))))> + (hooks (list linux-module-database))))))> + (return `(("kernel" ,kernel)> + ,@(if hurd `(("hurd" ,hurd)) '()))))))
It may be clearer to avoid ‘mbegin’ and instead write it this way:
(define (kernel-builder-configuration-modules config) (let* … (with-monad %store-monad (return …))))
Both work but I find this variant slightly clearer.
Ludo’.
L
L
Ludovic Courtès wrote on 10 Feb 15:17 +0100
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
87sg646m7y.fsf_-_@gnu.org
Hi!
I agree with 宋文武.
Still…
raid5atemyhomework <raid5atemyhomework@protonmail.com> skribis:
Toggle quote (23 lines)> (service-type (name 'file-systems)> (extensions> (list (service-extension shepherd-root-service-type> - file-system-shepherd-services)> + (lambda (value)> + (file-system-shepherd-services> + (filter file-system? value)> + (filter symbol? value))))> (service-extension fstab-service-type> - file-system-fstab-entries)> + (lambda (value)> + (file-system-fstab-entries> + (filter file-system? value))))> > ;; Have 'user-processes' depend on 'file-systems'.> (service-extension user-processes-service-type> (const '(file-systems)))))> +> + ;; Extensions consist of lists of <file-system> objects or> + ;; shepherd services’ names (symbols). In the latter case,> + ;; the provided shepherd services supposed to mount and> + ;; unmount some file systems themself.
Why do we need to extend with symbols?
In general it’s much clearer if extensions receive only one type ofobject (<file-system> records in this case). It’s also best to avoidpassing around symbolic names (that’s why we extend with <file-system>records rather than with Shepherd service names or whatever.)
Ludo’.
L
L
Ludovic Courtès wrote on 10 Feb 15:27 +0100
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
87eeho6ls3.fsf_-_@gnu.org
raid5atemyhomework <raid5atemyhomework@protonmail.com> skribis:
Toggle quote (13 lines)>>From d0e095136cd471f083d92416f12ca22b47301f31 Mon Sep 17 00:00:00 2001> From: raid5atemyhomework <raid5atemyhomework@protonmail.com>> Date: Wed, 6 Jan 2021 09:24:20 +0800> Subject: [PATCH 4/4] gnu: Add service to install ZFS.>> * gnu/services/file-systems.scm: New file.> (zfs-service-type): New variable.> (<zfs-configuration>): New type.> (%zfs-zvol-dependency): New variable.> * gnu/local.mk: Add gnu/services/file-systems.scm.> * gnu/services/base.scm (dependency->shepherd-service-name): Export.> * doc/guix.texi (ZFS file system): New subsection.
A bit of nitpicking on the documentation bits…
Toggle quote (3 lines)> +@node ZFS file system> +@subsection ZFS file system
Please follow the same typographical conventions as in the rest of themanual. In particular, capitalize section titles and leave two spacesafter end-of-sentence periods.
Here I recommend s/ZFS file system/ZFS/ in all the text.
Toggle quote (7 lines)> +The ZFS file system has a license incompatible with the Linux kernel,> +and thus cannot be distributed with the kernel. However, as a user> +you have the right to do whatever you want on your own hardware,> +including download the ZFS source code, compile it, link it to your> +own private copy of Linux, and run it. You can even use the Guix build> +system to automate this.
How about:
ZFS is free software; unfortunately its license is incompatible with the GNU General Public License (GPL), the license of the Linux kernel, which means they cannot be distributed together. However, as a user, you can choose to build ZFS and use it together with Linux; you can even rely on Guix to automate this task. See @uref{https://www.fsf.org/licensing/zfs-and-linux,this analysis by the Free Software Foundation} for more information.
?
Toggle quote (1 lines)> +As a large and complex kernel module, ZFS on Linux has to be compiled
^I think it’s a proper noun and is spelled “ZFS-on-Linux”.
Toggle quote (2 lines)> +with a specific version of the kernel. Often even the latest ZFS> +package available cannot be compiled with the latest Linux kernel
^s/Linux/Linux-libre/
Thanks,Ludo’.
R
R
raid5atemyhomework wrote on 10 Feb 15:32 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
ih7G5G36XJ1ajfM1OZo4G6oaKrwtRusZF9NJhKBtRudytmXCib41DxapatiWe6As3jl1miABW-Yp8oXy4wI4dq5xn4AXEFO9DAdXtFZYGyY=@protonmail.com
Hi Ludo,
Toggle quote (25 lines)> raid5atemyhomework raid5atemyhomework@protonmail.com skribis:>> > > From d0e095136cd471f083d92416f12ca22b47301f31 Mon Sep 17 00:00:00 2001> > > From: raid5atemyhomework raid5atemyhomework@protonmail.com> > > Date: Wed, 6 Jan 2021 09:24:20 +0800> > > Subject: [PATCH 4/4] gnu: Add service to install ZFS.> >> > - gnu/services/file-systems.scm: New file.> > (zfs-service-type): New variable.> > (<zfs-configuration>): New type.> > (%zfs-zvol-dependency): New variable.> >> > - gnu/local.mk: Add gnu/services/file-systems.scm.> > - gnu/services/base.scm (dependency->shepherd-service-name): Export.> > - doc/guix.texi (ZFS file system): New subsection.>> A bit of nitpicking on the documentation bits…>> > +@node ZFS file system> > +@subsection ZFS file system>> Please follow the same typographical conventions as in the rest of the> manual. In particular, capitalize section titles and leave two spaces> after end-of-sentence periods.
This was imitated from the `@subsection Btrfs file system`.
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 10 Feb 15:46 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)
HM9LDUftlNlIoFFvUdzDIRUsjIADdaZLVoCF0cg-l2EoW6hVu09JpTcsOBMbQbpefRhiRv0IKE4W5QMi3ww5NedWcg8eNiRjrzA3hhJ06N0=@protonmail.com
Toggle quote (7 lines)> Why do we need to extend with symbols?>> In general it’s much clearer if extensions receive only one type of> object (<file-system> records in this case). It’s also best to avoid> passing around symbolic names (that’s why we extend with <file-system>> records rather than with Shepherd service names or whatever.)
For this case, how would it be done?
ZFS file system, on other operating systems and distributions, is documented as automatically mounting filesystems, without management in an `fstab` or similar file, because the intent is that you would make lots of filesystems for various uses and managing an `fstab` would be too onerous. Thus, ZFS file system expects to mount multiple file systems with a single `zfs mount -a` command at startup.
Would the below sketch be acceptable?
```scheme; gnu/system/file-systems.scm(define-record-type* file-system #;... #;... (has-fstab-entry? file-system-has-fstab-entry? (default #t)))
;...; gnu/services/base,scm
(define file-system-service-type (service-type #;... (extensions (list #;... (service-extension fstab-service-type (lambda (file-systems) (filter file-system-has-fstab-entry? file-systems))) #;...)) #;...))
;...; gnu/services/file-systems.scm
(define zfs-service-type (service-type #;... (extensions (list #;... (service-extension file-system-service-type (const (list (file-system (device "dummy") (mount-point "zfs/*") (has-fstab-entry? #f))))))) #;...))```
Then there will be a Shepherd service providing `file-system-zfs/*` which would perform `zfs mount -a -l` on `start` and `zfs unmount -a -f` on `stop`.
Would that be acceptable? I am wary of this since it creates a dummy file-system and needs an additional field on every `file-system` record, one which is *only* used by ZFS. I feel the `file-system-target-service-type` is more generic and does not use trickery.
Thanksraid5atemyhomework
L
L
Ludovic Courtès wrote on 10 Feb 16:44 +0100
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
87zh0c53m6.fsf_-_@gnu.org
Hi,
raid5atemyhomework <raid5atemyhomework@protonmail.com> skribis:
Toggle quote (24 lines)>>From 4beb73c62995cf236b402dad8e1c36016027c781 Mon Sep 17 00:00:00 2001> From: raid5atemyhomework <raid5atemyhomework@protonmail.com>> Date: Tue, 5 Jan 2021 22:27:56 +0800> Subject: [PATCH 1/4] gnu: Allow services to install kernel-loadable modules.>> * gnu/system.scm (operating-system-directory-base-entries): Remove code> to handle generation of "kernel" and "hurd".> (operating-system-default-essential-services): Instantiate> kernel-loadable-module-service.> (hurd-default-essential-services): Instantiate> kernel-loadable-module-service.> (package-for-kernel): Move ...> * gnu/services.scm: ... to here.> (kernel-loadable-module-service-type): New variable.> (kernel-loadable-module-service): New procedure.> * gnu/tests/linux-modules.scm (run-loadable-kernel-modules-test): Move> code to ...> (run-loadable-kernel-modules-test-base): ... new procedure here.> (run-loadable-kernel-modules-service-test): New procedure.> (%test-loadable-kernel-modules-service-0): New variable.> (%test-loadable-kernel-modules-service-1): New variable.> (%test-loadable-kernel-modules-service-2): New variable.> * doc/guix.texi: Document kernel-loadable-module-service-type.
[…]
Toggle quote (6 lines)> +@defvr {Scheme Variable} kernel-loadable-module-service-type> +Type of the service that collects lists of packages containing> +kernel-loadable modules, and adds them to the set of kernel-loadable> +modules.> +@end defvr
Would be nice to expound a bit here, in particular by adding an example(along the lines of those used in system tests maybe?). Otherwise itcan be hard to fathom how this is meant to be used.
Toggle quote (10 lines)> +;; Configuration for the kernel builder.> +(define-record-type* <kernel-builder-configuration> kernel-builder-configuration> + make-kernel-builder-configuration> + kernel-builder-configuration?> + this-kernel-builder-configuration> +> + (kernel kernel-builder-configuration-kernel (default #f))> + (hurd kernel-builder-configuration-hurd (default #f))> + (modules kernel-builder-configuration-modules (default '())))
How about <linux-build-configuration> instead?
In general, throughout the project, we do not use “kernel” and “Linux”interchangeably. Since this is a Linux-only feature, let’s call it thatway and remove the ‘hurd’ field (the Hurd has no notion of in-kernelmodules since pretty much everything happens in user-space.)
Toggle quote (3 lines)> +(define kernel-loadable-module-service-type> + (service-type (name 'kernel-loadable-modules)
Same here: ‘linux-loadable-module-service-type’.
But… it’s not clear at first sight how this differs from the existing‘kernel-module-loader’. Perhaps ‘linux-build-service-type’ would bemore accurate? Or am I missing something?
Thanks,Ludo’.
L
L
Ludovic Courtès wrote on 10 Feb 16:47 +0100
control message for bug #45692
(address . control@debbugs.gnu.org)
87y2fv6i26.fsf@gnu.org
merge 45692 45703quit
L
L
Ludovic Courtès wrote on 10 Feb 16:47 +0100
(address . control@debbugs.gnu.org)
87wnvf6i1i.fsf@gnu.org
merge 45692 45643quit
R
R
raid5atemyhomework wrote on 10 Feb 17:49 +0100
Re: bug#45692: [PATCH 0/4] Even Better ZFS Support on Guix
(name . Ludovic Courtès)(address . ludo@gnu.org)
Y5MIZ9SDFP1f6SdbG-fjEvSAkjgni6w0dilDSaKyWAauhgFfK-swJ6TpVCsLQ7c0bYA8tnIe1Fo7OiFAcuu-vO_W2h2UQdpfxNRZ1w_uSL0=@protonmail.com
Toggle quote (18 lines)> > +;; Configuration for the kernel builder.> > +(define-record-type* <kernel-builder-configuration> kernel-builder-configuration> >> > - make-kernel-builder-configuration> > - kernel-builder-configuration?> > - this-kernel-builder-configuration> > -> > - (kernel kernel-builder-configuration-kernel (default #f))> > - (hurd kernel-builder-configuration-hurd (default #f))> > - (modules kernel-builder-configuration-modules (default '())))>> How about <linux-build-configuration> instead?>> In general, throughout the project, we do not use “kernel” and “Linux”> interchangeably. Since this is a Linux-only feature, let’s call it that> way and remove the ‘hurd’ field (the Hurd has no notion of in-kernel> modules since pretty much everything happens in user-space.)
The `operating-system` record uses `kernel-loadable-modules` as the record field name. I suggest changing that first, if you truly want to differentiate "kernel" from "linux" "throughout the project". Or deprecate it entirely and instead use the new `linux-loadable-modules-service-type`, in principle the only field needed in `operating-system` should be `services`.
Toggle quote (12 lines)>> > +(define kernel-loadable-module-service-type> >> > - (service-type (name 'kernel-loadable-modules)>> Same here: ‘linux-loadable-module-service-type’.>> But… it’s not clear at first sight how this differs from the existing> ‘kernel-module-loader’. Perhaps ‘linux-build-service-type’ would be> more accurate? Or am I missing something?

`kernel-module-loader` explicitly loads a module at startup, it does not make a non-Linux-libre-built-in module actually *loadable*. So there is a need for something to augment the `linux-loadable-modules` record field of `operating-system`.
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 13 Feb 02:49 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
20Uzjlb-aGAMYW_UfB_P-PUylV-NrHRtq3NpXLe6Fsfuuq-wssW64sgoWoRSzwQdj-t5g2P4ebBJR2fllYCiQc5LyLb8oc2YrDs021q9u-g=@protonmail.com
Hello Ludo',

Toggle quote (7 lines)> > +As a large and complex kernel module, ZFS on Linux has to be compiled>> ^>>> I think it’s a proper noun and is spelled “ZFS-on-Linux”.
FWIW ZFSonLinux is now folded into OpenZFS (which is now a package that includes both Linux and FreeBSD support, with plans to merge in the Illumos codebase as well) since 2.0.0. The documentation was written when Guix was still at 0.8.5 (when it was definitely ZoL), so I should probably change this to OpenZFS, and clarify whether I am referring to the ZFS file system or the OpenZFS package.
Thanksraid5atemyhomework
宋文武 wrote on 20 Feb 12:44 +0100
Re: ZFS on Guix, again
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
OS3P286MB066003E371477FEA2063411EA3839@OS3P286MB0660.JPNP286.PROD.OUTLOOK.COM
raid5atemyhomework <raid5atemyhomework@protonmail.com> writes:
Toggle quote (5 lines)> Hi guix-devel,>> I had some questions on the big ZFS guix bugpatches a week ago, and> did not find any response, so I am back here pestering everyone.
Hello, thank you for working on ZFS for guix!
Toggle quote (12 lines)>> [...]> There are two alternatives:>> * Go with what I already proposed which I think is more general-purpose and cleaner (there is a separate service type that accepts symbols, and a separate service type that accepts `<file-system>` records, and the latter just extends the former).> * Don't make a separate service type, but now we need to add some kind of `fstab?` field to `file-system` so that the ZFS shepherd service that mounts ZFS file systems will not be included in the `/etc/fstab`.>> I think overall that having lots of tiny service types that are then> combined together fits the functional design of Guix better. So I> would strongly propose my original design rather than hacks on top of> `file-system-service-type`.
Well, I think the 'file-system-service-type' should handle all filesystems related configurations, but my opion is not strong. Waitingludo to decide...
R
R
raid5atemyhomework wrote on 22 Mar 15:33 +0100
[PATCH v3 0/3] New patch series for Even Better ZFS Support on Guix
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
K_sz_FR8KdZSXDYUr6uI-jabQq0PqKjVANb1wWph8rK6Yy0wLIq8vj5cMNuRlWWNZKL4BFfRQoni1elqjzbCW7FYvCdLrSC2x-1N97_RU34=@protonmail.com
This is a new patch series, please completely forget the previous patch series.
Notable differences are:
* ZFS automounted datasets are mounted before `user-processes` instead of `file-systems`, since there is strong objection to modifying `file-systems` so it can be extended to allow ZFS automounted datasets to be mounted before `file-systems`. This prevents easy `/home` on ZFS (though see the added documentation in 3/3 for a workaround).* The old `kernel-loadable-module-service-type` is renamed to `linux-loadable-module-service-type`, despite the corresponding `operating-system` field being named `kernel-loadable-modules`, and the loader service type being named `kernel-module-loader-service-type` :rolleyes:.* There is a new `linux-builder-service-type`, which can be extended with a function to modify its `linux-builder-configuration`. The `linux-loadable-module-service-type` extends this base service. I think this design patttern should be more extensively used in Guix; currently services are only extensible via one field of their configuration, this pattern allows creating multiple service types for each extensible field of the configuration. In this particular case, the `kernel` of the `linux-builder-service-type` can be extended (by adding/removing `configure` or `make` flags, or added `inputs`) independently of the list of loadable modules using this design pattern.* Auto-scrub and auto-snapshot is now supported, with additional fields in `zfs-configuration` to support it.* The auto-mounting Shepherd service has been renamed `zfs-auto-mount` (previously `zfs-automount`).* Documentation includes a list of things that are currently not (fully) supported.
Personally I'm not particularly motivated to push this anymore. I have a frankenstein's `configuration.scm` that gets me the bits of ZFS functionality that I absolutely need, with the rest something I'm just living without, and while it would be *nice* to have *some* out-of-the-box support for ZFS (or at least make it easy to get a box to get support for ZFS out of), I personally don't need it at this point.
Patch 1/3 corresponds to the original Patch 1/4. Patch 2/3 has no corresponding patch in the original patch set, but is the same patch as https://issues.guix.gnu.org/47134which nobody is looking at, as usual. Patch 3/3 corresponds to the original Patch 4/4.
The original Patch 2/4 has been removed since it's what makes `/home` on ZFS work by making `file-systems` extensible. The original Patch 3/4 was already merged.
This patch series is minimally tested compared to the previous patch series; I have homework to store on my ZFS pool and should really stop fooling around with the operating system. I have this patch series running in a VM with a ZFS pool. Auto-scrub is not tested, though auto-snapshot looks working. Auto-mounting works as with previous patch series. Encrypted and compressed filesystems are untested but I don't expect it to be significantly regressed compared to the previous patch series.
More work can be done for ZFS support. But I think the current set gives a workable ZFS support; further work can be done piecemeal afterwards. The documentation includes notes for anyone who feels motivated enough to keep improving the ZFS support.
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 22 Mar 15:33 +0100
[PATCH v3 1/3] gnu: Allow services to install kernel-loadable modules.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
VQKkOa-eyRygylIYZoOrs8rI2MCV0e9iMe4ZXaFZZoEsT2UasU5NJjDRY4gge12_eBJrDYONBtVJ1qoojdiueq8LOmzSHl8wCZup8Fc9tR4=@protonmail.com
From d54d718dd83195041d9f536e8c675eb2bffdcb8d Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Mon, 22 Mar 2021 11:23:32 +0800Subject: [PATCH 1/3] gnu: Allow services to install kernel-loadable modules.
* gnu/system.scm (operating-system-directory-base-entries): Remove codeto handle generation of "kernel" for linux-libre kernels.(operating-system-default-essential-services): Instantiatelinux-builder-service-type.(package-for-kernel): Move ...* gnu/services.scm: ... to here.(linux-builder-service-type): New variable.(linux-builder-configuration): New type.(linux-loadable-module-service-type): New variable.* gnu/tests/linux-modules.scm (run-loadable-kernel-modules-test): Movecode to ...(run-loadable-kernel-modules-test-base): ... new procedure here.(run-loadable-kernel-modules-service-test): New procedure.(%test-loadable-kernel-modules-service-0): New variable.(%test-loadable-kernel-modules-service-1): New variable.(%test-loadable-kernel-modules-service-2): New variable.* doc/guix.texi: Document linux-loadable-module-service-type.--- doc/guix.texi | 22 +++++++++ gnu/services.scm | 90 +++++++++++++++++++++++++++++++++++++ gnu/system.scm | 34 ++++---------- gnu/tests/linux-modules.scm | 80 ++++++++++++++++++++++++++++----- 4 files changed, 191 insertions(+), 35 deletions(-)
Toggle diff (341 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 386169b2a5..86b22f3673 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -33967,6 +33967,28 @@ configuration when you use @command{guix system reconfigure}, @command{guix system init}, or @command{guix deploy}. @end defvr
+@defvr {Scheme Variable} linux-loadable-module-service-type+Type of the service that collects lists of packages containing+kernel-loadable modules, and adds them to the set of kernel-loadable+modules.++This service type is intended to be extended by other service types,+such as below:++@lisp+(define module-installing-service-type+ (service-type+ (name 'module-installing-service)+ (extensions (list (service-extension linux-loadable-module-service-type+ (const (list module-to-install-1+ module-to-install-2)))))+ (default-value #f)))+@end lisp++This does not actually load modules at bootup, only adds it to the+kernel profile so that it @emph{can} be loaded by other means.+@end defvr+ @node Shepherd Services @subsection Shepherd Services
diff --git a/gnu/services.scm b/gnu/services.scmindex ddd1bac30c..a20edeb8ec 100644--- a/gnu/services.scm+++ b/gnu/services.scm@@ -2,6 +2,7 @@ ;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org> ;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com> ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -33,6 +34,8 @@ #:use-module (guix diagnostics) #:autoload (guix openpgp) (openpgp-format-fingerprint) #:use-module (guix modules)+ #:use-module (guix packages)+ #:use-module (guix utils) #:use-module (gnu packages base) #:use-module (gnu packages bash) #:use-module (gnu packages hurd)@@ -106,6 +109,12 @@ profile-service-type firmware-service-type gc-root-service-type+ linux-builder-service-type+ linux-builder-configuration+ linux-builder-configuration?+ linux-builder-configuration-kernel+ linux-builder-configuration-modules+ linux-loadable-module-service-type
%boot-service %activation-service@@ -872,6 +881,87 @@ as Wifi cards."))) will not be reclaimed by the garbage collector.") (default-value '())))
+;; Configuration for the Linux kernel builder.+(define-record-type* <linux-builder-configuration>+ linux-builder-configuration+ make-linux-builder-configuration+ linux-builder-configuration?+ this-linux-builder-configuration++ (kernel linux-builder-configuration-kernel) ; package+ (modules linux-builder-configuration-modules (default '()))) ; list of packages++(define (package-for-kernel target-kernel module-package)+ "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if+possible (that is if there's a LINUX keyword argument in the build system)."+ (package+ (inherit module-package)+ (arguments+ (substitute-keyword-arguments (package-arguments module-package)+ ((#:linux kernel #f)+ target-kernel)))))++(define (linux-builder-configuration->system-entry config)+ "Return the kernel entry of the 'system' directory."+ (let* ((kernel (linux-builder-configuration-kernel config))+ (modules (linux-builder-configuration-modules config))+ (kernel (profile+ (content (packages->manifest+ (cons kernel+ (map (lambda (module)+ (cond+ ((package? module)+ (package-for-kernel kernel module))+ ;; support (,package "kernel-module-output")+ ((and (list? module) (package? (car module)))+ (cons (package-for-kernel kernel+ (car module))+ (cdr module)))+ (else+ module)))+ modules))))+ (hooks (list linux-module-database)))))+ (with-monad %store-monad+ (return `(("kernel" ,kernel))))))++(define linux-builder-service-type+ (service-type (name 'linux-builder)+ (extensions+ (list (service-extension system-service-type+ linux-builder-configuration->system-entry)))+ (default-value '())+ (compose identity)+ (extend (lambda (config modifiers)+ (if (null? modifiers)+ config+ ((apply compose modifiers) config))))+ (description "Builds the linux-libre kernel profile, containing+the kernel itself and any linux-loadable kernel modules. This can be extended+with a function that accepts the current configuration and returns a new+configuration.")))++(define (linux-loadable-module-builder-modifier modules)+ "Extends linux-builder-service-type by appending the given MODULES to the+configuration of linux-builder-service-type."+ (lambda (config)+ (linux-builder-configuration+ (inherit config)+ (modules (append (linux-builder-configuration-modules config)+ modules)))))++(define linux-loadable-module-service-type+ (service-type (name 'linux-loadable-modules)+ (extensions+ (list (service-extension linux-builder-service-type+ linux-loadable-module-builder-modifier)))+ (default-value '())+ (compose concatenate)+ (extend append)+ (description "Adds packages and package outputs as modules+included in the booted linux-libre profile. Other services can extend this+service type to add particular modules to the set of linux-loadable modules.")))++ ;;; ;;; Service folding.diff --git a/gnu/system.scm b/gnu/system.scmindex 5bf2a85272..7cc4f134b7 100644--- a/gnu/system.scm+++ b/gnu/system.scm@@ -13,6 +13,7 @@ ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <jannek@gnu.org> ;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il> ;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -601,16 +602,6 @@ OS." (file-append (operating-system-kernel os) "/" (system-linux-image-file-name))))
-(define (package-for-kernel target-kernel module-package)- "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if-possible (that is if there's a LINUX keyword argument in the build system)."- (package- (inherit module-package)- (arguments- (substitute-keyword-arguments (package-arguments module-package)- ((#:linux kernel #f)- target-kernel)))))- (define %default-modprobe-blacklist ;; List of kernel modules to blacklist by default. '("usbmouse" ;races with bcm5974, see <https://bugs.gnu.org/35574>@@ -628,23 +619,12 @@ value of the SYSTEM-SERVICE-TYPE service." (let* ((locale (operating-system-locale-directory os)) (kernel (operating-system-kernel os)) (hurd (operating-system-hurd os))- (modules (operating-system-kernel-loadable-modules os))- (kernel (if hurd- kernel- (profile- (content (packages->manifest- (cons kernel- (map (lambda (module)- (if (package? module)- (package-for-kernel kernel- module)- module))- modules))))- (hooks (list linux-module-database))))) (initrd (and (not hurd) (operating-system-initrd-file os))) (params (operating-system-boot-parameters-file os)))- `(("kernel" ,kernel)- ,@(if hurd `(("hurd" ,hurd)) '())+ `(,@(if hurd+ `(("hurd" ,hurd)+ ("kernel" ,kernel))+ '()) ("parameters" ,params) ,@(if initrd `(("initrd" ,initrd)) '()) ("locale" ,locale)))) ;used by libc@@ -664,6 +644,10 @@ bookkeeping." (host-name (host-name-service (operating-system-host-name os))) (entries (operating-system-directory-base-entries os))) (cons* (service system-service-type entries)+ (service linux-builder-service-type+ (linux-builder-configuration+ (kernel (operating-system-kernel os))+ (modules (operating-system-kernel-loadable-modules os)))) %boot-service
;; %SHEPHERD-ROOT-SERVICE must come last so that the gexp thatdiff --git a/gnu/tests/linux-modules.scm b/gnu/tests/linux-modules.scmindex 953b132ef7..30d8eae03b 100644--- a/gnu/tests/linux-modules.scm+++ b/gnu/tests/linux-modules.scm@@ -2,6 +2,7 @@ ;;; Copyright © 2019 Jakob L. Kreuze <zerodaysfordays@sdf.org> ;;; Copyright © 2020 Danny Milosavljevic <dannym@scratchpost.org> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -34,7 +35,10 @@ #:use-module (guix utils) #:export (%test-loadable-kernel-modules-0 %test-loadable-kernel-modules-1- %test-loadable-kernel-modules-2))+ %test-loadable-kernel-modules-2+ %test-loadable-kernel-modules-service-0+ %test-loadable-kernel-modules-service-1+ %test-loadable-kernel-modules-service-2))
;;; Commentary: ;;;@@ -66,17 +70,11 @@ that MODULES are actually loaded." (member module modules string=?)) '#$modules))))))
-(define* (run-loadable-kernel-modules-test module-packages module-names)- "Run a test of an OS having MODULE-PACKAGES, and verify that MODULE-NAMES-are loaded in memory."+(define* (run-loadable-kernel-modules-test-base base-os module-names)+ "Run a test of BASE-OS, verifying that MODULE-NAMES are loaded in memory." (define os (marionette-operating-system- (operating-system- (inherit (simple-operating-system))- (services (cons (service kernel-module-loader-service-type module-names)- (operating-system-user-services- (simple-operating-system))))- (kernel-loadable-modules module-packages))+ base-os #:imported-modules '((guix combinators)))) (define vm (virtual-machine os)) (define (test script)@@ -98,6 +96,36 @@ are loaded in memory." (gexp->derivation "loadable-kernel-modules" (test (modules-loaded?-program os module-names))))
+(define* (run-loadable-kernel-modules-test module-packages module-names)+ "Run a test of an OS having MODULE-PACKAGES, and verify that MODULE-NAMES+are loaded in memory."+ (run-loadable-kernel-modules-test-base+ (operating-system+ (inherit (simple-operating-system))+ (services (cons (service kernel-module-loader-service-type module-names)+ (operating-system-user-services+ (simple-operating-system))))+ (kernel-loadable-modules module-packages))+ module-names))++(define* (run-loadable-kernel-modules-service-test module-packages module-names)+ "Run a test of an OS having MODULE-PACKAGES, which are loaded by creating a+service that extends LINUXL-LOADABLE-MODULE-SERVICE-TYPE. Then verify that+MODULE-NAMES are loaded in memory."+ (define module-installing-service-type+ (service-type+ (name 'module-installing-service)+ (extensions (list (service-extension linux-loadable-module-service-type+ (const module-packages))))+ (default-value #f)))+ (run-loadable-kernel-modules-test-base+ (operating-system+ (inherit (simple-operating-system))+ (services (cons* (service module-installing-service-type)+ (operating-system-user-services+ (simple-operating-system)))))+ module-names))+ (define %test-loadable-kernel-modules-0 (system-test (name "loadable-kernel-modules-0")@@ -129,3 +157,35 @@ with two extra modules.") (package-arguments ddcci-driver-linux)))))) '("acpi_call" "ddcci")))))++(define %test-loadable-kernel-modules-service-0+ (system-test+ (name "loadable-kernel-modules-service-0")+ (description "Tests loadable kernel modules extensible service with no+extra modules.")+ (value (run-loadable-kernel-modules-service-test '() '()))))++(define %test-loadable-kernel-modules-service-1+ (system-test+ (name "loadable-kernel-modules-service-1")+ (description "Tests loadable kernel modules extensible service with one+extra module.")+ (value (run-loadable-kernel-modules-service-test+ (list ddcci-driver-linux)+ '("ddcci")))))++(define %test-loadable-kernel-modules-service-2+ (system-test+ (name "loadable-kernel-modules-service-2")+ (description "Tests loadable kernel modules extensible service with two+extra modules.")+ (value (run-loadable-kernel-modules-service-test+ (list acpi-call-linux-module+ (package+ (inherit ddcci-driver-linux)+ (arguments+ `(#:linux #f+ ,@(strip-keyword-arguments '(#:linux)+ (package-arguments+ ddcci-driver-linux))))))+ '("acpi_call" "ddcci")))))
--2.31.0
R
R
raid5atemyhomework wrote on 22 Mar 15:34 +0100
[PATCH v3 2/3] gnu: Add zfs-auto-snapshot.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
2c97vRukDGd4mnnPwPzLZOBq6qR5EqiBvaevyGcaoh1zc8NQkJO8y2DvtaMuafb6R9m5_Hhizx41nJjyggYPiHNXmnYKkQCejKsw_n1-wec=@protonmail.com
From 481dc87f77743b7a282777656e78615ea57d19cc Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Sun, 14 Mar 2021 16:40:47 +0800Subject: [PATCH 2/3] gnu: Add zfs-auto-snapshot.
* gnu/packages/file-systems.scm (zfs-auto-snapshot): New variable.--- gnu/packages/file-systems.scm | 75 +++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+)
Toggle diff (86 lines)diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scmindex 198653c639..6877f89f6e 100644--- a/gnu/packages/file-systems.scm+++ b/gnu/packages/file-systems.scm@@ -976,6 +976,81 @@ originally developed for Solaris and is now maintained by the OpenZFS community.") (license license:cddl1.0)))
+(define-public zfs-auto-snapshot+ (package+ (name "zfs-auto-snapshot")+ (version "1.2.4")+ (source+ (origin+ (method url-fetch)+ (uri (string-append+ "https://github.com/zfsonlinux/zfs-auto-snapshot/archive/upstream/"+ version ".tar.gz"))+ (sha256+ (base32 "16ry1w43i44xc67gr73x6fa48ninfhqxr498ad4m3kya93vp2zrh"))))+ (build-system gnu-build-system)+ (inputs+ ;; Note: if you are inheriting from the above zfs package in order+ ;; to provide a specific stable kernel version, you should also+ ;; inherit this package and replace the sole input below.+ `(("zfs" ,zfs)))+ (arguments+ `(#:tests? #f ; No tests+ #:phases+ (modify-phases %standard-phases+ (delete 'configure)+ (delete 'build)+ ;; Guix System may not have a traditional cron system, but+ ;; the cron scripts installed by this package are convenient+ ;; to use as targets for an mcron job specification, so make+ ;; sure they can be run in-store.+ (add-before 'install 'fix-scripts+ (lambda* (#:key outputs inputs #:allow-other-keys)+ (let* ((out (assoc-ref outputs "out"))+ (zfs-auto-snapshot (string-append+ out+ "/sbin/zfs-auto-snapshot"))+ (zfs-package (assoc-ref inputs "zfs"))+ (zpool (string-append+ zfs-package+ "/sbin/zpool"))+ (zfs (string-append+ zfs-package+ "/sbin/zfs")))+ (substitute* '("etc/zfs-auto-snapshot.cron.daily"+ "etc/zfs-auto-snapshot.cron.frequent"+ "etc/zfs-auto-snapshot.cron.hourly"+ "etc/zfs-auto-snapshot.cron.monthly"+ "etc/zfs-auto-snapshot.cron.weekly")+ (("zfs-auto-snapshot")+ zfs-auto-snapshot))+ (substitute* "src/zfs-auto-snapshot.sh"+ (("LC_ALL=C zfs list")+ (string-append "LC_ALL=C " zfs " list"))+ (("LC_ALL=C zpool status")+ (string-append "LC_ALL=C " zpool " status"))+ (("zfs snapshot")+ (string-append zfs " snapshot"))+ (("zfs destroy")+ (string-append zfs " destroy"))))))+ ;; Provide DESTDIR and PREFIX on make command.+ (replace 'install+ (lambda* (#:key outputs #:allow-other-keys)+ (let ((out (assoc-ref outputs "out")))+ (invoke "make" "install"+ "PREFIX="+ (string-append "DESTDIR=" out)))+ #t)))))+ (home-page "https://github.com/zfsonlinux/zfs-auto-snapshot")+ (synopsis "Automatically create, rotate, and destroy periodic ZFS snapshots")+ (description+ "An alternative implementation of the zfs-auto-snapshot service for Linux+that is compatible with zfs-linux (now OpenZFS) and zfs-fuse.++On Guix System, you will need to invoke the included shell scripts as @code{job}+definitions in your @code{operating-system} declaration.")+ (license license:gpl2+)))+ (define-public mergerfs (package (name "mergerfs")
--2.31.0
R
R
raid5atemyhomework wrote on 22 Mar 15:35 +0100
[PATCH v3 3/3] gnu: Add ZFS service type.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
ECVfxCKtS2pEf-XrbGUETYDOd1gNFzwvZc8s2AdO5zwbpHoFaK4XmW6QxYmDF1bhJsKB3_8sbnyNarpBm1v9gaqEAMbOkAK14OulJQpBoc0=@protonmail.com
From 7c039aee628bdb0188c0da27defb591f7791053e Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Mon, 22 Mar 2021 16:26:28 +0800Subject: [PATCH 3/3] gnu: Add ZFS service type.
* gnu/services/file-systems.scm: New file.* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.* gnu/services/base.scm: Export dependency->shepherd-service-name.* doc/guix.texi (ZFS File System): New subsection.--- doc/guix.texi | 351 ++++++++++++++++++++++++++++++++++ gnu/local.mk | 2 + gnu/services/base.scm | 4 +- gnu/services/file-systems.scm | 295 ++++++++++++++++++++++++++++ 4 files changed, 651 insertions(+), 1 deletion(-) create mode 100644 gnu/services/file-systems.scm
Toggle diff (705 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 86b22f3673..a2a092b31b 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -87,6 +87,7 @@ Copyright @copyright{} 2020 Daniel Brooks@* Copyright @copyright{} 2020 John Soo@* Copyright @copyright{} 2020 Jonathan Brielmaier@* Copyright @copyright{} 2020 Edgar Vincent@*+Copyright @copyright{} 2021 raid5atemyhomework@*
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or@@ -14095,6 +14096,356 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp
++@node ZFS File System+@subsection ZFS File System++Support for ZFS file systems is provided on Guix by the OpenZFS project.+OpenZFS currently only supports Linux-Libre and is not available on the+Hurd.++OpenZFS is free software; unfortunately its license is incompatible with+the GNU General Public License (GPL), the license of the Linux kernel,+which means they cannot be distributed together. However, as a user,+you can choose to build ZFS and use it together with Linux; you can+even rely on Guix to automate this task. See+@uref{https://www.fsf.org/licensing/zfs-and-linux, this analysis by+the Free Software Foundation} for more information.++As a large and complex kernel module, OpenZFS has to be compiled for a+specific version of Linux-Libre. At times, the latest OpenZFS package+available in Guix is not compatible with the latest Linux-Libre version.+Thus, directly installing the @code{zfs} package can fail.++Instead, you are recommended to select a specific older long-term-support+Linux-Libre kernel. Do not use @code{linux-libre-lts}, as even the+latest long-term-support kernel may be too new for @code{zfs}. Instead,+explicitly select a specific older version, such as @code{linux-libre-5.10},+and upgrade it manually later as new long-term-support kernels become+available that you have confirmed is compatible with the latest available+OpenZFS version on Guix.++For example, you can modify your system configuration file to a specific+Linux-Libre version and add the @code{zfs-service-type} service.++@lisp+(use-modules (gnu))+(use-package-modules+ #;@dots{}+ linux)+(use-service-modules+ #;@dots{}+ file-systems)++(define my-kernel linux-libre-5.10)++(operating-system+ (kernel my-kernel)+ #;@dots{}+ (services+ (cons* (service zfs-service-type+ (zfs-configuration+ (kernel my-kernel)))+ #;@dots{}+ %desktop-services))+ #;@dots{})+@end lisp++@defvr {Scheme Variable} zfs-service-type+This is the type for a service that adds ZFS support to your operating+system. The service is configured using a @code{zfs-configuration}+record.++Here is an example use:++@lisp+(service zfs-service-type+ (zfs-configuration+ (kernel linux-libre-5.4)))+@end lisp+@end defvr++@deftp {Data Type} zfs-configuration+This data type represents the configuration of the ZFS support in Guix+System. Its fields are:++@table @asis+@item @code{kernel}+The package of the Linux-Libre kernel to compile OpenZFS for. This field+is always required. It @emph{must} be the same kernel you use in your+@code{operating-system} form.++@item @code{base-zfs} (default: @code{zfs})+The OpenZFS package that will be compiled for the given Linux-Libre kernel.++@item @code{base-zfs-auto-snapshot} (default: @code{zfs-auto-snapshot})+The @code{zfs-auto-snapshot} package to use. It will be modified to+specifically use the OpenZFS compiled for your kernel.++@item @code{dependencies} (default: @code{'()})+A list of @code{<file-system>} or @code{<mapped-device>} records that must+be mounted or opened before OpenZFS scans for pools to import. For example,+if you have set up LUKS containers as leaf VDEVs in a pool, you have to+include their corresponding @code{<mapped-ddevice>} records so that OpenZFS+can import the pool correctly at bootup.++@item @code{auto-mount?} (default: @code{#t})+Whether to mount datasets with the ZFS @code{mountpoint} property automatically+at startup. This is the behavior that ZFS users usually expect. You might+set this to @code{#f} for an operating system intended as a ``rescue'' system+that is intended to help debug problems with the disks rather than actually+work in production.++@item @code{auto-scrub} (default: @code{'weekly})+Specifies how often to scrub all pools. Can be the symbols @code{'weekly} or+@code{'monthly}, or a schedule specification understood by+@xref{mcron, mcron job specifications,, mcron, GNU@tie{}mcron}, such as+@code{"0 3 * * 6"} for ``every 3AM on Saturday''.+It can also be @code{#f} to disable auto-scrubbing (@strong{not recommended}).++The general guideline is to scrub weekly when using consumer-quality drives, and+to scrub monthly when using enterprise-quality drives.++@code{'weekly} scrubs are done on Sunday midnight, while @code{monthly} scrubs+are done on midnight on the first day of each month.++@item @code{auto-snapshot?} (default: @code{#t})+Specifies whether to auto-snapshot by default. If @code{#t}, then snapshots+are automatically created except for ZFS datasets with the+@code{com.sun:auto-snapshot} ZFS vendor property set to @code{false}.++If @code{#f}, snapshots will not be automatically created, unless the ZFS+dataset has the @code{com.sun:auto-snapshot} ZFS vendor property set to+@code{true}.++@item @code{auto-snapshot-keep} (default: @code{'()})+Specifies an association list of symbol-number pairs, indicating the number+of automatically-created snapshots to retain for each frequency type.++If not specified via this field, by default there are 4 @code{frequent}, 24+@code{hourly}, 31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly} snapshots.++For example:++@lisp+(zfs-configuration+ (kernel my-kernel)+ (auto-snapshot-keep+ '((frequent . 8)+ (hourly . 12))))+@end lisp++The above will keep 8 @code{frequent} snapshots and 12 @code{hourly} snapshots.+@code{daily}, @code{weekly}, and @code{monthly} snapshots will keep their+defaults (31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly}).++@end table+@end deftp++@subsubsection ZFS Auto-Snapshot++The ZFS service on Guix System supports auto-snapshots as implemented in the+Solaris operating system.++@code{frequent} (every 15 minutes), @code{hourly}, @code{daily}, @code{weekly},+and @code{monthly} snapshots are created automatically for ZFS datasets that+have auto-snapshot enabled. They will be named, for example,+@code{zfs-auto-snap_frequent-2021-03-22-1415}. You can continue to use+manually-created snapshots as long as they do not conflict with the naming+convention used by auto-snapshot. You can also safely manually destroy+automatically-created snapshots, for example to free up space.++The @code{com.sun:auto-snapshot} ZFS property controls auto-snapshot on a+per-dataset level. Sub-datasets will inherit this property from their parent+dataset, but can have their own property.++You @emph{must} set this property to @code{true} or @code{false} exactly,+otherwise it will be treated as if the property is unset.++For example:++@example+# zfs list -o name+NAME+tank+tank/important-data+tank/tmp+# zfs set com.sun:auto-snapshot=true tank+# zfs set com.sun:auto-snapshot=false tank/tmp+@end example++The above will set @code{tank} and @code{tank/important-data} to be+auto-snapshot, while @code{tank/tmp} will not be auto-snapshot.++If the @code{com.sun:auto-snapshot} property is not set for a dataset+(the default when pools and datasets are created), then whether+auto-snapshot is done or not will depend on the @code{auto-snapshot?}+field of the @code{zfs-configuration} record.++There are also @code{com.sun:auto-snapshot:frequent},+@code{com.sun:auto-snapshot:hourly}, @code{com.sun:auto-snapshot:daily},+@code{com.sun:auto-snapshot:weekly}, and @code{com.sun:auto-snapshot:monthly}+properties that give finer-grained control of whether to auto-snapshot a+dataset at a particular schedule.++The number of snapshots kept for all datasets can be overridden via the+@code{auto-snapshot-keep} field of the @code{zfs-configuration} record.+There is currently no support to have different numbers of snapshots to+keep for different datasets.++@subsubsection ZVOLs++ZFS supports ZVOLs, block devices that ZFS exposes to the operating+system in the @code{/dev/zvol/} directory. The ZVOL will have the same+resilience and self-healing properties as other datasets on your ZFS pool.+ZVOLs can also be snapshotted (and will be included in auto-snapshotting+if enabled), which snapshots the state of the block device, effectively+snapshotting the hosted file system.++You can put any file system inside the ZVOL. However, in order to mount this+file system at system start, you need to add @code{%zfs-zvol-dependency} as a+dependency of each file system inside a ZVOL.++@defvr {Scheme Variable} %zfs-zvol-dependency+An artificial @code{<mapped-device>} which tells the file system mounting+service to wait for ZFS to provide ZVOLs before mounting the+@code{<file-system>} dependent on it.+@end defvr++For example, suppose you create a ZVOL and put an ext4 filesystem+inside it:++@example+# zfs create -V 100G tank/ext4-on-zfs+# mkfs.ext4 /dev/zvol/tank/ext4-on-zfs+# mkdir /ext4-on-zfs+# mount /dev/zvol/tank/ext4-on-zfs /ext4-on-zfs+@end example++You can then set this up to be mounted at boot by adding this to the+@code{file-systems} field of your @code{operating-system} record:++@lisp+(file-system+ (device "/dev/zvol/tank/ext4-on-zfs")+ (mount-point "/ext4-on-zfs")+ (type "ext4")+ (dependencies (list %zfs-zvol-dependency)))+@end lisp++You @emph{must not} add @code{%zfs-zvol-dependency} to your+@code{operating-system}'s @code{mapped-devices} field, and you @emph{must+not} add it (or any @code{<file-system>}s dependent on it) to the+@code{dependencies} field of @code{zfs-configuration}. Finally, you+@emph{must not} use @code{%zfs-zvol-dependency} unless you actually+instantiate @code{zfs-service-type} on your system.++@subsubsection Unsupported Features++Some common features and uses of ZFS are currently not supported, or not+fully supported, on Guix.++@enumerate+@item+Shepherd-managed daemons that are configured to read from or write to ZFS+mountpoints need to include @code{user-processes} in their @code{requirement}+field. This is the earliest that ZFS file systems are assured of being+mounted.++Generally, most daemons will, directly or indirectly, require+@code{networking}, or @code{user-processes}, or both. Most implementations+of @code{networking} will also require @code{user-processes} so daemons that+require only @code{networking} will also generally start up after+@code{user-processes}. A notable exception, however, is+@code{static-networking-service-type}. You will need to explicitly add+@code{user-processes} as a @code{requirement} of your @code{static-networking}+record.++@item+@code{mountpoint=legacy} ZFS file systems. The handlers for the Guix mounting+system have not yet been modified to support ZFS, and will expect @code{/dev}+paths in the @code{<file-system>}'s @code{device} field, but ZFS file systems+are referred to via non-path @code{pool/file/system} names. Such file systems+also need to be mounted @emph{after} OpenZFS has scanned for pools.++You can still manually mount these file systems after system boot; what is+only unsupported is mounting them automatically at system boot by specifying+them in @code{<file-system>} records of your @code{operating-system}.++@item+@code{/home} on ZFS. Guix will create home directories for users, but this+process currently cannot be scheduled after ZFS file systems are mounted.+Thus, the ZFS file system might be mounted @emph{after} Guix has created+home directories at boot, at which point OpenZFS will refuse to mount since+the mountpoint is not empty. However, you @emph{can} create an ext4, xfs,+btrfs, or other supported file system inside a ZVOL, have that depend on+@code{%zfs-zvol-dependency}, and set it to mount on the @code{/home}+directory; they will be scheduled to mount before the @code{user-homes}+process.++Similarly, other locations like @code{/var}, @code{/gnu/store} and so+on cannot be reliably put in a ZFS file system, though they may be+possible to create as other file systems inside ZVOL containers.++@item+@code{/} and @code{/boot} on ZFS. These require Guix to expose more of+the @code{initrd} very early boot process to services. It also requires+Guix to have the ability to explicitly load modules while still in+@code{initrd} (currently kernel modules loaded by+@code{kernel-module-loader-service-type} are loaded after @code{/} is+mounted). Further, since one of ZFS's main advantages is that it can+continue working despite the loss of one or more devices, it makes sense+to also support installing the bootloader on all devices of the pool that+contains the @code{/} and @code{/boot}; after all, if ZFS can survive the+loss of one device, the bootloader should also be able to survive the loss+of one device.++@item+ZVOL swap devices. Mapped swap devices need to be listed in+@code{mapped-devices} to ensure they are opened before the system attempts+to use them, but you cannot currently add @code{%zfs-zvol-dependency} to+@code{mapped-devices}.++This will also require significant amounts of testing, as various kernel+build options and patches may affect how swapping works, which are possibly+different on Guix System compared to other distributions that this feature is+known to work on.++@item+ZFS Event Daemon. Support for this has not been written yet, patches are+welcome. The main issue is how to design this in a Guix style while+supporting legacy shell-script styles as well. In particular, OpenZFS itself+comes with a number of shell scripts intended for ZFS Event Daemon, and we+need to figure out how the user can choose to use or not use the provided+scripts (and configure any settings they have) or override with their own+custom code (which could be shell scripts they have written and trusted from+previous ZFS installations).++As-is, you can create your own service that activates the ZFS Event Daemon+by creating the @file{/etc/zfs/zed} directory and filling it appropriately,+then launching @code{zed}.++@item+@file{/etc/zfs/zpool.cache}. Currently the ZFS support on Guix always forces+scanning of all devices at bootup to look for ZFS pools. For systems with+dozens or hundreds of storage devices, this can lead to slow bootup. One issue+is that tools should really not write to @code{/etc} which is supposed to be for+configuration; possibly it could be moved to @code{/var} instead. Another issue+is that if Guix ever supports @code{/} on ZFS, we would need to somehow keep the+@code{zpool.cache} file inside the @code{initrd} up-to-date with what is in the+@code{/} mount point.++@item+@code{zfs share}. This will require some (unknown amount of) work to integrate+into the Samba and NFS services of Guix. You @emph{can} manually set up Samba+and NFS to share any mounted ZFS datasets by setting up their configurations+properly; it just can't be done for you by @code{zfs share} and the+@code{sharesmb} and @code{sharenfs} properties.+@end enumerate++Hopefully, support for the above only requires code to be written, so users+are encouraged to hack on Guix to implement the above features.+ @node Mapped Devices @section Mapped Devices
diff --git a/gnu/local.mk b/gnu/local.mkindex 3d4147a879..6fd8284a9b 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -40,6 +40,7 @@ # Copyright © 2020 Malte Frank Gerdes <mate.f.gerdes@gmail.com> # Copyright © 2020 Vinicius Monego <monego@posteo.net> # Copyright © 2021 Björn Höfling <bjoern.hoefling@bjoernhoefling.de>+# Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> # # This file is part of GNU Guix. #@@ -607,6 +608,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/docker.scm \ %D%/services/authentication.scm \ %D%/services/file-sharing.scm \+ %D%/services/file-systems.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \ %D%/services/getmail.scm \diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex f50bcfdcb4..09a2d7c45b 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -187,7 +187,9 @@
references-file
- %base-services))+ %base-services++ dependency->shepherd-service-name))
;;; Commentary: ;;;diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scmnew file mode 100644index 0000000000..0b1aae38ac--- /dev/null+++ b/gnu/services/file-systems.scm@@ -0,0 +1,295 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu services file-systems)+ #:use-module (gnu packages file-systems)+ #:use-module (gnu services)+ #:use-module (gnu services base)+ #:use-module (gnu services linux)+ #:use-module (gnu services mcron)+ #:use-module (gnu services shepherd)+ #:use-module (gnu system mapped-devices)+ #:use-module (guix gexp)+ #:use-module (guix packages)+ #:use-module (guix records)+ #:export (zfs-service-type++ zfs-configuration+ zfs-configuration?+ zfs-configuration-kernel+ zfs-configuration-base-zfs+ zfs-configuration-base-zfs-auto-snapshot+ zfs-configuration-dependencies+ zfs-configuration-auto-mount?+ zfs-configuration-auto-scrub+ zfs-configuration-auto-snapshot?+ zfs-configuration-auto-snapshot-keep++ %zfs-zvol-dependency))++(define-record-type* <zfs-configuration>+ zfs-configuration+ make-zfs-configuration+ zfs-configuration?++ ; linux-libre kernel you want to compile the base-zfs module for.+ (kernel zfs-configuration-kernel)++ ; the OpenZFS package that will be modified to compile for the+ ; given kernel.+ (base-zfs zfs-configuration-base-zfs+ (default zfs))++ ; the zfs-auto-snapshot package that will be modified to compile+ ; for the given kernel.+ (base-zfs-auto-snapshot zfs-configuration-base-zfs-auto-snapshot+ (default zfs-auto-snapshot))++ ; list of <mapped-device> or <file-system> objects that must be+ ; opened/mounted before we import any ZFS pools.+ (dependencies zfs-configuration-dependencies+ (default '()))++ ; #t if mountable datasets are to be mounted automatically.+ ; #f if not mounting.+ ; #t is the expected behavior on other operating systems, the+ ; #f is only supported for "rescue" operating systems where+ ; the user wants lower-level control of when to mount.+ (auto-mount? zfs-configuration-auto-mount?+ (default #t))++ ; 'weekly for weekly scrubbing, 'monthly for monthly scrubbing, an+ ; mcron time specification that can be given to `job`, or #f to+ ; disable.+ (auto-scrub zfs-configuration-auto-scrub+ (default 'weekly))++ ; #t if auto-snapshot is default (and `com.sun:auto-snapshot=false`+ ; disables auto-snapshot per dataset), #f if no auto-snapshotting+ ; is default (and `com.sun:auto-snapshot=true` enables auto-snapshot+ ; per dataset).+ (auto-snapshot? zfs-configuration-auto-snapshot?+ (default #t))++ ; association list of symbol-number pairs to indicate the number+ ; of automatic snapshots to keep for each of 'frequent, 'hourly,+ ; 'daily, 'weekly, and 'monthly.+ ; e.g. '((frequent . 8) (hourly . 12))+ (auto-snapshot-keep zfs-configuration-auto-snapshot-keep+ (default '())))++(define %default-auto-snapshot-keep+ '((frequent . 4)+ (hourly . 24)+ (daily . 31)+ (weekly . 8)+ (monthly . 12)))++(define %auto-snapshot-mcron-schedule+ '((frequent . "0,15,30,45 * * * *")+ (hourly . "0 * * * *")+ (daily . "0 0 * * *")+ (weekly . "0 0 * * 7")+ (monthly . "0 0 1 * *")))++;; A synthetic and unusable MAPPED-DEVICE intended for use when+;; the user has created a mountable filesystem inside a ZFS+;; zvol and wants it mounted inside the configuration.scm.+(define %zfs-zvol-dependency+ (mapped-device+ (source '())+ (targets '("zvol/*"))+ (type #f)))++(define (make-zfs-package conf)+ (let ((kernel (zfs-configuration-kernel conf))+ (base-zfs (zfs-configuration-base-zfs conf)))+ (package+ (inherit base-zfs)+ (arguments (cons* #:linux kernel+ (package-arguments base-zfs))))))++(define (make-zfs-auto-snapshot-package conf)+ (let ((zfs (make-zfs-package conf))+ (base-zfs-auto-snapshot (zfs-configuration-base-zfs-auto-snapshot conf)))+ (package+ (inherit base-zfs-auto-snapshot)+ (inputs `(("zfs" ,zfs))))))++(define (zfs-loadable-modules conf)+ (list (list (make-zfs-package conf) "module")))++(define (zfs-shepherd-services conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (zfs (file-append zfs-package "/sbin/zfs"))+ (zvol_wait (file-append zfs-package "/bin/zvol_wait"))+ (scheme-modules `((srfi srfi-1)+ (srfi srfi-34)+ (srfi srfi-35)+ (rnrs io ports)+ ,@%default-modules)))+ (define zfs-scan+ (shepherd-service+ (provision '(zfs-scan))+ (requirement `(root-file-system+ kernel-module-loader+ udev+ ,@(map dependency->shepherd-service-name+ (zfs-configuration-dependencies conf))))+ (documentation "Scans for and imports ZFS pools.")+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "zfs: error importing pools: ~s~%"+ (condition-message c))+ #f))+ ; TODO: optionally use a cachefile.+ (invoke #$zpool "import" "-a" "-N"))))+ ;; Why not one-shot? Because we don't really want to rescan+ ;; this each time a requiring process is restarted, as scanning+ ;; can take a long time and a lot of I/O.+ (stop #~(const #f))))++ (define device-mapping-zvol/*+ (shepherd-service+ (provision '(device-mapping-zvol/*))+ (requirement '(zfs-scan))+ (documentation "Waits for all ZFS ZVOLs to be opened.")+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "zfs: error opening zvols: ~s~%"+ (condition-message c))+ #f))+ (invoke #$zvol_wait))))+ (stop #~(const #f))))++ (define zfs-auto-mount+ (shepherd-service+ (provision '(zfs-auto-mount))+ (requirement '(zfs-scan))+ (documentation "Mounts all non-legacy mounted ZFS filesystems.")+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "zfs: error mounting file systems: ~s~%"+ (condition-message c))+ #f))+ ;; Output to current-error-port, otherwise the+ ;; user will not see any prompts for passwords+ ;; of encrypted datasets.+ ;; XXX Maybe better to explicitly open /dev/console ?+ (with-output-to-port (current-error-port)+ (lambda ()+ (invoke #$zfs "mount" "-a" "-l"))))))+ (stop #~(lambda _+ ;; Make sure that Shepherd does not have a CWD that+ ;; is a mounted ZFS filesystem, which would prevent+ ;; unmounting.+ (chdir "/")+ (invoke #$zfs "unmount" "-a" "-f")))))++ `(,zfs-scan+ ,device-mapping-zvol/*+ ,@(if (zfs-configuration-auto-mount? conf)+ `(,zfs-auto-mount)+ '()))))++(define (zfs-user-processes conf)+ (if (zfs-configuration-auto-mount? conf)+ '(zfs-auto-mount)+ '(zfs-scan)))++(define (zfs-mcron-auto-snapshot-jobs conf)+ (let* ((user-auto-snapshot-keep (zfs-configuration-auto-snapshot-keep conf))+ ;; assoc-ref has earlier entries overriding later ones.+ (auto-snapshot-keep (append user-auto-snapshot-keep+ %default-auto-snapshot-keep))+ (auto-snapshot? (zfs-configuration-auto-snapshot? conf))+ (zfs-auto-snapshot-package (make-zfs-auto-snapshot-package conf))+ (zfs-auto-snapshot (file-append zfs-auto-snapshot-package+ "/sbin/zfs-auto-snapshot")))+ (map+ (lambda (label)+ (let ((keep (assoc-ref auto-snapshot-keep label))+ (sched (assoc-ref %auto-snapshot-mcron-schedule label)))+ #~(job '#$sched+ (string-append #$zfs-auto-snapshot+ " --quiet --syslog "+ " --label=" #$(symbol->string label)+ " --keep=" #$(number->string keep)+ " //"))))+ '(frequent hourly daily weekly monthly))))++(define (zfs-mcron-auto-scrub-jobs conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (auto-scrub (zfs-configuration-auto-scrub conf))+ (sched (cond+ ((eq? auto-scrub 'weekly) "0 0 * * 7")+ ((eq? auto-scrub 'monthly) "0 0 1 * *")+ (else auto-scrub))))+ (list+ #~(job '#$sched+ ;; Suppress errors: if there are no ZFS pools, the+ ;; scrub will not be given any arguments, which makes+ ;; it error out.+ (string-append "(" #$zpool " scrub `" #$zpool " list -o name -H` "+ "> /dev/null 2>&1) "+ "|| exit 0")))))++(define (zfs-mcron-jobs conf)+ (append (zfs-mcron-auto-snapshot-jobs conf)+ (if (zfs-configuration-auto-scrub conf)+ (zfs-mcron-auto-scrub-jobs conf)+ '())))++(define zfs-service-type+ (service-type+ (name 'zfs)+ (extensions+ (list ;; Install OpenZFS kernel module into kernel profile.+ (service-extension linux-loadable-module-service-type+ zfs-loadable-modules)+ ;; And load it.+ (service-extension kernel-module-loader-service-type+ (const '("zfs")))+ ;; Make sure ZFS pools and datasets are mounted at+ ;; boot.+ (service-extension shepherd-root-service-type+ zfs-shepherd-services)+ ;; Make sure user-processes don't start until+ ;; after ZFS does.+ (service-extension user-processes-service-type+ zfs-user-processes)+ ;; Install automated scrubbing and snapshotting.+ (service-extension mcron-service-type+ zfs-mcron-jobs)++ ;; Install ZFS management commands in the system+ ;; profile.+ (service-extension profile-service-type+ (compose list make-zfs-package))+ ;; Install ZFS udev rules.+ (service-extension udev-service-type+ (compose list make-zfs-package))))+ (description "Installs ZFS, an advanced filesystem and volume manager.")))
--2.31.0
L
L
Léo Le Bouter wrote on 28 Mar 14:55 +0200
Re: [bug#45692] [PATCH v3 0/3] New patch series for Even Better ZFS Support on Guix
ffa79fc37bb8f8f4152a22497060e2ebd6f9c19a.camel@zaclys.net
Thanks a lot for the patches!
On Mon, 2021-03-22 at 14:33 +0000, raid5atemyhomework via Guix-patchesvia wrote:
Toggle quote (8 lines)> Personally I'm not particularly motivated to push this anymore. I> have a frankenstein's `configuration.scm` that gets me the bits of> ZFS functionality that I absolutely need, with the rest something I'm> just living without, and while it would be *nice* to have *some* out-> of-the-box support for ZFS (or at least make it easy to get a box to> get support for ZFS out of), I personally don't need it at this> point.
I understand your frustration but also understand that the people whoare able to review your patches can't make themselves available in theway you desire, there's lots of pending patches to GNU Guix and as muchas ZFS support is valuable, there's also lots of other things that arevaluable and are in the backlog.
Toggle quote (5 lines)> Patch 1/3 corresponds to the original Patch 1/4. Patch 2/3 has no> corresponding patch in the original patch set, but is the same patch> as https://issues.guix.gnu.org/47134 which nobody is looking at, as> usual. Patch 3/3 corresponds to the original Patch 4/4.
Please do not blame or shame us for not looking at patches, it's notvery nice, we do our best, at all times. We are overworked and thepeople who are knowledgeable about GNU Guix and Scheme particularilyso. I don't feel like I am knowledgeable about Scheme and GNU Guixenough to review your patches, sorry for that.
Léo
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEFIvLi9gL+xax3g6RRaix6GvNEKYFAmBgfKcACgkQRaix6GvNEKYfYw/+N51EZdVvf6GLz3EyT1f6fTeAB+XoBj8iCKFLXdNPe1dop67AW2djKQDR4pIn1jN+gFqcP9P4NsyNLCT7KWyrYJ7AHa9vYKa0AZ5WEoNlmfsX+debqbLB3MlSR3oXPLoQcTqvDoR+s07g6hX5HcG42CQIlMVyHA3NzboCEC/RIzeWsJDZb50ctVYJ44yDgRQ1Ejhwx50T0bIk4OZR+/tpEgrqjPU+rYobcCi+YFDkqtkvFsiau7gVoNZ3uHJb8T9Yw21vUPtgQsP+T/2EUA8sUiVyeykXh2KKEcV/Z/C5BjYGTEpWZ2e7LM2vp1Vn/qxQVPUYNE/x+l7u7XH/cF2CyQw1o2dZdu/ZwosLcNxbzt6R9g1MSeXS9IKaLON8F1oTKKBVjAfhqvCov6uCuEHFfVWi7EFhGe6a2JIljApLK+0KXoFDTa5/4HKPERF0/MH9keTIKiWH1l35r4UIIyccU67gkxqCFNTgr6BlZW33qyHUhHqQljbUB5qribUdoF/R6/Whxk+aGJ5bnw32BxwKdT9oty7nq1rrVXC+zf5KN4p4A1Sb5aTY2EU09qL3OvF3Wd+u8Y+/7cbBwAA06HhJ1EQYrn5szKUp4hx909z0OuoUEkXUA81HE6FIU8jDvw3ZDNPfc9VW3+y8OoDSijd0A5IYuX1UcHyZPsgZXxQbdEA==qHhm-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 29 Mar 06:39 +0200
(name . Léo Le Bouter)(address . lle-bout@zaclys.net)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
6_CTNY0LKnK4gdEM41_839GBi_6dql-yEVzGPVMQDFvi6JNkQF1BoxQu9_6gRx6bTNqk6rSpUhxDtDWveZOBnxYdMMW9h87WPP8uWZPteZk=@protonmail.com
Toggle quote (28 lines)> On Mon, 2021-03-22 at 14:33 +0000, raid5atemyhomework via Guix-patches> via wrote:>> > Personally I'm not particularly motivated to push this anymore. I> > have a frankenstein's `configuration.scm` that gets me the bits of> > ZFS functionality that I absolutely need, with the rest something I'm> > just living without, and while it would be nice to have some out-> > of-the-box support for ZFS (or at least make it easy to get a box to> > get support for ZFS out of), I personally don't need it at this> > point.>> I understand your frustration but also understand that the people who> are able to review your patches can't make themselves available in the> way you desire, there's lots of pending patches to GNU Guix and as much> as ZFS support is valuable, there's also lots of other things that are> valuable and are in the backlog.>> > Patch 1/3 corresponds to the original Patch 1/4. Patch 2/3 has no> > corresponding patch in the original patch set, but is the same patch> > as https://issues.guix.gnu.org/47134 which nobody is looking at, as> > usual. Patch 3/3 corresponds to the original Patch 4/4.>> Please do not blame or shame us for not looking at patches, it's not> very nice, we do our best, at all times. We are overworked and the> people who are knowledgeable about GNU Guix and Scheme particularily> so. I don't feel like I am knowledgeable about Scheme and GNU Guix> enough to review your patches, sorry for that.
Okay.
Thanksraid5atemyhomework
D
D
Danny Milosavljevic wrote on 11 May 16:05 +0200
Re: [bug#45692] [PATCH v3 2/3] gnu: Add zfs-auto-snapshot.
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
20210511160434.6b88bba3@scratchpost.org
Hi,
Toggle quote (4 lines)> + (invoke "make" "install"> + "PREFIX="> + (string-append "DESTDIR=" out)))
Are you sure about that?
Usually, DESTDIR is in order to supply a temporary build root (which will notbe referred-to by the installed programs--because that directory will bedeleted after the build) and PREFIX is in order to supply an actualinstallation directory (installed programs can refer to that whenever theywant).
Is it different for this program?
Also, I'd suggest to use #:make-flags so other phases can also see the PREFIXchosen. Otherwise, those could erroneously default to another prefix (forloading of config files at runtime etc).
What do you think?
D
D
Danny Milosavljevic wrote on 11 May 16:17 +0200
Re: [bug#45692] [PATCH v3 1/3] gnu: Allow services to install kernel-loadable modules.
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
20210511161700.30208638@scratchpost.org
Pushed this patch to guix master as commit a3df382525ac515d1aa083f7f5bd3bd31eb1df40.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAmCakdwACgkQ5xo1VCwwuqW1Xwf+LDsPUcMzmaFmYOgNRFjOC5mfd/2cJnUDJEMBsedSkSd+VL0CVe0iI/G0ylTMzAIkNREGb9E6C79AR+yh5SxfwhgjOzR6J+Nq9WKt9ZBvP4ujPS8nkxOHdwrAegXmhxdty7Qu6GD1AGMG1+wzcBe7U1oH5CTlA1GvjbQTJ29uHIFgBNlLtHisWqZcbLN2C4yjv52Qjsd9uenhFHXcsvlWT/yp5BUdeJ4cJ1PBCNSI9+PADABAIQc39epDEf2/uhb0+XZKpZPmk3Z6F9EgjAEWMMvbiqxX+CT5/cb9ZVUMh9eCgo13GLFG6j+gxMRZMN3oFQn89i2eMjB2jnXbwF91jw===OiQJ-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 13 May 03:21 +0200
Re: [bug#45692] [PATCH v3 2/3] gnu: Add zfs-auto-snapshot.
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
WQzldY7UPtjV-mvfe9NX5iz4_vihtDIPFCKMwHPRr40rLtQBPzsfO0OEyqnicuhwdxvztREkhl4riP07QT2uX6PScQNTFq5fxZ9a93tREKM=@protonmail.com
Hello Danny,
Toggle quote (28 lines)> Hi,>> > - (invoke "make" "install"> >> >> > - "PREFIX="> >> >> > - (string-append "DESTDIR=" out)))> >> >>> Are you sure about that?>> Usually, DESTDIR is in order to supply a temporary build root (which will not> be referred-to by the installed programs--because that directory will be> deleted after the build) and PREFIX is in order to supply an actual> installation directory (installed programs can refer to that whenever they> want).>> Is it different for this program?>> Also, I'd suggest to use #:make-flags so other phases can also see the PREFIX> chosen. Otherwise, those could erroneously default to another prefix (for> loading of config files at runtime etc).>> What do you think?
Yes, I am quite sure; the program uses a custom `Makefile` that is fairly simple and can be trivially quoted here for your review:
```MakefilePREFIX := /usr/local
all:
install: install -d $(DESTDIR)/etc/cron.d install -d $(DESTDIR)/etc/cron.daily install -d $(DESTDIR)/etc/cron.hourly install -d $(DESTDIR)/etc/cron.weekly install -d $(DESTDIR)/etc/cron.monthly install -m 0644 etc/zfs-auto-snapshot.cron.frequent $(DESTDIR)/etc/cron.d/zfs-auto-snapshot install etc/zfs-auto-snapshot.cron.hourly $(DESTDIR)/etc/cron.hourly/zfs-auto-snapshot install etc/zfs-auto-snapshot.cron.daily $(DESTDIR)/etc/cron.daily/zfs-auto-snapshot install etc/zfs-auto-snapshot.cron.weekly $(DESTDIR)/etc/cron.weekly/zfs-auto-snapshot install etc/zfs-auto-snapshot.cron.monthly $(DESTDIR)/etc/cron.monthly/zfs-auto-snapshot install -d $(DESTDIR)$(PREFIX)/share/man/man8 install -m 0644 src/zfs-auto-snapshot.8 $(DESTDIR)$(PREFIX)/share/man/man8/zfs-auto-snapshot.8 install -d $(DESTDIR)$(PREFIX)/sbin install src/zfs-auto-snapshot.sh $(DESTDIR)$(PREFIX)/sbin/zfs-auto-snapshot
uninstall: rm $(DESTDIR)/etc/cron.d/zfs-auto-snapshot rm $(DESTDIR)/etc/cron.hourly/zfs-auto-snapshot rm $(DESTDIR)/etc/cron.daily/zfs-auto-snapshot rm $(DESTDIR)/etc/cron.weekly/zfs-auto-snapshot rm $(DESTDIR)/etc/cron.monthly/zfs-auto-snapshot rm $(DESTDIR)$(PREFIX)/share/man/man8/zfs-auto-snapshot.8 rm $(DESTDIR)$(PREFIX)/sbin/zfs-auto-snapshot```
Notice how it does not modify any of the source files at all, so the sources do not refer to anything provided in `$PREFIX`. However, it does create directories in `$DESTDIR`, so it has to use `$DESTDIR`.
Thanksraid5atemyhomework
D
D
Danny Milosavljevic wrote on 13 May 15:08 +0200
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
20210513150803.27df82a2@scratchpost.org
Hi,
thanks for the explanation!
Pushed zfs-auto-snapshot to guix master as commit ba3b295a3ee956ac7500b5f9bb1d151b28ab30ed.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAmCdJLMACgkQ5xo1VCwwuqV55AgAm7RU9/STq0MECt9hjSFZSYJY/k1A1+f0IaP9eWE6SPW8/C06opazXK/cxRnA+WFXadKnkgL+9C9heJu6S8BXJbpd6qWoKX5bUpVhEyTC4wAPCjuoWtTtxFPfnrdby0RnQSvWEHYafo5bHscD+deZV8sT6XniK+t5lgOSWzrVpIjfXX4rI3p/I7jYjoshzV4flFzcGZG6W5Vs759mQMmTRJAk3P5/czlnn4xMNbQxF8prq12rIs/81pQdb2s7zoD/xNqU59fG6yBApsp0BrcIm1tRSfYXx4je14V35URcv9XfuCvC6iahcVqgdP85SvmLV2fDGlWN8a0ev42/eLQIuw===INaF-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 23 Jul 17:11 +0200
Re: [bug#45692] [PATCH v3 0/3] New patch series for Even Better ZFS Support on Guix
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
I2RNgCmKJ864TRprR4B41C6P5TGTfPu--zIzvoKCVSoeD6AkXNVVhsDJP-Ck8mWCvd6tK1cvCQRyN13xDqFWUR-239BzqvNqErJb8G5okdI=@protonmail.com
Can I get any particular feedback here at this point, from anyone at all?
R
R
raid5atemyhomework wrote on 25 Jul 16:03 +0200
Re: [PATCH v3 3/3] gnu: Add ZFS service type.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
uiFKaz-Xf0_3ZvrgI6DeH7rHSFnWiJ6bw9L3qzudFIrMH297q-OTPNRYQT26c8l14uylsg3pWJiW4gFmtGSQ_bEbLNE53NR35QZoDHaF6CE=@protonmail.com
Because it has been so long, the latest patch has bitrotted.
I really think at this point shaming people into reviewing this is my only option.
Give me a few hours or days to get a new patch that applies cleanly to origin/master.
Thanksraid5atemyhomewkr
R
R
raid5atemyhomework wrote on 25 Jul 16:31 +0200
Re: [PATCH v4 3/3] gnu: Add ZFS service type.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
XEBTiombw9FV1thMVWHGYkA24tfj5sFr49f5kAQw7MoEai8i-TGy9bVYW6WGjPSmC8vd6BCX4MqzR2lPsEp2-PXFvE2d4PPrvkYyYIbX8RY=@protonmail.com
Hello nonexistent reviewer,
I updated this patch to latest `origin/master` because the previous version has bitrotted and will not `git am` cleanly anymore.
There are no changes relative to v3, just rebased it so that the patch applies cleanly.
Testing has been very minimal: I created a VM with the added service, then ran it in a VM session that included additional devices (qcow2 files) from a previous VM run that formatted those as devices of a ZFS pool, and confirmed that the new VM could read it and manage that pool.

Is there any chance this will get reviewed or should I just not bother and move on with my life and forget about Guix?

At this point as well, I would like to point out what I think is a failing of how the Guix distribution is managed.
Guix does not assign any particular committers to any particular tasks or areas.The intent is that any committer can review and commit any particular patch.
However, this fails as the Guix project has grown.
No single committer wants to review ***all*** the available patches, and this is understandable, as the Guix project has grown significantly and includes a wide variety of people with diverging interests and as an open-source project you cannot really require that particular people look at particular things.Unfortunately, I do not know *who* committers are, and more importantly, *which* committer might be interested in this ZFS service type.Because "any committer can review and commit any patch!!" there is no particular list or table I can refer to, to figure out who might be useful to ping for this patchset.
At the same time, because no committer is interested in *all* patches I cannot just ping some particular person and expect to somehow get on some list somewhere that tells me "you will be the 48486th patch that will be reviewed by <foo> who is interested in all patches".

It is very discouraging to work on this code for a few weeks, release it, not get any reviews, and end up in a situation where I have to make annoying small changes just to keep the patch from bitrotting.
I understand that there are few possible reviewers, but if potential new contributors get discouraged from contributing because they do not see their code actually getting in, then you really cannot expect the number of reviewers to increase, either.
I think it would be nice if I could at least be told some number of people who *might* be interested in this patch, or just throw the towel and not bother.

Thanksraid5atemyhomework

From 5351aa7c1c14d4fea032adad895c436e02d1f261 Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Mon, 22 Mar 2021 16:26:28 +0800Subject: [PATCH] gnu: Add ZFS service type.
* gnu/services/file-systems.scm: New file.* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.* gnu/services/base.scm: Export dependency->shepherd-service-name.* doc/guix.texi (ZFS File System): New subsection.--- doc/guix.texi | 351 ++++++++++++++++++++++++++++++++++ gnu/local.mk | 2 + gnu/services/base.scm | 4 +- gnu/services/file-systems.scm | 295 ++++++++++++++++++++++++++++ 4 files changed, 651 insertions(+), 1 deletion(-) create mode 100644 gnu/services/file-systems.scm
Toggle diff (705 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex b3c16e6507..e21c47d7ca 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -94,6 +94,7 @@ Copyright @copyright{} 2021 Xinglu Chen@* Copyright @copyright{} 2021 Raghav Gururajan@* Copyright @copyright{} 2021 Domagoj Stolfa@* Copyright @copyright{} 2021 Hui Lu@*+Copyright @copyright{} 2021 raid5atemyhomework@*
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or@@ -14265,6 +14266,356 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp
++@node ZFS File System+@subsection ZFS File System++Support for ZFS file systems is provided on Guix by the OpenZFS project.+OpenZFS currently only supports Linux-Libre and is not available on the+Hurd.++OpenZFS is free software; unfortunately its license is incompatible with+the GNU General Public License (GPL), the license of the Linux kernel,+which means they cannot be distributed together. However, as a user,+you can choose to build ZFS and use it together with Linux; you can+even rely on Guix to automate this task. See+@uref{https://www.fsf.org/licensing/zfs-and-linux, this analysis by+the Free Software Foundation} for more information.++As a large and complex kernel module, OpenZFS has to be compiled for a+specific version of Linux-Libre. At times, the latest OpenZFS package+available in Guix is not compatible with the latest Linux-Libre version.+Thus, directly installing the @code{zfs} package can fail.++Instead, you are recommended to select a specific older long-term-support+Linux-Libre kernel. Do not use @code{linux-libre-lts}, as even the+latest long-term-support kernel may be too new for @code{zfs}. Instead,+explicitly select a specific older version, such as @code{linux-libre-5.10},+and upgrade it manually later as new long-term-support kernels become+available that you have confirmed is compatible with the latest available+OpenZFS version on Guix.++For example, you can modify your system configuration file to a specific+Linux-Libre version and add the @code{zfs-service-type} service.++@lisp+(use-modules (gnu))+(use-package-modules+ #;@dots{}+ linux)+(use-service-modules+ #;@dots{}+ file-systems)++(define my-kernel linux-libre-5.10)++(operating-system+ (kernel my-kernel)+ #;@dots{}+ (services+ (cons* (service zfs-service-type+ (zfs-configuration+ (kernel my-kernel)))+ #;@dots{}+ %desktop-services))+ #;@dots{})+@end lisp++@defvr {Scheme Variable} zfs-service-type+This is the type for a service that adds ZFS support to your operating+system. The service is configured using a @code{zfs-configuration}+record.++Here is an example use:++@lisp+(service zfs-service-type+ (zfs-configuration+ (kernel linux-libre-5.4)))+@end lisp+@end defvr++@deftp {Data Type} zfs-configuration+This data type represents the configuration of the ZFS support in Guix+System. Its fields are:++@table @asis+@item @code{kernel}+The package of the Linux-Libre kernel to compile OpenZFS for. This field+is always required. It @emph{must} be the same kernel you use in your+@code{operating-system} form.++@item @code{base-zfs} (default: @code{zfs})+The OpenZFS package that will be compiled for the given Linux-Libre kernel.++@item @code{base-zfs-auto-snapshot} (default: @code{zfs-auto-snapshot})+The @code{zfs-auto-snapshot} package to use. It will be modified to+specifically use the OpenZFS compiled for your kernel.++@item @code{dependencies} (default: @code{'()})+A list of @code{<file-system>} or @code{<mapped-device>} records that must+be mounted or opened before OpenZFS scans for pools to import. For example,+if you have set up LUKS containers as leaf VDEVs in a pool, you have to+include their corresponding @code{<mapped-ddevice>} records so that OpenZFS+can import the pool correctly at bootup.++@item @code{auto-mount?} (default: @code{#t})+Whether to mount datasets with the ZFS @code{mountpoint} property automatically+at startup. This is the behavior that ZFS users usually expect. You might+set this to @code{#f} for an operating system intended as a ``rescue'' system+that is intended to help debug problems with the disks rather than actually+work in production.++@item @code{auto-scrub} (default: @code{'weekly})+Specifies how often to scrub all pools. Can be the symbols @code{'weekly} or+@code{'monthly}, or a schedule specification understood by+@xref{mcron, mcron job specifications,, mcron, GNU@tie{}mcron}, such as+@code{"0 3 * * 6"} for ``every 3AM on Saturday''.+It can also be @code{#f} to disable auto-scrubbing (@strong{not recommended}).++The general guideline is to scrub weekly when using consumer-quality drives, and+to scrub monthly when using enterprise-quality drives.++@code{'weekly} scrubs are done on Sunday midnight, while @code{monthly} scrubs+are done on midnight on the first day of each month.++@item @code{auto-snapshot?} (default: @code{#t})+Specifies whether to auto-snapshot by default. If @code{#t}, then snapshots+are automatically created except for ZFS datasets with the+@code{com.sun:auto-snapshot} ZFS vendor property set to @code{false}.++If @code{#f}, snapshots will not be automatically created, unless the ZFS+dataset has the @code{com.sun:auto-snapshot} ZFS vendor property set to+@code{true}.++@item @code{auto-snapshot-keep} (default: @code{'()})+Specifies an association list of symbol-number pairs, indicating the number+of automatically-created snapshots to retain for each frequency type.++If not specified via this field, by default there are 4 @code{frequent}, 24+@code{hourly}, 31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly} snapshots.++For example:++@lisp+(zfs-configuration+ (kernel my-kernel)+ (auto-snapshot-keep+ '((frequent . 8)+ (hourly . 12))))+@end lisp++The above will keep 8 @code{frequent} snapshots and 12 @code{hourly} snapshots.+@code{daily}, @code{weekly}, and @code{monthly} snapshots will keep their+defaults (31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly}).++@end table+@end deftp++@subsubsection ZFS Auto-Snapshot++The ZFS service on Guix System supports auto-snapshots as implemented in the+Solaris operating system.++@code{frequent} (every 15 minutes), @code{hourly}, @code{daily}, @code{weekly},+and @code{monthly} snapshots are created automatically for ZFS datasets that+have auto-snapshot enabled. They will be named, for example,+@code{zfs-auto-snap_frequent-2021-03-22-1415}. You can continue to use+manually-created snapshots as long as they do not conflict with the naming+convention used by auto-snapshot. You can also safely manually destroy+automatically-created snapshots, for example to free up space.++The @code{com.sun:auto-snapshot} ZFS property controls auto-snapshot on a+per-dataset level. Sub-datasets will inherit this property from their parent+dataset, but can have their own property.++You @emph{must} set this property to @code{true} or @code{false} exactly,+otherwise it will be treated as if the property is unset.++For example:++@example+# zfs list -o name+NAME+tank+tank/important-data+tank/tmp+# zfs set com.sun:auto-snapshot=true tank+# zfs set com.sun:auto-snapshot=false tank/tmp+@end example++The above will set @code{tank} and @code{tank/important-data} to be+auto-snapshot, while @code{tank/tmp} will not be auto-snapshot.++If the @code{com.sun:auto-snapshot} property is not set for a dataset+(the default when pools and datasets are created), then whether+auto-snapshot is done or not will depend on the @code{auto-snapshot?}+field of the @code{zfs-configuration} record.++There are also @code{com.sun:auto-snapshot:frequent},+@code{com.sun:auto-snapshot:hourly}, @code{com.sun:auto-snapshot:daily},+@code{com.sun:auto-snapshot:weekly}, and @code{com.sun:auto-snapshot:monthly}+properties that give finer-grained control of whether to auto-snapshot a+dataset at a particular schedule.++The number of snapshots kept for all datasets can be overridden via the+@code{auto-snapshot-keep} field of the @code{zfs-configuration} record.+There is currently no support to have different numbers of snapshots to+keep for different datasets.++@subsubsection ZVOLs++ZFS supports ZVOLs, block devices that ZFS exposes to the operating+system in the @code{/dev/zvol/} directory. The ZVOL will have the same+resilience and self-healing properties as other datasets on your ZFS pool.+ZVOLs can also be snapshotted (and will be included in auto-snapshotting+if enabled), which snapshots the state of the block device, effectively+snapshotting the hosted file system.++You can put any file system inside the ZVOL. However, in order to mount this+file system at system start, you need to add @code{%zfs-zvol-dependency} as a+dependency of each file system inside a ZVOL.++@defvr {Scheme Variable} %zfs-zvol-dependency+An artificial @code{<mapped-device>} which tells the file system mounting+service to wait for ZFS to provide ZVOLs before mounting the+@code{<file-system>} dependent on it.+@end defvr++For example, suppose you create a ZVOL and put an ext4 filesystem+inside it:++@example+# zfs create -V 100G tank/ext4-on-zfs+# mkfs.ext4 /dev/zvol/tank/ext4-on-zfs+# mkdir /ext4-on-zfs+# mount /dev/zvol/tank/ext4-on-zfs /ext4-on-zfs+@end example++You can then set this up to be mounted at boot by adding this to the+@code{file-systems} field of your @code{operating-system} record:++@lisp+(file-system+ (device "/dev/zvol/tank/ext4-on-zfs")+ (mount-point "/ext4-on-zfs")+ (type "ext4")+ (dependencies (list %zfs-zvol-dependency)))+@end lisp++You @emph{must not} add @code{%zfs-zvol-dependency} to your+@code{operating-system}'s @code{mapped-devices} field, and you @emph{must+not} add it (or any @code{<file-system>}s dependent on it) to the+@code{dependencies} field of @code{zfs-configuration}. Finally, you+@emph{must not} use @code{%zfs-zvol-dependency} unless you actually+instantiate @code{zfs-service-type} on your system.++@subsubsection Unsupported Features++Some common features and uses of ZFS are currently not supported, or not+fully supported, on Guix.++@enumerate+@item+Shepherd-managed daemons that are configured to read from or write to ZFS+mountpoints need to include @code{user-processes} in their @code{requirement}+field. This is the earliest that ZFS file systems are assured of being+mounted.++Generally, most daemons will, directly or indirectly, require+@code{networking}, or @code{user-processes}, or both. Most implementations+of @code{networking} will also require @code{user-processes} so daemons that+require only @code{networking} will also generally start up after+@code{user-processes}. A notable exception, however, is+@code{static-networking-service-type}. You will need to explicitly add+@code{user-processes} as a @code{requirement} of your @code{static-networking}+record.++@item+@code{mountpoint=legacy} ZFS file systems. The handlers for the Guix mounting+system have not yet been modified to support ZFS, and will expect @code{/dev}+paths in the @code{<file-system>}'s @code{device} field, but ZFS file systems+are referred to via non-path @code{pool/file/system} names. Such file systems+also need to be mounted @emph{after} OpenZFS has scanned for pools.++You can still manually mount these file systems after system boot; what is+only unsupported is mounting them automatically at system boot by specifying+them in @code{<file-system>} records of your @code{operating-system}.++@item+@code{/home} on ZFS. Guix will create home directories for users, but this+process currently cannot be scheduled after ZFS file systems are mounted.+Thus, the ZFS file system might be mounted @emph{after} Guix has created+home directories at boot, at which point OpenZFS will refuse to mount since+the mountpoint is not empty. However, you @emph{can} create an ext4, xfs,+btrfs, or other supported file system inside a ZVOL, have that depend on+@code{%zfs-zvol-dependency}, and set it to mount on the @code{/home}+directory; they will be scheduled to mount before the @code{user-homes}+process.++Similarly, other locations like @code{/var}, @code{/gnu/store} and so+on cannot be reliably put in a ZFS file system, though they may be+possible to create as other file systems inside ZVOL containers.++@item+@code{/} and @code{/boot} on ZFS. These require Guix to expose more of+the @code{initrd} very early boot process to services. It also requires+Guix to have the ability to explicitly load modules while still in+@code{initrd} (currently kernel modules loaded by+@code{kernel-module-loader-service-type} are loaded after @code{/} is+mounted). Further, since one of ZFS's main advantages is that it can+continue working despite the loss of one or more devices, it makes sense+to also support installing the bootloader on all devices of the pool that+contains the @code{/} and @code{/boot}; after all, if ZFS can survive the+loss of one device, the bootloader should also be able to survive the loss+of one device.++@item+ZVOL swap devices. Mapped swap devices need to be listed in+@code{mapped-devices} to ensure they are opened before the system attempts+to use them, but you cannot currently add @code{%zfs-zvol-dependency} to+@code{mapped-devices}.++This will also require significant amounts of testing, as various kernel+build options and patches may affect how swapping works, which are possibly+different on Guix System compared to other distributions that this feature is+known to work on.++@item+ZFS Event Daemon. Support for this has not been written yet, patches are+welcome. The main issue is how to design this in a Guix style while+supporting legacy shell-script styles as well. In particular, OpenZFS itself+comes with a number of shell scripts intended for ZFS Event Daemon, and we+need to figure out how the user can choose to use or not use the provided+scripts (and configure any settings they have) or override with their own+custom code (which could be shell scripts they have written and trusted from+previous ZFS installations).++As-is, you can create your own service that activates the ZFS Event Daemon+by creating the @file{/etc/zfs/zed} directory and filling it appropriately,+then launching @code{zed}.++@item+@file{/etc/zfs/zpool.cache}. Currently the ZFS support on Guix always forces+scanning of all devices at bootup to look for ZFS pools. For systems with+dozens or hundreds of storage devices, this can lead to slow bootup. One issue+is that tools should really not write to @code{/etc} which is supposed to be for+configuration; possibly it could be moved to @code{/var} instead. Another issue+is that if Guix ever supports @code{/} on ZFS, we would need to somehow keep the+@code{zpool.cache} file inside the @code{initrd} up-to-date with what is in the+@code{/} mount point.++@item+@code{zfs share}. This will require some (unknown amount of) work to integrate+into the Samba and NFS services of Guix. You @emph{can} manually set up Samba+and NFS to share any mounted ZFS datasets by setting up their configurations+properly; it just can't be done for you by @code{zfs share} and the+@code{sharesmb} and @code{sharenfs} properties.+@end enumerate++Hopefully, support for the above only requires code to be written, o users+are encouraged to hack on Guix to implement the above features.+ @node Mapped Devices @section Mapped Devices
diff --git a/gnu/local.mk b/gnu/local.mkindex b944c671af..a2ff871277 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -43,6 +43,7 @@ # Copyright © 2021 Philip McGrath <philip@philipmcgrath.com> # Copyright © 2021 Arun Isaac <arunisaac@systemreboot.net> # Copyright © 2021 Sharlatan Hellseher <sharlatanus@gmail.com>+# Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> # # This file is part of GNU Guix. #@@ -618,6 +619,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/docker.scm \ %D%/services/authentication.scm \ %D%/services/file-sharing.scm \+ %D%/services/file-systems.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \ %D%/services/getmail.scm \diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex ab3e441a7b..bcca24f93a 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -185,7 +185,9 @@
references-file
- %base-services))+ %base-services++ dependency->shepherd-service-name))
;;; Commentary: ;;;diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scmnew file mode 100644index 0000000000..0b1aae38ac--- /dev/null+++ b/gnu/services/file-systems.scm@@ -0,0 +1,295 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu services file-systems)+ #:use-module (gnu packages file-systems)+ #:use-module (gnu services)+ #:use-module (gnu services base)+ #:use-module (gnu services linux)+ #:use-module (gnu services mcron)+ #:use-module (gnu services shepherd)+ #:use-module (gnu system mapped-devices)+ #:use-module (guix gexp)+ #:use-module (guix packages)+ #:use-module (guix records)+ #:export (zfs-service-type++ zfs-configuration+ zfs-configuration?+ zfs-configuration-kernel+ zfs-configuration-base-zfs+ zfs-configuration-base-zfs-auto-snapshot+ zfs-configuration-dependencies+ zfs-configuration-auto-mount?+ zfs-configuration-auto-scrub+ zfs-configuration-auto-snapshot?+ zfs-configuration-auto-snapshot-keep++ %zfs-zvol-dependency))++(define-record-type* <zfs-configuration>+ zfs-configuration+ make-zfs-configuration+ zfs-configuration?++ ; linux-libre kernel you want to compile the base-zfs module for.+ (kernel zfs-configuration-kernel)++ ; the OpenZFS package that will be modified to compile for the+ ; given kernel.+ (base-zfs zfs-configuration-base-zfs+ (default zfs))++ ; the zfs-auto-snapshot package that will be modified to compile+ ; for the given kernel.+ (base-zfs-auto-snapshot zfs-configuration-base-zfs-auto-snapshot+ (default zfs-auto-snapshot))++ ; list of <mapped-device> or <file-system> objects that must be+ ; opened/mounted before we import any ZFS pools.+ (dependencies zfs-configuration-dependencies+ (default '()))++ ; #t if mountable datasets are to be mounted automatically.+ ; #f if not mounting.+ ; #t is the expected behavior on other operating systems, the+ ; #f is only supported for "rescue" operating systems where+ ; the user wants lower-level control of when to mount.+ (auto-mount? zfs-configuration-auto-mount?+ (default #t))++ ; 'weekly for weekly scrubbing, 'monthly for monthly scrubbing, an+ ; mcron time specification that can be given to `job`, or #f to+ ; disable.+ (auto-scrub zfs-configuration-auto-scrub+ (default 'weekly))++ ; #t if auto-snapshot is default (and `com.sun:auto-snapshot=false`+ ; disables auto-snapshot per dataset), #f if no auto-snapshotting+ ; is default (and `com.sun:auto-snapshot=true` enables auto-snapshot+ ; per dataset).+ (auto-snapshot? zfs-configuration-auto-snapshot?+ (default #t))++ ; association list of symbol-number pairs to indicate the number+ ; of automatic snapshots to keep for each of 'frequent, 'hourly,+ ; 'daily, 'weekly, and 'monthly.+ ; e.g. '((frequent . 8) (hourly . 12))+ (auto-snapshot-keep zfs-configuration-auto-snapshot-keep+ (default '())))++(define %default-auto-snapshot-keep+ '((frequent . 4)+ (hourly . 24)+ (daily . 31)+ (weekly . 8)+ (monthly . 12)))++(define %auto-snapshot-mcron-schedule+ '((frequent . "0,15,30,45 * * * *")+ (hourly . "0 * * * *")+ (daily . "0 0 * * *")+ (weekly . "0 0 * * 7")+ (monthly . "0 0 1 * *")))++;; A synthetic and unusable MAPPED-DEVICE intended for use when+;; the user has created a mountable filesystem inside a ZFS+;; zvol and wants it mounted inside the configuration.scm.+(define %zfs-zvol-dependency+ (mapped-device+ (source '())+ (targets '("zvol/*"))+ (type #f)))++(define (make-zfs-package conf)+ (let ((kernel (zfs-configuration-kernel conf))+ (base-zfs (zfs-configuration-base-zfs conf)))+ (package+ (inherit base-zfs)+ (arguments (cons* #:linux kernel+ (package-arguments base-zfs))))))++(define (make-zfs-auto-snapshot-package conf)+ (let ((zfs (make-zfs-package conf))+ (base-zfs-auto-snapshot (zfs-configuration-base-zfs-auto-snapshot conf)))+ (package+ (inherit base-zfs-auto-snapshot)+ (inputs `(("zfs" ,zfs))))))++(define (zfs-loadable-modules conf)+ (list (list (make-zfs-package conf) "module")))++(define (zfs-shepherd-services conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (zfs (file-append zfs-package "/sbin/zfs"))+ (zvol_wait (file-append zfs-package "/bin/zvol_wait"))+ (scheme-modules `((srfi srfi-1)+ (srfi srfi-34)+ (srfi srfi-35)+ (rnrs io ports)+ ,@%default-modules)))+ (define zfs-scan+ (shepherd-service+ (provision '(zfs-scan))+ (requirement `(root-file-system+ kernel-module-loader+ udev+ ,@(map dependency->shepherd-service-name+ (zfs-configuration-dependencies conf))))+ (documentation "Scans for and imports ZFS pools.")+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "zfs: error importing pools: ~s~%"+ (condition-message c))+ #f))+ ; TODO: optionally use a cachefile.+ (invoke #$zpool "import" "-a" "-N"))))+ ;; Why not one-shot? Because we don't really want to rescan+ ;; this each time a requiring process is restarted, as scanning+ ;; can take a long time and a lot of I/O.+ (stop #~(const #f))))++ (define device-mapping-zvol/*+ (shepherd-service+ (provision '(device-mapping-zvol/*))+ (requirement '(zfs-scan))+ (documentation "Waits for all ZFS ZVOLs to be opened.")+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "zfs: error opening zvols: ~s~%"+ (condition-message c))+ #f))+ (invoke #$zvol_wait))))+ (stop #~(const #f))))++ (define zfs-auto-mount+ (shepherd-service+ (provision '(zfs-auto-mount))+ (requirement '(zfs-scan))+ (documentation "Mounts all non-legacy mounted ZFS filesystems.")+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "zfs: error mounting file systems: ~s~%"+ (condition-message c))+ #f))+ ;; Output to current-error-port, otherwise the+ ;; user will not see any prompts for passwords+ ;; of encrypted datasets.+ ;; XXX Maybe better to explicitly open /dev/console ?+ (with-output-to-port (current-error-port)+ (lambda ()+ (invoke #$zfs "mount" "-a" "-l"))))))+ (stop #~(lambda _+ ;; Make sure that Shepherd does not have a CWD that+ ;; is a mounted ZFS filesystem, which would prevent+ ;; unmounting.+ (chdir "/")+ (invoke #$zfs "unmount" "-a" "-f")))))++ `(,zfs-scan+ ,device-mapping-zvol/*+ ,@(if (zfs-configuration-auto-mount? conf)+ `(,zfs-auto-mount)+ '()))))++(define (zfs-user-processes conf)+ (if (zfs-configuration-auto-mount? conf)+ '(zfs-auto-mount)+ '(zfs-scan)))++(define (zfs-mcron-auto-snapshot-jobs conf)+ (let* ((user-auto-snapshot-keep (zfs-configuration-auto-snapshot-keep conf))+ ;; assoc-ref has earlier entries overriding later ones.+ (auto-snapshot-keep (append user-auto-snapshot-keep+ %default-auto-snapshot-keep))+ (auto-snapshot? (zfs-configuration-auto-snapshot? conf))+ (zfs-auto-snapshot-package (make-zfs-auto-snapshot-package conf))+ (zfs-auto-snapshot (file-append zfs-auto-snapshot-package+ "/sbin/zfs-auto-snapshot")))+ (map+ (lambda (label)+ (let ((keep (assoc-ref auto-snapshot-keep label))+ (sched (assoc-ref %auto-snapshot-mcron-schedule label)))+ #~(job '#$sched+ (string-append #$zfs-auto-snapshot+ " --quiet --syslog "+ " --label=" #$(symbol->string label)+ " --keep=" #$(number->string keep)+ " //"))))+ '(frequent hourly daily weekly monthly))))++(define (zfs-mcron-auto-scrub-jobs conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (auto-scrub (zfs-configuration-auto-scrub conf))+ (sched (cond+ ((eq? auto-scrub 'weekly) "0 0 * * 7")+ ((eq? auto-scrub 'monthly) "0 0 1 * *")+ (else auto-scrub))))+ (list+ #~(job '#$sched+ ;; Suppress errors: if there are no ZFS pools, the+ ;; scrub will not be given any arguments, which makes+ ;; it error out.+ (string-append "(" #$zpool " scrub `" #$zpool " list -o name -H` "+ "> /dev/null 2>&1) "+ "|| exit 0")))))++(define (zfs-mcron-jobs conf)+ (append (zfs-mcron-auto-snapshot-jobs conf)+ (if (zfs-configuration-auto-scrub conf)+ (zfs-mcron-auto-scrub-jobs conf)+ '())))++(define zfs-service-type+ (service-type+ (name 'zfs)+ (extensions+ (list ;; Install OpenZFS kernel module into kernel profile.+ (service-extension linux-loadable-module-service-type+ zfs-loadable-modules)+ ;; And load it.+ (service-extension kernel-module-loader-service-type+ (const '("zfs")))+ ;; Make sure ZFS pools and datasets are mounted at+ ;; boot.+ (service-extension shepherd-root-service-type+ zfs-shepherd-services)+ ;; Make sure user-processes don't start until+ ;; after ZFS does.+ (service-extension user-processes-service-type+ zfs-user-processes)+ ;; Install automated scrubbing and snapshotting.+ (service-extension mcron-service-type+ zfs-mcron-jobs)++ ;; Install ZFS management commands in the system+ ;; profile.+ (service-extension profile-service-type+ (compose list make-zfs-package))+ ;; Install ZFS udev rules.+ (service-extension udev-service-type+ (compose list make-zfs-package))))+ (description "Installs ZFS, an advanced filesystem and volume manager.")))
--2.31.1
R
R
raid5atemyhomework wrote on 1 Aug 11:41 +0200
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
YIcuycFkyDDf7q5ttD0uN6nKeWVS3jtw-J_vmF0roEg_MgXbCAgoGKtMdTFRPkqO09C0B6hSIP3TWvUYZRhqng0bLwP88zkViwqyQQMMj5s=@protonmail.com
R
R
raid5atemyhomework wrote on 10 Aug 13:43 +0200
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
zPNx3dOuiEZ1x2lQZ8YojpSU7BLzUbtBfBsdaoDsOiArEJoPo6zNeVLk1uQP_rs9Gy4kMBtgtD47yd-0DS0FhrBSL-wNWsJCXGg2rswoifQ=@protonmail.com
BUMPITY BUMPITY BUMP BUMP BUMP
R
R
raid5atemyhomework wrote on 31 Aug 02:48 +0200
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
9Vr2H4nOppX9ANzsGjyQEbM0mUl4LcJAsmEwScnDh084gLHotpSwNrX3-TOt1QaqrOQwAsww03POWafkdxUVI-7DLfKDheVvGTL6AHUijFM=@protonmail.com
YOOOOOOOOOHOOOOOOOOOOOOOO
M
M
Maxime Devos wrote on 2 Sep 22:57 +0200
Re: [bug#45692] [PATCH v4 3/3] gnu: Add ZFS service type.
2020c2223378c7eb3635defb27e6b4545e048b9f.camel@telenet.be
Hi,
Some comments on the code. Spoiler: the code is presumably good,but there's a GPL violation.
Toggle quote (39 lines)> From 5351aa7c1c14d4fea032adad895c436e02d1f261 Mon Sep 17 00:00:00 2001> From: raid5atemyhomework <raid5atemyhomework@protonmail.com>> Date: Mon, 22 Mar 2021 16:26:28 +0800> Subject: [PATCH] gnu: Add ZFS service type.> > * gnu/services/file-systems.scm: New file.> * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.> * gnu/services/base.scm: Export dependency->shepherd-service-name.> * doc/guix.texi (ZFS File System): New subsection.> ---> doc/guix.texi | 351 ++++++++++++++++++++++++++++++++++> gnu/local.mk | 2 +> gnu/services/base.scm | 4 +-> gnu/services/file-systems.scm | 295 ++++++++++++++++++++++++++++> 4 files changed, 651 insertions(+), 1 deletion(-)> create mode 100644 gnu/services/file-systems.scm> > diff --git a/doc/guix.texi b/doc/guix.texi> index b3c16e6507..e21c47d7ca 100644> --- a/doc/guix.texi> +++ b/doc/guix.texi> @@ -94,6 +94,7 @@ Copyright @copyright{} 2021 Xinglu Chen@*> Copyright @copyright{} 2021 Raghav Gururajan@*> Copyright @copyright{} 2021 Domagoj Stolfa@*> Copyright @copyright{} 2021 Hui Lu@*> +Copyright @copyright{} 2021 raid5atemyhomework@*> > Permission is granted to copy, distribute and/or modify this document> under the terms of the GNU Free Documentation License, Version 1.3 or> @@ -14265,6 +14266,356 @@ a file system declaration such as:> compress-force=zstd,space_cache=v2"))> @end lisp> > +> +@node ZFS File System> +@subsection ZFS File System> +> +Support for ZFS file systems is provided on Guix by the OpenZFS project.
This is worded a bit misleading: it doesn't seem like OpenZFS is providingGuix-specific support, except in the more general sense that OpenZFS provides aZFS kernel module and utilities to _everyone_. Suggestion:
‘Support for ZFS file systems in Guix is based on the OpenZFS project.’
Toggle quote (11 lines)> +OpenZFS currently only supports Linux-Libre and is not available on the> +Hurd.> +> +OpenZFS is free software; unfortunately its license is incompatible with> +the GNU General Public License (GPL), the license of the Linux kernel,> +which means they cannot be distributed together. However, as a user,> +you can choose to build ZFS and use it together with Linux; you can> +even rely on Guix to automate this task. See> +@uref{https://www.fsf.org/licensing/zfs-and-linux, this analysis by> +the Free Software Foundation} for more information.
That analysis says explicitely that the CDDL is incompatible with the GPL,and that they cannot be legally linked together. E.g., see the second quotedparagraph:
‘A copyleft license, including any version of the GNU GPL or GNU AGPL, requiresaugmented versions to be free under the same license -- the same requirement itapplies to modification of the code.1 I wrote a copyleft license for GNU programsto ensure that all users of all versions of them would get the freedoms I intendedto give them.
It is not enough to require that the combined program be free software somehow. Itmust be released, as a whole, **under the original copyleft license**, to ensure that:(1) subsequent users get the exact same freedoms and (2) subsequent intermediariesdo not get more opportunity than first-stage intermediaries to make the program nonfree.’
(emphasis mine)
See also https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/.
The argument ‘it's the _user_ that's building & linking, not ‘the guix project’’ doesn'tseem convincing to me. We're still linking the kernel and OpenZFS kernel module,we're just doing it lazily (in the SRFI-45 sense).
Until Sun's copyright expires or Oracle relicenses their ZFS code,I don't think we can legally provide the ZFS kernel module in guix(the user space tools should be fine, but they are useless without the kernelmodules I assume, so maybe remove them too?).
Toggle quote (333 lines)> +As a large and complex kernel module, OpenZFS has to be compiled for a> +specific version of Linux-Libre. At times, the latest OpenZFS package> +available in Guix is not compatible with the latest Linux-Libre version.> +Thus, directly installing the @code{zfs} package can fail.> +> +Instead, you are recommended to select a specific older long-term-support> +Linux-Libre kernel. Do not use @code{linux-libre-lts}, as even the> +latest long-term-support kernel may be too new for @code{zfs}. Instead,> +explicitly select a specific older version, such as @code{linux-libre-5.10},> +and upgrade it manually later as new long-term-support kernels become> +available that you have confirmed is compatible with the latest available> +OpenZFS version on Guix.> +> +For example, you can modify your system configuration file to a specific> +Linux-Libre version and add the @code{zfs-service-type} service.> +> +@lisp> +(use-modules (gnu))> +(use-package-modules> + #;@dots{}> + linux)> +(use-service-modules> + #;@dots{}> + file-systems)> +> +(define my-kernel linux-libre-5.10)> +> +(operating-system> + (kernel my-kernel)> + #;@dots{}> + (services> + (cons* (service zfs-service-type> + (zfs-configuration> + (kernel my-kernel)))> + #;@dots{}> + %desktop-services))> + #;@dots{})> +@end lisp> +> +@defvr {Scheme Variable} zfs-service-type> +This is the type for a service that adds ZFS support to your operating> +system. The service is configured using a @code{zfs-configuration}> +record.> +> +Here is an example use:> +> +@lisp> +(service zfs-service-type> + (zfs-configuration> + (kernel linux-libre-5.4)))> +@end lisp> +@end defvr> +> +@deftp {Data Type} zfs-configuration> +This data type represents the configuration of the ZFS support in Guix> +System. Its fields are:> +> +@table @asis> +@item @code{kernel}> +The package of the Linux-Libre kernel to compile OpenZFS for. This field> +is always required. It @emph{must} be the same kernel you use in your> +@code{operating-system} form.> +> +@item @code{base-zfs} (default: @code{zfs})> +The OpenZFS package that will be compiled for the given Linux-Libre kernel.> +> +@item @code{base-zfs-auto-snapshot} (default: @code{zfs-auto-snapshot})> +The @code{zfs-auto-snapshot} package to use. It will be modified to> +specifically use the OpenZFS compiled for your kernel.> +> +@item @code{dependencies} (default: @code{'()})> +A list of @code{<file-system>} or @code{<mapped-device>} records that must> +be mounted or opened before OpenZFS scans for pools to import. For example,> +if you have set up LUKS containers as leaf VDEVs in a pool, you have to> +include their corresponding @code{<mapped-ddevice>} records so that OpenZFS> +can import the pool correctly at bootup.> +> +@item @code{auto-mount?} (default: @code{#t})> +Whether to mount datasets with the ZFS @code{mountpoint} property automatically> +at startup. This is the behavior that ZFS users usually expect. You might> +set this to @code{#f} for an operating system intended as a ``rescue'' system> +that is intended to help debug problems with the disks rather than actually> +work in production.> +> +@item @code{auto-scrub} (default: @code{'weekly})> +Specifies how often to scrub all pools. Can be the symbols @code{'weekly} or> +@code{'monthly}, or a schedule specification understood by> +@xref{mcron, mcron job specifications,, mcron, GNU@tie{}mcron}, such as> +@code{"0 3 * * 6"} for ``every 3AM on Saturday''.> +It can also be @code{#f} to disable auto-scrubbing (@strong{not recommended}).> +> +The general guideline is to scrub weekly when using consumer-quality drives, and> +to scrub monthly when using enterprise-quality drives.> +> +@code{'weekly} scrubs are done on Sunday midnight, while @code{monthly} scrubs> +are done on midnight on the first day of each month.> +> +@item @code{auto-snapshot?} (default: @code{#t})> +Specifies whether to auto-snapshot by default. If @code{#t}, then snapshots> +are automatically created except for ZFS datasets with the> +@code{com.sun:auto-snapshot} ZFS vendor property set to @code{false}.> +> +If @code{#f}, snapshots will not be automatically created, unless the ZFS> +dataset has the @code{com.sun:auto-snapshot} ZFS vendor property set to> +@code{true}.> +> +@item @code{auto-snapshot-keep} (default: @code{'()})> +Specifies an association list of symbol-number pairs, indicating the number> +of automatically-created snapshots to retain for each frequency type.> +> +If not specified via this field, by default there are 4 @code{frequent}, 24> +@code{hourly}, 31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly} snapshots.> +> +For example:> +> +@lisp> +(zfs-configuration> + (kernel my-kernel)> + (auto-snapshot-keep> + '((frequent . 8)> + (hourly . 12))))> +@end lisp> +> +The above will keep 8 @code{frequent} snapshots and 12 @code{hourly} snapshots.> +@code{daily}, @code{weekly}, and @code{monthly} snapshots will keep their> +defaults (31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly}).> +> +@end table> +@end deftp> +> +@subsubsection ZFS Auto-Snapshot> +> +The ZFS service on Guix System supports auto-snapshots as implemented in the> +Solaris operating system.> +> +@code{frequent} (every 15 minutes), @code{hourly}, @code{daily}, @code{weekly},> +and @code{monthly} snapshots are created automatically for ZFS datasets that> +have auto-snapshot enabled. They will be named, for example,> +@code{zfs-auto-snap_frequent-2021-03-22-1415}. You can continue to use> +manually-created snapshots as long as they do not conflict with the naming> +convention used by auto-snapshot. You can also safely manually destroy> +automatically-created snapshots, for example to free up space.> +> +The @code{com.sun:auto-snapshot} ZFS property controls auto-snapshot on a> +per-dataset level. Sub-datasets will inherit this property from their parent> +dataset, but can have their own property.> +> +You @emph{must} set this property to @code{true} or @code{false} exactly,> +otherwise it will be treated as if the property is unset.> +> +For example:> +> +@example> +# zfs list -o name> +NAME> +tank> +tank/important-data> +tank/tmp> +# zfs set com.sun:auto-snapshot=true tank> +# zfs set com.sun:auto-snapshot=false tank/tmp> +@end example> +> +The above will set @code{tank} and @code{tank/important-data} to be> +auto-snapshot, while @code{tank/tmp} will not be auto-snapshot.> +> +If the @code{com.sun:auto-snapshot} property is not set for a dataset> +(the default when pools and datasets are created), then whether> +auto-snapshot is done or not will depend on the @code{auto-snapshot?}> +field of the @code{zfs-configuration} record.> +> +There are also @code{com.sun:auto-snapshot:frequent},> +@code{com.sun:auto-snapshot:hourly}, @code{com.sun:auto-snapshot:daily},> +@code{com.sun:auto-snapshot:weekly}, and @code{com.sun:auto-snapshot:monthly}> +properties that give finer-grained control of whether to auto-snapshot a> +dataset at a particular schedule.> +> +The number of snapshots kept for all datasets can be overridden via the> +@code{auto-snapshot-keep} field of the @code{zfs-configuration} record.> +There is currently no support to have different numbers of snapshots to> +keep for different datasets.> +> +@subsubsection ZVOLs> +> +ZFS supports ZVOLs, block devices that ZFS exposes to the operating> +system in the @code{/dev/zvol/} directory. The ZVOL will have the same> +resilience and self-healing properties as other datasets on your ZFS pool.> +ZVOLs can also be snapshotted (and will be included in auto-snapshotting> +if enabled), which snapshots the state of the block device, effectively> +snapshotting the hosted file system.> +> +You can put any file system inside the ZVOL. However, in order to mount this> +file system at system start, you need to add @code{%zfs-zvol-dependency} as a> +dependency of each file system inside a ZVOL.> +> +@defvr {Scheme Variable} %zfs-zvol-dependency> +An artificial @code{<mapped-device>} which tells the file system mounting> +service to wait for ZFS to provide ZVOLs before mounting the> +@code{<file-system>} dependent on it.> +@end defvr> +> +For example, suppose you create a ZVOL and put an ext4 filesystem> +inside it:> +> +@example> +# zfs create -V 100G tank/ext4-on-zfs> +# mkfs.ext4 /dev/zvol/tank/ext4-on-zfs> +# mkdir /ext4-on-zfs> +# mount /dev/zvol/tank/ext4-on-zfs /ext4-on-zfs> +@end example> +> +You can then set this up to be mounted at boot by adding this to the> +@code{file-systems} field of your @code{operating-system} record:> +> +@lisp> +(file-system> + (device "/dev/zvol/tank/ext4-on-zfs")> + (mount-point "/ext4-on-zfs")> + (type "ext4")> + (dependencies (list %zfs-zvol-dependency)))> +@end lisp> +> +You @emph{must not} add @code{%zfs-zvol-dependency} to your> +@code{operating-system}'s @code{mapped-devices} field, and you @emph{must> +not} add it (or any @code{<file-system>}s dependent on it) to the> +@code{dependencies} field of @code{zfs-configuration}. Finally, you> +@emph{must not} use @code{%zfs-zvol-dependency} unless you actually> +instantiate @code{zfs-service-type} on your system.> +> +@subsubsection Unsupported Features> +> +Some common features and uses of ZFS are currently not supported, or not> +fully supported, on Guix.> +> +@enumerate> +@item> +Shepherd-managed daemons that are configured to read from or write to ZFS> +mountpoints need to include @code{user-processes} in their @code{requirement}> +field. This is the earliest that ZFS file systems are assured of being> +mounted.> +> +Generally, most daemons will, directly or indirectly, require> +@code{networking}, or @code{user-processes}, or both. Most implementations> +of @code{networking} will also require @code{user-processes} so daemons that> +require only @code{networking} will also generally start up after> +@code{user-processes}. A notable exception, however, is> +@code{static-networking-service-type}. You will need to explicitly add> +@code{user-processes} as a @code{requirement} of your @code{static-networking}> +record.> +> +@item> +@code{mountpoint=legacy} ZFS file systems. The handlers for the Guix mounting> +system have not yet been modified to support ZFS, and will expect @code{/dev}> +paths in the @code{<file-system>}'s @code{device} field, but ZFS file systems> +are referred to via non-path @code{pool/file/system} names. Such file systems> +also need to be mounted @emph{after} OpenZFS has scanned for pools.> +> +You can still manually mount these file systems after system boot; what is> +only unsupported is mounting them automatically at system boot by specifying> +them in @code{<file-system>} records of your @code{operating-system}.> +> +@item> +@code{/home} on ZFS. Guix will create home directories for users, but this> +process currently cannot be scheduled after ZFS file systems are mounted.> +Thus, the ZFS file system might be mounted @emph{after} Guix has created> +home directories at boot, at which point OpenZFS will refuse to mount since> +the mountpoint is not empty. However, you @emph{can} create an ext4, xfs,> +btrfs, or other supported file system inside a ZVOL, have that depend on> +@code{%zfs-zvol-dependency}, and set it to mount on the @code{/home}> +directory; they will be scheduled to mount before the @code{user-homes}> +process.> +> +Similarly, other locations like @code{/var}, @code{/gnu/store} and so> +on cannot be reliably put in a ZFS file system, though they may be> +possible to create as other file systems inside ZVOL containers.> +> +@item> +@code{/} and @code{/boot} on ZFS. These require Guix to expose more of> +the @code{initrd} very early boot process to services. It also requires> +Guix to have the ability to explicitly load modules while still in> +@code{initrd} (currently kernel modules loaded by> +@code{kernel-module-loader-service-type} are loaded after @code{/} is> +mounted). Further, since one of ZFS's main advantages is that it can> +continue working despite the loss of one or more devices, it makes sense> +to also support installing the bootloader on all devices of the pool that> +contains the @code{/} and @code{/boot}; after all, if ZFS can survive the> +loss of one device, the bootloader should also be able to survive the loss> +of one device.> +> +@item> +ZVOL swap devices. Mapped swap devices need to be listed in> +@code{mapped-devices} to ensure they are opened before the system attempts> +to use them, but you cannot currently add @code{%zfs-zvol-dependency} to> +@code{mapped-devices}.> +> +This will also require significant amounts of testing, as various kernel> +build options and patches may affect how swapping works, which are possibly> +different on Guix System compared to other distributions that this feature is> +known to work on.> +> +@item> +ZFS Event Daemon. Support for this has not been written yet, patches are> +welcome. The main issue is how to design this in a Guix style while> +supporting legacy shell-script styles as well. In particular, OpenZFS itself> +comes with a number of shell scripts intended for ZFS Event Daemon, and we> +need to figure out how the user can choose to use or not use the provided> +scripts (and configure any settings they have) or override with their own> +custom code (which could be shell scripts they have written and trusted from> +previous ZFS installations).> +> +As-is, you can create your own service that activates the ZFS Event Daemon> +by creating the @file{/etc/zfs/zed} directory and filling it appropriately,> +then launching @code{zed}.> +> +@item> +@file{/etc/zfs/zpool.cache}. Currently the ZFS support on Guix always forces> +scanning of all devices at bootup to look for ZFS pools. For systems with> +dozens or hundreds of storage devices, this can lead to slow bootup. One issue> +is that tools should really not write to @code{/etc} which is supposed to be for> +configuration; possibly it could be moved to @code{/var} instead. Another issue> +is that if Guix ever supports @code{/} on ZFS, we would need to somehow keep the> +@code{zpool.cache} file inside the @code{initrd} up-to-date with what is in the> +@code{/} mount point.> +> +@item> +@code{zfs share}. This will require some (unknown amount of) work to integrate> +into the Samba and NFS services of Guix. You @emph{can} manually set up Samba> +and NFS to share any mounted ZFS datasets by setting up their configurations> +properly; it just can't be done for you by @code{zfs share} and the> +@code{sharesmb} and @code{sharenfs} properties.> +@end enumerate> +> +Hopefully, support for the above only requires code to be written, o users
o users -> and users
Toggle quote (5 lines)> +are encouraged to hack on Guix to implement the above features.> +> @node Mapped Devices> @section Mapped Devices
I'm not familiar enough with ZFS to understand all that text above; I'll assumeit's fine.
Toggle quote (91 lines)> diff --git a/gnu/local.mk b/gnu/local.mk> index b944c671af..a2ff871277 100644> --- a/gnu/local.mk> +++ b/gnu/local.mk> @@ -43,6 +43,7 @@> # Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>> # Copyright © 2021 Arun Isaac <arunisaac@systemreboot.net>> # Copyright © 2021 Sharlatan Hellseher <sharlatanus@gmail.com>> +# Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>> #> # This file is part of GNU Guix.> #> @@ -618,6 +619,7 @@ GNU_SYSTEM_MODULES = \> %D%/services/docker.scm \> %D%/services/authentication.scm \> %D%/services/file-sharing.scm \> + %D%/services/file-systems.scm \> %D%/services/games.scm \> %D%/services/ganeti.scm \> %D%/services/getmail.scm \> diff --git a/gnu/services/base.scm b/gnu/services/base.scm> index ab3e441a7b..bcca24f93a 100644> --- a/gnu/services/base.scm> +++ b/gnu/services/base.scm> @@ -185,7 +185,9 @@> > references-file> > - %base-services))> + %base-services> +> + dependency->shepherd-service-name))> > ;;; Commentary:> ;;;> diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm> new file mode 100644> index 0000000000..0b1aae38ac> --- /dev/null> +++ b/gnu/services/file-systems.scm> @@ -0,0 +1,295 @@> +;;; GNU Guix --- Functional package management for GNU> +;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>> +;;;> +;;; This file is part of GNU Guix.> +;;;> +;;; GNU Guix is free software; you can redistribute it and/or modify it> +;;; under the terms of the GNU General Public License as published by> +;;; the Free Software Foundation; either version 3 of the License, or (at> +;;; your option) any later version.> +;;;> +;;; GNU Guix is distributed in the hope that it will be useful, but> +;;; WITHOUT ANY WARRANTY; without even the implied warranty of> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the> +;;; GNU General Public License for more details.> +;;;> +;;; You should have received a copy of the GNU General Public License> +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>;.> +> +(define-module (gnu services file-systems)> + #:use-module (gnu packages file-systems)> + #:use-module (gnu services)> + #:use-module (gnu services base)> + #:use-module (gnu services linux)> + #:use-module (gnu services mcron)> + #:use-module (gnu services shepherd)> + #:use-module (gnu system mapped-devices)> + #:use-module (guix gexp)> + #:use-module (guix packages)> + #:use-module (guix records)> + #:export (zfs-service-type> +> + zfs-configuration> + zfs-configuration?> + zfs-configuration-kernel> + zfs-configuration-base-zfs> + zfs-configuration-base-zfs-auto-snapshot> + zfs-configuration-dependencies> + zfs-configuration-auto-mount?> + zfs-configuration-auto-scrub> + zfs-configuration-auto-snapshot?> + zfs-configuration-auto-snapshot-keep> +> + %zfs-zvol-dependency))> +> +(define-record-type* <zfs-configuration>> + zfs-configuration> + make-zfs-configuration> + zfs-configuration?> +> + ; linux-libre kernel you want to compile the base-zfs module for.
If a comment is on a line of its own, it's normally ";;".";" is usually only for comments on the same line as some expression.
Toggle quote (19 lines)> + (kernel zfs-configuration-kernel)

> + ; the OpenZFS package that will be modified to compile for the> + ; given kernel.> + (base-zfs zfs-configuration-base-zfs> + (default zfs))> +> + ; the zfs-auto-snapshot package that will be modified to compile> + ; for the given kernel.> + (base-zfs-auto-snapshot zfs-configuration-base-zfs-auto-snapshot> + (default zfs-auto-snapshot))> +> + ; list of <mapped-device> or <file-system> objects that must be> + ; opened/mounted before we import any ZFS pools.> + (dependencies zfs-configuration-dependencies> + (default '()))> +> + ; #t if mountable datasets are to be mounted automatically.
Gramatically simpler: #t to mount all mountable datasets by default.
Toggle quote (14 lines)> + ; #f if not mounting.> + ; #t is the expected behavior on other operating systems, the> + ; #f is only supported for "rescue" operating systems where> + ; the user wants lower-level control of when to mount.> + (auto-mount? zfs-configuration-auto-mount?> + (default #t))> +> + ; 'weekly for weekly scrubbing, 'monthly for monthly scrubbing, an> + ; mcron time specification that can be given to `job`, or #f to> + ; disable.> + (auto-scrub zfs-configuration-auto-scrub> + (default 'weekly))> +> + ; #t if auto-snapshot is default (and `com.sun:auto-snapshot=false`
--> #t if auto-snapshot is the defaultor--> #t to auto-snapshot by default
Toggle quote (13 lines)> + ; disables auto-snapshot per dataset), #f if no auto-snapshotting> + ; is default (and `com.sun:auto-snapshot=true` enables auto-snapshot> + ; per dataset).> + (auto-snapshot? zfs-configuration-auto-snapshot?> + (default #t))> +> + ; association list of symbol-number pairs to indicate the number> + ; of automatic snapshots to keep for each of 'frequent, 'hourly,> + ; 'daily, 'weekly, and 'monthly.> + ; e.g. '((frequent . 8) (hourly . 12))> + (auto-snapshot-keep zfs-configuration-auto-snapshot-keep> + (default '())))
Seems a reasonable structure, though I'm not familiar with ZFS.
Toggle quote (121 lines)> +(define %default-auto-snapshot-keep> + '((frequent . 4)> + (hourly . 24)> + (daily . 31)> + (weekly . 8)> + (monthly . 12)))> +> +(define %auto-snapshot-mcron-schedule> + '((frequent . "0,15,30,45 * * * *")> + (hourly . "0 * * * *")> + (daily . "0 0 * * *")> + (weekly . "0 0 * * 7")> + (monthly . "0 0 1 * *")))> +> +;; A synthetic and unusable MAPPED-DEVICE intended for use when> +;; the user has created a mountable filesystem inside a ZFS> +;; zvol and wants it mounted inside the configuration.scm.> +(define %zfs-zvol-dependency> + (mapped-device> + (source '())> + (targets '("zvol/*"))> + (type #f)))> +> +(define (make-zfs-package conf)> + (let ((kernel (zfs-configuration-kernel conf))> + (base-zfs (zfs-configuration-base-zfs conf)))> + (package> + (inherit base-zfs)> + (arguments (cons* #:linux kernel> + (package-arguments base-zfs))))))> +> +(define (make-zfs-auto-snapshot-package conf)> + (let ((zfs (make-zfs-package conf))> + (base-zfs-auto-snapshot (zfs-configuration-base-zfs-auto-snapshot conf)))> + (package> + (inherit base-zfs-auto-snapshot)> + (inputs `(("zfs" ,zfs))))))> +> +(define (zfs-loadable-modules conf)> + (list (list (make-zfs-package conf) "module")))> +> +(define (zfs-shepherd-services conf)> + (let* ((zfs-package (make-zfs-package conf))> + (zpool (file-append zfs-package "/sbin/zpool"))> + (zfs (file-append zfs-package "/sbin/zfs"))> + (zvol_wait (file-append zfs-package "/bin/zvol_wait"))> + (scheme-modules `((srfi srfi-1)> + (srfi srfi-34)> + (srfi srfi-35)> + (rnrs io ports)> + ,@%default-modules)))> + (define zfs-scan> + (shepherd-service> + (provision '(zfs-scan))> + (requirement `(root-file-system> + kernel-module-loader> + udev> + ,@(map dependency->shepherd-service-name> + (zfs-configuration-dependencies conf))))> + (documentation "Scans for and imports ZFS pools.")> + (modules scheme-modules)> + (start #~(lambda _> + (guard (c ((message-condition? c)> + (format (current-error-port)> + "zfs: error importing pools: ~s~%"> + (condition-message c))> + #f))> + ; TODO: optionally use a cachefile.> + (invoke #$zpool "import" "-a" "-N"))))> + ;; Why not one-shot? Because we don't really want to rescan> + ;; this each time a requiring process is restarted, as scanning> + ;; can take a long time and a lot of I/O.> + (stop #~(const #f))))> +> + (define device-mapping-zvol/*> + (shepherd-service> + (provision '(device-mapping-zvol/*))> + (requirement '(zfs-scan))> + (documentation "Waits for all ZFS ZVOLs to be opened.")> + (modules scheme-modules)> + (start #~(lambda _> + (guard (c ((message-condition? c)> + (format (current-error-port)> + "zfs: error opening zvols: ~s~%"> + (condition-message c))> + #f))> + (invoke #$zvol_wait))))> + (stop #~(const #f))))> +> + (define zfs-auto-mount> + (shepherd-service> + (provision '(zfs-auto-mount))> + (requirement '(zfs-scan))> + (documentation "Mounts all non-legacy mounted ZFS filesystems.")> + (modules scheme-modules)> + (start #~(lambda _> + (guard (c ((message-condition? c)> + (format (current-error-port)> + "zfs: error mounting file systems: ~s~%"> + (condition-message c))> + #f))> + ;; Output to current-error-port, otherwise the> + ;; user will not see any prompts for passwords> + ;; of encrypted datasets.> + ;; XXX Maybe better to explicitly open /dev/console ?> + (with-output-to-port (current-error-port)> + (lambda ()> + (invoke #$zfs "mount" "-a" "-l"))))))> + (stop #~(lambda _> + ;; Make sure that Shepherd does not have a CWD that> + ;; is a mounted ZFS filesystem, which would prevent> + ;; unmounting.> + (chdir "/")> + (invoke #$zfs "unmount" "-a" "-f")))))> +> + `(,zfs-scan> + ,device-mapping-zvol/*> + ,@(if (zfs-configuration-auto-mount? conf)> + `(,zfs-auto-mount)> + '()))))
Service definition seems reasonable, but agan, I'm no ZFS expert.
Toggle quote (44 lines)> +(define (zfs-user-processes conf)> + (if (zfs-configuration-auto-mount? conf)> + '(zfs-auto-mount)> + '(zfs-scan)))> +> +(define (zfs-mcron-auto-snapshot-jobs conf)> + (let* ((user-auto-snapshot-keep (zfs-configuration-auto-snapshot-keep conf))> + ;; assoc-ref has earlier entries overriding later ones.> + (auto-snapshot-keep (append user-auto-snapshot-keep> + %default-auto-snapshot-keep))> + (auto-snapshot? (zfs-configuration-auto-snapshot? conf))> + (zfs-auto-snapshot-package (make-zfs-auto-snapshot-package conf))> + (zfs-auto-snapshot (file-append zfs-auto-snapshot-package> + "/sbin/zfs-auto-snapshot")))> + (map> + (lambda (label)> + (let ((keep (assoc-ref auto-snapshot-keep label))> + (sched (assoc-ref %auto-snapshot-mcron-schedule label)))> + #~(job '#$sched> + (string-append #$zfs-auto-snapshot> + " --quiet --syslog "> + " --label=" #$(symbol->string label)> + " --keep=" #$(number->string keep)> + " //"))))> + '(frequent hourly daily weekly monthly))))> +> +(define (zfs-mcron-auto-scrub-jobs conf)> + (let* ((zfs-package (make-zfs-package conf))> + (zpool (file-append zfs-package "/sbin/zpool"))> + (auto-scrub (zfs-configuration-auto-scrub conf))> + (sched (cond> + ((eq? auto-scrub 'weekly) "0 0 * * 7")> + ((eq? auto-scrub 'monthly) "0 0 1 * *")> + (else auto-scrub))))> + (list> + #~(job '#$sched> +> + ;; Suppress errors: if there are no ZFS pools, the> + ;; scrub will not be given any arguments, which makes> + ;; it error out.> + (string-append "(" #$zpool " scrub `" #$zpool " list -o name -H` "> + "> /dev/null 2>&1) "> + "|| exit 0")))))
I think people here would prefer something in Guile, e.g. using 'invoke'.For redirecting to "/dev/null", maybe with-output-to-port, with-error-to-portand (%make-void-port "w0") can be used. Doing this in Guile would avoid anypotential issues with whitespace and the escape rules for bash.
Toggle quote (33 lines)> +(define (zfs-mcron-jobs conf)> + (append (zfs-mcron-auto-snapshot-jobs conf)> + (if (zfs-configuration-auto-scrub conf)> + (zfs-mcron-auto-scrub-jobs conf)> + '())))> +> +(define zfs-service-type> + (service-type> + (name 'zfs)> + (extensions> + (list ;; Install OpenZFS kernel module into kernel profile.> + (service-extension linux-loadable-module-service-type> + zfs-loadable-modules)> + ;; And load it.> + (service-extension kernel-module-loader-service-type> + (const '("zfs")))> + ;; Make sure ZFS pools and datasets are mounted at> + ;; boot.> + (service-extension shepherd-root-service-type> + zfs-shepherd-services)> + ;; Make sure user-processes don't start until> + ;; after ZFS does.> + (service-extension user-processes-service-type> + zfs-user-processes)> + ;; Install automated scrubbing and snapshotting.> + (service-extension mcron-service-type> + zfs-mcron-jobs)> +> + ;; Install ZFS management commands in the system> + ;; profile.> + (service-extension profile-service-type> + (compose list make-zfs-package))
FWIW, I (YMMV) expect that the only packages guix installs in the systemprofile are those listed in the 'packages' field whenever feasible.Though adding zfs to the system profile when ZFS file systems seemsreasonable.
Apparently the consensus seems to be that putting the file system utilitiesinto the system profile automatically is useful and not problematic:https://issues.guix.gnu.org/issue/39505, https://issues.guix.gnu.org/49128 ,so putting it there is probably ok?
Toggle quote (5 lines)> + ;; Install ZFS udev rules.> + (service-extension udev-service-type> + (compose list make-zfs-package))))> + (description "Installs ZFS, an advanced filesystem and volume manager.")))
Also, a more general comment: ideally most top-level procedures have a docstringor comment explaining what they are for and how they are used.
Overall, the code seems to be quite reasonable (I haven't tested it though)with a minor issues.
If it weren't for the GPL violation, I was going to recommend that the patchwould be merged (after resolving the minor issues) --- there is the issue thatthere are very few people familiar enough with ZFS to review this, so I thoughtit might be reasonable to consider you ‘the guix expert on ZFS’, with the ideathat if someone has ZFS problems, they would be directed to you, and if someonehas ZFS-related patches, you would try to review them, and that over time, guixwill attract some reviewers understanding ZFS well-enough to review ZFS things,so after some time, you wouldn't have do all that on your own.
The GPL violation is very unfortunate. It would have been nice to have someZFS support in Guix.
Greetings,Maxime
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYTE6pRccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7m4EAQC3+kJXHerw2bosfiGR8KvicdZpP7Gc/H63OE4cJpAE4wD/StWqivX/tD02Iq4KlMI+m2b6ut3aMvkiBuc+FhRIOwo==Pcik-----END PGP SIGNATURE-----

M
M
Mason Loring Bliss wrote on 2 Sep 23:24 +0200
Gaslighting
(address . 45692@debbugs.gnu.org)
20210902212359.GU6420@blisses.org
Maxime writes, > The GPL violation is very unfortunate. It would have been nice to > have some ZFS support in Guix.
There is no GPL violation. Gaslighting in an attempt to sabotage theadoption of high-quality free software is somewhat poor form and not at alluseful.
To be clear, the one singular thing that would be a GPL violation would beGuix building Linux with ZFS built in and then distributing that binary.Users can build ZFS into their kernels and use them and that's fine as longas they don't distribute them. Guix can script this to make it easy forusers to build ZFS into their kernels and that would not be a violation.The only possible violation is distribution of a binary Linux kernel withZFS compiled into it.
Hope that helps.
-- Mason Loring Bliss mason@blisses.org Ewige Blumenkraft!(if awake 'sleep (aref #(sleep dream) (random 2))) -- Hamlet, Act III, Scene I
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEEXtBZz1axB5rEDCEnrJXcHbvJVUFAmExQO0ACgkQnrJXcHbvJVVz6A//ZGgw66e0hV0UbUOpIPoaj4gjaWPOReW44xgRyi8dP5qtwrpmzBIaCvBh5ONEnP8cOVQWJlF2bXk05pMiAp7c+n66ZY6IupEfMDbxqeCsHrUfJ9jAKWxEgJCK26EnK0qbA+RCPloHnkFiKYbgDsvIqB5Rv9c76XT8zL3QCw4Eo0gcJX5Gcm4CKPC4uQCd7rEDoowFJDZVfC/z+BtV6fzmomLpuceUlA3jvIIK/c2/umsdauwoB33Q8zaVhCt6CffpZEX7XmVv4wWJGKcCcojUDiqaBXN+8K7Z5aPNJFqLYMj6Vupvld2R4oyorq9mlqPu5jEmwF9I6N0QIO7F/4ndLGxB685Lzm3aXmzd4KD1HBQEtzUvuAustunnCENyh8mtdftM4sWhsFLKd4OSNYzHNDGG/2mm9JFDLqiHGP0CBZpr5JVQC9iBgRYqBn+IlUl9YmfRkMUUJOBIYrKfrQQCga3NPgvskCh6etiduXd48WVtnfIXot8BCeGTeJ49Q5r+D0V/Sx1wyVhgva+k1xkO3LkiQKSUob4kPumdlSUL4I+XYuLV3WtdA536crQlXeJIEAiYRXUZYNdK0xsRNj4HUE5E5FQhWmFQAuPR6f26CWbD2qjqh+/+Tm5/lVHDIFnqZAPsdXKDNye3fCeZNfZiZ7TUtXMw78/7KKoX3P4JgGw==Td5w-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 3 Sep 00:22 +0200
Re: [bug#45692] [PATCH v4 3/3] gnu: Add ZFS service type.
d89a03fc0603bd173476527a96cdb50f02454460.camel@telenet.be
Maxime Devos schreef op do 02-09-2021 om 22:57 [+0200]:
Toggle quote (5 lines)> Hi,> > Some comments on the code. Spoiler: the code is presumably good,> but there's a GPL violation.
Nevermind, it's apparently less of an issue than I expected?See the links to IRC at https://issues.guix.gnu.org/50347.
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYTFOrxccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hI3AQD9KtkmuVjtu3HdqJTsigvecanY0v6UU0A1ogwfdSD3YQEA/0kXZFSV+NbwfSXgEO3k0/I9AqQMcCpGSEm33Zx/AwQ==HNvs-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 3 Sep 14:22 +0200
Re: [bug#45692] Gaslighting
1061ab2fff72300fee6d29760b3c5e92c80969cb.camel@telenet.be
Mason Loring Bliss schreef op do 02-09-2021 om 17:24 [-0400]:
Toggle quote (9 lines)> Maxime writes,> > > The GPL violation is very unfortunate. It would have been nice to> > have some ZFS support in Guix.> > There is no GPL violation. Gaslighting in an attempt to sabotage the> adoption of high-quality free software is somewhat poor form and not at all> useful.
Indeed, gaslighing is poor form and not at all useful. But why are you suggestingI'm gaslightling here? Did you just read these last two sentences of the e-mail?If you read all of it, you'd have seen my explanation of why I believe there'sa GPL violation, and two relevant links to articles by SFLC and FSF.
Also see IRC logs:https://logs.guix.gnu.org/guix/2021-09-02.loghttps://logs.guix.gnu.org/guix/2021-09-03.log
While ultimately it's a matter for the courts to decide on,I believe I've reasonable grounds to believe it's a GPL violationand explained why, so I don't see any gaslightling here.
And I'm not ‘sabotaging’ anything. In fact, I'm _helping_ adoption of ZFS,by reviewing the patch and giving some suggestions. There is just the practicalproblem of the in-my-eyes probable GPL violation (your opinion on whether it'sa GPL violation might vary).
Toggle quote (8 lines)> To be clear, the one singular thing that would be a GPL violation would be> Guix building Linux with ZFS built in and then distributing that binary.> Users can build ZFS into their kernels and use them and that's fine as long> as they don't distribute them. Guix can script this to make it easy for> users to build ZFS into their kernels and that would not be a violation.> The only possible violation is distribution of a binary Linux kernel with> ZFS compiled into it.
See the previous mail and the IRC logs for why I find this reasoning ratherflimsy.
Toggle quote (2 lines)> Hope that helps.
No, your libel about ‘Maxime is gaslighting’ doesn't help. And your paragraphabout why you think there's no GPL violation here is nothing I haven't read before.
I probably won't be reading and replying to your mails anymore.
Bye,Maxime
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYTITdRccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7rHFAP40RtS29HE3qLvCXcNA+nJvJN0e50xYoGAaCo2P6HtwMwD/bQpz/sEfYZbo5NC8GEydG6suvedO+esfi69siO998wM==dfiW-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 3 Sep 14:41 +0200
Re: [bug#45692] [PATCH v4 3/3] gnu: Add ZFS service type.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
J8OknoQ6k_2BTBa5PUlSVldvmdbPN81V3IMT3ipJ3XfgPwfUV8TeC5K-8lIWVOHn4NkDSJnHu1X3hYXAT9AAJNeTZDTNwZtgpmEPfpOEmow=@protonmail.com
Greetings Maxime,
Toggle quote (9 lines)> Maxime Devos schreef op do 02-09-2021 om 22:57 [+0200]:>> > Hi,> > Some comments on the code. Spoiler: the code is presumably good,> > but there's a GPL violation.>> Nevermind, it's apparently less of an issue than I expected?> See the links to IRC at https://issues.guix.gnu.org/50347.
Note that this patch does ***not*** add ZFS to the Guix project.Instead, this patch creates a convenient service that uses the existing `zfs` package and builds the user system so that the user downloads the ZFS source code, compiles it, and links it to the kernel on the system.
In #50347, you refer to this analysis: https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/
I quote this part:
Toggle quote (12 lines)> ## Is The Analysis Different With Source-Only Distribution?>> ...>> Pure distribution of source with no binaries is undeniably different. When> distributing source code and no binaries, requirements in those sections of> GPLv2 and CDDLv1 that cover modification and/or binary (or “Executable”, as> CDDLv1 calls it) distribution do not activate. Therefore, the analysis is> simpler, and we find no specific clause in either license that prohibits> source-only redistribution of Linux and ZFS, even on the same distribution> media.
This is in line with the analysis already quoted in the documentation added: https://www.fsf.org/licensing/zfs-and-linux
Guix does ***not*** distribute any binaries; see the file `gnu/packages/file-systems.scm` in the **current** `master` branch of Guix:
Toggle quote (4 lines)> `(;; The ZFS kernel module should not be downloaded since the license> ;; terms don't allow for distributing it, only building it locally.> #:substitutable? #f
Note that the above code ***predates*** this patch: fe338d7f009 (Efraim Flashner 2019-12-19 11:47:49 +0200 1188)Also CCing Efraim here --- presumably he had some choice arguments about how `zfs` got into `gnu/packages/file-systems.scm` in the first place.
If the CDDL-GPL incompatibility is problematic, then why is it only being brought up now, why did it not get brought up in 2019, when Efraim was submitting the patch that put the ZFS package into Guix in the first place?
The code in this patch does not do anything that the user cannot do with their own scripts (indeed, I prototyped much of the code in my own `configuration.scm`).The code in this patch also does not link, directly or indirectly, into the ZFS kernel module.At worst, the code in this patch executes the binaries that are the output of compilation, but since it is invoked as a separate binary running in a separate process, there is no legal basis for considering this as "linking", as opposed to merely invoking a separate program (if merely invoking a separate program was enough to "link", then Windows cannot run any free software).
Your referred document then makes some speculation that even source distribution might be problematic.However, Guix does *not* even distribute sources, by my understanding --- the Guix build downloads from https://github.com/openzfs/zfs/releases/without going through any Guix servers, so this should be even less of a problem.If anyone is at legal risk, it is github, not Guix, for distributing the sources --- and it would be very strange for Oracle to not go after github for distributing source code that is intended to be linked to GPL code, but go after Guix; Guix is a much more niche project than the openzfs/zfs project on github.(my understanding is that Oracle implicitly allows the existence of openzfs/zfs, even has some maintainers of the project on their payroll, so it would be very strange to go after software that just downloads the source code from that project, compiles it, links it, and does ***not*** distribute it (`#:substitutable? #f`))
My understanding (and my argument) is that the already-existing code introduced in fe338d7f009 does not represent a distribution of any ZFS code, not even source:
* The existing fe338d7f009 code tells Guix to download from a github server, not from Guix.* The existing fe338d7f009 code specifically tells Cuirass to not provide a binary substitute, so Guix end-users must download from github, not from any Guix servers. * Thus, even if source distribution is legally problematic, Guix does not even distribute the source; the existing fe338d7f009 code just downloads it from an existing distributor. * That Oracle tolerates the continued existence of https://github.com/openzfs/zfsundermines any legal argument Oracle might make if somebody else builds a script that downloads ZFS from https://github.com/openzfs/zfd and compiles it and links it to GPL code locally without redistributing; that project already contains human-readable instructions on how to download, build, and link ZFS to a Linux kernel, the existing fe338d7f009 code merely translated that text to a machine-readable program. If Oracle thinks this is legally problematic, they should have demanded shutdown of the https://github.com/openzfs/zfsproject first, a much more popular project than Guix.* The actual code in this patch does not directly invoke the ZFS kernel module. * The actual code in this patch does link the ZFS kernel module to the kernel of the *local* system, but does not make this linked version available to others; it is only available locally on the system of the user that invokes this actual code. Again, this is merely a translation (to machine-readable text instructions) of human-readable text instructions on how to link the ZFS kernel module to the Linux kernel, text available publicly on https://github.com/openzfs/zfs. * The actual code in this patch invokes the ZFS tools as separate programs, thus does not link with them.
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 4 Sep 20:58 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
cS37T52dLINzNDyX2L5AnMaw2a0o1G1M3r16AKa9Piya2m5WzPRF-hThegq8C0vQ3KsfNIxeXCtMTKoVRd4KWsFbFPJtdhf0aqhvEPjXss8=@protonmail.com
Hello again Maxime,
Toggle quote (27 lines)> > +OpenZFS currently only supports Linux-Libre and is not available on the> > +Hurd.> > +> > +OpenZFS is free software; unfortunately its license is incompatible with> > +the GNU General Public License (GPL), the license of the Linux kernel,> > +which means they cannot be distributed together. However, as a user,> > +you can choose to build ZFS and use it together with Linux; you can> > +even rely on Guix to automate this task. See> > +@uref{https://www.fsf.org/licensing/zfs-and-linux, this analysis by> > +the Free Software Foundation} for more information.>> That analysis says explicitely that the CDDL is incompatible with the GPL,> and that they cannot be legally linked together. E.g., see the second quoted> paragraph:>> ‘A copyleft license, including any version of the GNU GPL or GNU AGPL, requires> augmented versions to be free under the same license -- the same requirement it> applies to modification of the code.1 I wrote a copyleft license for GNU programs> to ensure that all users of all versions of them would get the freedoms I intended> to give them.>> It is not enough to require that the combined program be free software somehow. It> must be released, as a whole, under the original copyleft license, to ensure that:> (1) subsequent users get the exact same freedoms and (2) subsequent intermediaries> do not get more opportunity than first-stage intermediaries to make the program nonfree.’

I think the key word you miss here is "released", i.e. "It must be released, as a whole, under the original copyleft license."
Looking at the GPLv2, the word "release" is never used, however I believe the "release" word in the FSF analysis would be considered as a synonym of "distribute" in this context.
The GPLv2 mentions "distribute" many times, but provides no definition of the word.My understanding is that "distribute" used in GPL means "to provide or make available to at least one person that asks for a copy from you, via some medium".The GPLv2 imposes many restrictions on the ability to "distribute", so it seems reasonable to consider it an important point.
Now, as I have pointed out, the existing package definition in `gnu/packages/file-systems.scm` specifically disables making a binary copy ("substitute" in Guix parlance) available.In addition, my understanding is that when compiling from source, the `source` field is what is used, and the `source` field in the `gnu/packages/file-systems.scm` refers to github.com, not any Guix server.
There is no text in GPLv2 which restricts compilation.However, it can be argued that compilation is a form of translation from source code to machine-executable binary, and the text does mention "translation is included without limitation in the term 'modification'".
GPLv2 restricts modification with three terms:
a. You should have prominent notices on modified files.b. Extra restriction on copies you ***distribute*** or ***publish***.c. Extra restriction if the program is interactive and prints copyright notices normally.
(a) does not apply since the linking process used (a form of dynamic linking) does not actually modify any files; presumably only in-memory tables or some such are modified.(b) does not apply if "distribute" is not what is being done by Guix here.(c) does not apply since the Linux kernel is not interactive (and even so, does not print copyright notices, not even in debug logs).
On the CDDL side, neither "compile" nor "translate" is ever used, but for completeness let us consider compile == translate == modify.Modifications are specifically allowed under conditions in section 3.However, again, section 3 is titled "distribution obligations", meaning they only apply on *distribution*.
So I think the issue here really is: Does Guix "distribute" the ZFS linked with Linux?My understanding is that the mere existence of code to perform that linking does not in fact *distribute* the code (linking is not the same as distributing); I believe the key point of "distribute" is that a third party gets a copy. And at least, the code I added in this patch does not provide any copy of the compiled code to anyone else; it just stores it on the local machine's disk, in a cpio archive that is used in the system's bootup. The copy, in the execution of the code I added, is never provided to anyone else, so I think my patch is unproblematic
Quick question: does `guix publish` respect `#:substitutable? #f`? If `guix publish` respects `#:substitutable? #f` then it seems to me that even the point "Guix should make at least an attempt to warn users of possibly legal gray areas when distributing" does not apply, too: Guix by itself would not (should not, really; that should be the point of `#:substituable? #f`) publish the compiled code anyway, users who specifically want to publish ZFS and Linux linked together would need to modify the `guix publish` code, and such a user would be running a fork of Guix, not Guix itself, thus should be made aware of this.
Indeed, if `guix publish` does *not* respect `#:substitutable? #f`, I think it would be more effective to protect users against potential legal gray areas for `guix publish` to respect that flag, and for us to then audit existing non-GPLv2-compatible kernel modules and ensure they are `#:substitutable? #f`, then to put up a warning; a warning might be overlooked, but an outright refusal to publish cannot.
Thanksraid5atemyhomework
X
X
Xinglu Chen wrote on 4 Sep 23:19 +0200
87k0jw3upm.fsf@yoctocell.xyz
On Sun, Jul 25 2021, raid5atemyhomework via Guix-patches via wrote:
Toggle quote (56 lines)> Hello nonexistent reviewer,>> I updated this patch to latest `origin/master` because the previous> version has bitrotted and will not `git am` cleanly anymore.>> There are no changes relative to v3, just rebased it so that the patch applies cleanly.>> Testing has been very minimal: I created a VM with the added service,> then ran it in a VM session that included additional devices (qcow2> files) from a previous VM run that formatted those as devices of a ZFS> pool, and confirmed that the new VM could read it and manage that> pool.>>> Is there any chance this will get reviewed or should I just not bother> and move on with my life and forget about Guix?>>> At this point as well, I would like to point out what I think is a> failing of how the Guix distribution is managed.>> Guix does not assign any particular committers to any particular tasks or areas.> The intent is that any committer can review and commit any particular patch.>> However, this fails as the Guix project has grown.>> No single committer wants to review ***all*** the available patches,> and this is understandable, as the Guix project has grown> significantly and includes a wide variety of people with diverging> interests and as an open-source project you cannot really require that> particular people look at particular things. Unfortunately, I do not> know *who* committers are, and more importantly, *which* committer> might be interested in this ZFS service type. Because "any committer> can review and commit any patch!!" there is no particular list or> table I can refer to, to figure out who might be useful to ping for> this patchset.>> At the same time, because no committer is interested in *all* patches> I cannot just ping some particular person and expect to somehow get on> some list somewhere that tells me "you will be the 48486th patch that> will be reviewed by <foo> who is interested in all patches".>>> It is very discouraging to work on this code for a few weeks, release> it, not get any reviews, and end up in a situation where I have to> make annoying small changes just to keep the patch from bitrotting.>> I understand that there are few possible reviewers, but if potential> new contributors get discouraged from contributing because they do not> see their code actually getting in, then you really cannot expect the> number of reviewers to increase, either.>> I think it would be nice if I could at least be told some number of> people who *might* be interested in this patch, or just throw the> towel and not bother.
You might want to bring up the topic of subsystem maintainers on theguix-devel mailing list to get some more attention.
I am just some random ZFS user, so maybe take my comments with a pinchof salt.
I haven’t followed the thread that closely, so apologies if some of myquestions have already been answered.
Toggle quote (184 lines)>> Thanks> raid5atemyhomework>>> From 5351aa7c1c14d4fea032adad895c436e02d1f261 Mon Sep 17 00:00:00 2001> From: raid5atemyhomework <raid5atemyhomework@protonmail.com>> Date: Mon, 22 Mar 2021 16:26:28 +0800> Subject: [PATCH] gnu: Add ZFS service type.>> * gnu/services/file-systems.scm: New file.> * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.> * gnu/services/base.scm: Export dependency->shepherd-service-name.> * doc/guix.texi (ZFS File System): New subsection.> ---> doc/guix.texi | 351 ++++++++++++++++++++++++++++++++++> gnu/local.mk | 2 +> gnu/services/base.scm | 4 +-> gnu/services/file-systems.scm | 295 ++++++++++++++++++++++++++++> 4 files changed, 651 insertions(+), 1 deletion(-)> create mode 100644 gnu/services/file-systems.scm>> diff --git a/doc/guix.texi b/doc/guix.texi> index b3c16e6507..e21c47d7ca 100644> --- a/doc/guix.texi> +++ b/doc/guix.texi> @@ -94,6 +94,7 @@ Copyright @copyright{} 2021 Xinglu Chen@*> Copyright @copyright{} 2021 Raghav Gururajan@*> Copyright @copyright{} 2021 Domagoj Stolfa@*> Copyright @copyright{} 2021 Hui Lu@*> +Copyright @copyright{} 2021 raid5atemyhomework@*>> Permission is granted to copy, distribute and/or modify this document> under the terms of the GNU Free Documentation License, Version 1.3 or> @@ -14265,6 +14266,356 @@ a file system declaration such as:> compress-force=zstd,space_cache=v2"))> @end lisp>> +> +@node ZFS File System> +@subsection ZFS File System> +> +Support for ZFS file systems is provided on Guix by the OpenZFS project.> +OpenZFS currently only supports Linux-Libre and is not available on the> +Hurd.> +> +OpenZFS is free software; unfortunately its license is incompatible with> +the GNU General Public License (GPL), the license of the Linux kernel,> +which means they cannot be distributed together. However, as a user,> +you can choose to build ZFS and use it together with Linux; you can> +even rely on Guix to automate this task. See> +@uref{https://www.fsf.org/licensing/zfs-and-linux, this analysis by> +the Free Software Foundation} for more information.> +> +As a large and complex kernel module, OpenZFS has to be compiled for a> +specific version of Linux-Libre. At times, the latest OpenZFS package> +available in Guix is not compatible with the latest Linux-Libre version.> +Thus, directly installing the @code{zfs} package can fail.> +> +Instead, you are recommended to select a specific older long-term-support> +Linux-Libre kernel. Do not use @code{linux-libre-lts}, as even the> +latest long-term-support kernel may be too new for @code{zfs}. Instead,> +explicitly select a specific older version, such as @code{linux-libre-5.10},> +and upgrade it manually later as new long-term-support kernels become> +available that you have confirmed is compatible with the latest available> +OpenZFS version on Guix.> +> +For example, you can modify your system configuration file to a specific> +Linux-Libre version and add the @code{zfs-service-type} service.> +> +@lisp> +(use-modules (gnu))> +(use-package-modules> + #;@dots{}> + linux)> +(use-service-modules> + #;@dots{}> + file-systems)> +> +(define my-kernel linux-libre-5.10)> +> +(operating-system> + (kernel my-kernel)> + #;@dots{}> + (services> + (cons* (service zfs-service-type> + (zfs-configuration> + (kernel my-kernel)))> + #;@dots{}> + %desktop-services))> + #;@dots{})> +@end lisp> +> +@defvr {Scheme Variable} zfs-service-type> +This is the type for a service that adds ZFS support to your operating> +system. The service is configured using a @code{zfs-configuration}> +record.> +> +Here is an example use:> +> +@lisp> +(service zfs-service-type> + (zfs-configuration> + (kernel linux-libre-5.4)))> +@end lisp> +@end defvr> +> +@deftp {Data Type} zfs-configuration> +This data type represents the configuration of the ZFS support in Guix> +System. Its fields are:> +> +@table @asis> +@item @code{kernel}> +The package of the Linux-Libre kernel to compile OpenZFS for. This field> +is always required. It @emph{must} be the same kernel you use in your> +@code{operating-system} form.> +> +@item @code{base-zfs} (default: @code{zfs})> +The OpenZFS package that will be compiled for the given Linux-Libre kernel.> +> +@item @code{base-zfs-auto-snapshot} (default: @code{zfs-auto-snapshot})> +The @code{zfs-auto-snapshot} package to use. It will be modified to> +specifically use the OpenZFS compiled for your kernel.> +> +@item @code{dependencies} (default: @code{'()})> +A list of @code{<file-system>} or @code{<mapped-device>} records that must> +be mounted or opened before OpenZFS scans for pools to import. For example,> +if you have set up LUKS containers as leaf VDEVs in a pool, you have to> +include their corresponding @code{<mapped-ddevice>} records so that OpenZFS> +can import the pool correctly at bootup.> +> +@item @code{auto-mount?} (default: @code{#t})> +Whether to mount datasets with the ZFS @code{mountpoint} property automatically> +at startup. This is the behavior that ZFS users usually expect. You might> +set this to @code{#f} for an operating system intended as a ``rescue'' system> +that is intended to help debug problems with the disks rather than actually> +work in production.> +> +@item @code{auto-scrub} (default: @code{'weekly})> +Specifies how often to scrub all pools. Can be the symbols @code{'weekly} or> +@code{'monthly}, or a schedule specification understood by> +@xref{mcron, mcron job specifications,, mcron, GNU@tie{}mcron}, such as> +@code{"0 3 * * 6"} for ``every 3AM on Saturday''.> +It can also be @code{#f} to disable auto-scrubbing (@strong{not recommended}).> +> +The general guideline is to scrub weekly when using consumer-quality drives, and> +to scrub monthly when using enterprise-quality drives.> +> +@code{'weekly} scrubs are done on Sunday midnight, while @code{monthly} scrubs> +are done on midnight on the first day of each month.> +> +@item @code{auto-snapshot?} (default: @code{#t})> +Specifies whether to auto-snapshot by default. If @code{#t}, then snapshots> +are automatically created except for ZFS datasets with the> +@code{com.sun:auto-snapshot} ZFS vendor property set to @code{false}.> +> +If @code{#f}, snapshots will not be automatically created, unless the ZFS> +dataset has the @code{com.sun:auto-snapshot} ZFS vendor property set to> +@code{true}.> +> +@item @code{auto-snapshot-keep} (default: @code{'()})> +Specifies an association list of symbol-number pairs, indicating the number> +of automatically-created snapshots to retain for each frequency type.> +> +If not specified via this field, by default there are 4 @code{frequent}, 24> +@code{hourly}, 31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly} snapshots.> +> +For example:> +> +@lisp> +(zfs-configuration> + (kernel my-kernel)> + (auto-snapshot-keep> + '((frequent . 8)> + (hourly . 12))))> +@end lisp> +> +The above will keep 8 @code{frequent} snapshots and 12 @code{hourly} snapshots.> +@code{daily}, @code{weekly}, and @code{monthly} snapshots will keep their> +defaults (31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly}).> +> +@end table> +@end deftp
IIUC, there is not way specify ZFS pools in the ‘file-systems’ field ofan <operating-system> record. Does this mean that ZFS as the root filesystem is not supported, and if so, is there a particular reason forthis?
Toggle quote (99 lines)> +> +@subsubsection ZFS Auto-Snapshot> +> +The ZFS service on Guix System supports auto-snapshots as implemented in the> +Solaris operating system.> +> +@code{frequent} (every 15 minutes), @code{hourly}, @code{daily}, @code{weekly},> +and @code{monthly} snapshots are created automatically for ZFS datasets that> +have auto-snapshot enabled. They will be named, for example,> +@code{zfs-auto-snap_frequent-2021-03-22-1415}. You can continue to use> +manually-created snapshots as long as they do not conflict with the naming> +convention used by auto-snapshot. You can also safely manually destroy> +automatically-created snapshots, for example to free up space.> +> +The @code{com.sun:auto-snapshot} ZFS property controls auto-snapshot on a> +per-dataset level. Sub-datasets will inherit this property from their parent> +dataset, but can have their own property.> +> +You @emph{must} set this property to @code{true} or @code{false} exactly,> +otherwise it will be treated as if the property is unset.> +> +For example:> +> +@example> +# zfs list -o name> +NAME> +tank> +tank/important-data> +tank/tmp> +# zfs set com.sun:auto-snapshot=true tank> +# zfs set com.sun:auto-snapshot=false tank/tmp> +@end example> +> +The above will set @code{tank} and @code{tank/important-data} to be> +auto-snapshot, while @code{tank/tmp} will not be auto-snapshot.> +> +If the @code{com.sun:auto-snapshot} property is not set for a dataset> +(the default when pools and datasets are created), then whether> +auto-snapshot is done or not will depend on the @code{auto-snapshot?}> +field of the @code{zfs-configuration} record.> +> +There are also @code{com.sun:auto-snapshot:frequent},> +@code{com.sun:auto-snapshot:hourly}, @code{com.sun:auto-snapshot:daily},> +@code{com.sun:auto-snapshot:weekly}, and @code{com.sun:auto-snapshot:monthly}> +properties that give finer-grained control of whether to auto-snapshot a> +dataset at a particular schedule.> +> +The number of snapshots kept for all datasets can be overridden via the> +@code{auto-snapshot-keep} field of the @code{zfs-configuration} record.> +There is currently no support to have different numbers of snapshots to> +keep for different datasets.> +> +@subsubsection ZVOLs> +> +ZFS supports ZVOLs, block devices that ZFS exposes to the operating> +system in the @code{/dev/zvol/} directory. The ZVOL will have the same> +resilience and self-healing properties as other datasets on your ZFS pool.> +ZVOLs can also be snapshotted (and will be included in auto-snapshotting> +if enabled), which snapshots the state of the block device, effectively> +snapshotting the hosted file system.> +> +You can put any file system inside the ZVOL. However, in order to mount this> +file system at system start, you need to add @code{%zfs-zvol-dependency} as a> +dependency of each file system inside a ZVOL.> +> +@defvr {Scheme Variable} %zfs-zvol-dependency> +An artificial @code{<mapped-device>} which tells the file system mounting> +service to wait for ZFS to provide ZVOLs before mounting the> +@code{<file-system>} dependent on it.> +@end defvr> +> +For example, suppose you create a ZVOL and put an ext4 filesystem> +inside it:> +> +@example> +# zfs create -V 100G tank/ext4-on-zfs> +# mkfs.ext4 /dev/zvol/tank/ext4-on-zfs> +# mkdir /ext4-on-zfs> +# mount /dev/zvol/tank/ext4-on-zfs /ext4-on-zfs> +@end example> +> +You can then set this up to be mounted at boot by adding this to the> +@code{file-systems} field of your @code{operating-system} record:> +> +@lisp> +(file-system> + (device "/dev/zvol/tank/ext4-on-zfs")> + (mount-point "/ext4-on-zfs")> + (type "ext4")> + (dependencies (list %zfs-zvol-dependency)))> +@end lisp> +> +You @emph{must not} add @code{%zfs-zvol-dependency} to your> +@code{operating-system}'s @code{mapped-devices} field, and you @emph{must> +not} add it (or any @code{<file-system>}s dependent on it) to the> +@code{dependencies} field of @code{zfs-configuration}. Finally, you> +@emph{must not} use @code{%zfs-zvol-dependency} unless you actually> +instantiate @code{zfs-service-type} on your system.
I am not familiar with ZVOLs, so I can’t really comment on that.
Toggle quote (60 lines)> +@subsubsection Unsupported Features> +> +Some common features and uses of ZFS are currently not supported, or not> +fully supported, on Guix.> +> +@enumerate> +@item> +Shepherd-managed daemons that are configured to read from or write to ZFS> +mountpoints need to include @code{user-processes} in their @code{requirement}> +field. This is the earliest that ZFS file systems are assured of being> +mounted.> +> +Generally, most daemons will, directly or indirectly, require> +@code{networking}, or @code{user-processes}, or both. Most implementations> +of @code{networking} will also require @code{user-processes} so daemons that> +require only @code{networking} will also generally start up after> +@code{user-processes}. A notable exception, however, is> +@code{static-networking-service-type}. You will need to explicitly add> +@code{user-processes} as a @code{requirement} of your @code{static-networking}> +record.> +> +@item> +@code{mountpoint=legacy} ZFS file systems. The handlers for the Guix mounting> +system have not yet been modified to support ZFS, and will expect @code{/dev}> +paths in the @code{<file-system>}'s @code{device} field, but ZFS file systems> +are referred to via non-path @code{pool/file/system} names. Such file systems> +also need to be mounted @emph{after} OpenZFS has scanned for pools.> +> +You can still manually mount these file systems after system boot; what is> +only unsupported is mounting them automatically at system boot by specifying> +them in @code{<file-system>} records of your @code{operating-system}.> +> +@item> +@code{/home} on ZFS. Guix will create home directories for users, but this> +process currently cannot be scheduled after ZFS file systems are mounted.> +Thus, the ZFS file system might be mounted @emph{after} Guix has created> +home directories at boot, at which point OpenZFS will refuse to mount since> +the mountpoint is not empty. However, you @emph{can} create an ext4, xfs,> +btrfs, or other supported file system inside a ZVOL, have that depend on> +@code{%zfs-zvol-dependency}, and set it to mount on the @code{/home}> +directory; they will be scheduled to mount before the @code{user-homes}> +process.> +> +Similarly, other locations like @code{/var}, @code{/gnu/store} and so> +on cannot be reliably put in a ZFS file system, though they may be> +possible to create as other file systems inside ZVOL containers.> +> +@item> +@code{/} and @code{/boot} on ZFS. These require Guix to expose more of> +the @code{initrd} very early boot process to services. It also requires> +Guix to have the ability to explicitly load modules while still in> +@code{initrd} (currently kernel modules loaded by> +@code{kernel-module-loader-service-type} are loaded after @code{/} is> +mounted). Further, since one of ZFS's main advantages is that it can> +continue working despite the loss of one or more devices, it makes sense> +to also support installing the bootloader on all devices of the pool that> +contains the @code{/} and @code{/boot}; after all, if ZFS can survive the> +loss of one device, the bootloader should also be able to survive the loss> +of one device.
Ah, OK, this answered my previous question.
Toggle quote (147 lines)> +@item> +ZVOL swap devices. Mapped swap devices need to be listed in> +@code{mapped-devices} to ensure they are opened before the system attempts> +to use them, but you cannot currently add @code{%zfs-zvol-dependency} to> +@code{mapped-devices}.> +> +This will also require significant amounts of testing, as various kernel> +build options and patches may affect how swapping works, which are possibly> +different on Guix System compared to other distributions that this feature is> +known to work on.> +> +@item> +ZFS Event Daemon. Support for this has not been written yet, patches are> +welcome. The main issue is how to design this in a Guix style while> +supporting legacy shell-script styles as well. In particular, OpenZFS itself> +comes with a number of shell scripts intended for ZFS Event Daemon, and we> +need to figure out how the user can choose to use or not use the provided> +scripts (and configure any settings they have) or override with their own> +custom code (which could be shell scripts they have written and trusted from> +previous ZFS installations).> +> +As-is, you can create your own service that activates the ZFS Event Daemon> +by creating the @file{/etc/zfs/zed} directory and filling it appropriately,> +then launching @code{zed}.> +> +@item> +@file{/etc/zfs/zpool.cache}. Currently the ZFS support on Guix always forces> +scanning of all devices at bootup to look for ZFS pools. For systems with> +dozens or hundreds of storage devices, this can lead to slow bootup. One issue> +is that tools should really not write to @code{/etc} which is supposed to be for> +configuration; possibly it could be moved to @code{/var} instead. Another issue> +is that if Guix ever supports @code{/} on ZFS, we would need to somehow keep the> +@code{zpool.cache} file inside the @code{initrd} up-to-date with what is in the> +@code{/} mount point.> +> +@item> +@code{zfs share}. This will require some (unknown amount of) work to integrate> +into the Samba and NFS services of Guix. You @emph{can} manually set up Samba> +and NFS to share any mounted ZFS datasets by setting up their configurations> +properly; it just can't be done for you by @code{zfs share} and the> +@code{sharesmb} and @code{sharenfs} properties.> +@end enumerate> +> +Hopefully, support for the above only requires code to be written, o users> +are encouraged to hack on Guix to implement the above features.> +> @node Mapped Devices> @section Mapped Devices>> diff --git a/gnu/local.mk b/gnu/local.mk> index b944c671af..a2ff871277 100644> --- a/gnu/local.mk> +++ b/gnu/local.mk> @@ -43,6 +43,7 @@> # Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>> # Copyright © 2021 Arun Isaac <arunisaac@systemreboot.net>> # Copyright © 2021 Sharlatan Hellseher <sharlatanus@gmail.com>> +# Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>> #> # This file is part of GNU Guix.> #> @@ -618,6 +619,7 @@ GNU_SYSTEM_MODULES = \> %D%/services/docker.scm \> %D%/services/authentication.scm \> %D%/services/file-sharing.scm \> + %D%/services/file-systems.scm \> %D%/services/games.scm \> %D%/services/ganeti.scm \> %D%/services/getmail.scm \> diff --git a/gnu/services/base.scm b/gnu/services/base.scm> index ab3e441a7b..bcca24f93a 100644> --- a/gnu/services/base.scm> +++ b/gnu/services/base.scm> @@ -185,7 +185,9 @@>> references-file>> - %base-services))> + %base-services> +> + dependency->shepherd-service-name))>> ;;; Commentary:> ;;;> diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm> new file mode 100644> index 0000000000..0b1aae38ac> --- /dev/null> +++ b/gnu/services/file-systems.scm> @@ -0,0 +1,295 @@> +;;; GNU Guix --- Functional package management for GNU> +;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>> +;;;> +;;; This file is part of GNU Guix.> +;;;> +;;; GNU Guix is free software; you can redistribute it and/or modify it> +;;; under the terms of the GNU General Public License as published by> +;;; the Free Software Foundation; either version 3 of the License, or (at> +;;; your option) any later version.> +;;;> +;;; GNU Guix is distributed in the hope that it will be useful, but> +;;; WITHOUT ANY WARRANTY; without even the implied warranty of> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the> +;;; GNU General Public License for more details.> +;;;> +;;; You should have received a copy of the GNU General Public License> +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.> +> +(define-module (gnu services file-systems)> + #:use-module (gnu packages file-systems)> + #:use-module (gnu services)> + #:use-module (gnu services base)> + #:use-module (gnu services linux)> + #:use-module (gnu services mcron)> + #:use-module (gnu services shepherd)> + #:use-module (gnu system mapped-devices)> + #:use-module (guix gexp)> + #:use-module (guix packages)> + #:use-module (guix records)> + #:export (zfs-service-type> +> + zfs-configuration> + zfs-configuration?> + zfs-configuration-kernel> + zfs-configuration-base-zfs> + zfs-configuration-base-zfs-auto-snapshot> + zfs-configuration-dependencies> + zfs-configuration-auto-mount?> + zfs-configuration-auto-scrub> + zfs-configuration-auto-snapshot?> + zfs-configuration-auto-snapshot-keep> +> + %zfs-zvol-dependency))> +> +(define-record-type* <zfs-configuration>> + zfs-configuration> + make-zfs-configuration> + zfs-configuration?> +> + ; linux-libre kernel you want to compile the base-zfs module for.> + (kernel zfs-configuration-kernel)> +> + ; the OpenZFS package that will be modified to compile for the> + ; given kernel.> + (base-zfs zfs-configuration-base-zfs> + (default zfs))
The field name usually just contains the package name, so ‘zfs’ and‘zfs-auto-snapshot’ instead of ‘base-zfs’ and ‘base-zfs-auto-snapshot’.
Toggle quote (144 lines)> + ; the zfs-auto-snapshot package that will be modified to compile> + ; for the given kernel.> + (base-zfs-auto-snapshot zfs-configuration-base-zfs-auto-snapshot> + (default zfs-auto-snapshot))> +> + ; list of <mapped-device> or <file-system> objects that must be> + ; opened/mounted before we import any ZFS pools.> + (dependencies zfs-configuration-dependencies> + (default '()))> +> + ; #t if mountable datasets are to be mounted automatically.> + ; #f if not mounting.> + ; #t is the expected behavior on other operating systems, the> + ; #f is only supported for "rescue" operating systems where> + ; the user wants lower-level control of when to mount.> + (auto-mount? zfs-configuration-auto-mount?> + (default #t))> +> + ; 'weekly for weekly scrubbing, 'monthly for monthly scrubbing, an> + ; mcron time specification that can be given to `job`, or #f to> + ; disable.> + (auto-scrub zfs-configuration-auto-scrub> + (default 'weekly))> +> + ; #t if auto-snapshot is default (and `com.sun:auto-snapshot=false`> + ; disables auto-snapshot per dataset), #f if no auto-snapshotting> + ; is default (and `com.sun:auto-snapshot=true` enables auto-snapshot> + ; per dataset).> + (auto-snapshot? zfs-configuration-auto-snapshot?> + (default #t))> +> + ; association list of symbol-number pairs to indicate the number> + ; of automatic snapshots to keep for each of 'frequent, 'hourly,> + ; 'daily, 'weekly, and 'monthly.> + ; e.g. '((frequent . 8) (hourly . 12))> + (auto-snapshot-keep zfs-configuration-auto-snapshot-keep> + (default '())))> +> +(define %default-auto-snapshot-keep> + '((frequent . 4)> + (hourly . 24)> + (daily . 31)> + (weekly . 8)> + (monthly . 12)))> +> +(define %auto-snapshot-mcron-schedule> + '((frequent . "0,15,30,45 * * * *")> + (hourly . "0 * * * *")> + (daily . "0 0 * * *")> + (weekly . "0 0 * * 7")> + (monthly . "0 0 1 * *")))> +> +;; A synthetic and unusable MAPPED-DEVICE intended for use when> +;; the user has created a mountable filesystem inside a ZFS> +;; zvol and wants it mounted inside the configuration.scm.> +(define %zfs-zvol-dependency> + (mapped-device> + (source '())> + (targets '("zvol/*"))> + (type #f)))> +> +(define (make-zfs-package conf)> + (let ((kernel (zfs-configuration-kernel conf))> + (base-zfs (zfs-configuration-base-zfs conf)))> + (package> + (inherit base-zfs)> + (arguments (cons* #:linux kernel> + (package-arguments base-zfs))))))> +> +(define (make-zfs-auto-snapshot-package conf)> + (let ((zfs (make-zfs-package conf))> + (base-zfs-auto-snapshot (zfs-configuration-base-zfs-auto-snapshot conf)))> + (package> + (inherit base-zfs-auto-snapshot)> + (inputs `(("zfs" ,zfs))))))> +> +(define (zfs-loadable-modules conf)> + (list (list (make-zfs-package conf) "module")))> +> +(define (zfs-shepherd-services conf)> + (let* ((zfs-package (make-zfs-package conf))> + (zpool (file-append zfs-package "/sbin/zpool"))> + (zfs (file-append zfs-package "/sbin/zfs"))> + (zvol_wait (file-append zfs-package "/bin/zvol_wait"))> + (scheme-modules `((srfi srfi-1)> + (srfi srfi-34)> + (srfi srfi-35)> + (rnrs io ports)> + ,@%default-modules)))> + (define zfs-scan> + (shepherd-service> + (provision '(zfs-scan))> + (requirement `(root-file-system> + kernel-module-loader> + udev> + ,@(map dependency->shepherd-service-name> + (zfs-configuration-dependencies conf))))> + (documentation "Scans for and imports ZFS pools.")> + (modules scheme-modules)> + (start #~(lambda _> + (guard (c ((message-condition? c)> + (format (current-error-port)> + "zfs: error importing pools: ~s~%"> + (condition-message c))> + #f))> + ; TODO: optionally use a cachefile.> + (invoke #$zpool "import" "-a" "-N"))))> + ;; Why not one-shot? Because we don't really want to rescan> + ;; this each time a requiring process is restarted, as scanning> + ;; can take a long time and a lot of I/O.> + (stop #~(const #f))))> +> + (define device-mapping-zvol/*> + (shepherd-service> + (provision '(device-mapping-zvol/*))> + (requirement '(zfs-scan))> + (documentation "Waits for all ZFS ZVOLs to be opened.")> + (modules scheme-modules)> + (start #~(lambda _> + (guard (c ((message-condition? c)> + (format (current-error-port)> + "zfs: error opening zvols: ~s~%"> + (condition-message c))> + #f))> + (invoke #$zvol_wait))))> + (stop #~(const #f))))> +> + (define zfs-auto-mount> + (shepherd-service> + (provision '(zfs-auto-mount))> + (requirement '(zfs-scan))> + (documentation "Mounts all non-legacy mounted ZFS filesystems.")> + (modules scheme-modules)> + (start #~(lambda _> + (guard (c ((message-condition? c)> + (format (current-error-port)> + "zfs: error mounting file systems: ~s~%"> + (condition-message c))> + #f))> + ;; Output to current-error-port, otherwise the> + ;; user will not see any prompts for passwords> + ;; of encrypted datasets.> + ;; XXX Maybe better to explicitly open /dev/console ?
Seeing this comment, I assume that encrypted pools are supported, right?
Toggle quote (42 lines)> + (with-output-to-port (current-error-port)> + (lambda ()> + (invoke #$zfs "mount" "-a" "-l"))))))> + (stop #~(lambda _> + ;; Make sure that Shepherd does not have a CWD that> + ;; is a mounted ZFS filesystem, which would prevent> + ;; unmounting.> + (chdir "/")> + (invoke #$zfs "unmount" "-a" "-f")))))> +> + `(,zfs-scan> + ,device-mapping-zvol/*> + ,@(if (zfs-configuration-auto-mount? conf)> + `(,zfs-auto-mount)> + '()))))> +> +(define (zfs-user-processes conf)> + (if (zfs-configuration-auto-mount? conf)> + '(zfs-auto-mount)> + '(zfs-scan)))> +> +(define (zfs-mcron-auto-snapshot-jobs conf)> + (let* ((user-auto-snapshot-keep (zfs-configuration-auto-snapshot-keep conf))> + ;; assoc-ref has earlier entries overriding later ones.> + (auto-snapshot-keep (append user-auto-snapshot-keep> + %default-auto-snapshot-keep))> + (auto-snapshot? (zfs-configuration-auto-snapshot? conf))> + (zfs-auto-snapshot-package (make-zfs-auto-snapshot-package conf))> + (zfs-auto-snapshot (file-append zfs-auto-snapshot-package> + "/sbin/zfs-auto-snapshot")))> + (map> + (lambda (label)> + (let ((keep (assoc-ref auto-snapshot-keep label))> + (sched (assoc-ref %auto-snapshot-mcron-schedule label)))> + #~(job '#$sched> + (string-append #$zfs-auto-snapshot> + " --quiet --syslog "> + " --label=" #$(symbol->string label)> + " --keep=" #$(number->string keep)> + " //"))))> + '(frequent hourly daily weekly monthly))))
Maybe use something like
(map first %default-auto-snapshot-keep)
to avoid having to changing it if things change in the future.
Toggle quote (56 lines)> +(define (zfs-mcron-auto-scrub-jobs conf)> + (let* ((zfs-package (make-zfs-package conf))> + (zpool (file-append zfs-package "/sbin/zpool"))> + (auto-scrub (zfs-configuration-auto-scrub conf))> + (sched (cond> + ((eq? auto-scrub 'weekly) "0 0 * * 7")> + ((eq? auto-scrub 'monthly) "0 0 1 * *")> + (else auto-scrub))))> + (list> + #~(job '#$sched> + ;; Suppress errors: if there are no ZFS pools, the> + ;; scrub will not be given any arguments, which makes> + ;; it error out.> + (string-append "(" #$zpool " scrub `" #$zpool " list -o name -H` "> + "> /dev/null 2>&1) "> + "|| exit 0")))))> +> +(define (zfs-mcron-jobs conf)> + (append (zfs-mcron-auto-snapshot-jobs conf)> + (if (zfs-configuration-auto-scrub conf)> + (zfs-mcron-auto-scrub-jobs conf)> + '())))> +> +(define zfs-service-type> + (service-type> + (name 'zfs)> + (extensions> + (list ;; Install OpenZFS kernel module into kernel profile.> + (service-extension linux-loadable-module-service-type> + zfs-loadable-modules)> + ;; And load it.> + (service-extension kernel-module-loader-service-type> + (const '("zfs")))> + ;; Make sure ZFS pools and datasets are mounted at> + ;; boot.> + (service-extension shepherd-root-service-type> + zfs-shepherd-services)> + ;; Make sure user-processes don't start until> + ;; after ZFS does.> + (service-extension user-processes-service-type> + zfs-user-processes)> + ;; Install automated scrubbing and snapshotting.> + (service-extension mcron-service-type> + zfs-mcron-jobs)> +> + ;; Install ZFS management commands in the system> + ;; profile.> + (service-extension profile-service-type> + (compose list make-zfs-package))> + ;; Install ZFS udev rules.> + (service-extension udev-service-type> + (compose list make-zfs-package))))> + (description "Installs ZFS, an advanced filesystem and volume manager.")))> --> 2.31.1
I haven’t tested anything, but the rest looks good! :-)
-----BEGIN PGP SIGNATURE-----
iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmEz4sUVHHB1YmxpY0B5b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x57YcQAKLhO/BZbp3/pfkF/9xrol6wZO3bnUY6rw3m1oi5mjEx7wLdSonltbkxoXQnZuAqnlIe1NXLldd7AQ5f4Lx3Uw9myB+lf0FjC1ks5Cp6yWhqgWp04o94mAxTEQ/yHB1jEMIhkM2PTaYyAIcil1dEOjJITD9bImx0tl93imfo+LAy1mBQvqb9ONimK1ixr6Xvu8IM6Cu85qTHVE18Pt3ugS25wvitFi+te5NAWMYHQ+2sbPQjGdlVCnTNYNEI7fL4Bse9+Y82WPdCeKPqay01MR4U0WbSP28KNmNqNtefL2dYoImARVckKdOP83NQzEtNaQY8VD32xNAcW/dfAKlcxLuLkfI+s1QUnYTnY9//AYhCao6b5MKn2iLAWJ68Ij7t2D8iEXLTFzi1l7zyGiKvJKlvmucNmWoKGCfmLOp0xxki4CC0mVkBGDRwA+QrPGqXDljbCWoAaOpEywnqX88cUxPq5+JwniConf+Bc/GYdmdqWbWp+NUqpXGPax96jsys9lBKoiEV/8DOGnyKsB3XLFtLv2oHwa4SoeeKexF9E7BIDABLts/kTPZG7VcWN4Vl0IVymIHF6aQBxpCk4GbHUH4voVNg/7oP2OZCQn9HOiClWN7awllCVISX7b/nmxijx/81UnsDzDWThtkacjtfyFqZalxX0d2CLc4s27HWzU6C=Ql/1-----END PGP SIGNATURE-----
Z
Z
zimoun wrote on 6 Sep 09:59 +0200
Re: [bug#45692]
(name . Mason Loring Bliss)(address . mason@blisses.org)
86lf4ajfsb.fsf_-_@gmail.com
Hi Mason,
On Thu, 02 Sep 2021 at 17:24, Mason Loring Bliss <mason@blisses.org> wrote:
Toggle quote (4 lines)> Gaslighting in an attempt to sabotage the> adoption of high-quality free software is somewhat poor form and not at all> useful.
Using such language (emphasized by the subject), you are not creating apositive environment. Please be respectful when differing viewpointsand experiences. And please assume good faith avoiding to jump innegative opinionated conclusions.
Thanks,simon
Z
Z
zimoun wrote on 6 Sep 10:08 +0200
Re: [bug#45692] [PATCH v4 3/3] gnu: Add ZFS service type.
(name . Maxime Devos)(address . maximedevos@telenet.be)
86bl56jfdn.fsf@gmail.com
Hi Maxime.
On Thu, 02 Sep 2021 at 22:57, Maxime Devos <maximedevos@telenet.be> wrote:
Toggle quote (2 lines)> See also https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/.
Reading the Software Freedom Conservancy analysis, from myunderstanding, the issue is about distributing the resulting *binary*Linux+ZFS. Other said, the distribution of the zfs.ko is an issue, notprovide a way to build it locally. Well, it does not appear to be a GPLviolation, IIUC the SFC analysis.
Thanks for taking the time to carefully review all the aspects.
Cheers,simon
M
M
Maxime Devos wrote on 6 Sep 12:40 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
a4eb0f76fc225c04039d8a2f83223e6de1a2b21b.camel@telenet.be
zimoun schreef op ma 06-09-2021 om 10:08 [+0200]:
Toggle quote (2 lines)> Hi Maxime.
Hi zimoun,
Toggle quote (9 lines)> > On Thu, 02 Sep 2021 at 22:57, Maxime Devos <maximedevos@telenet.be> wrote:> > > See also <https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/>;.> > Reading the Software Freedom Conservancy analysis, from my> understanding, the issue is about distributing the resulting *binary*> Linux+ZFS.
Previously I thought that ‘Is The Analysis Different With Source-Only Distribution?’somehow did not apply here, though I cannot remember the reasoning well (maybemy reasoning was bogus?).
However, SFC did note in that section:
‘Nevertheless, there may be arguments for contributory and/or indirect copyrightinfringement in many jurisdictions. We present no specific analysis ourselves onthe efficacy of a contributory infringement claim regarding source-only distributionsof ZFS and Linux. However, in our GPL litigation experience, we have noticed thatjudges are savvy at sniffing out attempts to circumvent legal requirements, and theyare skeptical about attempts to exploit loopholes. Furthermore, we cannot predictOracle's view — given its past willingness to enforce copyleft licenses, and Oracle'srecent attempts to adjudicate the limits of copyright in Court. Downstream users shouldconsider carefully before engaging in even source-only distribution.’
I'm completely unfamiliar with the notion of ‘contributory copyring infringement’or ‘indirect copyright infringement’. ‘Savvy judges skeptical at attempts toexploit loopholes’ seems plausible to me in my inexpert opinion.
Toggle quote (4 lines)> Other said, the distribution of the zfs.ko is an issue, not> provide a way to build it locally. Well, it does not appear to be a GPL> violation, IIUC the SFC analysis.
I neglected that the terms of the GPL that come into play depend on whetherone is only distributing source code (*) or also binaries, and whether one isdistributing _modified_ source code.
I don't quite see a GPL violation anymore if we only distribute unmodifiedsource code. However, what about freedom (1) and (3) (freedom to [...] andchange the program in source form and (3) distribute modified versions)?I was going to write something here about why that would not be (legally)possible because of the CDDL and GPL, but I forgot why it wouldn't bepossible. Maybe it is actually possible?
(*) raid5atemyhomework noted that guix does _not_ distribute source code,it only points to source code locations. I don't quite agree. From mypoint of view, on whose server the source code is hosted is merely atechnicality, guix is just ‘out-sourcing’ the source code distribution.Besides, ci.guix.gnu.org keeps a copy of the source code, and(guix download) will try to download from ci.guix.gnu.org.
Toggle quote (2 lines)> Thanks for taking the time to carefully review all the aspects.
Greetings,Maxime
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYTXwNRccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7kMuAP4643cECusxy+LaPDPqCdgvBOnOaJuyd5rIiDoRRariMQEA17L6e8O1pfR/eGQicTL/PYkK/USvvl34oo+nEvtAEAw==3dxq-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 6 Sep 12:52 +0200
(name . Xinglu Chen)(address . public@yoctocell.xyz)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
TkQVzGpSd-9nTC6ySmiXXhv44YUNayYAYe8w9Jtmqx_mlY4hQqgnGStksZG9HOaXjHTz9bHs3bhI43CLiQhHLEuBDA_xZGXnd6rGoEutyuk=@protonmail.com
Hello Xinglu Chen,
Thank you for your interest.
Toggle quote (3 lines)> You might want to bring up the topic of subsystem maintainers on the> guix-devel mailing list to get some more attention.
Not personally interested.

Toggle quote (12 lines)> > +The above will keep 8 @code{frequent} snapshots and 12 @code{hourly} snapshots.> > +@code{daily}, @code{weekly}, and @code{monthly} snapshots will keep their> > +defaults (31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly}).> > +> > +@end table> > +@end deftp>> IIUC, there is not way specify ZFS pools in the ‘file-systems’ field of> an <operating-system> record. Does this mean that ZFS as the root file> system is not supported, and if so, is there a particular reason for> this?
No, and as you saw, that requires some more work.
In particular, older versions of this patchset included the ability to add ZFS pools / datasets `file-system` objects, but the exact implementation was objected to, with no suggestion for what alternative to use instead. Since I thought it was contentious, I removed it entirely instead.
Note that ZFS-on-Root on Guix is even harder because of the need to put loading in `initrd`, and a lot of coding as well, not just `file-system`s support.
Given the sheer lack of review and etc etc, I am not encouraged to write more code that will remain unreviewed and unmerged. Maybe if this gets merged as-is, I will, but otherwise, I don't see the point.

Toggle quote (4 lines)> The field name usually just contains the package name, so ‘zfs’ and> ‘zfs-auto-snapshot’ instead of ‘base-zfs’ and ‘base-zfs-auto-snapshot’.

The point is that the service does **NOT** use `base-zfs` directly --- it creates a new version of that package, targeted towards the specific kernel you provided. This is necessary since internal kernel APIs and ABIs may change between versions, even minor v==ersions (Linux has no commitment to keeping kernel interfaces stable, it only has a commitment to keeping userspace interfaces stable, and OpenZFS **requires** the kernel interfaces, so it is safest to compile specifically to the kernel version that is used).
Thus the ***`base-`*** prefixes: the `zfs-service-type` does not use the `base-zfs` and `base-zfs-autosnapshot` packages as-is, they are instead used as the basis for the actual packages that are compiled and installed into the system. I thought this would be sufficiently different from other services, which use package names as-is (but use the packages as-is, without inheriting from them, unlike this service) that the `base-` prefix was justified.
Toggle quote (15 lines)> > - ;; Output to current-error-port, otherwise the> >> >> > - ;; user will not see any prompts for passwords> >> >> > - ;; of encrypted datasets.> >> >> > - ;; XXX Maybe better to explicitly open /dev/console ?> >> >>> Seeing this comment, I assume that encrypted pools are supported, right?
Encrypted datasets are supported, yes. If you set `keylocation=prompt` then the `init` process will pause and ask for the password on the console. You can even use `keylocation=file:///some/keyfile`, I also tested that in a VM. My own production setup (which doesn't use this service, but includes some code copy-pasted from this service) uses `keylocation=file:///*elided*`.

Toggle quote (10 lines)> > - '(frequent hourly daily weekly monthly))))> >> >>> Maybe use something like>> (map first %default-auto-snapshot-keep)>> to avoid having to changing it if things change in the future.
Good idea, thank you.
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 6 Sep 13:08 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)
84VZkrMdtpnHj3fTyW5llNzMcTOVnkpIKR7xkrJqC_6hjYVSSQCYchxbgIuLF6cP5L_4v5nDGjdFyP5_qeSt1e26Tv7Ga1r59ey0j-dvgt0=@protonmail.com
Hello Maxime,
Toggle quote (19 lines)> > Other said, the distribution of the zfs.ko is an issue, not> > provide a way to build it locally. Well, it does not appear to be a GPL> > violation, IIUC the SFC analysis.>> I neglected that the terms of the GPL that come into play depend on whether> one is only distributing source code () or also binaries, and whether one is> distributing modified source code.> I don't quite see a GPL violation anymore if we only distribute unmodified> source code. However, what about freedom (1) and (3) (freedom to [...] and> change the program in source form and (3) distribute modified versions)?> I was going to write something here about why that would not be (legally)> possible because of the CDDL and GPL, but I forgot why it wouldn't be> possible. Maybe it is actually possible?> () raid5atemyhomework noted that guix does not distribute source code,it only points to source code locations. I don't quite agree. From my> point of view, on whose server the source code is hosted is merely a> technicality, guix is just ‘out-sourcing’ the source code distribution.> Besides, ci.guix.gnu.org keeps a copy of the source code, and> (guix download) will try to download from ci.guix.gnu.org.
*shrug* if so, take note that the `zfs` package modifies the build system and even some of the source code in order to adapt it to the slightly different Guix environment (in particular, Guix does not give an FHS to packages being built, so we need to point some paths directly to absolute paths).
I am not really interested in modifying the patch unless the license issue is resolved, so I will wait until Guix maintainers can actually agree on whether there is a license violation or not.
Thanksraid5atemyhomework
X
X
Xinglu Chen wrote on 6 Sep 16:22 +0200
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
87a6kp238o.fsf@yoctocell.xyz
On Mon, Sep 06 2021, raid5atemyhomework via Guix-patches via wrote:
Toggle quote (4 lines)> Hello Xinglu Chen,>> Thank you for your interest.
You are welcome! Thank you for working on ZFS support!
Toggle quote (34 lines)>> You might want to bring up the topic of subsystem maintainers on the>> guix-devel mailing list to get some more attention.>> Not personally interested.>>>> > +The above will keep 8 @code{frequent} snapshots and 12 @code{hourly} snapshots.>> > +@code{daily}, @code{weekly}, and @code{monthly} snapshots will keep their>> > +defaults (31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly}).>> > +>> > +@end table>> > +@end deftp>>>> IIUC, there is not way specify ZFS pools in the ‘file-systems’ field of>> an <operating-system> record. Does this mean that ZFS as the root file>> system is not supported, and if so, is there a particular reason for>> this?>> No, and as you saw, that requires some more work.>> In particular, older versions of this patchset included the ability to> add ZFS pools / datasets `file-system` objects, but the exact> implementation was objected to, with no suggestion for what> alternative to use instead. Since I thought it was contentious, I> removed it entirely instead.>> Note that ZFS-on-Root on Guix is even harder because of the need to> put loading in `initrd`, and a lot of coding as well, not just> `file-system`s support.>> Given the sheer lack of review and etc etc, I am not encouraged to> write more code that will remain unreviewed and unmerged. Maybe if> this gets merged as-is, I will, but otherwise, I don't see the point.
I understand your feeling; hopefully we can get the patch merged soon!:-)
Toggle quote (21 lines)>> The field name usually just contains the package name, so ‘zfs’ and>> ‘zfs-auto-snapshot’ instead of ‘base-zfs’ and ‘base-zfs-auto-snapshot’.>>> The point is that the service does **NOT** use `base-zfs` directly ---> it creates a new version of that package, targeted towards the> specific kernel you provided. This is necessary since internal kernel> APIs and ABIs may change between versions, even minor v==ersions> (Linux has no commitment to keeping kernel interfaces stable, it only> has a commitment to keeping userspace interfaces stable, and OpenZFS> **requires** the kernel interfaces, so it is safest to compile> specifically to the kernel version that is used).>> Thus the ***`base-`*** prefixes: the `zfs-service-type` does not use> the `base-zfs` and `base-zfs-autosnapshot` packages as-is, they are> instead used as the basis for the actual packages that are compiled> and installed into the system. I thought this would be sufficiently> different from other services, which use package names as-is (but use> the packages as-is, without inheriting from them, unlike this service)> that the `base-` prefix was justified.
Ah, that makes sense, then I am fine with keeping it as-is.
Toggle quote (23 lines)>> > - ;; Output to current-error-port, otherwise the>> >>> >>> > - ;; user will not see any prompts for passwords>> >>> >>> > - ;; of encrypted datasets.>> >>> >>> > - ;; XXX Maybe better to explicitly open /dev/console ?>> >>> >>>>> Seeing this comment, I assume that encrypted pools are supported, right?>> Encrypted datasets are supported, yes. If you set> `keylocation=prompt` then the `init` process will pause and ask for> the password on the console. You can even use> `keylocation=file:///some/keyfile`, I also tested that in a VM. My> own production setup (which doesn't use this service, but includes> some code copy-pasted from this service) uses> `keylocation=file:///*elided*`.
Cool, hopefully I will get to test this soon. :-)
Toggle quote (14 lines)>> > - '(frequent hourly daily weekly monthly))))>> >>> >>>>> Maybe use something like>>>> (map first %default-auto-snapshot-keep)>>>> to avoid having to changing it if things change in the future.>> Good idea, thank you.>> Thanks> raid5atemyhomework
-----BEGIN PGP SIGNATURE-----
iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmE2JBcVHHB1YmxpY0B5b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x569AQAKOa1GC9J2NI4Giv8VSgLtSdUGjq5H9j6ad5sah7rnjtUX5khDTx2lovNjQA/85PSf/o90OVKruTyRikkfBTdj2Qcq8GBsWwXJBlDLO6aWwyBBNI/w9yyUh6+tSmJtKLZ6T0LZJuTEL7fWmTACDFazxi09dZmDGnZ3He8/fYo26PmsOp8HDMPLOj2qLlsHQJa1vCVYQLr58FEm/KpBNBwwVx/i8VhEZa4KBv71ZwxuTZ/Z6sT7Sm3IMQsSr45cjoyKnZ2EHFW1G/UKyOvBdyQKNQwe0M69Jt774j2rUOlVTiYtRnPMmbYZEkwiBMV3p5RT2aKJWwerbs2o6T6/cYmMmpmmzSNtcn87LlM1UwVmujCNUvrg8DLWTFsB0xHpoV2p+cFWsTS6Vt//RfFCravD+Pf9FcdMlyX033iXRu9MNuPQy+qCIsmqGbeRY70SfAcJ0uSLIlHi3a3GZMi+dziGEv47xibuYadS/ai+Z9FohPb89O0vUu8ZXdpZF1V1PojN8wyFoic32Fm1jEikBNB1FBOsUuibhvn+plkti+MfXrKr2FX7VD8L7ZXUdLLCN0TBZ/dk9l1c5V1q0prX88kdN3rDooQ9sfI795NUiab1kka9iC6I7PAfLOJyKNUh4Xs56w04itQauFl90YJ0luCKPVJJLnGOp9NNddgUuRaGMx=QVNI-----END PGP SIGNATURE-----
Z
Z
zimoun wrote on 6 Sep 19:17 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)
CAJ3okZ2US0-n_0DUuzWrZNjThSoi=K1Mz+yXLQ6o81FeUahMqw@mail.gmail.com
Hi Maxime,
On Mon, 6 Sept 2021 at 12:41, Maxime Devos <maximedevos@telenet.be> wrote:
Toggle quote (10 lines)> ‘Nevertheless, there may be arguments for contributory and/or indirect copyright> infringement in many jurisdictions. We present no specific analysis ourselves on> the efficacy of a contributory infringement claim regarding source-only distributions> of ZFS and Linux. However, in our GPL litigation experience, we have noticed that> judges are savvy at sniffing out attempts to circumvent legal requirements, and they> are skeptical about attempts to exploit loopholes. Furthermore, we cannot predict> Oracle's view — given its past willingness to enforce copyleft licenses, and Oracle's> recent attempts to adjudicate the limits of copyright in Court. Downstream users should> consider carefully before engaging in even source-only distribution.’
The « Nevertheless » is because the previous sentence is:
Toggle snippet (7 lines)[…] Therefore, the analysis is simpler, and we find no specific clausein either license that prohibits source-only redistribution of Linux andZFS, even on the same distribution media.
Nevertheless, […]
My understanding is:
- binary distribution violates licenses - source-only distribution appears to be fine - SFC cannot guarantee because all the arguments about source-onlydistribution have never been tested in Court.
Moreover, they write an explicit paragraph why « You cannot and shouldnot rely on this document as legal advice. » ;-)

Toggle quote (2 lines)> I don't quite see a GPL violation anymore if we only distribute unmodified
Good. :-)
Toggle quote (3 lines)> source code. However, what about freedom (1) and (3) (freedom to [...] and> change the program in source form and (3) distribute modified versions)?
Each license is free [1]. Therefore, they respect all the freedoms.The issue is about linking the result and distribute the binary.
1: https://directory.fsf.org/wiki/License:CDDL-1.0
Toggle quote (7 lines)> (*) raid5atemyhomework noted that guix does _not_ distribute source code,> it only points to source code locations. I don't quite agree. From my> point of view, on whose server the source code is hosted is merely a> technicality, guix is just ‘out-sourcing’ the source code distribution.> Besides, ci.guix.gnu.org keeps a copy of the source code, and> (guix download) will try to download from ci.guix.gnu.org.
Indeed, ci.guix.gnu.org keeps a copy of (as much as possible) sourcecode. But ci.guix.gnu.org does not distribute all the correspondingbinaries: see 'arguments' '#:substitutable? #f'. It is already thecase for the package 'zfs':
Toggle snippet (6 lines) (arguments `(;; The ZFS kernel module should not be downloaded since the license ;; terms don't allow for distributing it, only building it locally. #:substitutable? #f
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/file-systems.scm?id=a4ffe3d145b00736f5fdf53ee2c70a7e48592e83#n1175
As explained in the initial submission [2], this patch set is just a"glue" usable by the user locally. No binaries on the Guix side isinvolved. All the source-code is under free license.
2: http://issues.guix.gnu.org/45692#0

Cheers,simon
M
M
Maxime Devos wrote on 7 Sep 11:54 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
2d8f9c19f4faa5709cc1b6fb69d33cb8403c8c06.camel@telenet.be
Hi,
To be clear, I don't oppose the inclusion of the ZFS serviceon basis of the licensing anymore.
Greetings,Maxime.
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYTc23RccbWF4aW1lZGV2b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7qbmAP4yvk4xrMoXmrJXtKz6oiodwmQCJmzTBft+Bab9yikspQD8DAjVFCC1VZP8eopu+NvS1DZV5d0DC1bb+8hTOQZ9HgI==etW1-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 8 Sep 03:23 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)
4p6Z87-x-50VeiVbwVx1Rl6rBr5o_63kmCJomCUoeKxi6gMnKseYriTIrByXbQ5McOWp0mrt6U5Bd0QqvI0pcC17_3m5sRNNIZvNa6ow5jc=@protonmail.com
Good morning Maxime,
Toggle quote (5 lines)> Hi,>> To be clear, I don't oppose the inclusion of the ZFS service> on basis of the licensing anymore.
Okay. I'm busy right now, so will get around to updating the patch next week.
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 15 Sep 16:04 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)
rmMF4W9-i83r1QlsxtUWrmdYs6NCCNISRWGv148VJKP9RXrBF5KpBHSYBv5lNi20PMlHbz-TGom3M_m_M_qLTTvwiXxy-zX-sVXcjJoYudM=@protonmail.com
hello,
Toggle quote (7 lines)>> > Hi,> > To be clear, I don't oppose the inclusion of the ZFS service> > on basis of the licensing anymore.>> Okay. I'm busy right now, so will get around to updating the patch next week.
Scratch that, I'm busy this week as well, maybe next next week.
Thanksraid5atemyhomework
Z
Z
zimoun wrote on 21 Sep 11:42 +0200
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
865yuub6z2.fsf@gmail.com
Hi,
On Wed, 15 Sep 2021 at 14:04, raid5atemyhomework <raid5atemyhomework@protonmail.com> wrote:
Toggle quote (4 lines)>> Okay. I'm busy right now, so will get around to updating the patch next week.>> Scratch that, I'm busy this week as well, maybe next next week.
Hope that you will find the time next next week. :-)
Just a comment to ease the application of patch. Once rebased onorigin/master, you should create the patch set using:
git format-patch --base=origin/master
because then it ends with this, for instance,
Toggle snippet (5 lines)base-commit: d14221bf65cfbe7f8f5b7cac44132087cab70bf5-- 2.28.0
which provide the commit where the patch applies and thus ease work atreview time, IMHO. See details [1].
Cheers,simon

1: http://issues.guix.gnu.org/issue/43946
R
R
raid5atemyhomework wrote on 30 Sep 16:56 +0200
[PATCH v5 3/3] gnu: Add ZFS service type.
(name . zimoun)(address . zimon.toutoune@gmail.com)
w5Ck7et7YPcgboacWLbUnpTA7rbKz2c_psHpV6a6scRzJ1mTPb6MQ4A1RAcTFYJquJG3pmIVkZB1ySOnXaQu19ZUBi0BFHC33EtJg1Mx_wk=@protonmail.com
Sorry for the lateness everyone.
Hope this one gets reviewed and merged.
--
From 3803e046566278fe12d64f6e39564e9602bf434d Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Thu, 30 Sep 2021 16:58:46 +0800Subject: [PATCH] gnu: Add ZFS service type.
* gnu/services/file-systems.scm: New file.* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.* gnu/services/base.scm: Export dependency->shepherd-service-name.* doc/guix.texi (ZFS File System): New subsection.--- doc/guix.texi | 351 ++++++++++++++++++++++++++++++++ gnu/local.mk | 2 + gnu/services/base.scm | 4 +- gnu/services/file-systems.scm | 363 ++++++++++++++++++++++++++++++++++ 4 files changed, 719 insertions(+), 1 deletion(-) create mode 100644 gnu/services/file-systems.scm
Toggle diff (775 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex a72a726b54..dd38103953 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -97,6 +97,7 @@ Copyright @copyright{} 2021 Hui Lu@* Copyright @copyright{} 2021 pukkamustard@* Copyright @copyright{} 2021 Alice Brenon@* Copyright @copyright{} 2021 Andrew Tropin@*+Copyright @copyright{} 2021 raid5atemyhomework@*
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or@@ -14435,6 +14436,356 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp
++@node ZFS File System+@subsection ZFS File System++Support for ZFS file systems in Guix is based on the OpenZFS project.+OpenZFS currently only supports Linux-Libre and is not available on the+Hurd.++OpenZFS is free software; unfortunately its license is incompatible with+the GNU General Public License (GPL), the license of the Linux kernel,+which means they cannot be distributed together. However, as a user,+you can choose to build ZFS and use it together with Linux; you can+even rely on Guix to automate this task. See+@uref{https://www.fsf.org/licensing/zfs-and-linux, this analysis by+the Free Software Foundation} for more information.++As a large and complex kernel module, OpenZFS has to be compiled for a+specific version of Linux-Libre. At times, the latest OpenZFS package+available in Guix is not compatible with the latest Linux-Libre version.+Thus, directly installing the @code{zfs} package can fail.++Instead, you are recommended to select a specific older long-term-support+Linux-Libre kernel. Do not use @code{linux-libre-lts}, as even the+latest long-term-support kernel may be too new for @code{zfs}. Instead,+explicitly select a specific older version, such as @code{linux-libre-5.10},+and upgrade it manually later as new long-term-support kernels become+available that you have confirmed is compatible with the latest available+OpenZFS version on Guix.++For example, you can modify your system configuration file to a specific+Linux-Libre version and add the @code{zfs-service-type} service.++@lisp+(use-modules (gnu))+(use-package-modules+ #;@dots{}+ linux)+(use-service-modules+ #;@dots{}+ file-systems)++(define my-kernel linux-libre-5.10)++(operating-system+ (kernel my-kernel)+ #;@dots{}+ (services+ (cons* (service zfs-service-type+ (zfs-configuration+ (kernel my-kernel)))+ #;@dots{}+ %desktop-services))+ #;@dots{})+@end lisp++@defvr {Scheme Variable} zfs-service-type+This is the type for a service that adds ZFS support to your operating+system. The service is configured using a @code{zfs-configuration}+record.++Here is an example use:++@lisp+(service zfs-service-type+ (zfs-configuration+ (kernel linux-libre-5.4)))+@end lisp+@end defvr++@deftp {Data Type} zfs-configuration+This data type represents the configuration of the ZFS support in Guix+System. Its fields are:++@table @asis+@item @code{kernel}+The package of the Linux-Libre kernel to compile OpenZFS for. This field+is always required. It @emph{must} be the same kernel you use in your+@code{operating-system} form.++@item @code{base-zfs} (default: @code{zfs})+The OpenZFS package that will be compiled for the given Linux-Libre kernel.++@item @code{base-zfs-auto-snapshot} (default: @code{zfs-auto-snapshot})+The @code{zfs-auto-snapshot} package to use. It will be modified to+specifically use the OpenZFS compiled for your kernel.++@item @code{dependencies} (default: @code{'()})+A list of @code{<file-system>} or @code{<mapped-device>} records that must+be mounted or opened before OpenZFS scans for pools to import. For example,+if you have set up LUKS containers as leaf VDEVs in a pool, you have to+include their corresponding @code{<mapped-ddevice>} records so that OpenZFS+can import the pool correctly at bootup.++@item @code{auto-mount?} (default: @code{#t})+Whether to mount datasets with the ZFS @code{mountpoint} property automatically+at startup. This is the behavior that ZFS users usually expect. You might+set this to @code{#f} for an operating system intended as a ``rescue'' system+that is intended to help debug problems with the disks rather than actually+work in production.++@item @code{auto-scrub} (default: @code{'weekly})+Specifies how often to scrub all pools. Can be the symbols @code{'weekly} or+@code{'monthly}, or a schedule specification understood by+@xref{mcron, mcron job specifications,, mcron, GNU@tie{}mcron}, such as+@code{"0 3 * * 6"} for ``every 3AM on Saturday''.+It can also be @code{#f} to disable auto-scrubbing (@strong{not recommended}).++The general guideline is to scrub weekly when using consumer-quality drives, and+to scrub monthly when using enterprise-quality drives.++@code{'weekly} scrubs are done on Sunday midnight, while @code{monthly} scrubs+are done on midnight on the first day of each month.++@item @code{auto-snapshot?} (default: @code{#t})+Specifies whether to auto-snapshot by default. If @code{#t}, then snapshots+are automatically created except for ZFS datasets with the+@code{com.sun:auto-snapshot} ZFS vendor property set to @code{false}.++If @code{#f}, snapshots will not be automatically created, unless the ZFS+dataset has the @code{com.sun:auto-snapshot} ZFS vendor property set to+@code{true}.++@item @code{auto-snapshot-keep} (default: @code{'()})+Specifies an association list of symbol-number pairs, indicating the number+of automatically-created snapshots to retain for each frequency type.++If not specified via this field, by default there are 4 @code{frequent}, 24+@code{hourly}, 31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly} snapshots.++For example:++@lisp+(zfs-configuration+ (kernel my-kernel)+ (auto-snapshot-keep+ '((frequent . 8)+ (hourly . 12))))+@end lisp++The above will keep 8 @code{frequent} snapshots and 12 @code{hourly} snapshots.+@code{daily}, @code{weekly}, and @code{monthly} snapshots will keep their+defaults (31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly}).++@end table+@end deftp++@subsubsection ZFS Auto-Snapshot++The ZFS service on Guix System supports auto-snapshots as implemented in the+Solaris operating system.++@code{frequent} (every 15 minutes), @code{hourly}, @code{daily}, @code{weekly},+and @code{monthly} snapshots are created automatically for ZFS datasets that+have auto-snapshot enabled. They will be named, for example,+@code{zfs-auto-snap_frequent-2021-03-22-1415}. You can continue to use+manually-created snapshots as long as they do not conflict with the naming+convention used by auto-snapshot. You can also safely manually destroy+automatically-created snapshots, for example to free up space.++The @code{com.sun:auto-snapshot} ZFS property controls auto-snapshot on a+per-dataset level. Sub-datasets will inherit this property from their parent+dataset, but can have their own property.++You @emph{must} set this property to @code{true} or @code{false} exactly,+otherwise it will be treated as if the property is unset.++For example:++@example+# zfs list -o name+NAME+tank+tank/important-data+tank/tmp+# zfs set com.sun:auto-snapshot=true tank+# zfs set com.sun:auto-snapshot=false tank/tmp+@end example++The above will set @code{tank} and @code{tank/important-data} to be+auto-snapshot, while @code{tank/tmp} will not be auto-snapshot.++If the @code{com.sun:auto-snapshot} property is not set for a dataset+(the default when pools and datasets are created), then whether+auto-snapshot is done or not will depend on the @code{auto-snapshot?}+field of the @code{zfs-configuration} record.++There are also @code{com.sun:auto-snapshot:frequent},+@code{com.sun:auto-snapshot:hourly}, @code{com.sun:auto-snapshot:daily},+@code{com.sun:auto-snapshot:weekly}, and @code{com.sun:auto-snapshot:monthly}+properties that give finer-grained control of whether to auto-snapshot a+dataset at a particular schedule.++The number of snapshots kept for all datasets can be overridden via the+@code{auto-snapshot-keep} field of the @code{zfs-configuration} record.+There is currently no support to have different numbers of snapshots to+keep for different datasets.++@subsubsection ZVOLs++ZFS supports ZVOLs, block devices that ZFS exposes to the operating+system in the @code{/dev/zvol/} directory. The ZVOL will have the same+resilience and self-healing properties as other datasets on your ZFS pool.+ZVOLs can also be snapshotted (and will be included in auto-snapshotting+if enabled), which snapshots the state of the block device, effectively+snapshotting the hosted file system.++You can put any file system inside the ZVOL. However, in order to mount this+file system at system start, you need to add @code{%zfs-zvol-dependency} as a+dependency of each file system inside a ZVOL.++@defvr {Scheme Variable} %zfs-zvol-dependency+An artificial @code{<mapped-device>} which tells the file system mounting+service to wait for ZFS to provide ZVOLs before mounting the+@code{<file-system>} dependent on it.+@end defvr++For example, suppose you create a ZVOL and put an ext4 filesystem+inside it:++@example+# zfs create -V 100G tank/ext4-on-zfs+# mkfs.ext4 /dev/zvol/tank/ext4-on-zfs+# mkdir /ext4-on-zfs+# mount /dev/zvol/tank/ext4-on-zfs /ext4-on-zfs+@end example++You can then set this up to be mounted at boot by adding this to the+@code{file-systems} field of your @code{operating-system} record:++@lisp+(file-system+ (device "/dev/zvol/tank/ext4-on-zfs")+ (mount-point "/ext4-on-zfs")+ (type "ext4")+ (dependencies (list %zfs-zvol-dependency)))+@end lisp++You @emph{must not} add @code{%zfs-zvol-dependency} to your+@code{operating-system}'s @code{mapped-devices} field, and you @emph{must+not} add it (or any @code{<file-system>}s dependent on it) to the+@code{dependencies} field of @code{zfs-configuration}. Finally, you+@emph{must not} use @code{%zfs-zvol-dependency} unless you actually+instantiate @code{zfs-service-type} on your system.++@subsubsection Unsupported Features++Some common features and uses of ZFS are currently not supported, or not+fully supported, on Guix.++@enumerate+@item+Shepherd-managed daemons that are configured to read from or write to ZFS+mountpoints need to include @code{user-processes} in their @code{requirement}+field. This is the earliest that ZFS file systems are assured of being+mounted.++Generally, most daemons will, directly or indirectly, require+@code{networking}, or @code{user-processes}, or both. Most implementations+of @code{networking} will also require @code{user-processes} so daemons that+require only @code{networking} will also generally start up after+@code{user-processes}. A notable exception, however, is+@code{static-networking-service-type}. You will need to explicitly add+@code{user-processes} as a @code{requirement} of your @code{static-networking}+record.++@item+@code{mountpoint=legacy} ZFS file systems. The handlers for the Guix mounting+system have not yet been modified to support ZFS, and will expect @code{/dev}+paths in the @code{<file-system>}'s @code{device} field, but ZFS file systems+are referred to via non-path @code{pool/file/system} names. Such file systems+also need to be mounted @emph{after} OpenZFS has scanned for pools.++You can still manually mount these file systems after system boot; what is+only unsupported is mounting them automatically at system boot by specifying+them in @code{<file-system>} records of your @code{operating-system}.++@item+@code{/home} on ZFS. Guix will create home directories for users, but this+process currently cannot be scheduled after ZFS file systems are mounted.+Thus, the ZFS file system might be mounted @emph{after} Guix has created+home directories at boot, at which point OpenZFS will refuse to mount since+the mountpoint is not empty. However, you @emph{can} create an ext4, xfs,+btrfs, or other supported file system inside a ZVOL, have that depend on+@code{%zfs-zvol-dependency}, and set it to mount on the @code{/home}+directory; they will be scheduled to mount before the @code{user-homes}+process.++Similarly, other locations like @code{/var}, @code{/gnu/store} and so+on cannot be reliably put in a ZFS file system, though they may be+possible to create as other file systems inside ZVOL containers.++@item+@code{/} and @code{/boot} on ZFS. These require Guix to expose more of+the @code{initrd} very early boot process to services. It also requires+Guix to have the ability to explicitly load modules while still in+@code{initrd} (currently kernel modules loaded by+@code{kernel-module-loader-service-type} are loaded after @code{/} is+mounted). Further, since one of ZFS's main advantages is that it can+continue working despite the loss of one or more devices, it makes sense+to also support installing the bootloader on all devices of the pool that+contains the @code{/} and @code{/boot}; after all, if ZFS can survive the+loss of one device, the bootloader should also be able to survive the loss+of one device.++@item+ZVOL swap devices. Mapped swap devices need to be listed in+@code{mapped-devices} to ensure they are opened before the system attempts+to use them, but you cannot currently add @code{%zfs-zvol-dependency} to+@code{mapped-devices}.++This will also require significant amounts of testing, as various kernel+build options and patches may affect how swapping works, which are possibly+different on Guix System compared to other distributions that this feature is+known to work on.++@item+ZFS Event Daemon. Support for this has not been written yet, patches are+welcome. The main issue is how to design this in a Guix style while+supporting legacy shell-script styles as well. In particular, OpenZFS itself+comes with a number of shell scripts intended for ZFS Event Daemon, and we+need to figure out how the user can choose to use or not use the provided+scripts (and configure any settings they have) or override with their own+custom code (which could be shell scripts they have written and trusted from+previous ZFS installations).++As-is, you can create your own service that activates the ZFS Event Daemon+by creating the @file{/etc/zfs/zed} directory and filling it appropriately,+then launching @code{zed}.++@item+@file{/etc/zfs/zpool.cache}. Currently the ZFS support on Guix always forces+scanning of all devices at bootup to look for ZFS pools. For systems with+dozens or hundreds of storage devices, this can lead to slow bootup. One issue+is that tools should really not write to @code{/etc} which is supposed to be for+configuration; possibly it could be moved to @code{/var} instead. Another issue+is that if Guix ever supports @code{/} on ZFS, we would need to somehow keep the+@code{zpool.cache} file inside the @code{initrd} up-to-date with what is in the+@code{/} mount point.++@item+@code{zfs share}. This will require some (unknown amount of) work to integrate+into the Samba and NFS services of Guix. You @emph{can} manually set up Samba+and NFS to share any mounted ZFS datasets by setting up their configurations+properly; it just can't be done for you by @code{zfs share} and the+@code{sharesmb} and @code{sharenfs} properties.+@end enumerate++Hopefully, support for the above only requires code to be written, and users+are encouraged to hack on Guix to implement the above features.+ @node Mapped Devices @section Mapped Devices
diff --git a/gnu/local.mk b/gnu/local.mkindex d415b892e9..4147badd49 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -45,6 +45,7 @@ # Copyright © 2021 Sharlatan Hellseher <sharlatanus@gmail.com> # Copyright © 2021 Dmitry Polyakov <polyakov@liltechdude.xyz> # Copyright © 2021 Andrew Tropin <andrew@trop.in>+# Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> # # This file is part of GNU Guix. #@@ -633,6 +634,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/docker.scm \ %D%/services/authentication.scm \ %D%/services/file-sharing.scm \+ %D%/services/file-systems.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \ %D%/services/getmail.scm \diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex 50865055fe..d5d33aeada 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -186,7 +186,9 @@
references-file
- %base-services))+ %base-services++ dependency->shepherd-service-name))
;;; Commentary: ;;;diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scmnew file mode 100644index 0000000000..867349c3a5--- /dev/null+++ b/gnu/services/file-systems.scm@@ -0,0 +1,363 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu services file-systems)+ #:use-module (gnu packages file-systems)+ #:use-module (gnu services)+ #:use-module (gnu services base)+ #:use-module (gnu services linux)+ #:use-module (gnu services mcron)+ #:use-module (gnu services shepherd)+ #:use-module (gnu system mapped-devices)+ #:use-module (guix gexp)+ #:use-module (guix modules)+ #:use-module (guix packages)+ #:use-module (guix records)+ #:use-module (srfi srfi-1)+ #:export (zfs-service-type++ zfs-configuration+ zfs-configuration?+ zfs-configuration-kernel+ zfs-configuration-base-zfs+ zfs-configuration-base-zfs-auto-snapshot+ zfs-configuration-dependencies+ zfs-configuration-auto-mount?+ zfs-configuration-auto-scrub+ zfs-configuration-auto-snapshot?+ zfs-configuration-auto-snapshot-keep++ %zfs-zvol-dependency))++(define-record-type* <zfs-configuration>+ zfs-configuration+ make-zfs-configuration+ zfs-configuration?++ ;; linux-libre kernel you want to compile the base-zfs module for.+ (kernel zfs-configuration-kernel)++ ;; the OpenZFS package that will be modified to compile for the+ ;; given kernel.+ ;; Because it is modified and not the actual package that is used,+ ;; we prepend the name 'base-'.+ (base-zfs zfs-configuration-base-zfs+ (default zfs))++ ;; the zfs-auto-snapshot package that will be modified to compile+ ;; for the given kernel.+ ;; Because it is modified and not the actual package that is used,+ ;; we prepend the name 'base-'.+ (base-zfs-auto-snapshot zfs-configuration-base-zfs-auto-snapshot+ (default zfs-auto-snapshot))++ ;; list of <mapped-device> or <file-system> objects that must be+ ;; opened/mounted before we import any ZFS pools.+ (dependencies zfs-configuration-dependencies+ (default '()))++ ;; #t to mount all mountable datasets by default.+ ;; #f if not mounting.+ ;; #t is the expected behavior on other operating systems, the+ ;; #f is only supported for "rescue" operating systems where+ ;; the user wants lower-level control of when to mount.+ (auto-mount? zfs-configuration-auto-mount?+ (default #t))++ ;; 'weekly for weekly scrubbing, 'monthly for monthly scrubbing, an+ ;; mcron time specification that can be given to `job`, or #f to+ ;; disable.+ (auto-scrub zfs-configuration-auto-scrub+ (default 'weekly))++ ;; #t to auto-snapshot by default (and `com.sun:auto-snapshot=false`+ ;; disables auto-snapshot per dataset), #f to not auto-snapshot+ ;; by default (and `com.sun:auto-snapshot=true` enables auto-snapshot+ ;; per dataset).+ (auto-snapshot? zfs-configuration-auto-snapshot?+ (default #t))++ ;; association list of symbol-number pairs to indicate the number+ ;; of automatic snapshots to keep for each of 'frequent, 'hourly,+ ;; 'daily, 'weekly, and 'monthly.+ ;; e.g. '((frequent . 8) (hourly . 12))+ (auto-snapshot-keep zfs-configuration-auto-snapshot-keep+ (default '())))++(define %default-auto-snapshot-keep+ '((frequent . 4)+ (hourly . 24)+ (daily . 31)+ (weekly . 8)+ (monthly . 12)))++(define %auto-snapshot-mcron-schedule+ '((frequent . "0,15,30,45 * * * *")+ (hourly . "0 * * * *")+ (daily . "0 0 * * *")+ (weekly . "0 0 * * 7")+ (monthly . "0 0 1 * *")))++;; A synthetic and unusable MAPPED-DEVICE intended for use when+;; the user has created a mountable filesystem inside a ZFS+;; zvol and wants it mounted inside the configuration.scm.+(define %zfs-zvol-dependency+ (mapped-device+ (source '())+ (targets '("zvol/*"))+ (type #f)))++(define (make-zfs-package conf)+ "Creates a zfs package based on the given zfs-configuration.++ OpenZFS is a kernel package and to ensure best compatibility+ it should be compiled with the specific Linux-Libre kernel+ used on the system. This simply overrides the kernel used+ in compilation with that given in the configuration, which+ the user has to ensure is the same as in the operating-system."+ (let ((kernel (zfs-configuration-kernel conf))+ (base-zfs (zfs-configuration-base-zfs conf)))+ (package+ (inherit base-zfs)+ (arguments (cons* #:linux kernel+ (package-arguments base-zfs))))))++(define (make-zfs-auto-snapshot-package conf)+ "Creates a zfs-auto-snapshot package based on the given+ zfs-configuration.++ Since the OpenZFS tools above are compiled to a specific+ kernel version, zfs-auto-snapshot --- which calls into the+ OpenZFS tools --- has to be compiled with the specific+ modified OpenZFS package created in the make-zfs-package+ procedure."+ (let ((zfs (make-zfs-package conf))+ (base-zfs-auto-snapshot (zfs-configuration-base-zfs-auto-snapshot conf)))+ (package+ (inherit base-zfs-auto-snapshot)+ (inputs `(("zfs" ,zfs))))))++(define (zfs-loadable-modules conf)+ "Specifies that the specific 'module' output of the OpenZFS+ package is to be used; for use in indicating it as a+ loadable kernel module."+ (list (list (make-zfs-package conf) "module")))++(define (zfs-shepherd-services conf)+ "Constructs a list of Shepherd services that is installed+ by the ZFS Guix service.++ 'zfs-scan' scans all devices for ZFS pools, and makes them+ available to 'zpool' commands.+ 'device-mapping-zvol/*' waits for /dev/zvol/* to be+ populated by 'udev', and runs after 'zfs-scan'.+ 'zfs-auto-mount' mounts all ZFS datasets with a 'mount'+ property, which defaults to '/' followed by the name of+ the dataset.++ All the above behavior is expected by ZFS users from+ typical ZFS installations. A mild difference is that+ scanning is usually based on '/etc/zfs/zpool.cache'+ instead of the 'scan all devices' used below, but that+ file is questionable in Guix since ideally '/etc/'+ files are modified by the sysad directly;+ '/etc/zfs/zpool.cache' is modified by ZFS tools."+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (zfs (file-append zfs-package "/sbin/zfs"))+ (zvol_wait (file-append zfs-package "/bin/zvol_wait"))+ (scheme-modules `((srfi srfi-1)+ (srfi srfi-34)+ (srfi srfi-35)+ (rnrs io ports)+ ,@%default-modules)))+ (define zfs-scan+ (shepherd-service+ (provision '(zfs-scan))+ (requirement `(root-file-system+ kernel-module-loader+ udev+ ,@(map dependency->shepherd-service-name+ (zfs-configuration-dependencies conf))))+ (documentation "Scans for and imports ZFS pools.")+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "zfs: error importing pools: ~s~%"+ (condition-message c))+ #f))+ ;; TODO: optionally use a cachefile.+ (invoke #$zpool "import" "-a" "-N"))))+ ;; Why not one-shot? Because we don't really want to rescan+ ;; this each time a requiring process is restarted, as scanning+ ;; can take a long time and a lot of I/O.+ (stop #~(const #f))))++ (define device-mapping-zvol/*+ (shepherd-service+ (provision '(device-mapping-zvol/*))+ (requirement '(zfs-scan))+ (documentation "Waits for all ZFS ZVOLs to be opened.")+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "zfs: error opening zvols: ~s~%"+ (condition-message c))+ #f))+ (invoke #$zvol_wait))))+ (stop #~(const #f))))++ (define zfs-auto-mount+ (shepherd-service+ (provision '(zfs-auto-mount))+ (requirement '(zfs-scan))+ (documentation "Mounts all non-legacy mounted ZFS filesystems.")+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "zfs: error mounting file systems: ~s~%"+ (condition-message c))+ #f))+ ;; Output to current-error-port, otherwise the+ ;; user will not see any prompts for passwords+ ;; of encrypted datasets.+ ;; XXX Maybe better to explicitly open /dev/console ?+ (with-output-to-port (current-error-port)+ (lambda ()+ (invoke #$zfs "mount" "-a" "-l"))))))+ (stop #~(lambda _+ ;; Make sure that Shepherd does not have a CWD that+ ;; is a mounted ZFS filesystem, which would prevent+ ;; unmounting.+ (chdir "/")+ (invoke #$zfs "unmount" "-a" "-f")))))++ `(,zfs-scan+ ,device-mapping-zvol/*+ ,@(if (zfs-configuration-auto-mount? conf)+ `(,zfs-auto-mount)+ '()))))++(define (zfs-user-processes conf)+ "Provides the last Shepherd service that 'user-processes' has to+ wait for.++ If not auto-mounting, then user-processes should only wait for+ the device scan."+ (if (zfs-configuration-auto-mount? conf)+ '(zfs-auto-mount)+ '(zfs-scan)))++(define (zfs-mcron-auto-snapshot-jobs conf)+ "Creates a list of mcron jobs for auto-snapshotting, one for each+ of the standard durations."+ (let* ((user-auto-snapshot-keep (zfs-configuration-auto-snapshot-keep conf))+ ;; assoc-ref has earlier entries overriding later ones.+ (auto-snapshot-keep (append user-auto-snapshot-keep+ %default-auto-snapshot-keep))+ (auto-snapshot? (zfs-configuration-auto-snapshot? conf))+ (zfs-auto-snapshot-package (make-zfs-auto-snapshot-package conf))+ (zfs-auto-snapshot (file-append zfs-auto-snapshot-package+ "/sbin/zfs-auto-snapshot")))+ (map+ (lambda (label)+ (let ((keep (assoc-ref auto-snapshot-keep label))+ (sched (assoc-ref %auto-snapshot-mcron-schedule label)))+ #~(job '#$sched+ (lambda ()+ (system* #$zfs-auto-snapshot+ "--quiet"+ "--syslog"+ #$(string-append "--label="+ (symbol->string label))+ #$(string-append "--keep="+ (number->string keep))+ "//")))))+ (map first %auto-snapshot-mcron-schedule))))++(define (zfs-mcron-auto-scrub-jobs conf)+ "Creates a list of mcron jobs for auto-scrubbing."+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (auto-scrub (zfs-configuration-auto-scrub conf))+ (sched (cond+ ((eq? auto-scrub 'weekly) "0 0 * * 7")+ ((eq? auto-scrub 'monthly) "0 0 1 * *")+ (else auto-scrub))))+ (define code+ ;; We need to get access to (guix build utils) for the+ ;; invoke procedures.+ (with-imported-modules (source-module-closure '((guix build utils)))+ #~(begin+ (use-modules (guix build utils)+ (ice-9 ports))+ ;; The ZFS pools in the system.+ (define pools+ (invoke/quiet #$zpool "list" "-o" "name" "-H"))+ ;; Only scrub if there are actual ZFS pools, as the+ ;; zpool scrub command errors out if given an empty+ ;; argument list.+ (unless (null? pools)+ ;; zpool scrub only initiates the scrub and otherwise+ ;; prints nothing. Results are always seen on the+ ;; zpool status command.+ (apply invoke #$zpool "scrub" pools)))))+ (list+ #~(job '#$sched+ #$(program-file "mcron-zfs-scrub.scm" code)))))++(define (zfs-mcron-jobs conf)+ "Creates a list of mcron jobs for ZFS management."+ (append (zfs-mcron-auto-snapshot-jobs conf)+ (if (zfs-configuration-auto-scrub conf)+ (zfs-mcron-auto-scrub-jobs conf)+ '())))++(define zfs-service-type+ (service-type+ (name 'zfs)+ (extensions+ (list ;; Install OpenZFS kernel module into kernel profile.+ (service-extension linux-loadable-module-service-type+ zfs-loadable-modules)+ ;; And load it.+ (service-extension kernel-module-loader-service-type+ (const '("zfs")))+ ;; Make sure ZFS pools and datasets are mounted at+ ;; boot.+ (service-extension shepherd-root-service-type+ zfs-shepherd-services)+ ;; Make sure user-processes don't start until+ ;; after ZFS does.+ (service-extension user-processes-service-type+ zfs-user-processes)+ ;; Install automated scrubbing and snapshotting.+ (service-extension mcron-service-type+ zfs-mcron-jobs)++ ;; Install ZFS management commands in the system+ ;; profile.+ (service-extension profile-service-type+ (compose list make-zfs-package))+ ;; Install ZFS udev rules.+ (service-extension udev-service-type+ (compose list make-zfs-package))))+ (description "Installs ZFS, an advanced filesystem and volume manager.")))
base-commit: a939011b58c65f4192a10cde9e925e85702bacf4
--2.33.0
R
R
raid5atemyhomework wrote on 19 Oct 15:18 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
vKQTyz5EIfQSpWnnKn7BEJoaQRNhH-yuzsTBshVSM0PpZm1oXCs30kSE_vsO5IGqixUGgnlXReotz-oJWHk6IKokBJlPD4j8Gf8NNpjMzfg=@protonmail.com
***BUMP***
Toggle quote (1280 lines)> Sorry for the lateness everyone.>> Hope this one gets reviewed and merged.>> -------------------------------------------------------------------------->> From 3803e046566278fe12d64f6e39564e9602bf434d Mon Sep 17 00:00:00 2001> From: raid5atemyhomework raid5atemyhomework@protonmail.com> Date: Thu, 30 Sep 2021 16:58:46 +0800> Subject: [PATCH] gnu: Add ZFS service type.>> - gnu/services/file-systems.scm: New file.>> - gnu/local.mk (GNU_SYSTEM_MODULES): Add it.>> - gnu/services/base.scm: Export dependency->shepherd-service-name.>> - doc/guix.texi (ZFS File System): New subsection.>>> doc/guix.texi | 351 ++++++++++++++++++++++++++++++++> gnu/local.mk | 2 +> gnu/services/base.scm | 4 +-> gnu/services/file-systems.scm | 363 ++++++++++++++++++++++++++++++++++> 4 files changed, 719 insertions(+), 1 deletion(-)> create mode 100644 gnu/services/file-systems.scm>> diff --git a/doc/guix.texi b/doc/guix.texi> index a72a726b54..dd38103953 100644> --- a/doc/guix.texi> +++ b/doc/guix.texi> @@ -97,6 +97,7 @@ Copyright @copyright{} 2021 Hui Lu@*> Copyright @copyright{} 2021 pukkamustard@*> Copyright @copyright{} 2021 Alice Brenon@*> Copyright @copyright{} 2021 Andrew Tropin@*> +Copyright @copyright{} 2021 raid5atemyhomework@*>> Permission is granted to copy, distribute and/or modify this document> under the terms of the GNU Free Documentation License, Version 1.3 or> @@ -14435,6 +14436,356 @@ a file system declaration such as:> compress-force=zstd,space_cache=v2"))> @end lisp>> +> +@node ZFS File System> +@subsection ZFS File System> +> +Support for ZFS file systems in Guix is based on the OpenZFS project.> +OpenZFS currently only supports Linux-Libre and is not available on the> +Hurd.> +> +OpenZFS is free software; unfortunately its license is incompatible with> +the GNU General Public License (GPL), the license of the Linux kernel,> +which means they cannot be distributed together. However, as a user,> +you can choose to build ZFS and use it together with Linux; you can> +even rely on Guix to automate this task. See> +@uref{https://www.fsf.org/licensing/zfs-and-linux, this analysis by> +the Free Software Foundation} for more information.> +> +As a large and complex kernel module, OpenZFS has to be compiled for a> +specific version of Linux-Libre. At times, the latest OpenZFS package> +available in Guix is not compatible with the latest Linux-Libre version.> +Thus, directly installing the @code{zfs} package can fail.> +> +Instead, you are recommended to select a specific older long-term-support> +Linux-Libre kernel. Do not use @code{linux-libre-lts}, as even the> +latest long-term-support kernel may be too new for @code{zfs}. Instead,> +explicitly select a specific older version, such as @code{linux-libre-5.10},> +and upgrade it manually later as new long-term-support kernels become> +available that you have confirmed is compatible with the latest available> +OpenZFS version on Guix.> +> +For example, you can modify your system configuration file to a specific> +Linux-Libre version and add the @code{zfs-service-type} service.> +> +@lisp> +(use-modules (gnu))> +(use-package-modules>> - #;@dots{}> - linux)> +(use-service-modules>> - #;@dots{}> - file-systems)> ->> +(define my-kernel linux-libre-5.10)> +> +(operating-system>> - (kernel my-kernel)> - #;@dots{}> - (services> - (cons* (service zfs-service-type> - (zfs-configuration>>> - (kernel my-kernel)))>>> - #;@dots{}>>> - %desktop-services))>>> - #;@dots{})> +@end lisp>> ->> +@defvr {Scheme Variable} zfs-service-type> +This is the type for a service that adds ZFS support to your operating> +system. The service is configured using a @code{zfs-configuration}> +record.> +> +Here is an example use:> +> +@lisp> +(service zfs-service-type>> - (zfs-configuration> - (kernel linux-libre-5.4)))> +@end lisp> +@end defvr>> ->> +@deftp {Data Type} zfs-configuration> +This data type represents the configuration of the ZFS support in Guix> +System. Its fields are:> +> +@table @asis> +@item @code{kernel}> +The package of the Linux-Libre kernel to compile OpenZFS for. This field> +is always required. It @emph{must} be the same kernel you use in your> +@code{operating-system} form.> +> +@item @code{base-zfs} (default: @code{zfs})> +The OpenZFS package that will be compiled for the given Linux-Libre kernel.> +> +@item @code{base-zfs-auto-snapshot} (default: @code{zfs-auto-snapshot})> +The @code{zfs-auto-snapshot} package to use. It will be modified to> +specifically use the OpenZFS compiled for your kernel.> +> +@item @code{dependencies} (default: @code{'()})> +A list of @code{<file-system>} or @code{<mapped-device>} records that must> +be mounted or opened before OpenZFS scans for pools to import. For example,> +if you have set up LUKS containers as leaf VDEVs in a pool, you have to> +include their corresponding @code{<mapped-ddevice>} records so that OpenZFS> +can import the pool correctly at bootup.> +> +@item @code{auto-mount?} (default: @code{#t})> +Whether to mount datasets with the ZFS @code{mountpoint} property automatically> +at startup. This is the behavior that ZFS users usually expect. You might> +set this to @code{#f} for an operating system intended as a `rescue'' system +that is intended to help debug problems with the disks rather than actually +work in production. + +@item @code{auto-scrub} (default: @code{'weekly}) +Specifies how often to scrub all pools. Can be the symbols @code{'weekly} or +@code{'monthly}, or a schedule specification understood by +@xref{mcron, mcron job specifications,, mcron, GNU@tie{}mcron}, such as +@code{"0 3 * * 6"} for`every 3AM on Saturday''.> +It can also be @code{#f} to disable auto-scrubbing (@strong{not recommended}).> +> +The general guideline is to scrub weekly when using consumer-quality drives, and> +to scrub monthly when using enterprise-quality drives.> +> +@code{'weekly} scrubs are done on Sunday midnight, while @code{monthly} scrubs> +are done on midnight on the first day of each month.> +> +@item @code{auto-snapshot?} (default: @code{#t})> +Specifies whether to auto-snapshot by default. If @code{#t}, then snapshots> +are automatically created except for ZFS datasets with the> +@code{com.sun:auto-snapshot} ZFS vendor property set to @code{false}.> +> +If @code{#f}, snapshots will not be automatically created, unless the ZFS> +dataset has the @code{com.sun:auto-snapshot} ZFS vendor property set to> +@code{true}.> +> +@item @code{auto-snapshot-keep} (default: @code{'()})> +Specifies an association list of symbol-number pairs, indicating the number> +of automatically-created snapshots to retain for each frequency type.> +> +If not specified via this field, by default there are 4 @code{frequent}, 24> +@code{hourly}, 31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly} snapshots.> +> +For example:> +> +@lisp> +(zfs-configuration>> - (kernel my-kernel)> - (auto-snapshot-keep> - '((frequent . 8)> - (hourly . 12))))>>>> +@end lisp> +> +The above will keep 8 @code{frequent} snapshots and 12 @code{hourly} snapshots.> +@code{daily}, @code{weekly}, and @code{monthly} snapshots will keep their> +defaults (31 @code{daily}, 8 @code{weekly}, and 12 @code{monthly}).> +> +@end table> +@end deftp> +> +@subsubsection ZFS Auto-Snapshot> +> +The ZFS service on Guix System supports auto-snapshots as implemented in the> +Solaris operating system.> +> +@code{frequent} (every 15 minutes), @code{hourly}, @code{daily}, @code{weekly},> +and @code{monthly} snapshots are created automatically for ZFS datasets that> +have auto-snapshot enabled. They will be named, for example,> +@code{zfs-auto-snap_frequent-2021-03-22-1415}. You can continue to use> +manually-created snapshots as long as they do not conflict with the naming> +convention used by auto-snapshot. You can also safely manually destroy> +automatically-created snapshots, for example to free up space.> +> +The @code{com.sun:auto-snapshot} ZFS property controls auto-snapshot on a> +per-dataset level. Sub-datasets will inherit this property from their parent> +dataset, but can have their own property.> +> +You @emph{must} set this property to @code{true} or @code{false} exactly,> +otherwise it will be treated as if the property is unset.> +> +For example:> +> +@example> +# zfs list -o name> +NAME> +tank> +tank/important-data> +tank/tmp> +# zfs set com.sun:auto-snapshot=true tank> +# zfs set com.sun:auto-snapshot=false tank/tmp> +@end example> +> +The above will set @code{tank} and @code{tank/important-data} to be> +auto-snapshot, while @code{tank/tmp} will not be auto-snapshot.> +> +If the @code{com.sun:auto-snapshot} property is not set for a dataset> +(the default when pools and datasets are created), then whether> +auto-snapshot is done or not will depend on the @code{auto-snapshot?}> +field of the @code{zfs-configuration} record.> +> +There are also @code{com.sun:auto-snapshot:frequent},> +@code{com.sun:auto-snapshot:hourly}, @code{com.sun:auto-snapshot:daily},> +@code{com.sun:auto-snapshot:weekly}, and @code{com.sun:auto-snapshot:monthly}> +properties that give finer-grained control of whether to auto-snapshot a> +dataset at a particular schedule.> +> +The number of snapshots kept for all datasets can be overridden via the> +@code{auto-snapshot-keep} field of the @code{zfs-configuration} record.> +There is currently no support to have different numbers of snapshots to> +keep for different datasets.> +> +@subsubsection ZVOLs> +> +ZFS supports ZVOLs, block devices that ZFS exposes to the operating> +system in the @code{/dev/zvol/} directory. The ZVOL will have the same> +resilience and self-healing properties as other datasets on your ZFS pool.> +ZVOLs can also be snapshotted (and will be included in auto-snapshotting> +if enabled), which snapshots the state of the block device, effectively> +snapshotting the hosted file system.> +> +You can put any file system inside the ZVOL. However, in order to mount this> +file system at system start, you need to add @code{%zfs-zvol-dependency} as a> +dependency of each file system inside a ZVOL.> +> +@defvr {Scheme Variable} %zfs-zvol-dependency> +An artificial @code{<mapped-device>} which tells the file system mounting> +service to wait for ZFS to provide ZVOLs before mounting the> +@code{<file-system>} dependent on it.> +@end defvr> +> +For example, suppose you create a ZVOL and put an ext4 filesystem> +inside it:> +> +@example> +# zfs create -V 100G tank/ext4-on-zfs> +# mkfs.ext4 /dev/zvol/tank/ext4-on-zfs> +# mkdir /ext4-on-zfs> +# mount /dev/zvol/tank/ext4-on-zfs /ext4-on-zfs> +@end example> +> +You can then set this up to be mounted at boot by adding this to the> +@code{file-systems} field of your @code{operating-system} record:> +> +@lisp> +(file-system>> - (device "/dev/zvol/tank/ext4-on-zfs")> - (mount-point "/ext4-on-zfs")> - (type "ext4")> - (dependencies (list %zfs-zvol-dependency)))> +@end lisp>> ->> +You @emph{must not} add @code{%zfs-zvol-dependency} to your> +@code{operating-system}'s @code{mapped-devices} field, and you @emph{must> +not} add it (or any @code{<file-system>}s dependent on it) to the> +@code{dependencies} field of @code{zfs-configuration}. Finally, you> +@emph{must not} use @code{%zfs-zvol-dependency} unless you actually> +instantiate @code{zfs-service-type} on your system.> +> +@subsubsection Unsupported Features> +> +Some common features and uses of ZFS are currently not supported, or not> +fully supported, on Guix.> +> +@enumerate> +@item> +Shepherd-managed daemons that are configured to read from or write to ZFS> +mountpoints need to include @code{user-processes} in their @code{requirement}> +field. This is the earliest that ZFS file systems are assured of being> +mounted.> +> +Generally, most daemons will, directly or indirectly, require> +@code{networking}, or @code{user-processes}, or both. Most implementations> +of @code{networking} will also require @code{user-processes} so daemons that> +require only @code{networking} will also generally start up after> +@code{user-processes}. A notable exception, however, is> +@code{static-networking-service-type}. You will need to explicitly add> +@code{user-processes} as a @code{requirement} of your @code{static-networking}> +record.> +> +@item> +@code{mountpoint=legacy} ZFS file systems. The handlers for the Guix mounting> +system have not yet been modified to support ZFS, and will expect @code{/dev}> +paths in the @code{<file-system>}'s @code{device} field, but ZFS file systems> +are referred to via non-path @code{pool/file/system} names. Such file systems> +also need to be mounted @emph{after} OpenZFS has scanned for pools.> +> +You can still manually mount these file systems after system boot; what is> +only unsupported is mounting them automatically at system boot by specifying> +them in @code{<file-system>} records of your @code{operating-system}.>> ->> +@item> +@code{/home} on ZFS. Guix will create home directories for users, but this> +process currently cannot be scheduled after ZFS file systems are mounted.> +Thus, the ZFS file system might be mounted @emph{after} Guix has created> +home directories at boot, at which point OpenZFS will refuse to mount since> +the mountpoint is not empty. However, you @emph{can} create an ext4, xfs,> +btrfs, or other supported file system inside a ZVOL, have that depend on> +@code{%zfs-zvol-dependency}, and set it to mount on the @code{/home}> +directory; they will be scheduled to mount before the @code{user-homes}> +process.> +> +Similarly, other locations like @code{/var}, @code{/gnu/store} and so> +on cannot be reliably put in a ZFS file system, though they may be> +possible to create as other file systems inside ZVOL containers.> +> +@item> +@code{/} and @code{/boot} on ZFS. These require Guix to expose more of> +the @code{initrd} very early boot process to services. It also requires> +Guix to have the ability to explicitly load modules while still in> +@code{initrd} (currently kernel modules loaded by> +@code{kernel-module-loader-service-type} are loaded after @code{/} is> +mounted). Further, since one of ZFS's main advantages is that it can> +continue working despite the loss of one or more devices, it makes sense> +to also support installing the bootloader on all devices of the pool that> +contains the @code{/} and @code{/boot}; after all, if ZFS can survive the> +loss of one device, the bootloader should also be able to survive the loss> +of one device.> +> +@item> +ZVOL swap devices. Mapped swap devices need to be listed in> +@code{mapped-devices} to ensure they are opened before the system attempts> +to use them, but you cannot currently add @code{%zfs-zvol-dependency} to> +@code{mapped-devices}.> +> +This will also require significant amounts of testing, as various kernel> +build options and patches may affect how swapping works, which are possibly> +different on Guix System compared to other distributions that this feature is> +known to work on.> +> +@item> +ZFS Event Daemon. Support for this has not been written yet, patches are> +welcome. The main issue is how to design this in a Guix style while> +supporting legacy shell-script styles as well. In particular, OpenZFS itself> +comes with a number of shell scripts intended for ZFS Event Daemon, and we> +need to figure out how the user can choose to use or not use the provided> +scripts (and configure any settings they have) or override with their own> +custom code (which could be shell scripts they have written and trusted from> +previous ZFS installations).> +> +As-is, you can create your own service that activates the ZFS Event Daemon> +by creating the @file{/etc/zfs/zed} directory and filling it appropriately,> +then launching @code{zed}.> +> +@item> +@file{/etc/zfs/zpool.cache}. Currently the ZFS support on Guix always forces> +scanning of all devices at bootup to look for ZFS pools. For systems with> +dozens or hundreds of storage devices, this can lead to slow bootup. One issue> +is that tools should really not write to @code{/etc} which is supposed to be for> +configuration; possibly it could be moved to @code{/var} instead. Another issue> +is that if Guix ever supports @code{/} on ZFS, we would need to somehow keep the> +@code{zpool.cache} file inside the @code{initrd} up-to-date with what is in the> +@code{/} mount point.> +> +@item> +@code{zfs share}. This will require some (unknown amount of) work to integrate> +into the Samba and NFS services of Guix. You @emph{can} manually set up Samba> +and NFS to share any mounted ZFS datasets by setting up their configurations> +properly; it just can't be done for you by @code{zfs share} and the> +@code{sharesmb} and @code{sharenfs} properties.> +@end enumerate> +> +Hopefully, support for the above only requires code to be written, and users> +are encouraged to hack on Guix to implement the above features.> +> @node Mapped Devices> @section Mapped Devices>> diff --git a/gnu/local.mk b/gnu/local.mk> index d415b892e9..4147badd49 100644> --- a/gnu/local.mk> +++ b/gnu/local.mk> @@ -45,6 +45,7 @@>> Copyright © 2021 Sharlatan Hellseher sharlatanus@gmail.com>> ===========================================================>> Copyright © 2021 Dmitry Polyakov polyakov@liltechdude.xyz>> ==========================================================>> Copyright © 2021 Andrew Tropin andrew@trop.in>> ==============================================>> +# Copyright © 2021 raid5atemyhomework raid5atemyhomework@protonmail.com>> ==>> This file is part of GNU Guix.>> ===============================>> ==>> @@ -633,6 +634,7 @@ GNU_SYSTEM_MODULES = \> %D%/services/docker.scm \> %D%/services/authentication.scm \> %D%/services/file-sharing.scm \>> - %D%/services/file-systems.scm \> %D%/services/games.scm \> %D%/services/ganeti.scm \> %D%/services/getmail.scm \> diff --git a/gnu/services/base.scm b/gnu/services/base.scm> index 50865055fe..d5d33aeada 100644> --- a/gnu/services/base.scm> +++ b/gnu/services/base.scm> @@ -186,7 +186,9 @@>> references-file>>> - %base-services))>>>> - %base-services>>> -> - dependency->shepherd-service-name))>>>> ;;; Commentary:> ;;;> diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm> new file mode 100644> index 0000000000..867349c3a5> --- /dev/null> +++ b/gnu/services/file-systems.scm> @@ -0,0 +1,363 @@> +;;; GNU Guix --- Functional package management for GNU> +;;; Copyright © 2021 raid5atemyhomework raid5atemyhomework@protonmail.com> +;;;> +;;; This file is part of GNU Guix.> +;;;> +;;; GNU Guix is free software; you can redistribute it and/or modify it> +;;; under the terms of the GNU General Public License as published by> +;;; the Free Software Foundation; either version 3 of the License, or (at> +;;; your option) any later version.> +;;;> +;;; GNU Guix is distributed in the hope that it will be useful, but> +;;; WITHOUT ANY WARRANTY; without even the implied warranty of> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the> +;;; GNU General Public License for more details.> +;;;> +;;; You should have received a copy of the GNU General Public License> +;;; along with GNU Guix. If not, see http://www.gnu.org/licenses/.>> ->> +(define-module (gnu services file-systems)>> - #:use-module (gnu packages file-systems)> - #:use-module (gnu services)> - #:use-module (gnu services base)> - #:use-module (gnu services linux)> - #:use-module (gnu services mcron)> - #:use-module (gnu services shepherd)> - #:use-module (gnu system mapped-devices)> - #:use-module (guix gexp)> - #:use-module (guix modules)> - #:use-module (guix packages)> - #:use-module (guix records)> - #:use-module (srfi srfi-1)> - #:export (zfs-service-type> -> - zfs-configuration>>> - zfs-configuration?>>> - zfs-configuration-kernel>>> - zfs-configuration-base-zfs>>> - zfs-configuration-base-zfs-auto-snapshot>>> - zfs-configuration-dependencies>>> - zfs-configuration-auto-mount?>>> - zfs-configuration-auto-scrub>>> - zfs-configuration-auto-snapshot?>>> - zfs-configuration-auto-snapshot-keep>>> -> - %zfs-zvol-dependency))>>> ->> +(define-record-type* <zfs-configuration>>> - zfs-configuration>> - make-zfs-configuration>> - zfs-configuration?>> -> - ;; linux-libre kernel you want to compile the base-zfs module for.>> - (kernel zfs-configuration-kernel)>> -> - ;; the OpenZFS package that will be modified to compile for the>> - ;; given kernel.>> - ;; Because it is modified and not the actual package that is used,>> - ;; we prepend the name 'base-'.>> - (base-zfs zfs-configuration-base-zfs>> - (default zfs))>>> -> - ;; the zfs-auto-snapshot package that will be modified to compile>> - ;; for the given kernel.>> - ;; Because it is modified and not the actual package that is used,>> - ;; we prepend the name 'base-'.>> - (base-zfs-auto-snapshot zfs-configuration-base-zfs-auto-snapshot>> - (default zfs-auto-snapshot))>>> -> - ;; list of <mapped-device> or <file-system> objects that must be>> - ;; opened/mounted before we import any ZFS pools.>> - (dependencies zfs-configuration-dependencies>> - (default '()))>>> -> - ;; #t to mount all mountable datasets by default.>> - ;; #f if not mounting.>> - ;; #t is the expected behavior on other operating systems, the>> - ;; #f is only supported for "rescue" operating systems where>> - ;; the user wants lower-level control of when to mount.>> - (auto-mount? zfs-configuration-auto-mount?>> - (default #t))>>> -> - ;; 'weekly for weekly scrubbing, 'monthly for monthly scrubbing, an>> - ;; mcron time specification that can be given to `job`, or #f to>> - ;; disable.>> - (auto-scrub zfs-configuration-auto-scrub>> - (default 'weekly))>>> -> - ;; #t to auto-snapshot by default (and `com.sun:auto-snapshot=false`>> - ;; disables auto-snapshot per dataset), #f to not auto-snapshot>> - ;; by default (and `com.sun:auto-snapshot=true` enables auto-snapshot>> - ;; per dataset).>> - (auto-snapshot? zfs-configuration-auto-snapshot?>> - (default #t))>>> -> - ;; association list of symbol-number pairs to indicate the number>> - ;; of automatic snapshots to keep for each of 'frequent, 'hourly,>> - ;; 'daily, 'weekly, and 'monthly.>> - ;; e.g. '((frequent . 8) (hourly . 12))>> - (auto-snapshot-keep zfs-configuration-auto-snapshot-keep>> - (default '())))>>> ->> +(define %default-auto-snapshot-keep>> - '((frequent . 4)> - (hourly . 24)> - (daily . 31)> - (weekly . 8)> - (monthly . 12)))> ->> +(define %auto-snapshot-mcron-schedule>> - '((frequent . "0,15,30,45 * * * *")> - (hourly . "0 * * * *")> - (daily . "0 0 * * *")> - (weekly . "0 0 * * 7")> - (monthly . "0 0 1 * *")))> ->> +;; A synthetic and unusable MAPPED-DEVICE intended for use when> +;; the user has created a mountable filesystem inside a ZFS> +;; zvol and wants it mounted inside the configuration.scm.> +(define %zfs-zvol-dependency>> - (mapped-device> - (source '())> - (targets '("zvol/*"))> - (type #f)))> ->> +(define (make-zfs-package conf)>> - "Creates a zfs package based on the given zfs-configuration.> -> - OpenZFS is a kernel package and to ensure best compatibility> - it should be compiled with the specific Linux-Libre kernel> - used on the system. This simply overrides the kernel used> - in compilation with that given in the configuration, which> - the user has to ensure is the same as in the operating-system."> - (let ((kernel (zfs-configuration-kernel conf))> - (base-zfs (zfs-configuration-base-zfs conf)))>>> - (package> - (inherit base-zfs)>>> - (arguments (cons* #:linux kernel>>> - (package-arguments base-zfs))))))>>> ->> +(define (make-zfs-auto-snapshot-package conf)>> - "Creates a zfs-auto-snapshot package based on the given> - zfs-configuration.> -> - Since the OpenZFS tools above are compiled to a specific> - kernel version, zfs-auto-snapshot --- which calls into the> - OpenZFS tools --- has to be compiled with the specific> - modified OpenZFS package created in the make-zfs-package> - procedure."> - (let ((zfs (make-zfs-package conf))> - (base-zfs-auto-snapshot (zfs-configuration-base-zfs-auto-snapshot conf)))>>> - (package> - (inherit base-zfs-auto-snapshot)>>> - (inputs `(("zfs" ,zfs))))))>>> ->> +(define (zfs-loadable-modules conf)>> - "Specifies that the specific 'module' output of the OpenZFS> - package is to be used; for use in indicating it as a> - loadable kernel module."> - (list (list (make-zfs-package conf) "module")))> ->> +(define (zfs-shepherd-services conf)>> - "Constructs a list of Shepherd services that is installed>> - by the ZFS Guix service.>> -> - 'zfs-scan' scans all devices for ZFS pools, and makes them>> - available to 'zpool' commands.>> - 'device-mapping-zvol/' waits for /dev/zvol/ to be>> - populated by 'udev', and runs after 'zfs-scan'.>> - 'zfs-auto-mount' mounts all ZFS datasets with a 'mount'>> - property, which defaults to '/' followed by the name of>> - the dataset.>> -> - All the above behavior is expected by ZFS users from>> - typical ZFS installations. A mild difference is that>> - scanning is usually based on '/etc/zfs/zpool.cache'>> - instead of the 'scan all devices' used below, but that>> - file is questionable in Guix since ideally '/etc/'>> - files are modified by the sysad directly;>> - '/etc/zfs/zpool.cache' is modified by ZFS tools.">> - (let* ((zfs-package (make-zfs-package conf))>> - (zpool (file-append zfs-package "/sbin/zpool"))>>> - (zfs (file-append zfs-package "/sbin/zfs"))>>> - (zvol_wait (file-append zfs-package "/bin/zvol_wait"))>>> - (scheme-modules `((srfi srfi-1)>>> - (srfi srfi-34)>>> - (srfi srfi-35)>>> - (rnrs io ports)>>> - ,@%default-modules)))>>> - (define zfs-scan>> - (shepherd-service>>> - (provision '(zfs-scan))>>> - (requirement `(root-file-system>>> - kernel-module-loader>>> - udev>>> - ,@(map dependency->shepherd-service-name>>> - (zfs-configuration-dependencies conf))))>>> - (documentation "Scans for and imports ZFS pools.")>>> - (modules scheme-modules)>>> - (start #~(lambda _>>> - (guard (c ((message-condition? c)>>> - (format (current-error-port)>>> - "zfs: error importing pools: ~s~%">>> - (condition-message c))>>> - #f))>>> - ;; TODO: optionally use a cachefile.>>> - (invoke #$zpool "import" "-a" "-N"))))>>> - ;; Why not one-shot? Because we don't really want to rescan>>> - ;; this each time a requiring process is restarted, as scanning>>> - ;; can take a long time and a lot of I/O.>>> - (stop #~(const #f))))>>> -> - (define device-mapping-zvol/*>> - (shepherd-service>>> - (provision '(device-mapping-zvol/*))>>> - (requirement '(zfs-scan))>>> - (documentation "Waits for all ZFS ZVOLs to be opened.")>>> - (modules scheme-modules)>>> - (start #~(lambda _>>> - (guard (c ((message-condition? c)>>> - (format (current-error-port)>>> - "zfs: error opening zvols: ~s~%">>> - (condition-message c))>>> - #f))>>> - (invoke #$zvol_wait))))>>> - (stop #~(const #f))))>>> -> - (define zfs-auto-mount>> - (shepherd-service>>> - (provision '(zfs-auto-mount))>>> - (requirement '(zfs-scan))>>> - (documentation "Mounts all non-legacy mounted ZFS filesystems.")>>> - (modules scheme-modules)>>> - (start #~(lambda _>>> - (guard (c ((message-condition? c)>>> - (format (current-error-port)>>> - "zfs: error mounting file systems: ~s~%">>> - (condition-message c))>>> - #f))>>> - ;; Output to current-error-port, otherwise the>>> - ;; user will not see any prompts for passwords>>> - ;; of encrypted datasets.>>> - ;; XXX Maybe better to explicitly open /dev/console ?>>> - (with-output-to-port (current-error-port)>>> - (lambda ()>>> - (invoke #$zfs "mount" "-a" "-l"))))))>>> - (stop #~(lambda _>>> - ;; Make sure that Shepherd does not have a CWD that>>> - ;; is a mounted ZFS filesystem, which would prevent>>> - ;; unmounting.>>> - (chdir "/")>>> - (invoke #$zfs "unmount" "-a" "-f")))))>>> -> - `(,zfs-scan>> - ,device-mapping-zvol/*>>> - ,@(if (zfs-configuration-auto-mount? conf)>>> - `(,zfs-auto-mount)>>> - '()))))>>> ->> +(define (zfs-user-processes conf)>> - "Provides the last Shepherd service that 'user-processes' has to> - wait for.> -> - If not auto-mounting, then user-processes should only wait for> - the device scan."> - (if (zfs-configuration-auto-mount? conf)> - '(zfs-auto-mount)>>> - '(zfs-scan)))>>> ->> +(define (zfs-mcron-auto-snapshot-jobs conf)>> - "Creates a list of mcron jobs for auto-snapshotting, one for each>> - of the standard durations.">> - (let* ((user-auto-snapshot-keep (zfs-configuration-auto-snapshot-keep conf))>> - ;; assoc-ref has earlier entries overriding later ones.>>> - (auto-snapshot-keep (append user-auto-snapshot-keep>>> - %default-auto-snapshot-keep))>>> - (auto-snapshot? (zfs-configuration-auto-snapshot? conf))>>> - (zfs-auto-snapshot-package (make-zfs-auto-snapshot-package conf))>>> - (zfs-auto-snapshot (file-append zfs-auto-snapshot-package>>> - "/sbin/zfs-auto-snapshot")))>>> - (map>> - (lambda (label)>>> - (let ((keep (assoc-ref auto-snapshot-keep label))>>> - (sched (assoc-ref %auto-snapshot-mcron-schedule label)))>>> - #~(job '#$sched>>> - (lambda ()>>> - (system* #$zfs-auto-snapshot>>> - "--quiet">>> - "--syslog">>> - #$(string-append "--label=">>> - (symbol->string label))>>> - #$(string-append "--keep=">>> - (number->string keep))>>> - "//")))))>>> - (map first %auto-snapshot-mcron-schedule))))>>> ->> +(define (zfs-mcron-auto-scrub-jobs conf)>> - "Creates a list of mcron jobs for auto-scrubbing."> - (let* ((zfs-package (make-zfs-package conf))> - (zpool (file-append zfs-package "/sbin/zpool"))>>> - (auto-scrub (zfs-configuration-auto-scrub conf))>>> - (sched (cond>>> - ((eq? auto-scrub 'weekly) "0 0 * * 7")>>> - ((eq? auto-scrub 'monthly) "0 0 1 * *")>>> - (else auto-scrub))))>>> - (define code> - ;; We need to get access to (guix build utils) for the>>> - ;; invoke procedures.>>> - (with-imported-modules (source-module-closure '((guix build utils)))>>> - #~(begin>>> - (use-modules (guix build utils)>>> - (ice-9 ports))>>> - ;; The ZFS pools in the system.>>> - (define pools>>> - (invoke/quiet #$zpool "list" "-o" "name" "-H"))>>> - ;; Only scrub if there are actual ZFS pools, as the>>> - ;; zpool scrub command errors out if given an empty>>> - ;; argument list.>>> - (unless (null? pools)>>> - ;; zpool scrub only initiates the scrub and otherwise>>> - ;; prints nothing. Results are always seen on the>>> - ;; zpool status command.>>> - (apply invoke #$zpool "scrub" pools)))))>>> - (list> - #~(job '#$sched>>> - #$(program-file "mcron-zfs-scrub.scm" code)))))>>> ->> +(define (zfs-mcron-jobs conf)>> - "Creates a list of mcron jobs for ZFS management."> - (append (zfs-mcron-auto-snapshot-jobs conf)> - (if (zfs-configuration-auto-scrub conf)>>> - (zfs-mcron-auto-scrub-jobs conf)>>> - '())))>>> ->> +(define zfs-service-type>> - (service-type> - (name 'zfs)> - (extensions> - (list ;; Install OpenZFS kernel module into kernel profile.>>> - (service-extension linux-loadable-module-service-type>>> - zfs-loadable-modules)>>> - ;; And load it.>>> - (service-extension kernel-module-loader-service-type>>> - (const '("zfs")))>>> - ;; Make sure ZFS pools and datasets are mounted at>>> - ;; boot.>>> - (service-extension shepherd-root-service-type>>> - zfs-shepherd-services)>>> - ;; Make sure user-processes don't start until>>> - ;; after ZFS does.>>> - (service-extension user-processes-service-type>>> - zfs-user-processes)>>> - ;; Install automated scrubbing and snapshotting.>>> - (service-extension mcron-service-type>>> - zfs-mcron-jobs)>>> -> - ;; Install ZFS management commands in the system>>> - ;; profile.>>> - (service-extension profile-service-type>>> - (compose list make-zfs-package))>>> - ;; Install ZFS udev rules.>>> - (service-extension udev-service-type>>> - (compose list make-zfs-package))))>>> - (description "Installs ZFS, an advanced filesystem and volume manager.")))>> base-commit: a939011b58c65f4192a10cde9e925e85702bacf4> --> 2.33.0>
R
R
raid5atemyhomework wrote on 27 Oct 09:30 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
cdI3QeJup0QuZbhEU_KV5hfOx94QM7APxrzK8A-KHscrz63kKJLmnTuLec-qB3SC6yyc3aTadLClFu8SEN5QZcnQjaporMZXdTXxI05SFNg=@protonmail.com
WHEEEEEEEEEEEEEEEEEEEE ***BUMP***
P
P
pelzflorian (Florian Pelz) wrote on 27 Oct 18:38 +0200
Re: [bug#45692] [PATCH v5 3/3] gnu: Add ZFS service type.
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
20211027163827.4olhr3ks5s4sxkzp@pelzflorian.localdomain
On Wed, Oct 27, 2021 at 07:30:11AM +0000, raid5atemyhomework via Guix-patches via wrote:
Toggle quote (2 lines)> WHEEEEEEEEEEEEEEEEEEEE ***BUMP***
I’m sorry it takes so long (I’m not qualified to review), but pleasenote that Maxime’s e-mail setup broke down:
https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00057.html
Regards,Florian
R
R
raid5atemyhomework wrote 8 hours ago
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)
5YZ8GHQ55G_bk1ENRQbMKQT2bZq1jeplfowx_SukADa1rur5xYROOvtIFQRgmnahZ-mhXsy26cgvMhpAdHavnEGbmkeSH_jS54qn0EY5Ajk=@protonmail.com
Dear Santa,
For Christmas, I would like to have a reviewer for my patch.
I have been very good this year. I have been regularly scrubbing my ZFS pool every week and I keep monthly backups of my homework now. And, I don't store my homework on broken RAID5 implementations anymore.
Thanksraid5atemyhomework
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 45692@debbugs.gnu.org