[PATCH 0/4] Even Better ZFS Support on Guix

OpenSubmitted by raid5atemyhomework.
Details
4 participants
  • Danny Milosavljevic
  • 宋文武
  • Ludovic Courtès
  • raid5atemyhomework
Owner
unassigned
Severity
normal
Merged with
R
R
raid5atemyhomework wrote on 6 Jan 16:52 +0100
(name . guix-patches@gnu.org)(address . guix-patches@gnu.org)
f9YNSdmHSIAVHWwDRwnh8QVeNjUG4M6YMJMntulZ3s8cYPVCBHrVIirwkuGxpYIduWVNAyi13dFVlqUKs7_QXPDBvbDe_JNNEC_SoNBTprk=@protonmail.com
Hi all,
Sorry for the excessive number of patches.
This patchset contains 4 patches, two that add new features to Guix and do not involve ZFS at all, one bugfix of the ZFS package, and one which makes installing ZFS on Guix easy.
* Patch 1: Create an extensible service type that installs kernel-loadable modules into the kernel. * This allows a ZFS service to add the ZFS module to the kernel.* Patch 2: Make the `file-systems` shepherd service an extensible target, so that other services can add more requirements for `file-systems`. * This allows a ZFS service to make `file-systems` wait for the ZFS automount at startup, which is necessary in any case in order to ensure that ZFS has scanned for ZFS pools installed in the hardware.* Patch 3: Fix the ZFS package definition.* Patch 4: Add a ZFS service type that installs everything needed for basic ZFS usage!
I've tested the ZFS functionality in about a half-dozen VMs already, including checking that the ZFS setup persists across reboots of the VM:
* No ZFS installed.* With ZFS installed but without any ZFS pools.* With ZFS pool that is automounted.* With ZFS pool in legacy mode (needs manual `mount -t zfs poolname /mountpoint`).* Encrypted ZFS dataset (prompts passphrase at boot to mount).* Encrypted ZFS dataseet as `/home` (prompts passphrase at boot to mount, user `/home` directories are created and maintained).
Other bits of ZFS functionality would be *nice* but aren't in this patchset yet (and at least for my purposes are not really necessary, except perhaps the auto-scrubbing, I just need to wrap my head around `mcron`...):
* Auto-scrubbing of ZFS pools to ensure no data corruption.* Auto-snapshotting of ZFS datasets to protect against user error.* `/` on ZFS.* `/boot` on ZFS.* Managing ZFS "legacy" mountpoints on `(file-system ...)` declarations.* Managing ZFS pool setup in the `operating-system` declaration somehow (not sure how to do this).
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 6 Jan 16:54 +0100
[PATCH 1/4] gnu: Allow services to install kernel-loadable modules.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
2yOs7mxigs8_Vx_yi858pxAkoyXmU8IidizK5grcmX4M92O5QK-_Ywo9nrHMHvzLGj4U7B9ysl26eNbiPmzmeOQsuFvp5VjouVptRfwxKng=@protonmail.com
From 4beb73c62995cf236b402dad8e1c36016027c781 Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Tue, 5 Jan 2021 22:27:56 +0800Subject: [PATCH 1/4] gnu: Allow services to install kernel-loadable modules.
* gnu/system.scm (operating-system-directory-base-entries): Remove codeto handle generation of "kernel" and "hurd".(operating-system-default-essential-services): Instantiatekernel-loadable-module-service.(hurd-default-essential-services): Instantiatekernel-loadable-module-service.(package-for-kernel): Move ...* gnu/services.scm: ... to here.(kernel-loadable-module-service-type): New variable.(kernel-loadable-module-service): New procedure.* gnu/tests/linux-modules.scm (run-loadable-kernel-modules-test): Movecode to ...(run-loadable-kernel-modules-test-base): ... new procedure here.(run-loadable-kernel-modules-service-test): New procedure.(%test-loadable-kernel-modules-service-0): New variable.(%test-loadable-kernel-modules-service-1): New variable.(%test-loadable-kernel-modules-service-2): New variable.* doc/guix.texi: Document kernel-loadable-module-service-type.--- doc/guix.texi | 6 +++ gnu/services.scm | 70 ++++++++++++++++++++++++++++++++ gnu/system.scm | 37 +++++------------ gnu/tests/linux-modules.scm | 81 ++++++++++++++++++++++++++++++++----- 4 files changed, 157 insertions(+), 37 deletions(-)
Toggle diff (324 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 0f6e95a65a..78770151e3 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -32409,6 +32409,12 @@ configuration when you use @command{guix system reconfigure}, @command{guix system init}, or @command{guix deploy}. @end defvr
+@defvr {Scheme Variable} kernel-loadable-module-service-type+Type of the service that collects lists of packages containing+kernel-loadable modules, and adds them to the set of kernel-loadable+modules.+@end defvr+ @node Shepherd Services @subsection Shepherd Services
diff --git a/gnu/services.scm b/gnu/services.scmindex 13259dfaee..d7332a46b2 100644--- a/gnu/services.scm+++ b/gnu/services.scm@@ -2,6 +2,7 @@ ;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org> ;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com> ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -33,6 +34,8 @@ #:use-module (guix diagnostics) #:autoload (guix openpgp) (openpgp-format-fingerprint) #:use-module (guix modules)+ #:use-module (guix packages)+ #:use-module (guix utils) #:use-module (gnu packages base) #:use-module (gnu packages bash) #:use-module (gnu packages hurd)@@ -75,6 +78,7 @@ service-back-edges instantiate-missing-services fold-services+ kernel-loadable-module-service
service-error? missing-value-service-error?@@ -106,6 +110,7 @@ profile-service-type firmware-service-type gc-root-service-type+ kernel-loadable-module-service-type
%boot-service %activation-service@@ -864,6 +869,71 @@ as Wifi cards."))) will not be reclaimed by the garbage collector.") (default-value '())))
+;; Configuration for the kernel builder.+(define-record-type* <kernel-builder-configuration> kernel-builder-configuration+ make-kernel-builder-configuration+ kernel-builder-configuration?+ this-kernel-builder-configuration++ (kernel kernel-builder-configuration-kernel (default #f))+ (hurd kernel-builder-configuration-hurd (default #f))+ (modules kernel-builder-configuration-modules (default '())))++(define (package-for-kernel target-kernel module-package)+ "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if+possible (that is if there's a LINUX keyword argument in the build system)."+ (package+ (inherit module-package)+ (arguments+ (substitute-keyword-arguments (package-arguments module-package)+ ((#:linux kernel #f)+ target-kernel)))))++(define (kernel-builder-configuration->system-entry config)+ "Return the kernel and hurd entries of the 'system' directory."+ (mbegin %store-monad+ (let* ((kernel (kernel-builder-configuration-kernel config))+ (hurd (kernel-builder-configuration-hurd config))+ (modules (kernel-builder-configuration-modules config))+ (kernel (if hurd+ kernel+ (profile+ (content (packages->manifest+ (cons kernel+ (map (lambda (module)+ (if (package? module)+ (package-for-kernel kernel module)+ module))+ modules))))+ (hooks (list linux-module-database))))))+ (return `(("kernel" ,kernel)+ ,@(if hurd `(("hurd" ,hurd)) '()))))))++(define (kernel-builder-configuration-add-modules config modules)+ "Constructs a kernel builder configuration that has its modules extended."+ (kernel-builder-configuration+ (inherit config)+ (modules (append (kernel-builder-configuration-modules config) modules))))++(define kernel-loadable-module-service-type+ (service-type (name 'kernel-loadable-modules)+ (extensions+ (list (service-extension system-service-type+ kernel-builder-configuration->system-entry)))+ (compose concatenate)+ (extend kernel-builder-configuration-add-modules)+ (description+ "Register packages containing kernel-loadable modules and adds them+to the system.")))++(define (kernel-loadable-module-service kernel hurd modules)+ "Constructs the service that sets up kernel loadable modules."+ (service kernel-loadable-module-service-type+ (kernel-builder-configuration+ (kernel kernel)+ (hurd hurd)+ (modules modules))))+ ;;; ;;; Service folding.diff --git a/gnu/system.scm b/gnu/system.scmindex c284a18379..5c530f176e 100644--- a/gnu/system.scm+++ b/gnu/system.scm@@ -12,6 +12,7 @@ ;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@gmail.com> ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <jannek@gnu.org> ;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -600,16 +601,6 @@ OS." (file-append (operating-system-kernel os) "/" (system-linux-image-file-name))))
-(define (package-for-kernel target-kernel module-package)- "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if-possible (that is if there's a LINUX keyword argument in the build system)."- (package- (inherit module-package)- (arguments- (substitute-keyword-arguments (package-arguments module-package)- ((#:linux kernel #f)- target-kernel)))))- (define %default-modprobe-blacklist ;; List of kernel modules to blacklist by default. '("usbmouse" ;races with bcm5974, see <https://bugs.gnu.org/35574>@@ -625,26 +616,10 @@ possible (that is if there's a LINUX keyword argument in the build system)." "Return the basic entries of the 'system' directory of OS for use as the value of the SYSTEM-SERVICE-TYPE service." (let* ((locale (operating-system-locale-directory os))- (kernel (operating-system-kernel os)) (hurd (operating-system-hurd os))- (modules (operating-system-kernel-loadable-modules os))- (kernel (if hurd- kernel- (profile- (content (packages->manifest- (cons kernel- (map (lambda (module)- (if (package? module)- (package-for-kernel kernel- module)- module))- modules))))- (hooks (list linux-module-database))))) (initrd (and (not hurd) (operating-system-initrd-file os))) (params (operating-system-boot-parameters-file os)))- `(("kernel" ,kernel)- ,@(if hurd `(("hurd" ,hurd)) '())- ("parameters" ,params)+ `(("parameters" ,params) ,@(if initrd `(("initrd" ,initrd)) '()) ("locale" ,locale)))) ;used by libc
@@ -663,6 +638,10 @@ bookkeeping." (host-name (host-name-service (operating-system-host-name os))) (entries (operating-system-directory-base-entries os))) (cons* (service system-service-type entries)+ (kernel-loadable-module-service+ (operating-system-kernel os)+ (operating-system-hurd os)+ (operating-system-kernel-loadable-modules os)) %boot-service
;; %SHEPHERD-ROOT-SERVICE must come last so that the gexp that@@ -699,6 +678,10 @@ bookkeeping." (define (hurd-default-essential-services os) (let ((entries (operating-system-directory-base-entries os))) (list (service system-service-type entries)+ (kernel-loadable-module-service+ (operating-system-kernel os)+ (operating-system-hurd os)+ (operating-system-kernel-loadable-modules os)) %boot-service %hurd-startup-service %activation-servicediff --git a/gnu/tests/linux-modules.scm b/gnu/tests/linux-modules.scmindex 953b132ef7..9739e4124d 100644--- a/gnu/tests/linux-modules.scm+++ b/gnu/tests/linux-modules.scm@@ -2,6 +2,7 @@ ;;; Copyright © 2019 Jakob L. Kreuze <zerodaysfordays@sdf.org> ;;; Copyright © 2020 Danny Milosavljevic <dannym@scratchpost.org> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -34,7 +35,10 @@ #:use-module (guix utils) #:export (%test-loadable-kernel-modules-0 %test-loadable-kernel-modules-1- %test-loadable-kernel-modules-2))+ %test-loadable-kernel-modules-2+ %test-loadable-kernel-modules-service-0+ %test-loadable-kernel-modules-service-1+ %test-loadable-kernel-modules-service-2))
;;; Commentary: ;;;@@ -66,17 +70,11 @@ that MODULES are actually loaded." (member module modules string=?)) '#$modules))))))
-(define* (run-loadable-kernel-modules-test module-packages module-names)- "Run a test of an OS having MODULE-PACKAGES, and verify that MODULE-NAMES-are loaded in memory."+(define* (run-loadable-kernel-modules-test-base base-os module-names)+ "Run a test of BASE-OS, verifying that MODULE-NAMES are loaded in memory." (define os (marionette-operating-system- (operating-system- (inherit (simple-operating-system))- (services (cons (service kernel-module-loader-service-type module-names)- (operating-system-user-services- (simple-operating-system))))- (kernel-loadable-modules module-packages))+ base-os #:imported-modules '((guix combinators)))) (define vm (virtual-machine os)) (define (test script)@@ -98,6 +96,37 @@ are loaded in memory." (gexp->derivation "loadable-kernel-modules" (test (modules-loaded?-program os module-names))))
+(define* (run-loadable-kernel-modules-test module-packages module-names)+ "Run a test of an OS having MODULE-PACKAGES, and verify that MODULE-NAMES+are loaded in memory."+ (run-loadable-kernel-modules-test-base+ (operating-system+ (inherit (simple-operating-system))+ (services (cons (service kernel-module-loader-service-type module-names)+ (operating-system-user-services+ (simple-operating-system))))+ (kernel-loadable-modules module-packages))+ module-names))++(define* (run-loadable-kernel-modules-service-test module-packages module-names)+ "Run a test of an OS having MODULE-PACKAGES, which are loaded by creating a+service that extends KERNEL-LOADABLE-MODULE-SERVICE-TYPE. Then verify that+MODULE-NAMES are loaded in memory."+ (define module-installing-service-type+ (service-type+ (name 'module-installing-service)+ (extensions (list (service-extension kernel-loadable-module-service-type+ (const module-packages))))+ (default-value #f)))+ (run-loadable-kernel-modules-test-base+ (operating-system+ (inherit (simple-operating-system))+ (services (cons* (service kernel-module-loader-service-type module-names)+ (service module-installing-service-type)+ (operating-system-user-services+ (simple-operating-system)))))+ module-names))+ (define %test-loadable-kernel-modules-0 (system-test (name "loadable-kernel-modules-0")@@ -129,3 +158,35 @@ with two extra modules.") (package-arguments ddcci-driver-linux)))))) '("acpi_call" "ddcci")))))++(define %test-loadable-kernel-modules-service-0+ (system-test+ (name "loadable-kernel-modules-service-0")+ (description "Tests loadable kernel modules extensible service with no+extra modules.")+ (value (run-loadable-kernel-modules-service-test '() '()))))++(define %test-loadable-kernel-modules-service-1+ (system-test+ (name "loadable-kernel-modules-service-1")+ (description "Tests loadable kernel modules extensible service with one+extra module.")+ (value (run-loadable-kernel-modules-service-test+ (list ddcci-driver-linux)+ '("ddcci")))))++(define %test-loadable-kernel-modules-service-2+ (system-test+ (name "loadable-kernel-modules-service-2")+ (description "Tests loadable kernel modules extensible service with two+extra modules.")+ (value (run-loadable-kernel-modules-service-test+ (list acpi-call-linux-module+ (package+ (inherit ddcci-driver-linux)+ (arguments+ `(#:linux #f+ ,@(strip-keyword-arguments '(#:linux)+ (package-arguments+ ddcci-driver-linux))))))+ '("acpi_call" "ddcci")))))
--2.29.2
R
R
raid5atemyhomework wrote on 6 Jan 16:55 +0100
[PATCH 2/4] gnu: Make file-systems target extensible by services.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
Mg_8FIL2l-Zd7f6rM0kLocdknqSlZfYwXsKBY88f0vqvn_pqTW3otQGnYeTcg1SLbUXuzsO0P5-O35udXT5-c-gzr21W0E0XEYTAbAkdlpI=@protonmail.com
From 792a8f8efc95e4fe9a94d42f839ddcfb034b8540 Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Wed, 6 Jan 2021 08:15:54 +0800Subject: [PATCH 2/4] gnu: Make file-systems target extensible by services.
* gnu/services/base.scm (file-system-shepherd-services): Move file-systemsshepherd service to ...(file-systems-target-shepherd-services): ... new procedure here.(file-systems-target-service-type): New variable.(file-system-service-type): Extend file-systems-target service to add eachfile-system as a requirement.* gnu/system.scm (operating-system-default-essential-services): Instantiatefile-systems-target-service-type.(hurd-default-essential-services): Instantiate file-systems-target-service-type.--- gnu/services/base.scm | 37 +++++++++++++++++++++++++++---------- gnu/system.scm | 2 ++ 2 files changed, 29 insertions(+), 10 deletions(-)
Toggle diff (99 lines)diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex 945b546607..13cfb6a8a2 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -13,6 +13,7 @@ ;;; Copyright © 2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org> ;;; Copyright © 2020 Florian Pelz <pelzflorian@pelzflorian.de> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -67,6 +68,7 @@ #:export (fstab-service-type root-file-system-service file-system-service-type+ file-systems-target-service-type swap-service host-name-service console-keymap-service@@ -362,18 +364,29 @@ FILE-SYSTEM." (gnu system file-systems) ,@%default-modules)))))))
+(define (file-systems-target-shepherd-services requirements)+ (list+ (shepherd-service+ (provision '(file-systems))+ (requirement (cons* 'root-file-system 'user-file-systems requirements))+ (documentation "Target for all the initially-mounted file systems")+ (start #~(const #t))+ (stop #~(const #t)))))+(define file-systems-target-service-type+ (service-type+ (name 'file-systems)+ (extensions (list (service-extension shepherd-root-service-type+ file-systems-target-shepherd-services)))+ (compose concatenate)+ (extend append)+ ;; Extensions can add new values to this list.+ (default-value '())+ (description "The @code{file-systems} service is the target that is started+when all file systems have been mounted.")))+ (define (file-system-shepherd-services file-systems) "Return the list of Shepherd services for FILE-SYSTEMS." (let* ((file-systems (filter file-system-mount? file-systems)))- (define sink- (shepherd-service- (provision '(file-systems))- (requirement (cons* 'root-file-system 'user-file-systems- (map file-system->shepherd-service-name- file-systems)))- (documentation "Target for all the initially-mounted file systems")- (start #~(const #t))- (stop #~(const #f))))
(define known-mount-points (map file-system-mount-point file-systems))@@ -403,7 +416,7 @@ FILE-SYSTEM." (filter (negate known?) (mount-points))) #f))))
- (cons* sink user-unmount+ (cons* user-unmount (map file-system-shepherd-service file-systems))))
(define (file-system-fstab-entries file-systems)@@ -431,6 +444,10 @@ FILE-SYSTEM." (service-extension fstab-service-type file-system-fstab-entries)
+ ;; Have 'file-systems' depend on each declared file system.+ (service-extension file-systems-target-service-type+ (cut map file-system->shepherd-service-name <>))+ ;; Have 'user-processes' depend on 'file-systems'. (service-extension user-processes-service-type (const '(file-systems)))))diff --git a/gnu/system.scm b/gnu/system.scmindex 5c530f176e..6987641ee8 100644--- a/gnu/system.scm+++ b/gnu/system.scm@@ -667,6 +667,7 @@ bookkeeping." (operating-system-setuid-programs os)) (service profile-service-type (operating-system-packages os))+ (service file-systems-target-service-type) other-fs (append mappings swaps
@@ -691,6 +692,7 @@ bookkeeping." (operating-system-groups os)) (operating-system-skeletons os)) (root-file-system-service)+ (service file-systems-target-service-type) (service file-system-service-type '()) (service fstab-service-type (filter file-system-needed-for-boot?
--2.29.2
R
R
raid5atemyhomework wrote on 6 Jan 16:56 +0100
[PATCH 3/4] gnu: Fix ZFS package.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
acw65x5599Yvgtn3iJ_lyNBHCVnac88F4pw4NdemkLG-SFzy_URDm9Fp01CWjbIkNT-5UjqceOHDNBmZpakN0ifetm1ehaED4z65hHgUKwo=@protonmail.com
From d1c88161f2a32f4c03de57ee9d7582cc6c40f6bc Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Tue, 5 Jan 2021 20:00:50 +0800Subject: [PATCH 3/4] gnu: Fix ZFS package.
* gnu/packages/file-systems.scm (zfs): Correct "util-linux" input, addseparate "util-linux:lib" input.--- gnu/packages/file-systems.scm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Toggle diff (22 lines)diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scmindex 895ad069c5..2c5ad95d63 100644--- a/gnu/packages/file-systems.scm+++ b/gnu/packages/file-systems.scm@@ -6,6 +6,7 @@ ;;; Copyright © 2019, 2020 Efraim Flashner <efraim@flashner.co.il> ;;; Copyright © 2020 Raghav Gururajan <raghavgururajan@disroot.org> ;;; Copyright © 2020 Morgan Smith <Morgan.J.Smith@outlook.com>+;;; Copyright © 2021 raid5atemyhoemwork <raid5atemyhomework@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;;@@ -944,7 +945,8 @@ APFS.") ("openssl" ,openssl) ("python" ,python) ("python-cffi" ,python-cffi)- ("util-linux" ,util-linux "lib")+ ("util-linux" ,util-linux)+ ("util-linux:lib" ,util-linux "lib") ("zlib" ,zlib))) (home-page "https://zfsonlinux.org/") (synopsis "Native ZFS on Linux")
--2.29.2
R
R
raid5atemyhomework wrote on 6 Jan 16:57 +0100
[PATCH 4/4] gnu: Add ZFS service.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
6wemXB-PfHUqbuVr5-XRf0-tY4cKGGtKiUqrZPrIZYXoBw17L3xRuZrGOJQfTo5PKfFNCM8KyRTllidoc7asPE2x98BTiJSPVR7OSjxCuw8=@protonmail.com
From deded39fe88c44a18b0b66bab8c4300aca4c387e Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Wed, 6 Jan 2021 09:24:20 +0800Subject: [PATCH 4/4] gnu: Add ZFS service.
* gnu/services/file-systems.scm: New file.(zfs-service-type): New variable.(zfs-configuration): New variable.(zfs-configuration?): New procedure.* gnu/local.mk: Add gnu/services/file-systems.scm.* doc/guix.texi (ZFS file system): New subsection.--- doc/guix.texi | 107 ++++++++++++++++++++++++++ gnu/local.mk | 2 + gnu/services/file-systems.scm | 136 ++++++++++++++++++++++++++++++++++ 3 files changed, 245 insertions(+) create mode 100644 gnu/services/file-systems.scm
Toggle diff (280 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 78770151e3..62437083cb 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -13885,6 +13885,113 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp
+@node ZFS file system+@subsection ZFS file system++The ZFS file system has a license incompatible with the Linux kernel,+and thus cannot be distributed with the kernel. However, as a user+you have the right to do whatever you want on your own hardware,+including download the ZFS source code, compile it, link it to your+own private copy of Linux, and run it. You can even use the Guix build+system to automate this.++As a large and complex kernel module, ZFS on Linux has to be compiled+with a specific version of the kernel. Often even the latest ZFS+package available cannot be compiled with the latest Linux kernel+package provided by Guix System. Thus, installing the @code{zfs}+package is likely to fail.++Instead, you have to use an older long-term-support Linux kernel.+Do not use @code{linux-libre-lts} as the latest long-term-support+kernel might be too new for the ZFS package; instead, explicitly+select the version number, like @code{linux-libre-5.4}.++Then, you have to modify your system configuration file and use the+selected older kernel, and add the @code{zfs-service-type} service.++@lisp+(use-modules (gnu))+(use-package-modules+ ; @dots{}+ linux)+(use-service-modules+ ; @dots{}+ file-systems)++(define my-kernel linux-libre-5.4)++(operating-system+ (kernel my-kernel)+ ;; @dots{}+ (services+ (cons* (service zfs-service-type+ (zfs-configuration+ (kernel my-kernel)))+ ; @dots{}+ %desktop-services))+ ;; @dots{}+ )+@end lisp++@defvr {Scheme Variable} zfs-service-type+This is the type of the service to compile and install OpenZFS to+your operating system. It loads the ZFS module at startup, imports+pools, mounts automounted ZFS datasets, installs the ZFS command+line tools, and can provide module options for the ZFS module. Its+value must be a @code{zfs-configuration} record (see below).++Here is an example use:++@lisp+(service zfs-service-type+ (zfs-configuration+ (kernel linux-libre-5.4)+ (options '("zfs_arc_max=5000000000"))))+@end lisp+@end defvr++@deftp {Data Type} zfs-configuration+This data type represents the configuration of the ZFS service.+The available fields are:++@table @asis+@item @code{kernel}+The package of the Linux kernel to compile ZFS for. Required.++@item @code{base-zfs} (default: @code{zfs})+The ZFS package to use. It will be modified to use the indicated+kernel.++@item @code{options} (default: @code{'()})+A list of string options to pass as options to the ZFS module.+These will be put in a @file{/etc/modprobe.d/zfs.conf} file,+for example setting this to @code{'("zfs_admin_snapshot=1"+"zfs_trim_extent_bytes_min=0")} will create the following file:++@example+options zfs zfs_admin_snapshot=1 zfs_trim_extent_bytes_min=0+@end example+@end table+@end deftp++Once your system has been configured to include the ZFS service+and you have restarted the system, you can manage ZFS pools and+datasets with @code{zpool} and @code{zfs} commands.++ZFS datasets with an appropriate @code{mountpoint} property will+be automounted at startup after the root file system is started.+Encrypted datasets that are automounted will cause boot to pause+and prompt for the password to be provided on the console.++It's possible to have a ZFS dataset as your @code{/home} by simply+setting the @code{mountpoint} property. However, note that ZFS will+refuse to mount over a non-empty directory, so if your root+filesystem already has a non-empty @code{/home}, you should remove+it and its contents, then restart the system.++Having ZFS as a root filesystem or as @code{/boot} is not supported+yet.+ @node Mapped Devices @section Mapped Devices
diff --git a/gnu/local.mk b/gnu/local.mkindex 53a67e6ffe..49073ac2ac 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -39,6 +39,7 @@ # Copyright © 2020 Martin Becze <mjbecze@riseup.net> # Copyright © 2020 Malte Frank Gerdes <mate.f.gerdes@gmail.com> # Copyright © 2020 Vinicius Monego <monego@posteo.net>+# Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com> # # This file is part of GNU Guix. #@@ -601,6 +602,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/dict.scm \ %D%/services/dns.scm \ %D%/services/docker.scm \+ %D%/services/file-systems.scm \ %D%/services/authentication.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scmnew file mode 100644index 0000000000..7a92a2b9d5--- /dev/null+++ b/gnu/services/file-systems.scm@@ -0,0 +1,136 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu services file-systems)+ #:use-module (gnu packages file-systems)+ #:use-module (gnu services)+ #:use-module (gnu services base)+ #:use-module (gnu services shepherd)+ #:use-module (guix gexp)+ #:use-module (guix packages)+ #:use-module (guix records)+ #:export (zfs-service-type+ zfs-configuration+ zfs-configuration?))++(define-record-type* <zfs-configuration>+ zfs-configuration make-zfs-configuration zfs-configuration?+ ; kernel you want to compile the base-zfs module for.+ (kernel zfs-configuration-kernel)+ ; base package that will be compiled for the kernel+ (base-zfs zfs-configuration-base-zfs (default zfs))+ ; list of string options.+ (options zfs-configuration-options (default '())))++(define (make-zfs-package conf)+ (let ((base-zfs (zfs-configuration-base-zfs conf))+ (kernel (zfs-configuration-kernel conf)))+ (package+ (inherit base-zfs)+ (name (string-join (list (package-name base-zfs)+ "for"+ (package-name kernel)+ (package-version kernel)+ "version")+ "-"))+ (arguments (cons* #:linux kernel (package-arguments base-zfs))))))++(define (zfs-loadable-module conf)+ (list (list (make-zfs-package conf) "module")))++(define (zfs-shepherd-services conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool")))+ (list+ (shepherd-service+ (documentation "Scans for ZFS pools and automounts filesystems.")+ (provision '(zfs-scan-automount))+ (requirement '(root-file-system))+ (modules `((srfi srfi-1)+ (srfi srfi-34)+ (srfi srfi-35)+ (rnrs io ports)+ ,@%default-modules))+ (start #~(lambda _+ (and+ ;; You'd think we could've used kernel-module-loader-service-type,+ ;; but the kernel-module-loader shepherd service is dependent on+ ;; file-systems, and file-systems is made dependent on this+ ;; service. And we need the kernel module to be loaded before we+ ;; scan for ZFS pools. So break the dependency loop by just+ ;; loading ZFS module here by ourselves.+ (or (file-exists? "/proc/sys/kernel/modprobe")+ (begin+ (format (current-error-port) "error loading 'zfs' module: ~a~%"+ "Kernel is missing loadable module support.")+ #f))+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error loading 'zfs' module: ~a~%"+ (condition-message c))+ #f))+ (let ((modprobe (call-with-input-file "/proc/sys/kernel/modprobe"+ get-line)))+ (invoke/quiet modprobe "--" "zfs")))++ ; scan for pools and automount contained datasets.+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error importing zpools: ~a~%"+ (condition-message?))+ #f))+ ;; (current-output-port) is typically connected to /dev/klog,+ ;; so redirect it to (current-error-port) so that user can see+ ;; prompts for passphrases on console+ (with-output-to-port (current-error-port)+ (lambda ()+ (invoke #$zpool "import" "-a" "-l")))))))+ (stop #~(const #t))))))++(define (zfs-profile-service conf)+ (list (make-zfs-package conf)))++(define (zfs-etc-service conf)+ (let ((options (zfs-configuration-options conf)))+ (if (null? options)+ '()+ `(("modprobe.d/zfs.conf"+ ,(plain-file "zfs.conf"+ (string-join (cons "options zfs" options) " ")))))))++(define zfs-service-type+ (service-type (name 'zfs)+ (extensions+ (list+ ; install the kernel module+ (service-extension kernel-loadable-module-service-type+ zfs-loadable-module)+ ; load ZFS module, scan ZFS pools, and automount filesystems+ (service-extension shepherd-root-service-type+ zfs-shepherd-services)+ ; make sure automount occurs before file-systems target is reached+ (service-extension file-systems-target-service-type+ (const '(zfs-scan-automount)))+ ; install ZFS management tools+ (service-extension profile-service-type+ zfs-profile-service)+ ; install ZFS module options+ (service-extension etc-service-type+ zfs-etc-service)))+ (description+ "Install ZFS, an advanced filesystem and volume manager.")))
--2.29.2
D
D
Danny Milosavljevic wrote on 6 Jan 20:41 +0100
kernel-module-configuration-service for configuring kernel parameters
20210106204134.38e83db4@scratchpost.org
Hi raid5atemyhomework,Hi everyone,
@raid5atemyhomework, thanks for all the patches!
On Wed, 06 Jan 2021 15:57:19 +0000raid5atemyhomework via Guix-patches via <guix-patches@gnu.org> wrote:
Toggle quote (7 lines)> +@item @code{options} (default: @code{'()})> +A list of string options to pass as options to the ZFS module.
> +These will be put in a @file{/etc/modprobe.d/zfs.conf} file,> +for example setting this to @code{'("zfs_admin_snapshot=1"> +"zfs_trim_extent_bytes_min=0")} will create the following file:
Sure, but it would be better to create a way to configure those module parametersin Guix in a declarative way, first.
Your new kernel-loadable-module-service-type would be a good templateto write a kernel-module-configuration-service that can be extended byother services. The latter should allow users to parametrize the kernelin general via the operating-system form, and also allow services to extendsuch configuration (with merging, and conflict detection).
It would be used:
(1) To declaratively specify the contents of something like /etc/modprobe.d .It shouldn't even be called "/etc/modprobe.d"--it should also be in the storeinstead. This directory is only useful for non-Linux-builtins.
(2) If the "module" is built-in then the kernel command line must get theoptions instead. (in fact, it works as a kernel command line option alsoif it's a loadable module--so not sure we need /etc/modprobe.d at all--atleast at first. But there's probably a maximal length for the kernel commandline that we could exceed if we did that long term)
I know it's annoying that Guix doesn't have this facility already, but thetime to introduce an interface for /etc/modprobe.d and the kernel commandline for builtin modules is before other services introduce their ownad hoc way to create /etc/modprobe.d--like this tries to do here.
See also https://issues.guix.info/issue/42193for an earlier attempt (whichis already very far--but it has a bug somewhere). There's also already akernel profile thing like you wrote in that patchset.(Note that I would prefer there not to be a "LOAD?" in there because itconfuses loading the module (which is usually NOT started by user spacebut by the kernel on its own) and confguring the module (which has to bedone by user space because it's specifying policy, not mechanism))
Also, because the kernel usually loads loadable modules on its own (potentiallyreally early), /etc/modprobe.d has to be preset and known to the modprobeexecutable VERY EARLY (via environment variable MODPROBE_OPTIONS--see gnu/services.scm %modprobe-wrapper).
It is totally possible that some modules in the initrd need options, too(see load-linux-modules-from-directory for where this would need to go).load-linux-module/fd already accepts options and flags--but both are not givenon the call. For this, part of future kernel-module-configuration entries(the ones needed for modules in the initrd) should be copied into the initrd,too.
Then there's the handoff between initrd and main system. It would be badif the kernel tried and succeeded to load a module that is not in the initrdjust before the modprobe.d directory is set up (because it would be loadedwithout passing the options the user configured)--so that needs to be avoided.
@ludo: Could you help here?
Toggle quote (3 lines)> +@example> +options zfs zfs_admin_snapshot=1 zfs_trim_extent_bytes_min=0
Note:
This can be usefully put in a modprobe.d-like directory if zfs is a module, butnot if it's built into the kernel. But it can be put into the kernel commandline in both cases.
But I guess the ZFS Linux kernel module can't be built-in into the kernelanyway.
But that's a special case--in general, it's very much possible to make modulesbuilt-in.
Toggle quote (2 lines)> + ;; You'd think we could've used kernel-module-loader-service-type,
Definitely.
Toggle quote (3 lines)> + ;; but the kernel-module-loader shepherd service is dependent on> + ;; file-systems,
Yes--but why is that dependent on 'file-systems ? Is it because it needs /proc ?Or is it an oversight ? I would prefer to get rid of this dependency and thenuse kernel-module-loader-service-type.
Also, this manual loading of kernel modules is not supposed to be the way todo things in Linux. That a kernel module was compiled as a module isan *implementation detail*--so Linux should (and usually does) automaticallyload kernel modules the first time a device for them is accessed (after all,how would user space know whether something is compiled as a module orbuilt-in--that would be too much to ask).
Linux is not a microkernel, so the kind of modularily modprobe.d suggestsexists does not in fact exist in kernel space--even though Linux does a goodjob faking it: modprobe.d contains:
* "alias": a feature to configure aliases, with wildcards (only one level ofaliases allowed!)* "options" per module (also works for aliases with wildcards! That will be"fun" to map to Guix)* "install" in order to run some custom executable instead of loading themodule.* "remove" in order to run some custom executable instead of unloading themodule.* "blacklist" to ignore specific internal aliases of a module (that doesnot do what one would intuitively think!).
If the file name of the regular file under /etc/modprobe.d is not used foranything, then we can just have one file /gnu/store/*modprobe.d/guix.confin total in there.
Then there are sysctl kernel parameters--but those Guix already exposes viasysctl-service-type. But those should also be made able to be extendedby other services, and merge conflicts should be handled. For example, usersoften set net.ipv4.ip_forward=1 (for example via sysctl).
Thank you for all your effort to make ZFS work nicely in Guix.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/2Em4ACgkQ5xo1VCwwuqX7VQf/bHxCWBZt0FXRb5E0zvb0AZVwMPajkYvOBjfXJC+XcMbgjP3FKNW7VLGKFakRsyWnVX6gtFPxUDImjnCIGcY5LGYuaHxnDR4CTqzDBb06LRAMeJ1YTIyr+j+SZO17TG4hZa+xWLqM6MUpfEwuc1I/b5o9oN39XJJLJAWJOdXMi2ZGwOKY8dpYhaDwzDPoOb3xHj9oDyXyWjOfb/8VK8bF8+L9Fzy3Bjt2WiG58j5jXpkYz9ZQe3zA1hbHzwFmmKqkFbL2gFhTxGWuQrp6x29T60lzBwUcjG58waCQee23TVFR9K/v7fvjrBalkHMS+9/Dc8FBIdzlHAbxj8/74qYkUQ===nYdc-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 7 Jan 01:04 +0100
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
pilwAVCb0NEG83BVTSEqXkLCNEvBBrnb-QyKV_9PoF4xuCjSXKuF-F3_L5lSOke5BRjagVrUoSzJCST1fkigF162f9j76K12jBraNMAj7E4=@protonmail.com
Hello Danny,

Toggle quote (8 lines)> See also https://issues.guix.info/issue/42193for an earlier attempt (which> is already very far--but it has a bug somewhere). There's also already a> kernel profile thing like you wrote in that patchset.> (Note that I would prefer there not to be a "LOAD?" in there because it> confuses loading the module (which is usually NOT started by user space> but by the kernel on its own) and confguring the module (which has to be> done by user space because it's specifying policy, not mechanism))
Looks like that patchset was merged in, so basically I can just depend on that? So the first patch in this patchset would be dropped?
Toggle quote (6 lines)> But I guess the ZFS Linux kernel module can't be built-in into the kernel> anyway.>> But that's a special case--in general, it's very much possible to make modules> built-in.
ZFS *can* be built-in to the kernel, Ubuntu does it. You can't distribute it like that (Ubuntu distributes it like that but presumably they have enough lawyers to muddy the waters so that they can get away with it), but as the documentation in this patch notes: the user has every right to do whatever they want on the machine they own, including build a Linux kernel that has ZFS built-in and run it, they just can't make that version available to somebody else.
So to go whole-hog, we would have a service that replaces the kernel package and inserts kernel module sources in-tree somehow, then compiles Linux-libre on the user's machine. That would probably be a lot more painful to install ZFS with (the user has to recompile the whole kernel at each update of either kernel or ZFS, whereas with a kernel module the user has to recompile just the kernel module), so maybe kernel module is still better overall.

Toggle quote (17 lines)> > - ;; You'd think we could've used kernel-module-loader-service-type,> >> >>> Definitely.>> > - ;; but the kernel-module-loader shepherd service is dependent on> >> >> > - ;; file-systems,> >> >>> Yes--but why is that dependent on 'file-systems ? Is it because it needs /proc ?> Or is it an oversight ? I would prefer to get rid of this dependency and then> use kernel-module-loader-service-type.
Dunno --- one VM I tested, I removed the `zfs-scan-automount` shepherd service from the `file-systems` target, and the VM still wouldn't boot, claiming a stack overflow (the same error which I got when I was still trying to use kernel-module-loader-service-type here). Or maybe I just got confused with which VM was which, testing VMs wasn't a stress-free vacation. I just want ZFS, because MD RAID5 ate my homework, this is getting tiresome...
One thing I notice about `kernel-module-loader-service-type` is that it's not instantiated in essential services, or indeed anywhere in Guix. A few services *do* extend it. But my **very rough** understanding is that if you're going to extend a service, it had better be instantiated *once* in the list of services.
In particular I note that the documentation for `kernel-module-loader-service-type` shows an example where it uses `service` to program the `kernel-module-loader-service-type`, not `simple-service`. This suggests to me that `kernel-module-loader-service-type` is broken because it's not in the list of essential services but is extensible. Maybe. It's designed as an extensible service, but isn't instantiated at default. Maybe that's what really bit me and not the shepherd circular dependency loop? *shrug*

Toggle quote (8 lines)>> Also, this manual loading of kernel modules is not supposed to be the way to> do things in Linux. That a kernel module was compiled as a module is> an implementation detail--so Linux should (and usually does) automatically> load kernel modules the first time a device for them is accessed (after all,> how would user space know whether something is compiled as a module or> built-in--that would be too much to ask).
So how do I get ZFS loaded? Note that the devices it targets are block devices and it needs to scan for block devices that are formatted for ZFS. Do other filesystems have some autoload rule?
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 7 Jan 06:38 +0100
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
7kahfDckMJIQd8o2gH_zQ-G9ORP_P0w-VD44R2OJHNP6rJHss8przoTyObTYxx_UEI5CERTD3OlGORnTCBmZA2VMASWpF2Ea0Hx1ZAbUh-I=@protonmail.com
Hi Danny,
Toggle quote (10 lines)> > See also https://issues.guix.info/issue/42193for an earlier attempt (which> > is already very far--but it has a bug somewhere). There's also already a> > kernel profile thing like you wrote in that patchset.> > (Note that I would prefer there not to be a "LOAD?" in there because it> > confuses loading the module (which is usually NOT started by user space> > but by the kernel on its own) and confguring the module (which has to be> > done by user space because it's specifying policy, not mechanism))>> Looks like that patchset was merged in, so basically I can just depend on that? So the first patch in this patchset would be dropped?
No, sorry, my mistake, only one patch completely unrelated to the actual new service type was merged in.
In any case --- is this objection something that would block this patchset from being added to Guix? I appreciate that we should "do it right" --- but there's also an argument for "keep it simple", and the first patch in this patchset gets us a good part of the way to what is needed.
It seems that my patch is equivalent to the existing WIP 2/6 `kernel-profile-service-type`, so maybe I can just steal that patch for now, write tests specific for it, and get on with running ZFS in production?
Can we at least get patch 3/4 of *this* patchset merged since it's a trivial bugfix? How about patch 2/4, which is not so trivial, but does give some flexibility in case other filesystems want to be as ambitious as ZFS is?
I imagine a later, much more comprehensive `kernel-module-configuration-service-type` can be built on top of `kernel-profile-service-type`, `kernel-module-loader-service-type`, and a later `kernel-module-options-service-type`. Then `zfs-service-type` can be modified to use that `kernel-module-configuration-service-type` instead of rolling its own bits and pieces.
----
On the other hand, if we want to think of `/` on ZFS, then we need a notion of kernel modules that are added to the `initrd` file, rather than to the `kernel` directory.
Diving into `gnu/system/linux-initrd.scm`, I note that we can provide a kernel package and a list of modules to copy from the kernel package to the `initrd` by a `(flat-linux-module-directory linux modules)` function.
I imagine that it would be possible to modify this as follows:
* Have `flat-linux-module-directory` accept a list of packages from which to find module names, not just a single package.* Remove more code from `(operating-system-directory-base-entries os)`, and put the creation of the `"initrd"` file into a service, in much the same way that my patch does (unless there's an existing more-guixy way of putting files into `initrd`?). * Create a service type that gathers packages whose modules are to be added to the `initrd`, and if that list is non-empty, pass it to a new key `#:extra-linux-module-packages` of the `initrd` function. * `raw-initrd` would then pass that field together with the `linux` argument to the modified `flat-linux-module-directory` procedure. * Create another service type that gathers module names to be loaded at `boot` and appends them to the `#:linux-modules` to the `initrd` function. * Modify the `kernel-module-options-service-type` to pass in options via kernel command line always, maybe.
Then ZFS module can get loaded early, at boot, before the switch from `initrd` root to the "real" root.
* Create a service type that gathers additional pre-mount actions, which `raw-initrd` will accept as a new key `#:additional-pre-mount` and append to the `pre-mount` it passes to `boot-system`. * The `zfs-service-type` would then extend this service type to pass in an action to perform a `zpool import -a -N`, which makes ZFS scan for devices containing ZFS pools.* Somehow figure out a static build for the ZFS package, so we can use a statically-linked `zpool` in the above. * The `zfs-service-type` already inherits a ZFS package from the given `base-package`, I imagine it would be possible to make further inheritance which modifies the build to be static. It does require an additional build, though. Maybe an additional `root-on-zfs?` field in `zfs-configuration` can gate this, so that we don't add the static build and extend the boot script if `(not root-on-zfs?)` (i.e. use `kernel-module-configuration-service-type` if `(not root-on-zfs?)`, else use the new put-it-in-the-initrd service types).* Modify the various checks elsewhere so that ZFS poolnames can be passed as the `device` field of `file-system` records. See https://issues.guix.gnu.org/45643#3 * Then the installer can be modified so that if ZFS is included with the installer's profile, it can look at ZFS pools and offer to install to a ZFS pool instead of a `/dev` partition directly, then add `(file-system (device "rootpool") (mountpoint "/") (type "zfs"))` and the appropriate `zfs-service-type` and etc. and now we get `/` on ZFS.
But that can come later, much much later, I just need ZFS, *any* ZFS, before RAID5 eats more of my homework.
Thanksraid5atemyhomework

Toggle quote (43 lines)> > But I guess the ZFS Linux kernel module can't be built-in into the kernel> > anyway.> > But that's a special case--in general, it's very much possible to make modules> > built-in.>> ZFScan be built-in to the kernel, Ubuntu does it. You can't distribute it like that (Ubuntu distributes it like that but presumably they have enough lawyers to muddy the waters so that they can get away with it), but as the documentation in this patch notes: the user has every right to do whatever they want on the machine they own, including build a Linux kernel that has ZFS built-in and run it, they just can't make that version available to somebody else.>> So to go whole-hog, we would have a service that replaces the kernel package and inserts kernel module sources in-tree somehow, then compiles Linux-libre on the user's machine. That would probably be a lot more painful to install ZFS with (the user has to recompile the whole kernel at each update of either kernel or ZFS, whereas with a kernel module the user has to recompile just the kernel module), so maybe kernel module is still better overall.>> > > - ;; You'd think we could've used kernel-module-loader-service-type,> > >> > >> >> > Definitely.> >> > > - ;; but the kernel-module-loader shepherd service is dependent on> > >> > >> > > - ;; file-systems,> > >> > >> >> > Yes--but why is that dependent on 'file-systems ? Is it because it needs /proc ?> > Or is it an oversight ? I would prefer to get rid of this dependency and then> > use kernel-module-loader-service-type.>> Dunno --- one VM I tested, I removed the`zfs-scan-automount` shepherd service from the `file-systems` target, and the VM still wouldn't boot, claiming a stack overflow (the same error which I got when I was still trying to use kernel-module-loader-service-type here). Or maybe I just got confused with which VM was which, testing VMs wasn't a stress-free vacation. I just want ZFS, because MD RAID5 ate my homework, this is getting tiresome...>> One thing I notice about `kernel-module-loader-service-type` is that it's not instantiated in essential services, or indeed anywhere in Guix. A few services do extend it. But my very rough understanding is that if you're going to extend a service, it had better be instantiated once in the list of services.>> In particular I note that the documentation for `kernel-module-loader-service-type` shows an example where it uses `service` to program the `kernel-module-loader-service-type`, not `simple-service`. This suggests to me that `kernel-module-loader-service-type` is broken because it's not in the list of essential services but is extensible. Maybe. It's designed as an extensible service, but isn't instantiated at default. Maybe that's what really bit me and not the shepherd circular dependency loop? shrug>> > Also, this manual loading of kernel modules is not supposed to be the way to> > do things in Linux. That a kernel module was compiled as a module is> > an implementation detail--so Linux should (and usually does) automatically> > load kernel modules the first time a device for them is accessed (after all,> > how would user space know whether something is compiled as a module or> > built-in--that would be too much to ask).>> So how do I get ZFS loaded? Note that the devices it targets are block devices and it needs to scan for block devices that are formatted for ZFS. Do other filesystems have some autoload rule?>> Thanks> raid5atemyhomework
D
D
Danny Milosavljevic wrote on 7 Jan 09:23 +0100
Re: [bug#45692] [PATCH 3/4] gnu: Fix ZFS package.
20210107092331.0b2fc752@scratchpost.org
Pushed this one patch that fixes package "zfs" to guix master ascommit 8d3184c5bcf1a590f59a701f0484ab5cec673188.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl/2xQMACgkQ5xo1VCwwuqU2tQgAmIWZQ+ODdXCvUgDlwdVR5olRbLAZeTC7ZIHErRte4VzFAjJyvUkk1mAr0k/o9Y+SFda5TRza7GWHwgfZXiAoSyGgqUyrKH6MO0YKJlAljN9W69LSyigDuSvZSPQxRW8WHhDCJFrKkaW8cgXN2Im/QJ2Pd2kZDav266riE3Jk4JyRpS60VHHfnuE82jNT2UfnJuPK/SgAH8Q3G7z8+wd9iiMWKxpkkapHFGEdPUxJGsupNX8XtnWcCSegWQhRglVn0yY7gFhMODJcfjfqGLoDce6OjOWab4Snnv3Wjzmq3fn1gMUSzv1avIwXKbmSBldqAUk+VjiPrI6Si3ixZjK+xQ===ilmp-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 8 Jan 16:02 +0100
Re: [PATCH 4/4] gnu: Add ZFS service.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
LmlpVtErWRRqKI9LIqHvswFA3ig_rjwIk9u8S96Bp6A9VG_l0bAnQdhIQsxf_fB5CAJxMnoClNXDObuPloDX7JDDOl4ZuFGn6u54FmJ5K2o=@protonmail.com
This new version is dependent on https://issues.guix.info/45723and https://issues.guix.info/45722
New features:
* We can now delay pool importation after particular mapped devices are opened or file systems are mounted. This is useful if you want to make a ZFS pool out of LUKS containers, or mount an encrypted ZFS dataset with the encryption key in another (hopefully also encrypted) filesystem.* We can now put other non-ZFS filesystems inside ZVOLs, block devices provided by a ZFS pool, and have them mounted after ZFS has scanned pools and opened such ZVOLs for use by the ystem.
It's now possible to have a bunch of LUKS containers in a ZPOOL that provides a ZVOL that contains an EXT4 filesystem (though why would you do that, just use builtin ZFS encryption).
This uses my [PATCH 1/4] instead of https://issues.guix.info/42193#4. Should I use the latter instead? What would get ZFS on Guix faster?
I decided to remove ZFS parameter options, because I don't know whether I should wait for https://issues.guix.info/42193and how long that would take.

From d0e095136cd471f083d92416f12ca22b47301f31 Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Wed, 6 Jan 2021 09:24:20 +0800Subject: [PATCH 4/4] gnu: Add service to install ZFS.
* gnu/services/file-systems.scm: New file.(zfs-service-type): New variable.(<zfs-configuration>): New type.(%zfs-zvol-dependency): New variable.* gnu/local.mk: Add gnu/services/file-systems.scm.* gnu/services/base.scm (dependency->shepherd-service-name): Export.* doc/guix.texi (ZFS file system): New subsection.--- doc/guix.texi | 107 +++++++++++++++++++++ gnu/local.mk | 1 + gnu/services/base.scm | 4 +- gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++ 4 files changed, 285 insertions(+), 1 deletion(-) create mode 100644 gnu/services/file-systems.scm
Toggle diff (325 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex a31d355780..714274b8c9 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -13886,6 +13886,113 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp
+@node ZFS file system+@subsection ZFS file system++The ZFS file system has a license incompatible with the Linux kernel,+and thus cannot be distributed with the kernel. However, as a user+you have the right to do whatever you want on your own hardware,+including download the ZFS source code, compile it, link it to your+own private copy of Linux, and run it. You can even use the Guix build+system to automate this.++As a large and complex kernel module, ZFS on Linux has to be compiled+with a specific version of the kernel. Often even the latest ZFS+package available cannot be compiled with the latest Linux kernel+package provided by Guix System. Thus, installing the @code{zfs}+package is likely to fail.++Instead, you have to use an older long-term-support Linux kernel.+Do not use @code{linux-libre-lts} as the latest long-term-support+kernel might be too new for the ZFS package; instead, explicitly+select the version number, like @code{linux-libre-5.4}.++Then, you have to modify your system configuration file and use the+selected older kernel, and add the @code{zfs-service-type} service.++@lisp+(use-modules (gnu))+(use-package-modules+ ; @dots{}+ linux)+(use-service-modules+ ; @dots{}+ file-systems)++(define my-kernel linux-libre-5.4)++(operating-system+ (kernel my-kernel)+ ;; @dots{}+ (services+ (cons* (service zfs-service-type+ (zfs-configuration+ (kernel my-kernel)))+ ; @dots{}+ %desktop-services))+ ;; @dots{}+ )+@end lisp++@defvr {Scheme Variable} zfs-service-type+This is the type of the service to compile and install OpenZFS to+your operating system. It loads the ZFS module at startup, imports+pools, mounts automounted ZFS datasets, installs the ZFS command+line tools, and can provide module options for the ZFS module. Its+value must be a @code{zfs-configuration} record (see below).++Here is an example use:++@lisp+(service zfs-service-type+ (zfs-configuration+ (kernel linux-libre-5.4)+ (options '("zfs_arc_max=5000000000"))))+@end lisp+@end defvr++@deftp {Data Type} zfs-configuration+This data type represents the configuration of the ZFS service.+The available fields are:++@table @asis+@item @code{kernel}+The package of the Linux kernel to compile ZFS for. Required.++@item @code{base-zfs} (default: @code{zfs})+The ZFS package to use. It will be modified to use the indicated+kernel.++@item @code{options} (default: @code{'()})+A list of string options to pass as options to the ZFS module.+These will be put in a @file{/etc/modprobe.d/zfs.conf} file,+for example setting this to @code{'("zfs_admin_snapshot=1"+"zfs_trim_extent_bytes_min=0")} will create the following file:++@example+options zfs zfs_admin_snapshot=1 zfs_trim_extent_bytes_min=0+@end example+@end table+@end deftp++Once your system has been configured to include the ZFS service+and you have restarted the system, you can manage ZFS pools and+datasets with @code{zpool} and @code{zfs} commands.++ZFS datasets with an appropriate @code{mountpoint} property will+be automounted at startup after the root file system is started.+Encrypted datasets that are automounted will cause boot to pause+and prompt for the password to be provided on the console.++It's possible to have a ZFS dataset as your @code{/home} by simply+setting the @code{mountpoint} property. However, note that ZFS will+refuse to mount over a non-empty directory, so if your root+filesystem already has a non-empty @code{/home}, you should remove+it and its contents, then restart the system.++Having ZFS as a root filesystem or as @code{/boot} is not supported+yet.+ @node Mapped Devices @section Mapped Devices
diff --git a/gnu/local.mk b/gnu/local.mkindex 1151d4642e..5aeb45c4c2 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -602,6 +602,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/dict.scm \ %D%/services/dns.scm \ %D%/services/docker.scm \+ %D%/services/file-systems.scm \ %D%/services/authentication.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex 13cfb6a8a2..ef3680583b 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -188,7 +188,9 @@
references-file
- %base-services))+ %base-services++ dependency->shepherd-service-name))
;;; Commentary: ;;;diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scmnew file mode 100644index 0000000000..9061ab9582--- /dev/null+++ b/gnu/services/file-systems.scm@@ -0,0 +1,174 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu services file-systems)+ #:use-module (gnu packages file-systems)+ #:use-module (gnu services)+ #:use-module (gnu services base)+ #:use-module (gnu services linux)+ #:use-module (gnu services shepherd)+ #:use-module (gnu system mapped-devices)+ #:use-module (guix gexp)+ #:use-module (guix packages)+ #:use-module (guix records)+ #:export (zfs-service-type++ zfs-configuration+ zfs-configuration?+ zfs-configuration-kernel+ zfs-configuration-base-zfs+ zfs-configuration-dependencies++ %zfs-zvol-dependency))++(define-record-type* <zfs-configuration>+ zfs-configuration make-zfs-configuration zfs-configuration?+ ; kernel you want to compile the base-zfs module for.+ (kernel zfs-configuration-kernel)+ ; base package that will be compiled for the kernel+ (base-zfs zfs-configuration-base-zfs+ (default zfs))+ ; list of <mapped-device> | <file-system> we should wait for,+ ; before scanning for ZFS pools.+ (dependencies zfs-configuration-dependencies+ (default '())))++;; This is a synthetic and unusable MAPPED-DEVICE; its only use+;; is to be added as a (dependency ...) of some FILE-SYSTEM.+(define %zfs-zvol-dependency+ (mapped-device+ (source '())+ ;; The /* prevents naming conflict with non-ZFS device mappings,+ ;; since it is not a valid name for mapped devices, and also+ ;; implies "all zvols" in terms of globs.+ (targets '("zvol/*"))+ (type #f)))++(define (make-zfs-package conf)+ (let ((base-zfs (zfs-configuration-base-zfs conf))+ (kernel (zfs-configuration-kernel conf)))+ (package+ (inherit base-zfs)+ (name (string-join (list (package-name base-zfs)+ "for"+ (package-name kernel)+ (package-version kernel)+ "version")+ "-"))+ (arguments (cons* #:linux kernel (package-arguments base-zfs))))))++(define (zfs-loadable-module conf)+ (list (list (make-zfs-package conf) "module")))++(define (zfs-shepherd-services conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (zfs (file-append zfs-package "/sbin/zfs"))+ (zvol_wait (file-append zfs-package "/bin/zvol_wait"))+ (scheme-modules `((srfi srfi-1)+ (srfi srfi-34)+ (srfi srfi-35)+ (rnrs io ports)+ ,@%default-modules)))++ (define zfs-scan+ (shepherd-service+ (provision '(zfs-scan))+ (documentation "Scans for ZFS pools.")+ (requirement `(kernel-module-loader+ root-file-system+ ,@(map dependency->shepherd-service-name+ (zfs-configuration-dependencies conf))))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error importing zpools: ~a~%"+ (condition-message c))+ #f))+ ; TODO: optionally use a cachefile, for systems with dozens or+ ; hundreds of devices.+ (invoke/quiet #$zpool "import" "-a" "-N"))))+ (stop #~(const #t))))++ (define device-mapping-zvol/*+ (shepherd-service+ (provision '(device-mapping-zvol/*))+ (documentation "Waits for ZFS ZVOL devices to appear.")+ (requirement '(zfs-scan))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error waiting for zvols: ~a~%"+ (condition-message c))+ #f))+ (invoke/quiet #$zvol_wait))))+ (stop #~(const #t))))++ (define zfs-automount+ (shepherd-service+ (provision '(zfs-automount))+ (documentation "Automounts ZFS datasets.")+ (requirement '(zfs-scan))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error automounting zfs: ~a~$")+ #f))+ ; (current-output-port) is typically connected to /dev/klog,+ ; so redirect it to (current-error-port) so that user can see+ ; prompts for passphrases on console+ (with-output-to-port (current-error-port)+ (lambda ()+ (invoke #$zfs "mount" "-a" "-l"))))))+ (stop #~(lambda _+ ;; make sure we don't keep any ZFS mountpoints busy.+ (chdir "/")+ ;; unmount everything.+ (invoke/quiet #$zfs "unmount" "-a" "-f")))))++ (list zfs-scan+ device-mapping-zvol/*+ zfs-automount)))++(define zfs-service-type+ (service-type (name 'zfs)+ (extensions+ (list+ ; install the kernel module+ (service-extension kernel-loadable-module-service-type+ zfs-loadable-module)+ ; load the kernel module+ (service-extension kernel-module-loader-service-type+ (const '("zfs")))+ ; scan ZFS pools, automount filesystem, wait for zvols.+ (service-extension shepherd-root-service-type+ zfs-shepherd-services)+ ; make sure automount occurs before file-systems target is reached+ (service-extension file-systems-target-service-type+ (const '(zfs-automount)))+ ; install ZFS management tools+ (service-extension profile-service-type+ (compose list make-zfs-package))+ ; install ZFS udev rules+ (service-extension udev-service-type+ (compose list make-zfs-package))))+ (description+ "Install ZFS, an advanced filesystem and volume manager.")))
--2.30.0
R
R
raid5atemyhomework wrote on 8 Jan 17:16 +0100
Re: [PATCH 1/4] gnu: Allow services to install kernel-loadable modules.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
dczqwG2jPjcW8kLCB2rWgE7UxjX4eGFliB97Uug4pxwzaEVw7Wtrwm3S4fO1tRv3AG816TT0pAZWaaZY-3gE5RmAaDV9NLbCKyuW4HeBoZA=@protonmail.com
Is this patch acceptable? https://lists.nongnu.org/archive/html/guix-devel/2021-01/msg00070.htmlI reported that I compared a `guix system build` result of an `operating-system` that used the existing `(kernel-loadable-modules ...)` field, with and without this patch.
The resulting builds resulted in different hashes, but with exactly the same contents in the build --- `diff -r` very quickly reported no differences because it saw that nearly all the symlinks pointed to the same gnu store items.
R
R
raid5atemyhomework wrote on 9 Jan 09:31 +0100
Re: [PATCH 4/4] gnu: Add ZFS service.
(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
4XtBUBZvLlaRj3Qw8d44cVUm2QWePWKlxR4G7yH_bwXA62SoedCr5VHkch5VkOXfEd5frGaddsTLEEyiVQC0kQmearhfiiveI626gGbVjts=@protonmail.com
New version again. Change: Fix documentation `doc/guix.texi` to remove mentions of `options` and add documentation about the new `dependencies` features.

From dfe9ad7512d348933beb5b42041965ff604422b5 Mon Sep 17 00:00:00 2001From: raid5atemyhomework <raid5atemyhomework@protonmail.com>Date: Wed, 6 Jan 2021 09:24:20 +0800Subject: [PATCH 1/2] gnu: Add service to install ZFS.
* gnu/services/file-systems.scm: New file.(zfs-service-type): New variable.(<zfs-configuration>): New type.(%zfs-zvol-dependency): New variable.* gnu/local.mk: Add gnu/services/file-systems.scm.* gnu/services/base.scm (dependency->shepherd-service-name): Export.* doc/guix.texi (ZFS file system): New subsection.--- doc/guix.texi | 129 +++++++++++++++++++++++++ gnu/local.mk | 1 + gnu/services/base.scm | 4 +- gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++ 4 files changed, 307 insertions(+), 1 deletion(-) create mode 100644 gnu/services/file-systems.scm
Toggle diff (347 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex a31d355780..7004efe3c4 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -13886,6 +13886,135 @@ a file system declaration such as: compress-force=zstd,space_cache=v2")) @end lisp
+@node ZFS file system+@subsection ZFS file system++The ZFS file system has a license incompatible with the Linux kernel,+and thus cannot be distributed with the kernel. However, as a user+you have the right to do whatever you want on your own hardware,+including download the ZFS source code, compile it, link it to your+own private copy of Linux, and run it. You can even use the Guix build+system to automate this.++As a large and complex kernel module, ZFS on Linux has to be compiled+with a specific version of the kernel. Often even the latest ZFS+package available cannot be compiled with the latest Linux kernel+package provided by Guix System. Thus, installing the @code{zfs}+package is likely to fail.++Instead, you have to use an older long-term-support Linux kernel.+Do not use @code{linux-libre-lts}, since the latest long-term-support+kernel might be too new for the ZFS package; instead, explicitly+select the version number, like @code{linux-libre-5.4}, and upgrade+it manually later when you have verified that the ZFS version+available on Guix can be compiled with a later LTS kernel.++Then, you have to modify your system configuration file and use the+selected older kernel, and add the @code{zfs-service-type} service.++@lisp+(use-modules (gnu))+(use-package-modules+ ; @dots{}+ linux)+(use-service-modules+ ; @dots{}+ file-systems)++(define my-kernel linux-libre-5.4)++(operating-system+ (kernel my-kernel)+ ;; @dots{}+ (services+ (cons* (service zfs-service-type+ (zfs-configuration+ (kernel my-kernel)))+ ; @dots{}+ %desktop-services))+ ;; @dots{}+ )+@end lisp++@defvr {Scheme Variable} zfs-service-type+This is the type of the service to compile and install OpenZFS to+your operating system. It loads the ZFS module at startup, imports+pools, mounts automounted ZFS datasets, and installs the ZFS command+line tools. Its value must be a @code{zfs-configuration} record+(see below).++Here is an example use:++@lisp+(service zfs-service-type+ (zfs-configuration+ (kernel linux-libre-5.4)))+@end lisp+@end defvr++@deftp {Data Type} zfs-configuration+This data type represents the configuration of the ZFS service.+The available fields are:++@table @asis+@item @code{kernel}+The package of the Linux kernel to compile ZFS for. Required. It+@emph{must} be the same kernel you use in your operating system.++@item @code{base-zfs} (default: @code{zfs})+The ZFS package to use. It will be modified to use the indicated+kernel.++@item @code{dependencies} (default: @code{'()})+A list of @code{<file-system>}s or @code{<mapped-device>}s that+must be mounted or opened before ZFS scans for pools to import.+For example, you might have LUKS containers as the leaf VDEVs of+a ZFS pool.+@end table+@end deftp++Once your system has been configured to include the ZFS service+and you have restarted the system, you can manage ZFS pools and+datasets with @code{zpool} and @code{zfs} commands.++ZFS datasets with an appropriate @code{mountpoint} property will+be automounted at startup after the root file system is started.+Encrypted datasets that are automounted will cause boot to pause+and prompt for the password to be provided on the console.++It's possible to have a ZFS dataset as your @code{/home} by simply+setting the @code{mountpoint} ZFS property. However, note that ZFS+will refuse to mount over a non-empty directory, so if your root+filesystem already has a non-empty @code{/home}, you should remove+it and its contents, then restart the system.++ZFS features @dfn{ZVOL}s, which are block devices that ZFS exposes+to the system. You can put any file system inside a ZVOL. In order+to mount such a filesystem at boot, you can declare it as dependent+on @code{%zfs-zvol-dependency}.++@defvr {Scheme Variable} %zfs-zvol-dependency+This is an artificial @code{<mapped-device>} object which tells+the file mounting service to wait for ZFS to provide ZVOLs before+mounting the file system that is dependent on it. For example:++@lisp+(file-system+ (device "/dev/zvol/pool-name/zvol-name")+ (mount-point "/ext4-on-zfs")+ (type "ext4")+ (dependencies (list %zfs-zvol-dependency)))+@end lisp++Do @emph{not} add @code{%zfs-zvol-dependency} to your+@code{mapped-devices} declaration, In addition, only use it as a+@code{file-system} dependency if you instantiate a+@code{zfs-service-type} service in your operating system.+@end defvr++Having ZFS as a root filesystem or as @code{/boot} is not supported+yet.+ @node Mapped Devices @section Mapped Devices
diff --git a/gnu/local.mk b/gnu/local.mkindex 1151d4642e..5aeb45c4c2 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -602,6 +602,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/dict.scm \ %D%/services/dns.scm \ %D%/services/docker.scm \+ %D%/services/file-systems.scm \ %D%/services/authentication.scm \ %D%/services/games.scm \ %D%/services/ganeti.scm \diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex 13cfb6a8a2..ef3680583b 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -188,7 +188,9 @@
references-file
- %base-services))+ %base-services++ dependency->shepherd-service-name))
;;; Commentary: ;;;diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scmnew file mode 100644index 0000000000..9061ab9582--- /dev/null+++ b/gnu/services/file-systems.scm@@ -0,0 +1,174 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (gnu services file-systems)+ #:use-module (gnu packages file-systems)+ #:use-module (gnu services)+ #:use-module (gnu services base)+ #:use-module (gnu services linux)+ #:use-module (gnu services shepherd)+ #:use-module (gnu system mapped-devices)+ #:use-module (guix gexp)+ #:use-module (guix packages)+ #:use-module (guix records)+ #:export (zfs-service-type++ zfs-configuration+ zfs-configuration?+ zfs-configuration-kernel+ zfs-configuration-base-zfs+ zfs-configuration-dependencies++ %zfs-zvol-dependency))++(define-record-type* <zfs-configuration>+ zfs-configuration make-zfs-configuration zfs-configuration?+ ; kernel you want to compile the base-zfs module for.+ (kernel zfs-configuration-kernel)+ ; base package that will be compiled for the kernel+ (base-zfs zfs-configuration-base-zfs+ (default zfs))+ ; list of <mapped-device> | <file-system> we should wait for,+ ; before scanning for ZFS pools.+ (dependencies zfs-configuration-dependencies+ (default '())))++;; This is a synthetic and unusable MAPPED-DEVICE; its only use+;; is to be added as a (dependency ...) of some FILE-SYSTEM.+(define %zfs-zvol-dependency+ (mapped-device+ (source '())+ ;; The /* prevents naming conflict with non-ZFS device mappings,+ ;; since it is not a valid name for mapped devices, and also+ ;; implies "all zvols" in terms of globs.+ (targets '("zvol/*"))+ (type #f)))++(define (make-zfs-package conf)+ (let ((base-zfs (zfs-configuration-base-zfs conf))+ (kernel (zfs-configuration-kernel conf)))+ (package+ (inherit base-zfs)+ (name (string-join (list (package-name base-zfs)+ "for"+ (package-name kernel)+ (package-version kernel)+ "version")+ "-"))+ (arguments (cons* #:linux kernel (package-arguments base-zfs))))))++(define (zfs-loadable-module conf)+ (list (list (make-zfs-package conf) "module")))++(define (zfs-shepherd-services conf)+ (let* ((zfs-package (make-zfs-package conf))+ (zpool (file-append zfs-package "/sbin/zpool"))+ (zfs (file-append zfs-package "/sbin/zfs"))+ (zvol_wait (file-append zfs-package "/bin/zvol_wait"))+ (scheme-modules `((srfi srfi-1)+ (srfi srfi-34)+ (srfi srfi-35)+ (rnrs io ports)+ ,@%default-modules)))++ (define zfs-scan+ (shepherd-service+ (provision '(zfs-scan))+ (documentation "Scans for ZFS pools.")+ (requirement `(kernel-module-loader+ root-file-system+ ,@(map dependency->shepherd-service-name+ (zfs-configuration-dependencies conf))))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error importing zpools: ~a~%"+ (condition-message c))+ #f))+ ; TODO: optionally use a cachefile, for systems with dozens or+ ; hundreds of devices.+ (invoke/quiet #$zpool "import" "-a" "-N"))))+ (stop #~(const #t))))++ (define device-mapping-zvol/*+ (shepherd-service+ (provision '(device-mapping-zvol/*))+ (documentation "Waits for ZFS ZVOL devices to appear.")+ (requirement '(zfs-scan))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error waiting for zvols: ~a~%"+ (condition-message c))+ #f))+ (invoke/quiet #$zvol_wait))))+ (stop #~(const #t))))++ (define zfs-automount+ (shepherd-service+ (provision '(zfs-automount))+ (documentation "Automounts ZFS datasets.")+ (requirement '(zfs-scan))+ (modules scheme-modules)+ (start #~(lambda _+ (guard (c ((message-condition? c)+ (format (current-error-port)+ "error automounting zfs: ~a~$")+ #f))+ ; (current-output-port) is typically connected to /dev/klog,+ ; so redirect it to (current-error-port) so that user can see+ ; prompts for passphrases on console+ (with-output-to-port (current-error-port)+ (lambda ()+ (invoke #$zfs "mount" "-a" "-l"))))))+ (stop #~(lambda _+ ;; make sure we don't keep any ZFS mountpoints busy.+ (chdir "/")+ ;; unmount everything.+ (invoke/quiet #$zfs "unmount" "-a" "-f")))))++ (list zfs-scan+ device-mapping-zvol/*+ zfs-automount)))++(define zfs-service-type+ (service-type (name 'zfs)+ (extensions+ (list+ ; install the kernel module+ (service-extension kernel-loadable-module-service-type+ zfs-loadable-module)+ ; load the kernel module+ (service-extension kernel-module-loader-service-type+ (const '("zfs")))+ ; scan ZFS pools, automount filesystem, wait for zvols.+ (service-extension shepherd-root-service-type+ zfs-shepherd-services)+ ; make sure automount occurs before file-systems target is reached+ (service-extension file-systems-target-service-type+ (const '(zfs-automount)))+ ; install ZFS management tools+ (service-extension profile-service-type+ (compose list make-zfs-package))+ ; install ZFS udev rules+ (service-extension udev-service-type+ (compose list make-zfs-package))))+ (description+ "Install ZFS, an advanced filesystem and volume manager.")))
--2.30.0
宋文武 wrote on 23 Jan 14:05 +0100
Re: [bug#45692] [PATCH 2/4] gnu: Make file-systems target extensible by services.
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
OSZP286MB066439455D4B38ABA5F04337A3BF0@OSZP286MB0664.JPNP286.PROD.OUTLOOK.COM
Hello!
raid5atemyhomework <raid5atemyhomework@protonmail.com> writes:
Toggle quote (5 lines)>>From 792a8f8efc95e4fe9a94d42f839ddcfb034b8540 Mon Sep 17 00:00:00 2001> From: raid5atemyhomework <raid5atemyhomework@protonmail.com>> Date: Wed, 6 Jan 2021 08:15:54 +0800> Subject: [PATCH 2/4] gnu: Make file-systems target extensible by services.
It’s not clear to me what “file-systems target” is, and why we’reextending ‘file-systems-target-service-type’… I think what we want isto extend the ‘file-system-service-type’ with shepherd services’ names,which means some shepherd services that will handle file systemsmounting themself instead of the usual <file-system> objects, fstabentries, mounted by kernel.

So I write with this patch to extend file-system-service-type directlyinstead of introducing a new ‘file-systems-target-service-type’:
From 44ee1e470a2f9d4985af4d51654d9f943caa0f24 Mon Sep 17 00:00:00 2001From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>Date: Sat, 23 Jan 2021 20:39:06 +0800Subject: [PATCH] services: Allow 'file-system-service-type' extensible by service name.MIME-Version: 1.0Content-Type: text/plain; charset=UTF-8Content-Transfer-Encoding: 8bit
* gnu/services/base.scm (file-system-shepherd-services): Add'extra-services-names' paramater.(file-system-service-type): Handle services’ names from extensions.--- gnu/services/base.scm | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)
Toggle diff (53 lines)diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex f6a490f712..7bddef5034 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -364,15 +364,16 @@ FILE-SYSTEM." (gnu system file-systems) ,@%default-modules))))))) -(define (file-system-shepherd-services file-systems)+(define (file-system-shepherd-services file-systems extra-services-names) "Return the list of Shepherd services for FILE-SYSTEMS." (let* ((file-systems (filter file-system-mount? file-systems))) (define sink (shepherd-service (provision '(file-systems))- (requirement (cons* 'root-file-system 'user-file-systems- (map file-system->shepherd-service-name- file-systems)))+ (requirement (append '(root-file-system user-file-systems)+ (map file-system->shepherd-service-name+ file-systems)+ extra-services-names)) (documentation "Target for all the initially-mounted file systems") (start #~(const #t)) (stop #~(const #f))))@@ -429,13 +430,23 @@ FILE-SYSTEM." (service-type (name 'file-systems) (extensions (list (service-extension shepherd-root-service-type- file-system-shepherd-services)+ (lambda (value)+ (file-system-shepherd-services+ (filter file-system? value)+ (filter symbol? value)))) (service-extension fstab-service-type- file-system-fstab-entries)+ (lambda (value)+ (file-system-fstab-entries+ (filter file-system? value)))) ;; Have 'user-processes' depend on 'file-systems'. (service-extension user-processes-service-type (const '(file-systems)))))++ ;; Extensions consist of lists of <file-system> objects or+ ;; shepherd services’ names (symbols). In the latter case,+ ;; the provided shepherd services supposed to mount and+ ;; unmount some file systems themself. (compose concatenate) (extend append) (description-- 2.29.2
What do you think? Thank you!
R
R
raid5atemyhomework wrote on 25 Jan 01:18 +0100
(name . 宋文武)(address . iyzsong@outlook.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
yrjG3wstz9GpyH32kV3JqsEiG-KdxmP9_JHe6ghdMoizrBWpIZhd3hl2VPjFuWagt1WLPXMqSGdMZZWTcvE6J7qxVjdcvq3zTK-Wccd3wEs=@protonmail.com
Hello as well,
That is certainly another possibility, would this be more palatable to Guix?
Thanksraid5atemyhomework
Toggle quote (20 lines)> Hello!>> raid5atemyhomework raid5atemyhomework@protonmail.com writes:>> > > From 792a8f8efc95e4fe9a94d42f839ddcfb034b8540 Mon Sep 17 00:00:00 2001> > > From: raid5atemyhomework raid5atemyhomework@protonmail.com> > > Date: Wed, 6 Jan 2021 08:15:54 +0800> > > Subject: [PATCH 2/4] gnu: Make file-systems target extensible by services.>> It’s not clear to me what “file-systems target” is, and why we’re> extending ‘file-systems-target-service-type’… I think what we want is> to extend the ‘file-system-service-type’ with shepherd services’ names,> which means some shepherd services that will handle file systems> mounting themself instead of the usual <file-system> objects, fstab> entries, mounted by kernel.>> So I write with this patch to extend file-system-service-type directly> instead of introducing a new ‘file-systems-target-service-type’:>> What do you think? Thank you!
From 44ee1e470a2f9d4985af4d51654d9f943caa0f24 Mon Sep 17 00:00:00 2001From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>Date: Sat, 23 Jan 2021 20:39:06 +0800Subject: [PATCH] services: Allow 'file-system-service-type' extensible by service name.MIME-Version: 1.0Content-Type: text/plain; charset=UTF-8Content-Transfer-Encoding: 8bit
* gnu/services/base.scm (file-system-shepherd-services): Add'extra-services-names' paramater.(file-system-service-type): Handle services’ names from extensions.--- gnu/services/base.scm | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-)
Toggle diff (53 lines)diff --git a/gnu/services/base.scm b/gnu/services/base.scmindex f6a490f712..7bddef5034 100644--- a/gnu/services/base.scm+++ b/gnu/services/base.scm@@ -364,15 +364,16 @@ FILE-SYSTEM." (gnu system file-systems) ,@%default-modules))))))) -(define (file-system-shepherd-services file-systems)+(define (file-system-shepherd-services file-systems extra-services-names) "Return the list of Shepherd services for FILE-SYSTEMS." (let* ((file-systems (filter file-system-mount? file-systems))) (define sink (shepherd-service (provision '(file-systems))- (requirement (cons* 'root-file-system 'user-file-systems- (map file-system->shepherd-service-name- file-systems)))+ (requirement (append '(root-file-system user-file-systems)+ (map file-system->shepherd-service-name+ file-systems)+ extra-services-names)) (documentation "Target for all the initially-mounted file systems") (start #~(const #t)) (stop #~(const #f))))@@ -429,13 +430,23 @@ FILE-SYSTEM." (service-type (name 'file-systems) (extensions (list (service-extension shepherd-root-service-type- file-system-shepherd-services)+ (lambda (value)+ (file-system-shepherd-services+ (filter file-system? value)+ (filter symbol? value)))) (service-extension fstab-service-type- file-system-fstab-entries)+ (lambda (value)+ (file-system-fstab-entries+ (filter file-system? value)))) ;; Have 'user-processes' depend on 'file-systems'. (service-extension user-processes-service-type (const '(file-systems)))))++ ;; Extensions consist of lists of <file-system> objects or+ ;; shepherd services’ names (symbols). In the latter case,+ ;; the provided shepherd services supposed to mount and+ ;; unmount some file systems themself. (compose concatenate) (extend append) (description-- 2.29.2
D
D
Danny Milosavljevic wrote on 8 Feb 04:31 +0100
Re: [bug#45692] [PATCH 4/4] gnu: Add ZFS service.
(name . raid5atemyhomework viaGuix-patchesvia)(address . guix-patches@gnu.org)
20210208043152.7cdd5900@scratchpost.org
Hello,
is it necessary to manually load the ZFS Linux kernel module? Usually, Linuxshould autoload its drivers when the device files are accessed--module or not.
If that fails, didn't you patch the zfs package so it refers to modprobe anyway?
(I wanna prevent having 543 places in Guix where modprobe is invoked)
If it doesn't autoload, we can totally modprobe it, though.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAmAgsKgACgkQ5xo1VCwwuqWYlAf+LmpUSOE+T8s2fUOta0h6kXCDklfZ7ehCrHa83LfNAuOl0jVQS9OeoabcIN6y9gujcUY345NJI/QvWTqiKoU6deogK2Ut4VDLHfzhiD5H4J7WYGm1Z//ygFEqku7+YL1rbMTo/qe0s40U0MeMNq7V0PEDyg3XvKZmL7GdtYa71sAKRID3RMQ6M5pdGJUUWOlNOs9p+VQOcobReTS1t9NMRhrCHg6tjrVtTrnc7aVQxuEg7y+BSGeNC6MpLA/SHxYcu8w6vomlNyGQL7oY6xNMNc6NihiToWrinkV56bG7x7ssLenJGYsWOCOG5xqXwUy5VmcTHDYADCmx0q2v/4iJvQ===jbFl-----END PGP SIGNATURE-----

R
R
raid5atemyhomework wrote on 8 Feb 07:25 +0100
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
yKAqBHv-O0IjcNxi3O1RSMfxY_53Le1QYhA8AS6AIp8JYPpR7eeK3wdWvvBluJhoA13O0iUD904p_DXG78_1XP4mQtE2pKixjvtO7tXSQv0=@protonmail.com
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐On Monday, February 8, 2021 3:31 AM, Danny Milosavljevic <dannym@scratchpost.org> wrote:
Toggle quote (7 lines)> Hello,>> is it necessary to manually load the ZFS Linux kernel module? Usually, Linux> should autoload its drivers when the device files are accessed--module or not.>> If that fails, didn't you patch the zfs package so it refers to modprobe anyway?
It does, for `udev`. Not certain if it would trigger the load, though. Presumably it would, but I have not tested.
A lot of this code was written in 0.8.5, where the ZFS tools will *not* attempt to `modprobe` the module first, they just error out. I think in 2.0.1 upstream added code to load the module if the ZFS tool(s) were invoked while the module was not loaded, which is why additional patch was added by me for the 2.0.1 to patch in `modprobe`. I have not tested if the `modprobe`-ing by the ZFS tools actually *works*, though.
Maybe when I find some amount of motivation I can go test it.
Toggle quote (6 lines)>> (I wanna prevent having 543 places in Guix where modprobe is invoked)>> If it doesn't autoload, we can totally modprobe it, though.

Thanksraid5atemyhomework
L
L
Ludovic Courtès wrote on 10 Feb 15:13 +0100
Re: bug#45692: [PATCH 0/4] Even Better ZFS Support on Guix
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
87zh0c6mf4.fsf_-_@gnu.org
raid5atemyhomework <raid5atemyhomework@protonmail.com> skribis:
Toggle quote (20 lines)> +(define (kernel-builder-configuration->system-entry config)> + "Return the kernel and hurd entries of the 'system' directory."> + (mbegin %store-monad> + (let* ((kernel (kernel-builder-configuration-kernel config))> + (hurd (kernel-builder-configuration-hurd config))> + (modules (kernel-builder-configuration-modules config))> + (kernel (if hurd> + kernel> + (profile> + (content (packages->manifest> + (cons kernel> + (map (lambda (module)> + (if (package? module)> + (package-for-kernel kernel module)> + module))> + modules))))> + (hooks (list linux-module-database))))))> + (return `(("kernel" ,kernel)> + ,@(if hurd `(("hurd" ,hurd)) '()))))))
It may be clearer to avoid ‘mbegin’ and instead write it this way:
(define (kernel-builder-configuration-modules config) (let* … (with-monad %store-monad (return …))))
Both work but I find this variant slightly clearer.
Ludo’.
L
L
Ludovic Courtès wrote on 10 Feb 15:17 +0100
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
87sg646m7y.fsf_-_@gnu.org
Hi!
I agree with 宋文武.
Still…
raid5atemyhomework <raid5atemyhomework@protonmail.com> skribis:
Toggle quote (23 lines)> (service-type (name 'file-systems)> (extensions> (list (service-extension shepherd-root-service-type> - file-system-shepherd-services)> + (lambda (value)> + (file-system-shepherd-services> + (filter file-system? value)> + (filter symbol? value))))> (service-extension fstab-service-type> - file-system-fstab-entries)> + (lambda (value)> + (file-system-fstab-entries> + (filter file-system? value))))> > ;; Have 'user-processes' depend on 'file-systems'.> (service-extension user-processes-service-type> (const '(file-systems)))))> +> + ;; Extensions consist of lists of <file-system> objects or> + ;; shepherd services’ names (symbols). In the latter case,> + ;; the provided shepherd services supposed to mount and> + ;; unmount some file systems themself.
Why do we need to extend with symbols?
In general it’s much clearer if extensions receive only one type ofobject (<file-system> records in this case). It’s also best to avoidpassing around symbolic names (that’s why we extend with <file-system>records rather than with Shepherd service names or whatever.)
Ludo’.
L
L
Ludovic Courtès wrote on 10 Feb 15:27 +0100
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
87eeho6ls3.fsf_-_@gnu.org
raid5atemyhomework <raid5atemyhomework@protonmail.com> skribis:
Toggle quote (13 lines)>>From d0e095136cd471f083d92416f12ca22b47301f31 Mon Sep 17 00:00:00 2001> From: raid5atemyhomework <raid5atemyhomework@protonmail.com>> Date: Wed, 6 Jan 2021 09:24:20 +0800> Subject: [PATCH 4/4] gnu: Add service to install ZFS.>> * gnu/services/file-systems.scm: New file.> (zfs-service-type): New variable.> (<zfs-configuration>): New type.> (%zfs-zvol-dependency): New variable.> * gnu/local.mk: Add gnu/services/file-systems.scm.> * gnu/services/base.scm (dependency->shepherd-service-name): Export.> * doc/guix.texi (ZFS file system): New subsection.
A bit of nitpicking on the documentation bits…
Toggle quote (3 lines)> +@node ZFS file system> +@subsection ZFS file system
Please follow the same typographical conventions as in the rest of themanual. In particular, capitalize section titles and leave two spacesafter end-of-sentence periods.
Here I recommend s/ZFS file system/ZFS/ in all the text.
Toggle quote (7 lines)> +The ZFS file system has a license incompatible with the Linux kernel,> +and thus cannot be distributed with the kernel. However, as a user> +you have the right to do whatever you want on your own hardware,> +including download the ZFS source code, compile it, link it to your> +own private copy of Linux, and run it. You can even use the Guix build> +system to automate this.
How about:
ZFS is free software; unfortunately its license is incompatible with the GNU General Public License (GPL), the license of the Linux kernel, which means they cannot be distributed together. However, as a user, you can choose to build ZFS and use it together with Linux; you can even rely on Guix to automate this task. See @uref{https://www.fsf.org/licensing/zfs-and-linux,this analysis by the Free Software Foundation} for more information.
?
Toggle quote (1 lines)> +As a large and complex kernel module, ZFS on Linux has to be compiled
^I think it’s a proper noun and is spelled “ZFS-on-Linux”.
Toggle quote (2 lines)> +with a specific version of the kernel. Often even the latest ZFS> +package available cannot be compiled with the latest Linux kernel
^s/Linux/Linux-libre/
Thanks,Ludo’.
R
R
raid5atemyhomework wrote on 10 Feb 15:32 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
ih7G5G36XJ1ajfM1OZo4G6oaKrwtRusZF9NJhKBtRudytmXCib41DxapatiWe6As3jl1miABW-Yp8oXy4wI4dq5xn4AXEFO9DAdXtFZYGyY=@protonmail.com
Hi Ludo,
Toggle quote (25 lines)> raid5atemyhomework raid5atemyhomework@protonmail.com skribis:>> > > From d0e095136cd471f083d92416f12ca22b47301f31 Mon Sep 17 00:00:00 2001> > > From: raid5atemyhomework raid5atemyhomework@protonmail.com> > > Date: Wed, 6 Jan 2021 09:24:20 +0800> > > Subject: [PATCH 4/4] gnu: Add service to install ZFS.> >> > - gnu/services/file-systems.scm: New file.> > (zfs-service-type): New variable.> > (<zfs-configuration>): New type.> > (%zfs-zvol-dependency): New variable.> >> > - gnu/local.mk: Add gnu/services/file-systems.scm.> > - gnu/services/base.scm (dependency->shepherd-service-name): Export.> > - doc/guix.texi (ZFS file system): New subsection.>> A bit of nitpicking on the documentation bits…>> > +@node ZFS file system> > +@subsection ZFS file system>> Please follow the same typographical conventions as in the rest of the> manual. In particular, capitalize section titles and leave two spaces> after end-of-sentence periods.
This was imitated from the `@subsection Btrfs file system`.
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 10 Feb 15:46 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)
HM9LDUftlNlIoFFvUdzDIRUsjIADdaZLVoCF0cg-l2EoW6hVu09JpTcsOBMbQbpefRhiRv0IKE4W5QMi3ww5NedWcg8eNiRjrzA3hhJ06N0=@protonmail.com
Toggle quote (7 lines)> Why do we need to extend with symbols?>> In general it’s much clearer if extensions receive only one type of> object (<file-system> records in this case). It’s also best to avoid> passing around symbolic names (that’s why we extend with <file-system>> records rather than with Shepherd service names or whatever.)
For this case, how would it be done?
ZFS file system, on other operating systems and distributions, is documented as automatically mounting filesystems, without management in an `fstab` or similar file, because the intent is that you would make lots of filesystems for various uses and managing an `fstab` would be too onerous. Thus, ZFS file system expects to mount multiple file systems with a single `zfs mount -a` command at startup.
Would the below sketch be acceptable?
```scheme; gnu/system/file-systems.scm(define-record-type* file-system #;... #;... (has-fstab-entry? file-system-has-fstab-entry? (default #t)))
;...; gnu/services/base,scm
(define file-system-service-type (service-type #;... (extensions (list #;... (service-extension fstab-service-type (lambda (file-systems) (filter file-system-has-fstab-entry? file-systems))) #;...)) #;...))
;...; gnu/services/file-systems.scm
(define zfs-service-type (service-type #;... (extensions (list #;... (service-extension file-system-service-type (const (list (file-system (device "dummy") (mount-point "zfs/*") (has-fstab-entry? #f))))))) #;...))```
Then there will be a Shepherd service providing `file-system-zfs/*` which would perform `zfs mount -a -l` on `start` and `zfs unmount -a -f` on `stop`.
Would that be acceptable? I am wary of this since it creates a dummy file-system and needs an additional field on every `file-system` record, one which is *only* used by ZFS. I feel the `file-system-target-service-type` is more generic and does not use trickery.
Thanksraid5atemyhomework
L
L
Ludovic Courtès wrote on 10 Feb 16:44 +0100
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
87zh0c53m6.fsf_-_@gnu.org
Hi,
raid5atemyhomework <raid5atemyhomework@protonmail.com> skribis:
Toggle quote (24 lines)>>From 4beb73c62995cf236b402dad8e1c36016027c781 Mon Sep 17 00:00:00 2001> From: raid5atemyhomework <raid5atemyhomework@protonmail.com>> Date: Tue, 5 Jan 2021 22:27:56 +0800> Subject: [PATCH 1/4] gnu: Allow services to install kernel-loadable modules.>> * gnu/system.scm (operating-system-directory-base-entries): Remove code> to handle generation of "kernel" and "hurd".> (operating-system-default-essential-services): Instantiate> kernel-loadable-module-service.> (hurd-default-essential-services): Instantiate> kernel-loadable-module-service.> (package-for-kernel): Move ...> * gnu/services.scm: ... to here.> (kernel-loadable-module-service-type): New variable.> (kernel-loadable-module-service): New procedure.> * gnu/tests/linux-modules.scm (run-loadable-kernel-modules-test): Move> code to ...> (run-loadable-kernel-modules-test-base): ... new procedure here.> (run-loadable-kernel-modules-service-test): New procedure.> (%test-loadable-kernel-modules-service-0): New variable.> (%test-loadable-kernel-modules-service-1): New variable.> (%test-loadable-kernel-modules-service-2): New variable.> * doc/guix.texi: Document kernel-loadable-module-service-type.
[…]
Toggle quote (6 lines)> +@defvr {Scheme Variable} kernel-loadable-module-service-type> +Type of the service that collects lists of packages containing> +kernel-loadable modules, and adds them to the set of kernel-loadable> +modules.> +@end defvr
Would be nice to expound a bit here, in particular by adding an example(along the lines of those used in system tests maybe?). Otherwise itcan be hard to fathom how this is meant to be used.
Toggle quote (10 lines)> +;; Configuration for the kernel builder.> +(define-record-type* <kernel-builder-configuration> kernel-builder-configuration> + make-kernel-builder-configuration> + kernel-builder-configuration?> + this-kernel-builder-configuration> +> + (kernel kernel-builder-configuration-kernel (default #f))> + (hurd kernel-builder-configuration-hurd (default #f))> + (modules kernel-builder-configuration-modules (default '())))
How about <linux-build-configuration> instead?
In general, throughout the project, we do not use “kernel” and “Linux”interchangeably. Since this is a Linux-only feature, let’s call it thatway and remove the ‘hurd’ field (the Hurd has no notion of in-kernelmodules since pretty much everything happens in user-space.)
Toggle quote (3 lines)> +(define kernel-loadable-module-service-type> + (service-type (name 'kernel-loadable-modules)
Same here: ‘linux-loadable-module-service-type’.
But… it’s not clear at first sight how this differs from the existing‘kernel-module-loader’. Perhaps ‘linux-build-service-type’ would bemore accurate? Or am I missing something?
Thanks,Ludo’.
L
L
Ludovic Courtès wrote on 10 Feb 16:47 +0100
control message for bug #45692
(address . control@debbugs.gnu.org)
87y2fv6i26.fsf@gnu.org
merge 45692 45703quit
L
L
Ludovic Courtès wrote on 10 Feb 16:47 +0100
(address . control@debbugs.gnu.org)
87wnvf6i1i.fsf@gnu.org
merge 45692 45643quit
R
R
raid5atemyhomework wrote on 10 Feb 17:49 +0100
Re: bug#45692: [PATCH 0/4] Even Better ZFS Support on Guix
(name . Ludovic Courtès)(address . ludo@gnu.org)
Y5MIZ9SDFP1f6SdbG-fjEvSAkjgni6w0dilDSaKyWAauhgFfK-swJ6TpVCsLQ7c0bYA8tnIe1Fo7OiFAcuu-vO_W2h2UQdpfxNRZ1w_uSL0=@protonmail.com
Toggle quote (18 lines)> > +;; Configuration for the kernel builder.> > +(define-record-type* <kernel-builder-configuration> kernel-builder-configuration> >> > - make-kernel-builder-configuration> > - kernel-builder-configuration?> > - this-kernel-builder-configuration> > -> > - (kernel kernel-builder-configuration-kernel (default #f))> > - (hurd kernel-builder-configuration-hurd (default #f))> > - (modules kernel-builder-configuration-modules (default '())))>> How about <linux-build-configuration> instead?>> In general, throughout the project, we do not use “kernel” and “Linux”> interchangeably. Since this is a Linux-only feature, let’s call it that> way and remove the ‘hurd’ field (the Hurd has no notion of in-kernel> modules since pretty much everything happens in user-space.)
The `operating-system` record uses `kernel-loadable-modules` as the record field name. I suggest changing that first, if you truly want to differentiate "kernel" from "linux" "throughout the project". Or deprecate it entirely and instead use the new `linux-loadable-modules-service-type`, in principle the only field needed in `operating-system` should be `services`.
Toggle quote (12 lines)>> > +(define kernel-loadable-module-service-type> >> > - (service-type (name 'kernel-loadable-modules)>> Same here: ‘linux-loadable-module-service-type’.>> But… it’s not clear at first sight how this differs from the existing> ‘kernel-module-loader’. Perhaps ‘linux-build-service-type’ would be> more accurate? Or am I missing something?

`kernel-module-loader` explicitly loads a module at startup, it does not make a non-Linux-libre-built-in module actually *loadable*. So there is a need for something to augment the `linux-loadable-modules` record field of `operating-system`.
Thanksraid5atemyhomework
R
R
raid5atemyhomework wrote on 13 Feb 02:49 +0100
(name . Ludovic Courtès)(address . ludo@gnu.org)(name . 45692@debbugs.gnu.org)(address . 45692@debbugs.gnu.org)
20Uzjlb-aGAMYW_UfB_P-PUylV-NrHRtq3NpXLe6Fsfuuq-wssW64sgoWoRSzwQdj-t5g2P4ebBJR2fllYCiQc5LyLb8oc2YrDs021q9u-g=@protonmail.com
Hello Ludo',

Toggle quote (7 lines)> > +As a large and complex kernel module, ZFS on Linux has to be compiled>> ^>>> I think it’s a proper noun and is spelled “ZFS-on-Linux”.
FWIW ZFSonLinux is now folded into OpenZFS (which is now a package that includes both Linux and FreeBSD support, with plans to merge in the Illumos codebase as well) since 2.0.0. The documentation was written when Guix was still at 0.8.5 (when it was definitely ZoL), so I should probably change this to OpenZFS, and clarify whether I am referring to the ZFS file system or the OpenZFS package.
Thanksraid5atemyhomework
宋文武 wrote on 20 Feb 12:44 +0100
Re: ZFS on Guix, again
(name . raid5atemyhomework)(address . raid5atemyhomework@protonmail.com)
OS3P286MB066003E371477FEA2063411EA3839@OS3P286MB0660.JPNP286.PROD.OUTLOOK.COM
raid5atemyhomework <raid5atemyhomework@protonmail.com> writes:
Toggle quote (5 lines)> Hi guix-devel,>> I had some questions on the big ZFS guix bugpatches a week ago, and> did not find any response, so I am back here pestering everyone.
Hello, thank you for working on ZFS for guix!
Toggle quote (12 lines)>> [...]> There are two alternatives:>> * Go with what I already proposed which I think is more general-purpose and cleaner (there is a separate service type that accepts symbols, and a separate service type that accepts `<file-system>` records, and the latter just extends the former).> * Don't make a separate service type, but now we need to add some kind of `fstab?` field to `file-system` so that the ZFS shepherd service that mounts ZFS file systems will not be included in the `/etc/fstab`.>> I think overall that having lots of tiny service types that are then> combined together fits the functional design of Guix better. So I> would strongly propose my original design rather than hacks on top of> `file-system-service-type`.
Well, I think the 'file-system-service-type' should handle all filesystems related configurations, but my opion is not strong. Waitingludo to decide...
?