[PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’

OpenSubmitted by Brian Cully.
Details
4 participants
  • Brian Cully
  • Kaelyn
  • Ludovic Courtès
  • Maxime Devos
Owner
unassigned
Severity
normal
B
B
Brian Cully wrote on 2 May 21:11 +0200
[PATCH v1] initrd: Allow extra search paths w ith ‘initrd-extra-module-paths’
(address . guix-patches@gnu.org)
87wnf3pv87.fsf@ditto.jhoto.spork.org
This patch allows copying of out-of-tree kernel modules to the Linux
initrd.

For out-of-tree modules to found, an extra slot has been added to
‘operating-system’: ‘initrd-extra-module-paths’, which contains a list
containing items of either FILE-LIKE or (FILE-LIKE OUTPUT) which will be
searched for modules in addition to the standard Linux module path. The
required modules can then be added to ‘initrd-modules’ as normal and all paths
will be searched for it, including for any modules depended on.

* gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to
DIRECTORIES. Now takes a list of directories to search, rather than a single
one.
* gnu/system.scm <operating-system>: Add INITRD-EXTRA-MODULE-PATHS
field and accessor. Takes a list of file-like objects.
* gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX
argument to PACKAGES. Now contains a list of file-likes to search for modules.
(raw-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
to (flat-linux-extra-module-paths) along with the selected LINUX package.
(base-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
to (raw-initrd).
---
gnu/build/linux-modules.scm | 19 ++++++++------
gnu/system.scm | 5 ++++
gnu/system/linux-initrd.scm | 52 +++++++++++++++++++++++++------------
3 files changed, 51 insertions(+), 25 deletions(-)

Toggle diff (194 lines)
diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm
index 053720574b..97b7e429ea 100644
--- a/gnu/build/linux-modules.scm
+++ b/gnu/build/linux-modules.scm
@@ -225,8 +225,8 @@ (define (file-name->module-name file)
 '.ko[.gz|.xz]' and normalizing it."
   (normalize-module-name (strip-extension (basename file))))
 
-(define (find-module-file directory module)
-  "Lookup module NAME under DIRECTORY, and return its absolute file name.
+(define (find-module-file directories module)
+  "Lookup module NAME under DIRECTORIES, and return its absolute file name.
 NAME can be a file name with or without '.ko', or it can be a module name.
 Raise an error if it could not be found.
 
@@ -247,16 +247,19 @@ (define names
                            (else chr)))
                        module))))
 
-  (match (find-files directory
-                     (lambda (file stat)
-                       (member (strip-extension
-                                (basename file)) names)))
+  (match (append-map (lambda (directory)
+                       (find-files directory
+                                   (lambda (file _stat)
+                                     (member (strip-extension
+                                              (basename file))
+                                             names))))
+                       directories)
     ((file)
      file)
     (()
-     (error "kernel module not found" module directory))
+     (error "kernel module not found" module directories))
     ((_ ...)
-     (error "several modules by that name" module directory))))
+     (error "several modules by that name" module directories))))
 
 (define* (recursive-module-dependencies files
                                         #:key (lookup-module dot-ko))
diff --git a/gnu/system.scm b/gnu/system.scm
index c3810cbeeb..15ac30c933 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -103,6 +103,7 @@ (define-module (gnu system)
             operating-system-label
             operating-system-default-label
             operating-system-initrd-modules
+            operating-system-initrd-extra-module-paths
             operating-system-initrd
             operating-system-users
             operating-system-groups
@@ -232,6 +233,8 @@ (define-record-type* <operating-system> operating-system
   (initrd-modules operating-system-initrd-modules ; list of strings
                   (thunked)                       ; it's system-dependent
                   (default %base-initrd-modules))
+  (initrd-extra-module-paths operating-system-initrd-extra-module-paths ; list of file-likes
+                             (default '()))
 
   (firmware operating-system-firmware             ; list of packages
             (default %base-firmware))
@@ -1307,6 +1310,8 @@ (define make-initrd
                #:linux (operating-system-kernel os)
                #:linux-modules
                (operating-system-initrd-modules os)
+               #:linux-extra-module-paths
+               (operating-system-initrd-extra-module-paths os)
                #:mapped-devices mapped-devices
                #:keyboard-layout (operating-system-keyboard-layout os)))
 
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 4c4c78e444..50a182d7d5 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -119,13 +119,23 @@ (define builder
                               `(#:references-graphs (("closure" ,init))))
                "/initrd.cpio.gz"))
 
-(define (flat-linux-module-directory linux modules)
+(define (flat-linux-module-directory packages modules)
   "Return a flat directory containing the Linux kernel modules listed in
-MODULES and taken from LINUX."
+MODULES and taken from PACKAGES."
   (define imported-modules
     (source-module-closure '((gnu build linux-modules)
                              (guix build utils))))
 
+  (define (package+out->input package out)
+    (gexp-input package out))
+
+  (define package-inputs
+    (map (lambda (p)
+           (match p
+             ((p o) (package+out->input p o))
+             (p     (package+out->input p "out"))))
+         packages))
+
   (define build-exp
     (with-imported-modules imported-modules
       (with-extensions (list guile-zlib)
@@ -135,11 +145,12 @@ (define build-exp
                          (srfi srfi-1)
                          (srfi srfi-26))
 
-            (define module-dir
-              (string-append #$linux "/lib/modules"))
+            (define module-dirs
+              (map (cut string-append <> "/lib/modules")
+                   '#$package-inputs))
 
             (define modules
-              (let* ((lookup  (cut find-module-file module-dir <>))
+              (let* ((lookup  (cut find-module-file module-dirs <>))
                      (modules (map lookup '#$modules)))
                 (append modules
                         (recursive-module-dependencies
@@ -172,20 +183,23 @@ (define* (raw-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-paths '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       (helper-packages '())
                       qemu-networking?
                       volatile-root?
                       (on-error 'debug))
-  "Return as a file-like object a raw initrd, with kernel
-modules taken from LINUX.  FILE-SYSTEMS is a list of file-systems to be
-mounted by the initrd, possibly in addition to the root file system specified
-on the kernel command line via 'root'.  LINUX-MODULES is a list of kernel
-modules to be loaded at boot time. MAPPED-DEVICES is a list of device
-mappings to realize before FILE-SYSTEMS are mounted.
-HELPER-PACKAGES is a list of packages to be copied in the initrd. It may include
-e2fsck/static or other packages needed by the initrd to check root partition.
+  "Return as a file-like object a raw initrd, with kernel modules taken from
+LINUX.  FILE-SYSTEMS is a list of file-systems to be mounted by the initrd,
+possibly in addition to the root file system specified on the kernel command
+line via 'root'.  LINUX-MODULES is a list of kernel modules to be loaded at
+boot time. LINUX-EXTRA-MODULE-PATHS is a list of file-like objects which will
+be searched for modules in addition to the linux kernel. MAPPED-DEVICES is a
+list of device mappings to realize before FILE-SYSTEMS are mounted.
+HELPER-PACKAGES is a list of packages to be copied in the initrd. It may
+include e2fsck/static or other packages needed by the initrd to check root
+partition.
 
 When true, KEYBOARD-LAYOUT is a <keyboard-layout> record denoting the desired
 console keyboard layout.  This is done before MAPPED-DEVICES are set up and
@@ -221,7 +235,8 @@ (define file-system-scan-commands
           #~())))
 
   (define kodir
-    (flat-linux-module-directory linux linux-modules))
+    (flat-linux-module-directory (cons linux linux-extra-module-paths)
+                                 linux-modules))
 
   (expression->initrd
    (with-imported-modules (source-module-closure
@@ -366,6 +381,7 @@ (define* (base-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-paths '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       qemu-networking?
@@ -386,9 +402,10 @@ (define* (base-initrd file-systems
 QEMU-NETWORKING? and VOLATILE-ROOT? behaves as in raw-initrd.
 
 The initrd is automatically populated with all the kernel modules necessary
-for FILE-SYSTEMS and for the given options.  Additional kernel
-modules can be listed in LINUX-MODULES.  They will be added to the initrd, and
-loaded at boot time in the order in which they appear."
+for FILE-SYSTEMS and for the given options.  Additional kernel modules can be
+listed in LINUX-MODULES.  Additional search paths for modules can be listed in
+LINUX-EXTRA-MODULE-PATHS.  They will be added to the initrd, and loaded at
+boot time in the order in which they appear."
   (define linux-modules*
     ;; Modules added to the initrd and loaded from the initrd.
     `(,@linux-modules
@@ -408,6 +425,7 @@ (define helper-packages
   (raw-initrd file-systems
               #:linux linux
               #:linux-modules linux-modules*
+              #:linux-extra-module-paths linux-extra-module-paths
               #:mapped-devices mapped-devices
               #:helper-packages helper-packages
               #:keyboard-layout keyboard-layout

base-commit: 0a64b629ae8512790d532158a72a4a25698e8157
prerequisite-patch-id: 213ce2b9743f11d45836fe0794f117f3bb84063d
-- 
2.35.1
M
M
Maxime Devos wrote on 3 May 00:03 +0200
Re: [bug#55231] [PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’
c87a1c647289c0f0fb60773e2fd50916861b4d8b.camel@telenet.be
Brian Cully via Guix-patches via schreef op ma 02-05-2022 om 15:11 [-
0400]:
Toggle quote (3 lines)
> This patch allows copying of out-of-tree kernel modules to the Linux
> initrd.

This needs some information in the manual -- when does the field need
to be set? For what kernel modules? Is this required by v4l2loopback-
linux-module and librem-ec-acpi-linux-module? Things like that.
As-is, this functionality is hard to discover.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYnBVPhccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hICAP9CDCAYffBUwkoR9lfzmRg/llBf
MbwZUM4GSZzfcnSn5QEAyofUp0BgfQaywlQmN6VYwaebfeGIVq7RjHqV4Dn2EAk=
=fPl7
-----END PGP SIGNATURE-----


B
B
Brian Cully wrote on 3 May 00:53 +0200
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 55231@debbugs.gnu.org)
87sfprpmt3.fsf@ditto.jhoto.spork.org
Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (7 lines)
> This needs some information in the manual -- when does the field
> need
> to be set? For what kernel modules? Is this required by
> v4l2loopback-
> linux-module and librem-ec-acpi-linux-module? Things like that.
> As-is, this functionality is hard to discover.

I knew I missed something. How’s this look?

---
doc/guix.texi | 42 ++++++++++++++++++++++++++++++++++--------
1 file changed, 34 insertions(+), 8 deletions(-)

Toggle diff (107 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 5399584cb0..1ee2c1b4a3 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -15173,6 +15173,16 @@ Window System.
 The list of Linux kernel modules that need to be available in the
 initial RAM disk.  @xref{Initial RAM Disk}.
 
+@item @code{initrd-extra-module-paths} (default: @code{'()})
+@cindex initrd
+@cindex initial RAM disk
+A list of paths outside of the Linux kernel tree to search for 
Linux
+kernel modules.
+
+The items in this may be either file-like objects (usually 
packages), or
+a list where the first element is a package and the second is an
+output--e.g. @code{(list (list zfs "module"))}.
+
 @item @code{initrd} (default: @code{base-initrd})
 A procedure that returns an initial RAM disk for the Linux
 kernel.  This field is provided to support low-level 
 customization and
@@ -35516,6 +35526,19 @@ file system, you would write:
   (initrd-modules (cons "megaraid_sas" %base-initrd-modules)))
 @end lisp
 
+If a module listed in @code{initrd-modules} is not included in 
the
+Linux-libre kernel, then the location to it must be added to the
+@code{initrd-extra-module-paths} list.  For example, if your root 
file
+system exists on a ZFS pool, then your configuration might look 
like the
+following:
+
+@lisp
+(operating-system
+  ;; @dots{}
+  (initrd-modules (cons "zfs" %base-initrd-modules))
+  (initrd-extra-module-paths (list (list zfs "module"))))
+@end lisp
+
 @defvr {Scheme Variable} %base-initrd-modules
 This is the list of kernel modules included in the initrd by 
 default.
 @end defvr
@@ -35629,13 +35652,15 @@ here is how to use it and customize it 
further.
 @cindex initrd
 @cindex initial RAM disk
 @deffn {Scheme Procedure} raw-initrd @var{file-systems} @
-       [#:linux-modules '()] [#:mapped-devices '()] @
-       [#:keyboard-layout #f] @
+       [#:linux-modules '()] [#:linux-extra-module-paths '()] @
+       [#:mapped-devices '()] [#:keyboard-layout #f] @
        [#:helper-packages '()] [#:qemu-networking? #f] 
        [#:volatile-root? #f]
 Return a derivation that builds a raw initrd.  @var{file-systems} 
 is
 a list of file systems to be mounted by the initrd, possibly in 
 addition to
 the root file system specified on the kernel command line via 
 @option{root}.
 @var{linux-modules} is a list of kernel modules to be loaded at 
 boot time.
+@var{linux-extra-module-paths} is a list of file-like objects to 
be searched
+for kernel modules.
 @var{mapped-devices} is a list of device mappings to realize 
 before
 @var{file-systems} are mounted (@pxref{Mapped Devices}).
 @var{helper-packages} is a list of packages to be copied in the 
 initrd.
@@ -35660,12 +35685,13 @@ to it are lost.
 @deffn {Scheme Procedure} base-initrd @var{file-systems} @
        [#:mapped-devices '()] [#:keyboard-layout #f] @
        [#:qemu-networking? #f] [#:volatile-root? #f] @
-       [#:linux-modules '()]
-Return as a file-like object a generic initrd, with kernel
-modules taken from @var{linux}.  @var{file-systems} is a list of 
 file-systems to be
-mounted by the initrd, possibly in addition to the root file 
 system specified
-on the kernel command line via @option{root}. 
 @var{mapped-devices} is a list of device
-mappings to realize before @var{file-systems} are mounted.
+       [#:linux-modules '()] [#:linux-extra-module-paths '()]
+Return as a file-like object a generic initrd, with kernel 
modules taken
+from @var{linux} and @var{linux-extra-module-paths}. 
@var{file-systems}
+is a list of file-systems to be mounted by the initrd, possibly 
in
+addition to the root file system specified on the kernel command 
line
+via @option{root}.  @var{mapped-devices} is a list of device 
mappings to
+realize before @var{file-systems} are mounted.
 
 When true, @var{keyboard-layout} is a @code{<keyboard-layout>} 
 record denoting
 the desired console keyboard layout.  This is done before 
 @var{mapped-devices}
-- 
2.35.1
K
K
Kaelyn wrote on 13 May 21:46 +0200
[PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’
(name . 55231@debbugs.gnu.org)(address . 55231@debbugs.gnu.org)
j3O4Vb5dVlLA_P-LaSEH-2wB7aPJM3Zlupj1seuNjR34OijQwAiM9ZECRx309TmNwIejOzHv_W5lkUkeQC8MqGTbK3kqjcZa3fCx440oWcw=@protonmail.com
I've read through the original patch and the doc patch, and based on my own tinkering with guix initrd generation, the patches look good to me. I'm also happy to see the functionality being added, and already have plans for using it with zfs (to further the goal of an encrypted multi-disk zfs root).

Cheers,
Kaelyn
M
M
Maxime Devos wrote on 13 May 22:25 +0200
(address . 55231@debbugs.gnu.org)
f31190b82fc43348535ee873bcc29a9ff501a6f7.camel@telenet.be
(for whatever reason, looks like this didn't end up in my inbox)
Toggle quote (8 lines)
> +@item @code{initrd-extra-module-paths} (default: @code{'()})
> [...]
> The items in this may be either file-like objects (usually
> packages), or
> +a list where the first element is a package and the second is an
> +output--e.g. @code{(list (list zfs "module"))}.


As-is, I don't think this is a good example, because
'expression->initrd' does not set #:substitutable? #false,
the 'zfs' package has the comment (*)

`(;; The ZFS kernel module should not be downloaded since the
license
;; terms don't allow for distributing it, only building it
locally.
#:substitutable? #f [...])

and IIUC, the code inside expression->initrd copies the kernel module
into
the new store item, so it looks like this accidentally suggest people
to
commit copyvios, and copyvios are currently against the law.

Greetings,
Maxime.

(*) Though I don't understand that comment: Guix is a distribution, so
by definition
it's distributing zfs -- unless it's only talking about the binaries
and not the
source code ...
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYn6+whccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7tLWAP43VVfdGmeFc9n0cZfORPFbnUiG
b2Lnn8OyvxuhJtatbQEAlkKAnfaCenTjzuTbHXXUmzswBC/4ZzWpl4AtlxOWSQ8=
=lAgN
-----END PGP SIGNATURE-----


B
B
Brian Cully wrote on 21 May 21:12 +0200
[PATCH v2] initrd: Allow extra search paths w ith ‘initrd-extra-module-paths’
(address . 55231@debbugs.gnu.org)(name . Brian Cully)(address . bjc@kublai.com)
2053c4ab42dfe2719cfc377934ac2fb9bcb500a9.1653160364.git.bjc@spork.org
From: Brian Cully <bjc@kublai.com>

This patch allows copying of out-of-tree kernel modules to the Linux
initrd.

For out-of-tree modules to found, an extra slot has been added to
‘operating-system’: ‘initrd-extra-module-paths’, which contains a list
containing items of either FILE-LIKE or (FILE-LIKE OUTPUT) which will be
searched for modules in addition to the standard Linux module path. The
required modules can then be added to ‘initrd-modules’ as normal and all paths
will be searched for it, including for any modules depended on.

* doc: Update documentation for ‘initrd-extra-module-paths’.
* gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to
DIRECTORIES. Now takes a list of directories to search, rather than a single
one.
* gnu/system.scm <operating-system>: Add INITRD-EXTRA-MODULE-PATHS
field and accessor. Takes a list of file-like objects.
* gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX
argument to PACKAGES. Now contains a list of file-likes to search for modules.
(raw-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
to (flat-linux-extra-module-paths) along with the selected LINUX package.
(base-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
to (raw-initrd).
---
This version includes both the previous patch and the documentation.

doc/guix.texi | 42 ++++++++++++++++++++++++------
gnu/build/linux-modules.scm | 19 ++++++++------
gnu/system.scm | 5 ++++
gnu/system/linux-initrd.scm | 52 +++++++++++++++++++++++++------------
4 files changed, 85 insertions(+), 33 deletions(-)

Toggle diff (270 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index faa35060ef..9fd45ea209 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -15490,6 +15490,16 @@ operating-system Reference
 The list of Linux kernel modules that need to be available in the
 initial RAM disk.  @xref{Initial RAM Disk}.
 
+@item @code{initrd-extra-module-paths} (default: @code{'()})
+@cindex initrd
+@cindex initial RAM disk
+A list of paths outside of the Linux kernel tree to search for Linux
+kernel modules.
+
+The items in this may be either file-like objects (usually packages), or
+a list where the first element is a package and the second is an
+output--e.g. @code{(list (list zfs "module"))}.
+
 @item @code{initrd} (default: @code{base-initrd})
 A procedure that returns an initial RAM disk for the Linux
 kernel.  This field is provided to support low-level customization and
@@ -35816,6 +35826,19 @@ Initial RAM Disk
   (initrd-modules (cons "megaraid_sas" %base-initrd-modules)))
 @end lisp
 
+If a module listed in @code{initrd-modules} is not included in the
+Linux-libre kernel, then the location to it must be added to the
+@code{initrd-extra-module-paths} list.  For example, if your root file
+system exists on a ZFS pool, then your configuration might look like the
+following:
+
+@lisp
+(operating-system
+  ;; @dots{}
+  (initrd-modules (cons "zfs" %base-initrd-modules))
+  (initrd-extra-module-paths (list (list zfs "module"))))
+@end lisp
+
 @defvr {Scheme Variable} %base-initrd-modules
 This is the list of kernel modules included in the initrd by default.
 @end defvr
@@ -35929,13 +35952,15 @@ Initial RAM Disk
 @cindex initrd
 @cindex initial RAM disk
 @deffn {Scheme Procedure} raw-initrd @var{file-systems} @
-       [#:linux-modules '()] [#:mapped-devices '()] @
-       [#:keyboard-layout #f] @
+       [#:linux-modules '()] [#:linux-extra-module-paths '()] @
+       [#:mapped-devices '()] [#:keyboard-layout #f] @
        [#:helper-packages '()] [#:qemu-networking? #f] [#:volatile-root? #f]
 Return a derivation that builds a raw initrd.  @var{file-systems} is
 a list of file systems to be mounted by the initrd, possibly in addition to
 the root file system specified on the kernel command line via @option{root}.
 @var{linux-modules} is a list of kernel modules to be loaded at boot time.
+@var{linux-extra-module-paths} is a list of file-like objects to be searched
+for kernel modules.
 @var{mapped-devices} is a list of device mappings to realize before
 @var{file-systems} are mounted (@pxref{Mapped Devices}).
 @var{helper-packages} is a list of packages to be copied in the initrd.
@@ -35960,12 +35985,13 @@ Initial RAM Disk
 @deffn {Scheme Procedure} base-initrd @var{file-systems} @
        [#:mapped-devices '()] [#:keyboard-layout #f] @
        [#:qemu-networking? #f] [#:volatile-root? #f] @
-       [#:linux-modules '()]
-Return as a file-like object a generic initrd, with kernel
-modules taken from @var{linux}.  @var{file-systems} is a list of file-systems to be
-mounted by the initrd, possibly in addition to the root file system specified
-on the kernel command line via @option{root}.  @var{mapped-devices} is a list of device
-mappings to realize before @var{file-systems} are mounted.
+       [#:linux-modules '()] [#:linux-extra-module-paths '()]
+Return as a file-like object a generic initrd, with kernel modules taken
+from @var{linux} and @var{linux-extra-module-paths}.  @var{file-systems}
+is a list of file-systems to be mounted by the initrd, possibly in
+addition to the root file system specified on the kernel command line
+via @option{root}.  @var{mapped-devices} is a list of device mappings to
+realize before @var{file-systems} are mounted.
 
 When true, @var{keyboard-layout} is a @code{<keyboard-layout>} record denoting
 the desired console keyboard layout.  This is done before @var{mapped-devices}
diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm
index 053720574b..97b7e429ea 100644
--- a/gnu/build/linux-modules.scm
+++ b/gnu/build/linux-modules.scm
@@ -225,8 +225,8 @@ (define (file-name->module-name file)
 '.ko[.gz|.xz]' and normalizing it."
   (normalize-module-name (strip-extension (basename file))))
 
-(define (find-module-file directory module)
-  "Lookup module NAME under DIRECTORY, and return its absolute file name.
+(define (find-module-file directories module)
+  "Lookup module NAME under DIRECTORIES, and return its absolute file name.
 NAME can be a file name with or without '.ko', or it can be a module name.
 Raise an error if it could not be found.
 
@@ -247,16 +247,19 @@ (define (find-module-file directory module)
                            (else chr)))
                        module))))
 
-  (match (find-files directory
-                     (lambda (file stat)
-                       (member (strip-extension
-                                (basename file)) names)))
+  (match (append-map (lambda (directory)
+                       (find-files directory
+                                   (lambda (file _stat)
+                                     (member (strip-extension
+                                              (basename file))
+                                             names))))
+                       directories)
     ((file)
      file)
     (()
-     (error "kernel module not found" module directory))
+     (error "kernel module not found" module directories))
     ((_ ...)
-     (error "several modules by that name" module directory))))
+     (error "several modules by that name" module directories))))
 
 (define* (recursive-module-dependencies files
                                         #:key (lookup-module dot-ko))
diff --git a/gnu/system.scm b/gnu/system.scm
index ba3a1865d7..6c712e5cea 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -107,6 +107,7 @@ (define-module (gnu system)
             operating-system-label
             operating-system-default-label
             operating-system-initrd-modules
+            operating-system-initrd-extra-module-paths
             operating-system-initrd
             operating-system-users
             operating-system-groups
@@ -236,6 +237,8 @@ (define-record-type* <operating-system> operating-system
   (initrd-modules operating-system-initrd-modules ; list of strings
                   (thunked)                       ; it's system-dependent
                   (default %base-initrd-modules))
+  (initrd-extra-module-paths operating-system-initrd-extra-module-paths ; list of file-likes
+                             (default '()))
 
   (firmware operating-system-firmware             ; list of packages
             (default %base-firmware))
@@ -1312,6 +1315,8 @@ (define (operating-system-initrd-file os)
                #:linux (operating-system-kernel os)
                #:linux-modules
                (operating-system-initrd-modules os)
+               #:linux-extra-module-paths
+               (operating-system-initrd-extra-module-paths os)
                #:mapped-devices mapped-devices
                #:keyboard-layout (operating-system-keyboard-layout os)))
 
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 4c4c78e444..50a182d7d5 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -119,13 +119,23 @@ (define* (expression->initrd exp
                               `(#:references-graphs (("closure" ,init))))
                "/initrd.cpio.gz"))
 
-(define (flat-linux-module-directory linux modules)
+(define (flat-linux-module-directory packages modules)
   "Return a flat directory containing the Linux kernel modules listed in
-MODULES and taken from LINUX."
+MODULES and taken from PACKAGES."
   (define imported-modules
     (source-module-closure '((gnu build linux-modules)
                              (guix build utils))))
 
+  (define (package+out->input package out)
+    (gexp-input package out))
+
+  (define package-inputs
+    (map (lambda (p)
+           (match p
+             ((p o) (package+out->input p o))
+             (p     (package+out->input p "out"))))
+         packages))
+
   (define build-exp
     (with-imported-modules imported-modules
       (with-extensions (list guile-zlib)
@@ -135,11 +145,12 @@ (define (flat-linux-module-directory linux modules)
                          (srfi srfi-1)
                          (srfi srfi-26))
 
-            (define module-dir
-              (string-append #$linux "/lib/modules"))
+            (define module-dirs
+              (map (cut string-append <> "/lib/modules")
+                   '#$package-inputs))
 
             (define modules
-              (let* ((lookup  (cut find-module-file module-dir <>))
+              (let* ((lookup  (cut find-module-file module-dirs <>))
                      (modules (map lookup '#$modules)))
                 (append modules
                         (recursive-module-dependencies
@@ -172,20 +183,23 @@ (define* (raw-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-paths '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       (helper-packages '())
                       qemu-networking?
                       volatile-root?
                       (on-error 'debug))
-  "Return as a file-like object a raw initrd, with kernel
-modules taken from LINUX.  FILE-SYSTEMS is a list of file-systems to be
-mounted by the initrd, possibly in addition to the root file system specified
-on the kernel command line via 'root'.  LINUX-MODULES is a list of kernel
-modules to be loaded at boot time. MAPPED-DEVICES is a list of device
-mappings to realize before FILE-SYSTEMS are mounted.
-HELPER-PACKAGES is a list of packages to be copied in the initrd. It may include
-e2fsck/static or other packages needed by the initrd to check root partition.
+  "Return as a file-like object a raw initrd, with kernel modules taken from
+LINUX.  FILE-SYSTEMS is a list of file-systems to be mounted by the initrd,
+possibly in addition to the root file system specified on the kernel command
+line via 'root'.  LINUX-MODULES is a list of kernel modules to be loaded at
+boot time. LINUX-EXTRA-MODULE-PATHS is a list of file-like objects which will
+be searched for modules in addition to the linux kernel. MAPPED-DEVICES is a
+list of device mappings to realize before FILE-SYSTEMS are mounted.
+HELPER-PACKAGES is a list of packages to be copied in the initrd. It may
+include e2fsck/static or other packages needed by the initrd to check root
+partition.
 
 When true, KEYBOARD-LAYOUT is a <keyboard-layout> record denoting the desired
 console keyboard layout.  This is done before MAPPED-DEVICES are set up and
@@ -221,7 +235,8 @@ (define* (raw-initrd file-systems
           #~())))
 
   (define kodir
-    (flat-linux-module-directory linux linux-modules))
+    (flat-linux-module-directory (cons linux linux-extra-module-paths)
+                                 linux-modules))
 
   (expression->initrd
    (with-imported-modules (source-module-closure
@@ -366,6 +381,7 @@ (define* (base-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-paths '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       qemu-networking?
@@ -386,9 +402,10 @@ (define* (base-initrd file-systems
 QEMU-NETWORKING? and VOLATILE-ROOT? behaves as in raw-initrd.
 
 The initrd is automatically populated with all the kernel modules necessary
-for FILE-SYSTEMS and for the given options.  Additional kernel
-modules can be listed in LINUX-MODULES.  They will be added to the initrd, and
-loaded at boot time in the order in which they appear."
+for FILE-SYSTEMS and for the given options.  Additional kernel modules can be
+listed in LINUX-MODULES.  Additional search paths for modules can be listed in
+LINUX-EXTRA-MODULE-PATHS.  They will be added to the initrd, and loaded at
+boot time in the order in which they appear."
   (define linux-modules*
     ;; Modules added to the initrd and loaded from the initrd.
     `(,@linux-modules
@@ -408,6 +425,7 @@ (define* (base-initrd file-systems
   (raw-initrd file-systems
               #:linux linux
               #:linux-modules linux-modules*
+              #:linux-extra-module-paths linux-extra-module-paths
               #:mapped-devices mapped-devices
               #:helper-packages helper-packages
               #:keyboard-layout keyboard-layout
-- 
2.36.1
K
K
Kaelyn wrote on 30 May 22:14 +0200
[PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’
(name . 55231@debbugs.gnu.org)(address . 55231@debbugs.gnu.org)
5RCll7W7NfLNRPRFsqgPxB3zS1_6W-9QJbOmXc5RW3E8oZ4Pm_JTzO0SZXsMkZtC811Rw5G0ISt75vP_YKll5MGU7_Krm3Y6IWoXYL9xR3A=@protonmail.com
As the earlier patch discussion seemed to have focused on ZFS licensing issues that are largely orothogonal to the functionality being added, I'd like to again voice my support for the proposed patch. While ZFS is indeed a bad example due to open questions about module licensing and redistribution, it isn't the only out-of-tree module currently packaged in Guix which could be used with the extra initrd support (not counting future packages or external channels). Within the guix repo, the packages using the `linux-module-build-system` aside from zfs are:
- acpi-call-linux-module
- bbswitch-module
- ddcci-driver-linux
- rtl8812au-aircrack-ng-linux-module
- rtl8821ce-linux-module
- ttyebus-linux-module
- v4l2loopback-linux-module
- vhba-module
- wiregard-linux-compat

Of those nine, at least four look to be for specific hardware, and I can come up with theoretical use cases for why someone might want those or wiregard-linux-compat (though the cases are definitely contrived straw-men that may or may not be implementable in practice or actually useful in an initrd context, such as network booting with an older kernel and the root filesystem can only be accessed over a wireguard connection).

Cheers,
Kaelyn
L
L
Ludovic Courtès wrote on 1 Jun 17:54 +0200
(name . Brian Cully)(address . bjc@spork.org)
87ilpkwf8x.fsf_-_@gnu.org
Hello Brian,

Brian Cully <bjc@spork.org> skribis:

Toggle quote (25 lines)
> From: Brian Cully <bjc@kublai.com>
>
> This patch allows copying of out-of-tree kernel modules to the Linux
> initrd.
>
> For out-of-tree modules to found, an extra slot has been added to
> ‘operating-system’: ‘initrd-extra-module-paths’, which contains a list
> containing items of either FILE-LIKE or (FILE-LIKE OUTPUT) which will be
> searched for modules in addition to the standard Linux module path. The
> required modules can then be added to ‘initrd-modules’ as normal and all paths
> will be searched for it, including for any modules depended on.
>
> * doc: Update documentation for ‘initrd-extra-module-paths’.
> * gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to
> DIRECTORIES. Now takes a list of directories to search, rather than a single
> one.
> * gnu/system.scm <operating-system>: Add INITRD-EXTRA-MODULE-PATHS
> field and accessor. Takes a list of file-like objects.
> * gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX
> argument to PACKAGES. Now contains a list of file-likes to search for modules.
> (raw-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
> to (flat-linux-extra-module-paths) along with the selected LINUX package.
> (base-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
> to (raw-initrd).

Nice, it looks like a welcome addition.

Toggle quote (13 lines)
> +If a module listed in @code{initrd-modules} is not included in the
> +Linux-libre kernel, then the location to it must be added to the
> +@code{initrd-extra-module-paths} list. For example, if your root file
> +system exists on a ZFS pool, then your configuration might look like the
> +following:
> +
> +@lisp
> +(operating-system
> + ;; @dots{}
> + (initrd-modules (cons "zfs" %base-initrd-modules))
> + (initrd-extra-module-paths (list (list zfs "module"))))
> +@end lisp

I wonder if we could reuse the ‘kernel-loadable-modules’ field for this
purpose instead of introducing a new field. We’d need to pass it to the
initrd procedures and have them search in there in addition to the
kernel package, pretty much like this patch already does actually.

WDYT?

Nitpick: the GNU convention is to use “path” to denote “search paths”,
and other “file”, “file name”, or similar. In this case, that’d be
“kernel module” or “Linux module”.

Thank you!

Ludo’.
B
B
Brian Cully wrote on 2 Jun 22:40 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
87a6aug4p2.fsf@ditto.jhoto.spork.org
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (9 lines)
> I wonder if we could reuse the ‘kernel-loadable-modules’ field
> for this
> purpose instead of introducing a new field. We’d need to pass
> it to the
> initrd procedures and have them search in there in addition to
> the
> kernel package, pretty much like this patch already does
> actually.

This sounds like it could be made to work as you suggest. My
feeling is that the two contexts are slightly different, though,
as the Linux modules are a superset of the initrd modules, so I’d
prefer not to mix them as it might be confusing to people who are
used to other Linux distros where the initrd modules are called
out separately. I admit I’m probably being silly here, and don’t
have any serious objection in principle.

Toggle quote (6 lines)
> Nitpick: the GNU convention is to use “path” to denote “search
> paths”,
> and other “file”, “file name”, or similar. In this case, that’d
> be
> “kernel module” or “Linux module”.

I struggled with this a fair amount, actually. What these
file-likes actually represent is an element of a search path, even
if they come in the odd form of a file-like object, which is why I
used ‘path’. ‘file’ seems wrong, as it implies (to me) that it’s
the ‘initrd-extra-module-files’ option itself that would include
the module, rather than the ‘initrd-modules’ option.

Of course, all this goes away if we just reuse
‘kernel-loadable-modules’ as an additional input, rather than
adding another option, so that’s a distinct mark in favor of doing
that.

When I get some time (hopefully soon!) I’ll try to thread
‘kernel-loadable-modules’ through instead and see how far I can
get with that approach. Do you think the documentation for it will
need to be updated to specify that it’s also used as a search path
for initrd building? Or maybe the better option is to update the
documentation for ‘initrd-modules’ to say that it uses
‘kernel-loadable-modules’ as input?

-bjc
K
K
Kaelyn wrote on 2 Jun 23:23 +0200
[PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’
(name . 55231@debbugs.gnu.org)(address . 55231@debbugs.gnu.org)
jMhPvkR0keO7tVdAcEW2mIuI_3iopqkDVl73kagJHUdr7sa5CdxbqMwQ2j2CqDQymRUDaSXCPO_8CIv0qSDhOyD37Nc5rsnRyrNUBrVgQM4=@protonmail.com
Hi,

My $0.02 on 'initrd-extra-module-paths' vs 'kernel-loadable-modules': if the initrd code is smart enough to not include the entire packages and only includes the requested modules from the packages (which I think is already true based on behavior observed some time ago), then not having to duplicate the list would be preferable.

I found this issue in the tracker after being surprised that the initrd builder ignored 'kernel-loadable-modules' with 'initrd-modules' only working for modules that are included in the primary linux kernel package. That surprise over the incompatibility between 'initrd-modules' and 'kernel-loadable-modules' is also why I've been relatively vocal about the patch. ;)

Cheers,
Kaelyn
L
L
Ludovic Courtès wrote on 3 Jun 09:27 +0200
(name . Brian Cully)(address . bjc@spork.org)
87fskmky0d.fsf@gnu.org
Hi,

Brian Cully <bjc@spork.org> skribis:

Toggle quote (13 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> I wonder if we could reuse the ‘kernel-loadable-modules’ field for
>> this
>> purpose instead of introducing a new field. We’d need to pass it to
>> the
>> initrd procedures and have them search in there in addition to the
>> kernel package, pretty much like this patch already does actually.
>
> This sounds like it could be made to work as you suggest. My feeling
> is that the two contexts are slightly different, though, as the Linux
> modules are a superset of the initrd modules,

Right. Here, we want the initrd machinery to search for modules in any
place where they can be found, which is either ‘kernel’ or
‘kernel-loadable-modules’.

One could define ‘initrd-extra-module-paths’ to be different from
‘kernel-loadable-modules’, for example if you can be sure the module
will be loaded from the initrd and not after, but in general both are
likely to have the same value, no?

Toggle quote (12 lines)
>> Nitpick: the GNU convention is to use “path” to denote “search
>> paths”,
>> and other “file”, “file name”, or similar. In this case, that’d be
>> “kernel module” or “Linux module”.
>
> I struggled with this a fair amount, actually. What these file-likes
> actually represent is an element of a search path, even if they come
> in the odd form of a file-like object, which is why I used
> ‘path’. ‘file’ seems wrong, as it implies (to me) that it’s the
> ‘initrd-extra-module-files’ option itself that would include the
> module, rather than the ‘initrd-modules’ option.

Hmm right, you have a point! :-)

Toggle quote (8 lines)
> When I get some time (hopefully soon!) I’ll try to thread
> ‘kernel-loadable-modules’ through instead and see how far I can get
> with that approach. Do you think the documentation for it will need to
> be updated to specify that it’s also used as a search path for initrd
> building? Or maybe the better option is to update the documentation
> for ‘initrd-modules’ to say that it uses ‘kernel-loadable-modules’ as
> input?

I think you should update the documentation in the commit that changes
things, so that the patch is self-contained.

It may be enough to state in the documentation of the ‘initrd-modules’
field that its value is a list of module names that are searched for in
‘kernel’ and ‘kernel-loadable-modules’.

WDYT?

Thanks,
Ludo’.
B
B
Brian Cully wrote on 17 Jun 03:51 +0200
[PATCH v2 1/2] Allows copying of out-of-tree modules to the Linux initrd.
(address . 55231@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
88d91cf303fd82c3667149ee8a647527f44bf571.1655430718.git.bjc@spork.org
With this patch, modules for ‘initrd-modules’ will not only be searched for in
the in-tree Linux modules, but also any additional modules specified in
‘kernel-loadable-modules’.

* gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to
DIRECTORIES. Now takes a list of directories to search, rather than a single
one.
* gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX
argument to PACKAGES. Now contains a list of file-likes to search for modules.
(raw-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
to (flat-linux-extra-module-paths) along with the selected LINUX package.
(base-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
to (raw-initrd).
* gnu/system.scm (operating-system-initrd-file): pass in operating system
definition's kernel-loadable-modules into (make-initrd) as
LINUX-EXTRA-MODULE-PATHS.
---

I've removed the new operating-system slot in preference to re-using
kernel-loadable-modules, as discussed.

I did leave the old argument names for the lower level routines in place,
as I feel that they more accurately represent how the data are being
used.

I've also pulled out the documentation. Once this patch set is beaten
into acceptability, the documentation can be adjusted if that's deemed
necessary.

gnu/build/linux-modules.scm | 19 ++++++++------
gnu/system.scm | 2 ++
gnu/system/linux-initrd.scm | 52 +++++++++++++++++++++++++------------
3 files changed, 48 insertions(+), 25 deletions(-)

Toggle diff (174 lines)
diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm
index 053720574b..97b7e429ea 100644
--- a/gnu/build/linux-modules.scm
+++ b/gnu/build/linux-modules.scm
@@ -225,8 +225,8 @@ (define (file-name->module-name file)
 '.ko[.gz|.xz]' and normalizing it."
   (normalize-module-name (strip-extension (basename file))))
 
-(define (find-module-file directory module)
-  "Lookup module NAME under DIRECTORY, and return its absolute file name.
+(define (find-module-file directories module)
+  "Lookup module NAME under DIRECTORIES, and return its absolute file name.
 NAME can be a file name with or without '.ko', or it can be a module name.
 Raise an error if it could not be found.
 
@@ -247,16 +247,19 @@ (define (find-module-file directory module)
                            (else chr)))
                        module))))
 
-  (match (find-files directory
-                     (lambda (file stat)
-                       (member (strip-extension
-                                (basename file)) names)))
+  (match (append-map (lambda (directory)
+                       (find-files directory
+                                   (lambda (file _stat)
+                                     (member (strip-extension
+                                              (basename file))
+                                             names))))
+                       directories)
     ((file)
      file)
     (()
-     (error "kernel module not found" module directory))
+     (error "kernel module not found" module directories))
     ((_ ...)
-     (error "several modules by that name" module directory))))
+     (error "several modules by that name" module directories))))
 
 (define* (recursive-module-dependencies files
                                         #:key (lookup-module dot-ko))
diff --git a/gnu/system.scm b/gnu/system.scm
index ba1b7b5152..6f52377c8d 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -1313,6 +1313,8 @@ (define (operating-system-initrd-file os)
                #:linux (operating-system-kernel os)
                #:linux-modules
                (operating-system-initrd-modules os)
+               #:linux-extra-module-paths
+               (operating-system-kernel-loadable-modules os)
                #:mapped-devices mapped-devices
                #:keyboard-layout (operating-system-keyboard-layout os)))
 
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 4c4c78e444..50a182d7d5 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -119,13 +119,23 @@ (define* (expression->initrd exp
                               `(#:references-graphs (("closure" ,init))))
                "/initrd.cpio.gz"))
 
-(define (flat-linux-module-directory linux modules)
+(define (flat-linux-module-directory packages modules)
   "Return a flat directory containing the Linux kernel modules listed in
-MODULES and taken from LINUX."
+MODULES and taken from PACKAGES."
   (define imported-modules
     (source-module-closure '((gnu build linux-modules)
                              (guix build utils))))
 
+  (define (package+out->input package out)
+    (gexp-input package out))
+
+  (define package-inputs
+    (map (lambda (p)
+           (match p
+             ((p o) (package+out->input p o))
+             (p     (package+out->input p "out"))))
+         packages))
+
   (define build-exp
     (with-imported-modules imported-modules
       (with-extensions (list guile-zlib)
@@ -135,11 +145,12 @@ (define (flat-linux-module-directory linux modules)
                          (srfi srfi-1)
                          (srfi srfi-26))
 
-            (define module-dir
-              (string-append #$linux "/lib/modules"))
+            (define module-dirs
+              (map (cut string-append <> "/lib/modules")
+                   '#$package-inputs))
 
             (define modules
-              (let* ((lookup  (cut find-module-file module-dir <>))
+              (let* ((lookup  (cut find-module-file module-dirs <>))
                      (modules (map lookup '#$modules)))
                 (append modules
                         (recursive-module-dependencies
@@ -172,20 +183,23 @@ (define* (raw-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-paths '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       (helper-packages '())
                       qemu-networking?
                       volatile-root?
                       (on-error 'debug))
-  "Return as a file-like object a raw initrd, with kernel
-modules taken from LINUX.  FILE-SYSTEMS is a list of file-systems to be
-mounted by the initrd, possibly in addition to the root file system specified
-on the kernel command line via 'root'.  LINUX-MODULES is a list of kernel
-modules to be loaded at boot time. MAPPED-DEVICES is a list of device
-mappings to realize before FILE-SYSTEMS are mounted.
-HELPER-PACKAGES is a list of packages to be copied in the initrd. It may include
-e2fsck/static or other packages needed by the initrd to check root partition.
+  "Return as a file-like object a raw initrd, with kernel modules taken from
+LINUX.  FILE-SYSTEMS is a list of file-systems to be mounted by the initrd,
+possibly in addition to the root file system specified on the kernel command
+line via 'root'.  LINUX-MODULES is a list of kernel modules to be loaded at
+boot time. LINUX-EXTRA-MODULE-PATHS is a list of file-like objects which will
+be searched for modules in addition to the linux kernel. MAPPED-DEVICES is a
+list of device mappings to realize before FILE-SYSTEMS are mounted.
+HELPER-PACKAGES is a list of packages to be copied in the initrd. It may
+include e2fsck/static or other packages needed by the initrd to check root
+partition.
 
 When true, KEYBOARD-LAYOUT is a <keyboard-layout> record denoting the desired
 console keyboard layout.  This is done before MAPPED-DEVICES are set up and
@@ -221,7 +235,8 @@ (define* (raw-initrd file-systems
           #~())))
 
   (define kodir
-    (flat-linux-module-directory linux linux-modules))
+    (flat-linux-module-directory (cons linux linux-extra-module-paths)
+                                 linux-modules))
 
   (expression->initrd
    (with-imported-modules (source-module-closure
@@ -366,6 +381,7 @@ (define* (base-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-paths '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       qemu-networking?
@@ -386,9 +402,10 @@ (define* (base-initrd file-systems
 QEMU-NETWORKING? and VOLATILE-ROOT? behaves as in raw-initrd.
 
 The initrd is automatically populated with all the kernel modules necessary
-for FILE-SYSTEMS and for the given options.  Additional kernel
-modules can be listed in LINUX-MODULES.  They will be added to the initrd, and
-loaded at boot time in the order in which they appear."
+for FILE-SYSTEMS and for the given options.  Additional kernel modules can be
+listed in LINUX-MODULES.  Additional search paths for modules can be listed in
+LINUX-EXTRA-MODULE-PATHS.  They will be added to the initrd, and loaded at
+boot time in the order in which they appear."
   (define linux-modules*
     ;; Modules added to the initrd and loaded from the initrd.
     `(,@linux-modules
@@ -408,6 +425,7 @@ (define* (base-initrd file-systems
   (raw-initrd file-systems
               #:linux linux
               #:linux-modules linux-modules*
+              #:linux-extra-module-paths linux-extra-module-paths
               #:mapped-devices mapped-devices
               #:helper-packages helper-packages
               #:keyboard-layout keyboard-layout
-- 
2.36.1
L
L
Ludovic Courtès wrote on 17 Jun 22:34 +0200
Re: bug#55231: [PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’
(name . Brian Cully)(address . bjc@spork.org)(address . 55231@debbugs.gnu.org)
878rpvdo5w.fsf_-_@gnu.org
Hi Brian,

Brian Cully <bjc@spork.org> skribis:

Toggle quote (29 lines)
> With this patch, modules for ‘initrd-modules’ will not only be searched for in
> the in-tree Linux modules, but also any additional modules specified in
> ‘kernel-loadable-modules’.
>
> * gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to
> DIRECTORIES. Now takes a list of directories to search, rather than a single
> one.
> * gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX
> argument to PACKAGES. Now contains a list of file-likes to search for modules.
> (raw-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
> to (flat-linux-extra-module-paths) along with the selected LINUX package.
> (base-initrd): Add LINUX-EXTRA-MODULE-PATHS keyword argument. Pass it
> to (raw-initrd).
> * gnu/system.scm (operating-system-initrd-file): pass in operating system
> definition's kernel-loadable-modules into (make-initrd) as
> LINUX-EXTRA-MODULE-PATHS.
> ---
>
> I've removed the new operating-system slot in preference to re-using
> kernel-loadable-modules, as discussed.
>
> I did leave the old argument names for the lower level routines in place,
> as I feel that they more accurately represent how the data are being
> used.
>
> I've also pulled out the documentation. Once this patch set is beaten
> into acceptability, the documentation can be adjusted if that's deemed
> necessary.

Alright!

It looks great to me. I have two cosmetic comments:

Toggle quote (3 lines)
> + (define (package+out->input package out)
> + (gexp-input package out))

I think you can remove this definition and use ‘gexp-input’ directly.

Toggle quote (6 lines)
> @@ -172,20 +183,23 @@ (define* (raw-initrd file-systems
> #:key
> (linux linux-libre)
> (linux-modules '())
> + (linux-extra-module-paths '())

Nitpick: I’d call it #:linux-extra-module-path (singular, as in “search
path”) or simply #:linux-extra-module-directories.

If you could introduce a couple of lines in doc/guix.texi to explain
where modules are searched for, that’d be perfect.

Thanks for taking the time to prepare this revision!

Ludo’.
B
B
Brian Cully wrote on 18 Jun 21:11 +0200
[PATCH v3 1/2] Allows copying of out-of-tree modules to the Linux initrd.
(address . 55231@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
10a6842d1a6bde797d82d6b8107660bedce8b956.1655579477.git.bjc@spork.org
With this patch, modules for ‘initrd-modules’ will not only be searched for in
the in-tree Linux modules, but also any additional modules specified in
‘kernel-loadable-modules’.

* gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to
DIRECTORIES. Now takes a list of directories to search, rather than a single
one.
* gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX
argument to PACKAGES. Now contains a list of file-likes to search for modules.
(raw-initrd): Add LINUX-EXTRA-MODULE-DIRECTORIES keyword argument. Pass it
to (flat-linux-module-directory) along with the selected LINUX package.
(base-initrd): Add LINUX-EXTRA-MODULE-DIRECTORIES keyword argument. Pass it
to (raw-initrd).
* gnu/system.scm (operating-system-initrd-file): pass in operating system
definition's kernel-loadable-modules into (make-initrd) as
LINUX-EXTRA-MODULE-DIRECTORIES.
---

This version removes the redundant (package+out->input) procedure, and
renames ‘linux-extra-module-paths’ to ‘linux-extra-module-directories’.

gnu/build/linux-modules.scm | 19 ++++++++------
gnu/system.scm | 2 ++
gnu/system/linux-initrd.scm | 49 ++++++++++++++++++++++++-------------
3 files changed, 45 insertions(+), 25 deletions(-)

Toggle diff (171 lines)
diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm
index 053720574b..97b7e429ea 100644
--- a/gnu/build/linux-modules.scm
+++ b/gnu/build/linux-modules.scm
@@ -225,8 +225,8 @@ (define (file-name->module-name file)
 '.ko[.gz|.xz]' and normalizing it."
   (normalize-module-name (strip-extension (basename file))))
 
-(define (find-module-file directory module)
-  "Lookup module NAME under DIRECTORY, and return its absolute file name.
+(define (find-module-file directories module)
+  "Lookup module NAME under DIRECTORIES, and return its absolute file name.
 NAME can be a file name with or without '.ko', or it can be a module name.
 Raise an error if it could not be found.
 
@@ -247,16 +247,19 @@ (define (find-module-file directory module)
                            (else chr)))
                        module))))
 
-  (match (find-files directory
-                     (lambda (file stat)
-                       (member (strip-extension
-                                (basename file)) names)))
+  (match (append-map (lambda (directory)
+                       (find-files directory
+                                   (lambda (file _stat)
+                                     (member (strip-extension
+                                              (basename file))
+                                             names))))
+                       directories)
     ((file)
      file)
     (()
-     (error "kernel module not found" module directory))
+     (error "kernel module not found" module directories))
     ((_ ...)
-     (error "several modules by that name" module directory))))
+     (error "several modules by that name" module directories))))
 
 (define* (recursive-module-dependencies files
                                         #:key (lookup-module dot-ko))
diff --git a/gnu/system.scm b/gnu/system.scm
index ba1b7b5152..2439560671 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -1313,6 +1313,8 @@ (define (operating-system-initrd-file os)
                #:linux (operating-system-kernel os)
                #:linux-modules
                (operating-system-initrd-modules os)
+               #:linux-extra-module-directories
+               (operating-system-kernel-loadable-modules os)
                #:mapped-devices mapped-devices
                #:keyboard-layout (operating-system-keyboard-layout os)))
 
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 4c4c78e444..f6e8f75efa 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -119,13 +119,20 @@ (define* (expression->initrd exp
                               `(#:references-graphs (("closure" ,init))))
                "/initrd.cpio.gz"))
 
-(define (flat-linux-module-directory linux modules)
+(define (flat-linux-module-directory packages modules)
   "Return a flat directory containing the Linux kernel modules listed in
-MODULES and taken from LINUX."
+MODULES and taken from PACKAGES."
   (define imported-modules
     (source-module-closure '((gnu build linux-modules)
                              (guix build utils))))
 
+  (define package-inputs
+    (map (lambda (p)
+           (match p
+             ((p o) (gexp-input p o))
+             (p     (gexp-input p "out"))))
+         packages))
+
   (define build-exp
     (with-imported-modules imported-modules
       (with-extensions (list guile-zlib)
@@ -135,11 +142,12 @@ (define (flat-linux-module-directory linux modules)
                          (srfi srfi-1)
                          (srfi srfi-26))
 
-            (define module-dir
-              (string-append #$linux "/lib/modules"))
+            (define module-dirs
+              (map (cut string-append <> "/lib/modules")
+                   '#$package-inputs))
 
             (define modules
-              (let* ((lookup  (cut find-module-file module-dir <>))
+              (let* ((lookup  (cut find-module-file module-dirs <>))
                      (modules (map lookup '#$modules)))
                 (append modules
                         (recursive-module-dependencies
@@ -172,20 +180,23 @@ (define* (raw-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-directories '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       (helper-packages '())
                       qemu-networking?
                       volatile-root?
                       (on-error 'debug))
-  "Return as a file-like object a raw initrd, with kernel
-modules taken from LINUX.  FILE-SYSTEMS is a list of file-systems to be
-mounted by the initrd, possibly in addition to the root file system specified
-on the kernel command line via 'root'.  LINUX-MODULES is a list of kernel
-modules to be loaded at boot time. MAPPED-DEVICES is a list of device
-mappings to realize before FILE-SYSTEMS are mounted.
-HELPER-PACKAGES is a list of packages to be copied in the initrd. It may include
-e2fsck/static or other packages needed by the initrd to check root partition.
+  "Return as a file-like object a raw initrd, with kernel modules taken from
+LINUX.  FILE-SYSTEMS is a list of file-systems to be mounted by the initrd,
+possibly in addition to the root file system specified on the kernel command
+line via 'root'.  LINUX-MODULES is a list of kernel modules to be loaded at
+boot time. LINUX-EXTRA-MODULE-DIRECTORIES is a list of file-like objects which
+will be searched for modules in addition to the linux kernel. MAPPED-DEVICES
+is a list of device mappings to realize before FILE-SYSTEMS are mounted.
+HELPER-PACKAGES is a list of packages to be copied in the initrd. It may
+include e2fsck/static or other packages needed by the initrd to check root
+partition.
 
 When true, KEYBOARD-LAYOUT is a <keyboard-layout> record denoting the desired
 console keyboard layout.  This is done before MAPPED-DEVICES are set up and
@@ -221,7 +232,8 @@ (define* (raw-initrd file-systems
           #~())))
 
   (define kodir
-    (flat-linux-module-directory linux linux-modules))
+    (flat-linux-module-directory (cons linux linux-extra-module-directories)
+                                 linux-modules))
 
   (expression->initrd
    (with-imported-modules (source-module-closure
@@ -366,6 +378,7 @@ (define* (base-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-directories '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       qemu-networking?
@@ -386,9 +399,10 @@ (define* (base-initrd file-systems
 QEMU-NETWORKING? and VOLATILE-ROOT? behaves as in raw-initrd.
 
 The initrd is automatically populated with all the kernel modules necessary
-for FILE-SYSTEMS and for the given options.  Additional kernel
-modules can be listed in LINUX-MODULES.  They will be added to the initrd, and
-loaded at boot time in the order in which they appear."
+for FILE-SYSTEMS and for the given options.  Additional kernel modules can be
+listed in LINUX-MODULES.  Additional directories for modules can be listed in
+LINUX-EXTRA-MODULE-DIRECTORIES.  They will be added to the initrd, and loaded
+at boot time in the order in which they appear."
   (define linux-modules*
     ;; Modules added to the initrd and loaded from the initrd.
     `(,@linux-modules
@@ -408,6 +422,7 @@ (define* (base-initrd file-systems
   (raw-initrd file-systems
               #:linux linux
               #:linux-modules linux-modules*
+              #:linux-extra-module-directories linux-extra-module-directories
               #:mapped-devices mapped-devices
               #:helper-packages helper-packages
               #:keyboard-layout keyboard-layout
-- 
2.36.1
B
B
Brian Cully wrote on 18 Jun 21:11 +0200
[PATCH v3 2/2] doc: ‘initrd-modules’ will search ‘kernel-loadable-modules’.
(address . 55231@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
65cf1060a7ce60b1b91a25f809af6264abdcfa59.1655579477.git.bjc@spork.org
---
doc/guix.texi | 13 +++++++++++++
1 file changed, 13 insertions(+)

Toggle diff (26 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index eda0956260..d97909695f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -36461,6 +36461,19 @@ Initial RAM Disk
   (initrd-modules (cons "megaraid_sas" %base-initrd-modules)))
 @end lisp
 
+If a module listed in @code{initrd-modules} is not included in the
+Linux-libre kernel, then the location to it must be added to the
+@code{kernel-loadable-modules} list.  For example, if your root file
+system exists on a ZFS pool, then your configuration might look like the
+following:
+
+@lisp
+(operating-system
+  ;; @dots{}
+  (initrd-modules (cons "zfs" %base-initrd-modules))
+  (kernel-loadable-modules (list (list zfs "module"))))
+@end lisp
+
 @defvr {Scheme Variable} %base-initrd-modules
 This is the list of kernel modules included in the initrd by default.
 @end defvr
-- 
2.36.1
M
M
Maxime Devos wrote on 18 Jun 22:34 +0200
Re: [bug#55231] [PATCH v3 2/2] doc: ‘initrd-modules’ will search ‘kernel-loadable-modules’.
d9e2175bb610845bc2ab662eaff69d439e03f1a1.camel@telenet.be
Brian Cully via Guix-patches via schreef op za 18-06-2022 om 15:11 [-
0400]:
Toggle quote (13 lines)
> +If a module listed in @code{initrd-modules} is not included in the
> +Linux-libre kernel, then the location to it must be added to the
> +@code{kernel-loadable-modules} list.  For example, if your root file
> +system exists on a ZFS pool, then your configuration might look like the
> +following:
> +
> +@lisp
> +(operating-system
> +  ;; @dots{}
> +  (initrd-modules (cons "zfs" %base-initrd-modules))
> +  (kernel-loadable-modules (list (list zfs "module"))))
> +@end lisp

As written previously, this is not a good example, because:

Toggle quote (15 lines)
> As-is, I don't think this is a good example, because
> 'expression->initrd' does not set #:substitutable? #false,
> the 'zfs' package has the comment (*)
>
> `(;; The ZFS kernel module should not be downloaded since the
> license
> ;; terms don't allow for distributing it, only building it
> locally.
> #:substitutable? #f [...])
>
> and IIUC, the code inside expression->initrd copies the kernel module
> into the new store item, so it looks like this accidentally suggest
> people to commit copyvios, and copyvios are currently against the
> law.

and because the defense for not considering the ZFS license to be a
problem consists of the ZFS module not being distributed in binary
form, whereas this suggestion would in some situations cause it to be
distributed in binary form.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYq427RccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7vuRAP93e+IazsNTgVuVDhMRelp9dHd9
9osZoFuZBA8dInijdAEAscxvbiaqTecbAuzG+bVfZ7Vf5XaiqqzlGlCl7lHS7wM=
=gS+J
-----END PGP SIGNATURE-----


B
B
Brian Cully wrote on 18 Jun 22:43 +0200
Re: [bug#55231] [PATCH v3 2/2] doc: ‘initrd-mod ules’ will search ‘kernel-loadable-modules’.
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 55231@debbugs.gnu.org)
87czf566hu.fsf@ditto.jhoto.spork.org
Toggle quote (8 lines)
> and because the defense for not considering the ZFS license to
> be a
> problem consists of the ZFS module not being distributed in
> binary
> form, whereas this suggestion would in some situations cause it
> to be
> distributed in binary form.

If you have another example, I'll put it in instead. I used this
one, because I know this one works, and that's all.

That said, I'm not sure how this would cause the module to be
distributed as a binary. In order for it to be added to the
initrd, it will still need to built using the DKMS mechanism, and
thus compiled on (or at least for) the target Guix installation.

If the complaint is that one could generate a USB stick, or some
such, with ZFS in the initrd, then yes, that's possible. But
that's also possible by using the existing
‘kernel-loadable-modules’ mechanism to generate an image with
‘guix system image’ and distributing that. I don't think it's our
job to try and prevent such things, since, even if desirable, it's
not really feasible.

If the complaint is merely that it's in the documentation, then
ok, I'll change it to whatever module you want.

-bjc
M
M
Maxime Devos wrote on 19 Jun 00:34 +0200
Re: [bug#55231] [PATCH v3 2/2] doc: ‘initrd-modules’ will search ‘kernel-loadable-modules’.
(name . Brian Cully)(address . bjc@spork.org)(address . 55231@debbugs.gnu.org)
45da2e782470b7bc69fd3014a8c9ced154a58e49.camel@telenet.be
Brian Cully schreef op za 18-06-2022 om 16:43 [-0400]:
Toggle quote (3 lines)
> That said, I'm not sure how this would cause the module to be
> distributed as a binary. [...]

Because #:substitutable? isn't set appropriately in dependents,
substitute servers exist, ZFS can be used on substitute servers and
someone using the substitute server might have a sufficiently similar
system configuration to do substitution of a store item containing a
copy of the ZFS binary.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYq5TBRccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hp7AQCihs5fFUXl54bTaZgaSYFETXaw
UNZbSrcVKTEp4xPvpAD/T1Xx/Un/AzQKpEh3zZvofBsPncHBsb9ASaA6PfhkNAw=
=POZ3
-----END PGP SIGNATURE-----


B
B
Brian Cully wrote on 19 Jun 01:11 +0200
Re: [bug#55231] [PATCH v3 2/2] doc: ‘initrd-mod ules’ will search ‘kernel-loadable-modules’.
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 55231@debbugs.gnu.org)
878rpt5zmq.fsf@ditto.jhoto.spork.org
Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (9 lines)
> Because #:substitutable? isn't set appropriately in dependents,
> substitute servers exist, ZFS can be used on substitute servers
> and
> someone using the substitute server might have a sufficiently
> similar
> system configuration to do substitution of a store item
> containing a
> copy of the ZFS binary.

So, simply using ZFS in ‘kernel-loadable-modules’ would be enough
to trigger this misbehavior? That sounds like a pretty serious
issue. Would it be possible to have the substitute servers filter
on the #:substitutable flag?

This is getting out of scope, though. I don't really want this
patch to go in without /some/ documentation regarding how modules
get looked up, so if anyone has an alternate module and use-case,
I'll just swap it in for the ZFS one.

-bjc
M
M
Maxime Devos wrote on 19 Jun 14:05 +0200
Re: [bug#55231] [PATCH v3 2/2] doc: ‘initrd-modules’ will search ‘kernel-loadable-modules’.
(name . Brian Cully)(address . bjc@spork.org)(address . 55231@debbugs.gnu.org)
23328359aaef00b461b27e46b5e716655a8ff015.camel@telenet.be
Brian Cully schreef op za 18-06-2022 om 19:11 [-0400]:
Toggle quote (5 lines)
> So, simply using ZFS in ‘kernel-loadable-modules’ would be enough
> to trigger this misbehavior? That sounds like a pretty serious
> issue. Would it be possible to have the substitute servers filter
> on the #:substitutable flag?

That's exactly what the #:substitutable flag is for (IIUC)!
However, the problem is that currently, that flag isn't set for
derivations that make a copy of the ZFS module. As I wrote previously:

Toggle quote (8 lines)
> > and IIUC, the code inside expression->initrd copies the kernel
> > module into the new store item,

> This is getting out of scope, though. I don't really want this
> patch to go in without /some/ documentation regarding how modules
> get looked up, so if anyone has an alternate module and use-case,
> I'll just swap it in for the ZFS one.

I don't have any.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYq8RHhccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7ryiAQCxJDyPWK8uOhO8EAe6QkxQ4VVJ
5x6qwjNw/pYnZctswQD/TqLj9GzZ0VfrT1S4sot56loPrdZ0aBiehxkpKBhOBw8=
=a5R3
-----END PGP SIGNATURE-----


K
K
Kaelyn wrote on 21 Jun 14:34 +0200
Re: [bug#55231][PATCH v1] initrd: Allow extra search paths with ‘initrd-extra-module-paths’
(name . 55231@debbugs.gnu.org)(address . 55231@debbugs.gnu.org)
62rRCnWUAiOvJIODcOCnx0Vw7EUYR3YHxLrkEfMBUHjURBOWn_JfZf76LODMt2W8syVn0hnIaeU4ysQ-kfi5-4X3wyM7oGTnOO4t2JPwRzY=@protonmail.com
Hi again,

Regarding the documentation example, one (contrived) alternative is:

"""
For example, if you need the driver for a Realtek RTL8821CE wireless network adapter for mounting the root filesystem over NFS, your configuration might include the following:

@lisp
(operating-system
;; @dots{}
(initrd-modules (cons "8821ce" %base-initrd-modules))
(kernel-loadable-modules (list (list rtl8821ce-linux-module "module"))))
@end lisp
"""

While I don't have the hardware, I did verify the kernel module name by building the rtl8821ce-linux-module package.

Cheers,
Kaelyn
B
B
Brian Cully wrote on 24 Jun 15:28 +0200
[PATCH v4 1/2] Allows copying of out-of-tree modules to the Linux initrd.
(address . 55231@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
10a6842d1a6bde797d82d6b8107660bedce8b956.1656077306.git.bjc@spork.org
With this patch, modules for ‘initrd-modules’ will not only be searched for in
the in-tree Linux modules, but also any additional modules specified in
‘kernel-loadable-modules’.

* gnu/build/linux-modules.scm (find-module-file): change DIRECTORY argument to
DIRECTORIES. Now takes a list of directories to search, rather than a single
one.
* gnu/system/linux-initrd.scm (flat-linux-module-directory): change LINUX
argument to PACKAGES. Now contains a list of file-likes to search for modules.
(raw-initrd): Add LINUX-EXTRA-MODULE-DIRECTORIES keyword argument. Pass it
to (flat-linux-module-directory) along with the selected LINUX package.
(base-initrd): Add LINUX-EXTRA-MODULE-DIRECTORIES keyword argument. Pass it
to (raw-initrd).
* gnu/system.scm (operating-system-initrd-file): pass in operating system
definition's kernel-loadable-modules into (make-initrd) as
LINUX-EXTRA-MODULE-DIRECTORIES.
---
gnu/build/linux-modules.scm | 19 ++++++++------
gnu/system.scm | 2 ++
gnu/system/linux-initrd.scm | 49 ++++++++++++++++++++++++-------------
3 files changed, 45 insertions(+), 25 deletions(-)

Toggle diff (171 lines)
diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm
index 053720574b..97b7e429ea 100644
--- a/gnu/build/linux-modules.scm
+++ b/gnu/build/linux-modules.scm
@@ -225,8 +225,8 @@ (define (file-name->module-name file)
 '.ko[.gz|.xz]' and normalizing it."
   (normalize-module-name (strip-extension (basename file))))
 
-(define (find-module-file directory module)
-  "Lookup module NAME under DIRECTORY, and return its absolute file name.
+(define (find-module-file directories module)
+  "Lookup module NAME under DIRECTORIES, and return its absolute file name.
 NAME can be a file name with or without '.ko', or it can be a module name.
 Raise an error if it could not be found.
 
@@ -247,16 +247,19 @@ (define (find-module-file directory module)
                            (else chr)))
                        module))))
 
-  (match (find-files directory
-                     (lambda (file stat)
-                       (member (strip-extension
-                                (basename file)) names)))
+  (match (append-map (lambda (directory)
+                       (find-files directory
+                                   (lambda (file _stat)
+                                     (member (strip-extension
+                                              (basename file))
+                                             names))))
+                       directories)
     ((file)
      file)
     (()
-     (error "kernel module not found" module directory))
+     (error "kernel module not found" module directories))
     ((_ ...)
-     (error "several modules by that name" module directory))))
+     (error "several modules by that name" module directories))))
 
 (define* (recursive-module-dependencies files
                                         #:key (lookup-module dot-ko))
diff --git a/gnu/system.scm b/gnu/system.scm
index ba1b7b5152..2439560671 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -1313,6 +1313,8 @@ (define (operating-system-initrd-file os)
                #:linux (operating-system-kernel os)
                #:linux-modules
                (operating-system-initrd-modules os)
+               #:linux-extra-module-directories
+               (operating-system-kernel-loadable-modules os)
                #:mapped-devices mapped-devices
                #:keyboard-layout (operating-system-keyboard-layout os)))
 
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 4c4c78e444..f6e8f75efa 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -119,13 +119,20 @@ (define* (expression->initrd exp
                               `(#:references-graphs (("closure" ,init))))
                "/initrd.cpio.gz"))
 
-(define (flat-linux-module-directory linux modules)
+(define (flat-linux-module-directory packages modules)
   "Return a flat directory containing the Linux kernel modules listed in
-MODULES and taken from LINUX."
+MODULES and taken from PACKAGES."
   (define imported-modules
     (source-module-closure '((gnu build linux-modules)
                              (guix build utils))))
 
+  (define package-inputs
+    (map (lambda (p)
+           (match p
+             ((p o) (gexp-input p o))
+             (p     (gexp-input p "out"))))
+         packages))
+
   (define build-exp
     (with-imported-modules imported-modules
       (with-extensions (list guile-zlib)
@@ -135,11 +142,12 @@ (define (flat-linux-module-directory linux modules)
                          (srfi srfi-1)
                          (srfi srfi-26))
 
-            (define module-dir
-              (string-append #$linux "/lib/modules"))
+            (define module-dirs
+              (map (cut string-append <> "/lib/modules")
+                   '#$package-inputs))
 
             (define modules
-              (let* ((lookup  (cut find-module-file module-dir <>))
+              (let* ((lookup  (cut find-module-file module-dirs <>))
                      (modules (map lookup '#$modules)))
                 (append modules
                         (recursive-module-dependencies
@@ -172,20 +180,23 @@ (define* (raw-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-directories '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       (helper-packages '())
                       qemu-networking?
                       volatile-root?
                       (on-error 'debug))
-  "Return as a file-like object a raw initrd, with kernel
-modules taken from LINUX.  FILE-SYSTEMS is a list of file-systems to be
-mounted by the initrd, possibly in addition to the root file system specified
-on the kernel command line via 'root'.  LINUX-MODULES is a list of kernel
-modules to be loaded at boot time. MAPPED-DEVICES is a list of device
-mappings to realize before FILE-SYSTEMS are mounted.
-HELPER-PACKAGES is a list of packages to be copied in the initrd. It may include
-e2fsck/static or other packages needed by the initrd to check root partition.
+  "Return as a file-like object a raw initrd, with kernel modules taken from
+LINUX.  FILE-SYSTEMS is a list of file-systems to be mounted by the initrd,
+possibly in addition to the root file system specified on the kernel command
+line via 'root'.  LINUX-MODULES is a list of kernel modules to be loaded at
+boot time. LINUX-EXTRA-MODULE-DIRECTORIES is a list of file-like objects which
+will be searched for modules in addition to the linux kernel. MAPPED-DEVICES
+is a list of device mappings to realize before FILE-SYSTEMS are mounted.
+HELPER-PACKAGES is a list of packages to be copied in the initrd. It may
+include e2fsck/static or other packages needed by the initrd to check root
+partition.
 
 When true, KEYBOARD-LAYOUT is a <keyboard-layout> record denoting the desired
 console keyboard layout.  This is done before MAPPED-DEVICES are set up and
@@ -221,7 +232,8 @@ (define* (raw-initrd file-systems
           #~())))
 
   (define kodir
-    (flat-linux-module-directory linux linux-modules))
+    (flat-linux-module-directory (cons linux linux-extra-module-directories)
+                                 linux-modules))
 
   (expression->initrd
    (with-imported-modules (source-module-closure
@@ -366,6 +378,7 @@ (define* (base-initrd file-systems
                       #:key
                       (linux linux-libre)
                       (linux-modules '())
+                      (linux-extra-module-directories '())
                       (mapped-devices '())
                       (keyboard-layout #f)
                       qemu-networking?
@@ -386,9 +399,10 @@ (define* (base-initrd file-systems
 QEMU-NETWORKING? and VOLATILE-ROOT? behaves as in raw-initrd.
 
 The initrd is automatically populated with all the kernel modules necessary
-for FILE-SYSTEMS and for the given options.  Additional kernel
-modules can be listed in LINUX-MODULES.  They will be added to the initrd, and
-loaded at boot time in the order in which they appear."
+for FILE-SYSTEMS and for the given options.  Additional kernel modules can be
+listed in LINUX-MODULES.  Additional directories for modules can be listed in
+LINUX-EXTRA-MODULE-DIRECTORIES.  They will be added to the initrd, and loaded
+at boot time in the order in which they appear."
   (define linux-modules*
     ;; Modules added to the initrd and loaded from the initrd.
     `(,@linux-modules
@@ -408,6 +422,7 @@ (define* (base-initrd file-systems
   (raw-initrd file-systems
               #:linux linux
               #:linux-modules linux-modules*
+              #:linux-extra-module-directories linux-extra-module-directories
               #:mapped-devices mapped-devices
               #:helper-packages helper-packages
               #:keyboard-layout keyboard-layout
-- 
2.36.1
B
B
Brian Cully wrote on 24 Jun 15:28 +0200
[PATCH v4 2/2] doc: ‘initrd-modules’ will search ‘kernel-loadable-modules’.
(address . 55231@debbugs.gnu.org)(name . Brian Cully)(address . bjc@spork.org)
decfa9ca4ff55b14673e7b1013ceac4cb3bf9371.1656077306.git.bjc@spork.org
---
doc/guix.texi | 15 +++++++++++++++
1 file changed, 15 insertions(+)

Toggle diff (28 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index eda0956260..7c4682a76d 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -36461,6 +36461,21 @@ Initial RAM Disk
   (initrd-modules (cons "megaraid_sas" %base-initrd-modules)))
 @end lisp
 
+If a module listed in @code{initrd-modules} is not included in the
+Linux-libre kernel, then the location to it must be added to the
+@code{kernel-loadable-modules} list.
+
+For example, if you need the driver for a Realtek RTL8821CE wireless
+network adapter for mounting the root filesystem over NFS, your
+configuration might include the following:
+
+@lisp
+(operating-system
+  ;; @dots{}
+  (initrd-modules (cons "8821ce" %base-initrd-modules))
+  (kernel-loadable-modules (list (list rtl8821ce-linux-module "module"))))
+@end lisp
+
 @defvr {Scheme Variable} %base-initrd-modules
 This is the list of kernel modules included in the initrd by default.
 @end defvr
-- 
2.36.1
?