[PATCH 0/8] image: Add MBR based boot support.

  • Done
  • quality assurance status badge
Details
4 participants
  • Jan Nieuwenhuizen
  • Ludovic Courtès
  • Mathieu Othacehe
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Mathieu Othacehe
Severity
normal
M
M
Mathieu Othacehe wrote on 27 May 2020 09:22
(address . guix-patches@gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200527072219.25576-1-othacehe@gnu.org
Hello,

Until now, only Grub EFI bootloader was supported in (gnu system
image). Installing Grub EFI is as simple as copying a binary (created with
grub-mkstandalone) in a dedicated partition.

Now, for MBR based booting, one needs to run grub-install that does require
root permissions. To overcome this issue, I used a hack inspired from OpenWrt
and Buildroot.

As grub-install is in fact a wrapper around grub-mkimage and grub-bios-setup,
it is possible, with some plumbing and using those two tools, to install Grub
on a raw disk-image, without root permissions.

Thanks,

Mathieu

Mathieu Othacehe (8):
bootloader: Add 'disk-image-installer'.
bootloader: grub: Do not run grub-install when creating a disk-image.
bootloader: grub: Use inheritance to define grub-minimal-bootloader.
image: Add bootloader installation support.
system: image: Correct genimage configuration file indentation.
image: Use grub-efi to install the EFI bootloader.
system: image: Fix image-with-os.
image: Do not use VM to create disk-images.

gnu/bootloader.scm | 5 +-
gnu/bootloader/grub.scm | 80 ++++++++++++++++++++++------
gnu/build/image.scm | 9 +++-
gnu/ci.scm | 20 +++----
gnu/system/image.scm | 113 ++++++++++++++++++----------------------
gnu/tests/install.scm | 8 +--
guix/scripts/system.scm | 16 +++---
guix/utils.scm | 5 ++
8 files changed, 155 insertions(+), 101 deletions(-)

--
2.26.2
M
M
Mathieu Othacehe wrote on 27 May 2020 09:24
[PATCH 2/8] bootloader: grub: Do not run grub-install when creating a disk-image.
(address . 41560@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200527072420.26140-2-othacehe@gnu.org
* gnu/bootloader/grub.scm (install-grub): When creating a disk-image,
grub-install will fail because it lacks root permissions. In that case, do not
run grub-install and only copy Grub modules to the /boot directory.
---
gnu/bootloader/grub.scm | 26 ++++++++++++++++----------
1 file changed, 16 insertions(+), 10 deletions(-)

Toggle diff (41 lines)
diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
index 74dc00480f..29bec92196 100644
--- a/gnu/bootloader/grub.scm
+++ b/gnu/bootloader/grub.scm
@@ -423,18 +423,24 @@ fi~%"))))
(define install-grub
#~(lambda (bootloader device mount-point)
- ;; Install GRUB on DEVICE which is mounted at MOUNT-POINT.
(let ((grub (string-append bootloader "/sbin/grub-install"))
(install-dir (string-append mount-point "/boot")))
- ;; Tell 'grub-install' that there might be a LUKS-encrypted /boot or
- ;; root partition.
- (setenv "GRUB_ENABLE_CRYPTODISK" "y")
-
- ;; Hide potentially confusing messages from the user, such as
- ;; "Installing for i386-pc platform."
- (invoke/quiet grub "--no-floppy" "--target=i386-pc"
- "--boot-directory" install-dir
- device))))
+ ;; Install GRUB on DEVICE which is mounted at MOUNT-POINT. If device
+ ;; is #f we are creating a disk-image.
+ (if device
+ (begin
+ ;; Tell 'grub-install' that there might be a LUKS-encrypted
+ ;; /boot or root partition.
+ (setenv "GRUB_ENABLE_CRYPTODISK" "y")
+
+ ;; Hide potentially confusing messages from the user, such as
+ ;; "Installing for i386-pc platform."
+ (invoke/quiet grub "--no-floppy" "--target=i386-pc"
+ "--boot-directory" install-dir
+ device))
+ ;; When creating a disk-image, only install Grub modules.
+ (copy-recursively (string-append bootloader "/lib/")
+ install-dir)))))
(define install-grub-disk-image
#~(lambda (bootloader root-index image)
--
2.26.2
M
M
Mathieu Othacehe wrote on 27 May 2020 09:24
[PATCH 1/8] bootloader: Add 'disk-image-installer'.
(address . 41560@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200527072420.26140-1-othacehe@gnu.org
* gnu/bootloader.scm (<bootloader>)[disk-image-installer]: New field,
(bootloader-disk-image-installer): export it.
* gnu/bootloader/grub.scm (install-grub-disk-image): New procedure ...
(grub-bootloader): ... used as "disk-image-installer" here.
(grub-efi-bootloader): set "disk-image-installer" to #f.
* gnu/system/image.scm (root-partition?, find-root-partition): Move to
"Helpers" section.
(root-partition-index): New procedure.
(system-disk-image): Honor disk-image-installer, and
use it to install the bootloader directly on the disk-image, if supported.
---
gnu/bootloader.scm | 5 ++++-
gnu/bootloader/grub.scm | 45 ++++++++++++++++++++++++++++++++++++++++-
gnu/system/image.scm | 33 +++++++++++++++++++++---------
3 files changed, 71 insertions(+), 12 deletions(-)

Toggle diff (167 lines)
diff --git a/gnu/bootloader.scm b/gnu/bootloader.scm
index 01bdd4acaa..668caa7fc3 100644
--- a/gnu/bootloader.scm
+++ b/gnu/bootloader.scm
@@ -1,6 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2017 David Craven <david@craven.ch>
-;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
+;;; Copyright © 2017, 2020 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2017 Leo Famulari <leo@famulari.name>
;;; Copyright © 2019 Ludovic Courtès <ludo@gnu.org>
;;;
@@ -42,6 +42,7 @@
bootloader-name
bootloader-package
bootloader-installer
+ bootloader-disk-image-installer
bootloader-configuration-file
bootloader-configuration-file-generator
@@ -125,6 +126,8 @@ record."
(name bootloader-name)
(package bootloader-package)
(installer bootloader-installer)
+ (disk-image-installer bootloader-disk-image-installer
+ (default #f))
(configuration-file bootloader-configuration-file)
(configuration-file-generator bootloader-configuration-file-generator))
diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
index bb40c551a7..74dc00480f 100644
--- a/gnu/bootloader/grub.scm
+++ b/gnu/bootloader/grub.scm
@@ -2,7 +2,7 @@
;;; Copyright © 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com>
;;; Copyright © 2017 Leo Famulari <leo@famulari.name>
-;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
+;;; Copyright © 2017, 2020 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2019, 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;;
@@ -436,6 +436,47 @@ fi~%"))))
"--boot-directory" install-dir
device))))
+(define install-grub-disk-image
+ #~(lambda (bootloader root-index image)
+ ;; Install GRUB on the given IMAGE. The root partition index is
+ ;; ROOT-INDEX.
+ (let ((grub-mkimage
+ (string-append bootloader "/bin/grub-mkimage"))
+ (modules '("biosdisk" "part_msdos" "fat" "ext2"))
+ (grub-bios-setup
+ (string-append bootloader "/sbin/grub-bios-setup"))
+ (root-device (format #f "hd0,msdos~a" root-index))
+ (boot-img (string-append bootloader "/lib/grub/i386-pc/boot.img"))
+ (device-map "device.map"))
+
+ ;; Create a minimal, standalone Grub image that will be written
+ ;; directly in the MBR-GAP (space between the end of the MBR and the
+ ;; first partition).
+ (apply invoke grub-mkimage
+ "-O" "i386-pc"
+ "-o" "core.img"
+ "-p" (format #f "(~a)/boot/grub" root-device)
+ modules)
+
+ ;; Create a device mapping file.
+ (call-with-output-file device-map
+ (lambda (port)
+ (format port "(hd0) ~a~%" image)))
+
+ ;; Copy the default boot.img, that will be written on the MBR sector
+ ;; by GRUB-BIOS-SETUP.
+ (copy-file boot-img "boot.img")
+
+ ;; Install both the "boot.img" and the "core.img" files on the given
+ ;; IMAGE. On boot, the MBR sector will execute the minimal Grub
+ ;; written in the MBR-GAP. Grub configuration and missing modules will
+ ;; be read from ROOT-DEVICE.
+ (invoke grub-bios-setup
+ "-m" device-map
+ "-r" root-device
+ "-d" "."
+ image))))
+
(define install-grub-efi
#~(lambda (bootloader efi-dir mount-point)
;; Install GRUB onto the EFI partition mounted at EFI-DIR, for the
@@ -465,6 +506,7 @@ fi~%"))))
(name 'grub)
(package grub)
(installer install-grub)
+ (disk-image-installer install-grub-disk-image)
(configuration-file "/boot/grub/grub.cfg")
(configuration-file-generator grub-configuration-file)))
@@ -480,6 +522,7 @@ fi~%"))))
(bootloader
(inherit grub-bootloader)
(installer install-grub-efi)
+ (disk-image-installer #f)
(name 'grub-efi)
(package grub-efi)))
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index a1214dd20a..ffc746fcf5 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -147,6 +147,19 @@
(guix build utils))
gexp* ...))))
+(define (root-partition? partition)
+ "Return true if PARTITION is the root partition, false otherwise."
+ (member 'boot (partition-flags partition)))
+
+(define (find-root-partition image)
+ "Return the root partition of the given IMAGE."
+ (srfi-1:find root-partition? (image-partitions image)))
+
+(define (root-partition-index image)
+ "Return the index of the root partition of the given IMAGE."
+ (1+ (srfi-1:list-index identity
+ (map root-partition? (image-partitions image)))))
+
;;
;; Disk image.
@@ -276,9 +289,17 @@ image ~a {
(let* ((substitutable? (image-substitutable? image))
(builder
(with-imported-modules*
- (let ((inputs '#+(list genimage coreutils findutils)))
+ (let ((inputs '#+(list genimage coreutils findutils))
+ (bootloader-installer
+ #+(bootloader-disk-image-installer bootloader)))
(set-path-environment-variable "PATH" '("bin" "sbin") inputs)
- (genimage #$(image->genimage-cfg image) #$output))))
+ (genimage #$(image->genimage-cfg image) #$output)
+ ;; Install the bootloader directly on the disk-image.
+ (when bootloader-installer
+ (bootloader-installer
+ #+(bootloader-package bootloader)
+ #$(root-partition-index image)
+ (string-append #$output "/" #$genimage-name))))))
(image-dir (computed-file "image-dir" builder)))
(computed-file name
#~(symlink
@@ -371,14 +392,6 @@ used in the image. "
;; Image creation.
;;
-(define (root-partition? partition)
- "Return true if PARTITION is the root partition, false otherwise."
- (member 'boot (partition-flags partition)))
-
-(define (find-root-partition image)
- "Return the root partition of the given IMAGE."
- (srfi-1:find root-partition? (image-partitions image)))
-
(define (image->root-file-system image)
"Return the IMAGE root partition file-system type."
(let ((format (image-format image)))
--
2.26.2
M
M
Mathieu Othacehe wrote on 27 May 2020 09:24
[PATCH 3/8] bootloader: grub: Use inheritance to define grub-minimal-bootloader.
(address . 41560@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200527072420.26140-3-othacehe@gnu.org
* gnu/bootloader/grub.scm (grub-minimal-bootloader): Inherit from
grub-bootloader to avoid field redefinition.
---
gnu/bootloader/grub.scm | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)

Toggle diff (23 lines)
diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
index 29bec92196..388f29d3a8 100644
--- a/gnu/bootloader/grub.scm
+++ b/gnu/bootloader/grub.scm
@@ -516,13 +516,10 @@ fi~%"))))
(configuration-file "/boot/grub/grub.cfg")
(configuration-file-generator grub-configuration-file)))
-(define grub-minimal-bootloader
+(define* grub-minimal-bootloader
(bootloader
- (name 'grub)
- (package grub-minimal)
- (installer install-grub)
- (configuration-file "/boot/grub/grub.cfg")
- (configuration-file-generator grub-configuration-file)))
+ (inherit grub-bootloader)
+ (package grub-minimal)))
(define* grub-efi-bootloader
(bootloader
--
2.26.2
M
M
Mathieu Othacehe wrote on 27 May 2020 09:24
[PATCH 4/8] image: Add bootloader installation support.
(address . 41560@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200527072420.26140-4-othacehe@gnu.org
* gnu/build/image.scm (initialize-root-partition): Add bootloader-package and
bootloader-installer arguments. Run the bootloader-installer if defined.
* gnu/system/image.scm (system-disk-image): Adapt the partition initializer
call accordingly.
---
gnu/build/image.scm | 5 +++++
gnu/system/image.scm | 4 +++-
2 files changed, 8 insertions(+), 1 deletion(-)

Toggle diff (40 lines)
diff --git a/gnu/build/image.scm b/gnu/build/image.scm
index b37ea9332a..49faeab466 100644
--- a/gnu/build/image.scm
+++ b/gnu/build/image.scm
@@ -155,6 +155,8 @@ deduplicates files common to CLOSURE and the rest of PREFIX."
#:key
bootcfg
bootcfg-location
+ bootloader-package
+ bootloader-installer
(deduplicate? #t)
references-graphs
(register-closures? #t)
@@ -178,6 +180,9 @@ of the directory of the 'system' derivation."
#:deduplicate? deduplicate?))
references-graphs))
+ (when bootloader-installer
+ (display "installing bootloader...\n")
+ (bootloader-installer bootloader-package #f root))
(when bootcfg
(install-boot-config bootcfg bootcfg-location root)))
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index ffc746fcf5..6608991fdc 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -236,7 +236,9 @@ used in the image."
#:deduplicate? #f
#:system-directory #$os
#:bootloader-package
- #$(bootloader-package bootloader)
+ #+(bootloader-package bootloader)
+ #:bootloader-installer
+ #+(bootloader-installer bootloader)
#:bootcfg #$bootcfg
#:bootcfg-location
#$(bootloader-configuration-file bootloader)))))
--
2.26.2
M
M
Mathieu Othacehe wrote on 27 May 2020 09:24
[PATCH 5/8] system: image: Correct genimage configuration file indentation.
(address . 41560@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200527072420.26140-5-othacehe@gnu.org
* gnu/system/image.scm (system-disk-image): Fix genimage configuration file
indentation.
---
gnu/system/image.scm | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

Toggle diff (21 lines)
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 6608991fdc..7ac998d861 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -262,10 +262,10 @@ used in the image."
(image (partition-image partition))
(offset (partition-offset partition)))
#~(format #f "~/partition ~a {
- ~/~/partition-type = ~a
- ~/~/image = \"~a\"
- ~/~/offset = \"~a\"
- ~/}"
+~/~/partition-type = ~a
+~/~/image = \"~a\"
+~/~/offset = \"~a\"
+~/}"
#$label
#$dos-type
#$image
--
2.26.2
M
M
Mathieu Othacehe wrote on 27 May 2020 09:24
[PATCH 6/8] image: Use grub-efi to install the EFI bootloader.
(address . 41560@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200527072420.26140-6-othacehe@gnu.org
* guix/utils.scm (target-intel?): New exported procedure.
* gnu/build/image.scm (initialize-efi-partition): Rename bootloader-package
argument to grub-efi.
* gnu/system/image.scm (system-disk-image): Adapt accordingly to pass
grub-efi package. Make sure that grub-efi is not built if we are not targeting
an Intel system.
---
gnu/build/image.scm | 4 ++--
gnu/system/image.scm | 4 ++++
guix/utils.scm | 5 +++++
3 files changed, 11 insertions(+), 2 deletions(-)

Toggle diff (57 lines)
diff --git a/gnu/build/image.scm b/gnu/build/image.scm
index 49faeab466..a8594e202b 100644
--- a/gnu/build/image.scm
+++ b/gnu/build/image.scm
@@ -146,10 +146,10 @@ deduplicates files common to CLOSURE and the rest of PREFIX."
(define* (initialize-efi-partition root
#:key
- bootloader-package
+ grub-efi
#:allow-other-keys)
"Install in ROOT directory, an EFI loader using BOOTLOADER-PACKAGE."
- (install-efi-loader bootloader-package root))
+ (install-efi-loader grub-efi root))
(define* (initialize-root-partition root
#:key
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 7ac998d861..a706f872a8 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -235,6 +235,10 @@ used in the image."
#:references-graphs '#$graph
#:deduplicate? #f
#:system-directory #$os
+ #:grub-efi #$(let-system (system target)
+ (and (target-intel?
+ (or target system))
+ grub-efi))
#:bootloader-package
#+(bootloader-package bootloader)
#:bootloader-installer
diff --git a/guix/utils.scm b/guix/utils.scm
index d7b197fa44..fb3b233286 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -74,6 +74,7 @@
%current-target-system
package-name->name+version
target-mingw?
+ target-intel?
target-arm32?
target-aarch64?
target-arm?
@@ -490,6 +491,10 @@ a character other than '@'."
(and target
(string-suffix? "-mingw32" target)))
+(define* (target-intel? #:optional (target (or (%current-target-system)
+ (%current-system))))
+ (any (cut string-prefix? <> target) '("x86_64" "i686")))
+
(define* (target-arm32? #:optional (target (or (%current-target-system)
(%current-system))))
(string-prefix? "arm" target))
--
2.26.2
M
M
Mathieu Othacehe wrote on 27 May 2020 09:24
[PATCH 7/8] system: image: Fix image-with-os.
(address . 41560@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200527072420.26140-7-othacehe@gnu.org
* gnu/system/image.scm (image-with-os): Do not reorder partitions, as we want
them to be created according to definition order.
---
gnu/system/image.scm | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)

Toggle diff (37 lines)
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index a706f872a8..97124a4699 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -424,18 +424,18 @@ to OS. Also set the UUID and the size of the root partition."
(string=? (file-system-mount-point fs) "/"))
(operating-system-file-systems os)))
- (let*-values (((partitions) (image-partitions base-image))
- ((root-partition other-partitions)
- (srfi-1:partition root-partition? partitions)))
- (image
- (inherit base-image)
- (operating-system os)
- (partitions
- (cons (partition
- (inherit (car root-partition))
- (uuid (file-system-device root-file-system))
- (size (root-size base-image)))
- other-partitions)))))
+ (image
+ (inherit base-image)
+ (operating-system os)
+ (partitions
+ (map (lambda (p)
+ (if (root-partition? p)
+ (partition
+ (inherit p)
+ (uuid (file-system-device root-file-system))
+ (size (root-size base-image)))
+ p))
+ (image-partitions base-image)))))
(define (operating-system-for-image image)
"Return an operating-system based on the one specified in IMAGE, but
--
2.26.2
M
M
Mathieu Othacehe wrote on 27 May 2020 09:24
[PATCH 8/8] image: Do not use VM to create disk-images.
(address . 41560@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200527072420.26140-8-othacehe@gnu.org
Now that installing Grub on raw disk-images is supported, we do not need to
rely on (gnu system vm) module.

* gnu/system/image.scm (make-system-image): Rename to ...
(system-image): ... this, and remove the compatibility wrapper.
(find-image): Turn to a monadic procedure. This will become useful when
introducing Hurd support, to be able to detect the target system.
* gnu/ci.scm (qemu-jobs): Use lower-object now that system-image returns a
file-like object.
* gnu/tests/install.scm (run-install): Ditto.
* guix/scripts/system.scm (system-derivation-for-action): Add a 'base-image'
argument,
(perform-action): adapt accordingly.
---
gnu/ci.scm | 20 +++++++++++---------
gnu/system/image.scm | 40 ++++++----------------------------------
gnu/tests/install.scm | 8 ++++----
guix/scripts/system.scm | 16 +++++++++-------
4 files changed, 30 insertions(+), 54 deletions(-)

Toggle diff (162 lines)
diff --git a/gnu/ci.scm b/gnu/ci.scm
index b61181be51..fa67168e22 100644
--- a/gnu/ci.scm
+++ b/gnu/ci.scm
@@ -219,19 +219,21 @@ system.")
(run-with-store store
(mbegin %store-monad
(set-guile-for-build (default-guile))
- (system-image
- (image
- (inherit efi-disk-image)
- (size (* 1500 MiB))
- (operating-system installation-os))))))
+ (lower-object
+ (system-image
+ (image
+ (inherit efi-disk-image)
+ (size (* 1500 MiB))
+ (operating-system installation-os)))))))
(->job 'iso9660-image
(run-with-store store
(mbegin %store-monad
(set-guile-for-build (default-guile))
- (system-image
- (image
- (inherit iso9660-image)
- (operating-system installation-os)))))))
+ (lower-object
+ (system-image
+ (image
+ (inherit iso9660-image)
+ (operating-system installation-os))))))))
'()))
(define channel-build-system
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 97124a4699..8bb8412f16 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -488,7 +488,7 @@ it can be used for bootloading."
(type root-file-system-type))
file-systems-to-keep)))))
-(define* (make-system-image image)
+(define* (system-image image)
"Return the derivation of IMAGE. It can be a raw disk-image or an ISO9660
image, depending on IMAGE format."
(define substitutable? (image-substitutable? image))
@@ -521,38 +521,10 @@ image, depending on IMAGE format."
"Find and return an image that could match the given FILE-SYSTEM-TYPE. This
is useful to adapt to interfaces written before the addition of the <image>
record."
- ;; XXX: Add support for system and target here, or in the caller.
- (match file-system-type
- ("iso9660" iso9660-image)
- (_ efi-disk-image)))
-
-(define (system-image image)
- "Wrap 'make-system-image' call, so that it is used only if the given IMAGE
-is supported. Otherwise, fallback to image creation in a VM. This is
-temporary and should be removed once 'make-system-image' is able to deal with
-all types of images."
- (define substitutable? (image-substitutable? image))
- (define volatile-root? (image-volatile-root? image))
-
- (let* ((image-os (image-operating-system image))
- (image-root-filesystem-type (image->root-file-system image))
- (bootloader (bootloader-configuration-bootloader
- (operating-system-bootloader image-os)))
- (bootloader-name (bootloader-name bootloader))
- (size (image-size image))
- (format (image-format image)))
- (mbegin %store-monad
- (if (and (or (eq? bootloader-name 'grub)
- (eq? bootloader-name 'extlinux))
- (eq? format 'disk-image))
- ;; Fallback to image creation in a VM when it is not yet supported
- ;; by this module.
- (system-disk-image-in-vm image-os
- #:disk-image-size size
- #:file-system-type image-root-filesystem-type
- #:volatile? volatile-root?
- #:substitutable? substitutable?)
- (lower-object
- (make-system-image image))))))
+ (mbegin %store-monad
+ (return
+ (match file-system-type
+ ("iso9660" iso9660-image)
+ (_ efi-disk-image)))))
;;; image.scm ends here
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index cea26c8ef3..6bd8c7d3d2 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -228,18 +228,18 @@ packages defined in installation-os."
(mlet* %store-monad ((_ (set-grafting #f))
(system (current-system))
(target (operating-system-derivation target-os))
+ (base-image (find-image
+ installation-disk-image-file-system-type))
;; Since the installation system has no network access,
;; we cheat a little bit by adding TARGET to its GC
;; roots. This way, we know 'guix system init' will
;; succeed. Also add guile-final, which is pulled in
;; through provenance.drv and may not always be present.
- (image
+ (image ->
(system-image
(image
- (inherit
- (find-image
- installation-disk-image-file-system-type))
+ (inherit base-image)
(size install-size)
(operating-system
(operating-system-with-gc-roots
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 3efd113ac8..3d7aa77cb7 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -670,7 +670,7 @@ checking this by themselves in their 'check' procedure."
;;; Action.
;;;
-(define* (system-derivation-for-action os action
+(define* (system-derivation-for-action os base-image action
#:key image-size file-system-type
full-boot? container-shared-network?
mappings)
@@ -694,11 +694,12 @@ checking this by themselves in their 'check' procedure."
(* 70 (expt 2 20)))
#:mappings mappings))
((disk-image)
- (system-image
- (image
- (inherit (find-image file-system-type))
- (size image-size)
- (operating-system os))))
+ (lower-object
+ (system-image
+ (image
+ (inherit base-image)
+ (size image-size)
+ (operating-system os)))))
((docker-image)
(system-docker-image os #:shared-network? container-shared-network?))))
@@ -800,7 +801,8 @@ static checks."
(check-initrd-modules os)))
(mlet* %store-monad
- ((sys (system-derivation-for-action os action
+ ((image (find-image file-system-type))
+ (sys (system-derivation-for-action os image action
#:file-system-type file-system-type
#:image-size image-size
#:full-boot? full-boot?
--
2.26.2
J
J
Jan Nieuwenhuizen wrote on 27 May 2020 10:09
Re: [bug#41560] [PATCH 0/8] image: Add MBR based boot support.
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87d06pbxz8.fsf@gnu.org
Mathieu Othacehe writes:

Hello Mathieu,

Toggle quote (12 lines)
> Until now, only Grub EFI bootloader was supported in (gnu system
> image). Installing Grub EFI is as simple as copying a binary (created with
> grub-mkstandalone) in a dedicated partition.
>
> Now, for MBR based booting, one needs to run grub-install that does require
> root permissions. To overcome this issue, I used a hack inspired from OpenWrt
> and Buildroot.
>
> As grub-install is in fact a wrapper around grub-mkimage and grub-bios-setup,
> it is possible, with some plumbing and using those two tools, to install Grub
> on a raw disk-image, without root permissions.

Thanks for your amazing piece of work on disk-image; for me this means
adding GNU/Hurd support much sooner than I could hope for!

What can I say, not only LGTM, but also WFM (on a soon-to-be-reset
wip-hurd-vm).

Greetings,
Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 28 May 2020 23:32
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87y2pb925g.fsf@gnu.org
Hi,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (12 lines)
> Until now, only Grub EFI bootloader was supported in (gnu system
> image). Installing Grub EFI is as simple as copying a binary (created with
> grub-mkstandalone) in a dedicated partition.
>
> Now, for MBR based booting, one needs to run grub-install that does require
> root permissions. To overcome this issue, I used a hack inspired from OpenWrt
> and Buildroot.
>
> As grub-install is in fact a wrapper around grub-mkimage and grub-bios-setup,
> it is possible, with some plumbing and using those two tools, to install Grub
> on a raw disk-image, without root permissions.

Amazing! This is really cool, thank you!

Looking in more detail at the rest now…

Ludo’.
L
L
Ludovic Courtès wrote on 28 May 2020 23:37
Re: [bug#41560] [PATCH 1/8] bootloader: Add 'disk-image-installer'.
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87mu5r91xa.fsf@gnu.org
Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (11 lines)
> * gnu/bootloader.scm (<bootloader>)[disk-image-installer]: New field,
> (bootloader-disk-image-installer): export it.
> * gnu/bootloader/grub.scm (install-grub-disk-image): New procedure ...
> (grub-bootloader): ... used as "disk-image-installer" here.
> (grub-efi-bootloader): set "disk-image-installer" to #f.
> * gnu/system/image.scm (root-partition?, find-root-partition): Move to
> "Helpers" section.
> (root-partition-index): New procedure.
> (system-disk-image): Honor disk-image-installer, and
> use it to install the bootloader directly on the disk-image, if supported.

[...]

Toggle quote (3 lines)
> + (disk-image-installer bootloader-disk-image-installer
> + (default #f))

My only concern here is that we’re making an interface that’s only
implemented by one bootloader, and I fear bitrot of the other
bootloaders longer term. I guess we’ll see, this concern shouldn’t
block progress.

Toggle quote (22 lines)
> +(define install-grub-disk-image
> + #~(lambda (bootloader root-index image)
> + ;; Install GRUB on the given IMAGE. The root partition index is
> + ;; ROOT-INDEX.
> + (let ((grub-mkimage
> + (string-append bootloader "/bin/grub-mkimage"))
> + (modules '("biosdisk" "part_msdos" "fat" "ext2"))
> + (grub-bios-setup
> + (string-append bootloader "/sbin/grub-bios-setup"))
> + (root-device (format #f "hd0,msdos~a" root-index))
> + (boot-img (string-append bootloader "/lib/grub/i386-pc/boot.img"))
> + (device-map "device.map"))
> +
> + ;; Create a minimal, standalone Grub image that will be written
> + ;; directly in the MBR-GAP (space between the end of the MBR and the
> + ;; first partition).
> + (apply invoke grub-mkimage
> + "-O" "i386-pc"
> + "-o" "core.img"
> + "-p" (format #f "(~a)/boot/grub" root-device)
> + modules)

Very smart. s/Grub/GRUB/ everywhere please. :-)

Toggle quote (5 lines)
> +(define (root-partition-index image)
> + "Return the index of the root partition of the given IMAGE."
> + (1+ (srfi-1:list-index identity
> + (map root-partition? (image-partitions image)))))

Isn’t it just (list-index root-partition (image-partitions image))?

Otherwise LGTM!
L
L
Ludovic Courtès wrote on 28 May 2020 23:40
Re: [bug#41560] [PATCH 2/8] bootloader: grub: Do not run grub-install when creating a disk-image.
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87imgf91rt.fsf@gnu.org
Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (4 lines)
> * gnu/bootloader/grub.scm (install-grub): When creating a disk-image,
> grub-install will fail because it lacks root permissions. In that case, do not
> run grub-install and only copy Grub modules to the /boot directory.

[...]

Toggle quote (1 lines)
> + ;; Install GRUB on DEVICE which is mounted at MOUNT-POINT. If device
^
DEVICE

Toggle quote (2 lines)
> + ;; is #f we are creating a disk-image.

Maybe:

… is #f, then populate the disk image rooted at MOUNT-POINT.
L
L
Ludovic Courtès wrote on 28 May 2020 23:44
Re: [bug#41560] [PATCH 6/8] image: Use grub-efi to install the EFI bootloader.
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87d06n91kx.fsf@gnu.org
Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (7 lines)
> * guix/utils.scm (target-intel?): New exported procedure.
> * gnu/build/image.scm (initialize-efi-partition): Rename bootloader-package
> argument to grub-efi.
> * gnu/system/image.scm (system-disk-image): Adapt accordingly to pass
> grub-efi package. Make sure that grub-efi is not built if we are not targeting
> an Intel system.

[...]

Toggle quote (12 lines)
> +++ b/gnu/build/image.scm
> @@ -146,10 +146,10 @@ deduplicates files common to CLOSURE and the rest of PREFIX."
>
> (define* (initialize-efi-partition root
> #:key
> - bootloader-package
> + grub-efi
> #:allow-other-keys)
> "Install in ROOT directory, an EFI loader using BOOTLOADER-PACKAGE."
> - (install-efi-loader bootloader-package root))
> + (install-efi-loader grub-efi root))

Does it have to be GRUB?

Toggle quote (10 lines)
> +++ b/gnu/system/image.scm
> @@ -235,6 +235,10 @@ used in the image."
> #:references-graphs '#$graph
> #:deduplicate? #f
> #:system-directory #$os
> + #:grub-efi #$(let-system (system target)
> + (and (target-intel?
> + (or target system))
> + grub-efi))

Some AArch64 systems such as the SoftIron OverDrive 1000 use EFI too.
So I don’t think the above is correct.

Toggle quote (4 lines)
> +(define* (target-intel? #:optional (target (or (%current-target-system)
> + (%current-system))))
> + (any (cut string-prefix? <> target) '("x86_64" "i686")))

Shouldn’t it include i[345]6 as well?

Also, I think no 32-bit Intel systems use EFI.

Actually, why do we need to guess, can’t we just use the bootloader
specified in the <operating-system> record? (Naïve question, I haven’t
checked…)
L
L
Ludovic Courtès wrote on 28 May 2020 23:47
Re: [bug#41560] [PATCH 8/8] image: Do not use VM to create disk-images.
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
878shb91fx.fsf@gnu.org
Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (14 lines)
> Now that installing Grub on raw disk-images is supported, we do not need to
> rely on (gnu system vm) module.
>
> * gnu/system/image.scm (make-system-image): Rename to ...
> (system-image): ... this, and remove the compatibility wrapper.
> (find-image): Turn to a monadic procedure. This will become useful when
> introducing Hurd support, to be able to detect the target system.
> * gnu/ci.scm (qemu-jobs): Use lower-object now that system-image returns a
> file-like object.
> * gnu/tests/install.scm (run-install): Ditto.
> * guix/scripts/system.scm (system-derivation-for-action): Add a 'base-image'
> argument,
> (perform-action): adapt accordingly.

Yay! \o/

Thank you,
Ludo’.
M
M
Mathieu Othacehe wrote on 29 May 2020 09:17
Re: [bug#41560] [PATCH 1/8] bootloader: Add 'disk-image-installer'.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 41560@debbugs.gnu.org)
874krzyzuz.fsf@gnu.org
Hey Ludo!

Thanks for the review :)

Toggle quote (8 lines)
>> + (disk-image-installer bootloader-disk-image-installer
>> + (default #f))
>
> My only concern here is that we’re making an interface that’s only
> implemented by one bootloader, and I fear bitrot of the other
> bootloaders longer term. I guess we’ll see, this concern shouldn’t
> block progress.

I plan to implement it for extlinux and u-boot soon, so we should be
fine.

Toggle quote (2 lines)
> Very smart. s/Grub/GRUB/ everywhere please. :-)

Yes, thank you :)

Toggle quote (8 lines)
>
>> +(define (root-partition-index image)
>> + "Return the index of the root partition of the given IMAGE."
>> + (1+ (srfi-1:list-index identity
>> + (map root-partition? (image-partitions image)))))
>
> Isn’t it just (list-index root-partition (image-partitions image))?

Of course!

Mathieu
M
M
Mathieu Othacehe wrote on 29 May 2020 09:25
Re: [bug#41560] [PATCH 6/8] image: Use grub-efi to install the EFI bootloader.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 41560@debbugs.gnu.org)
87zh9rxkvx.fsf@gnu.org
Toggle quote (6 lines)
>> "Install in ROOT directory, an EFI loader using BOOTLOADER-PACKAGE."
>> - (install-efi-loader bootloader-package root))
>> + (install-efi-loader grub-efi root))
>
> Does it have to be GRUB?

No any EFI compatible bootloader would do the trick. However, for now
"install-efi-loader" calls "grub-mkstandalone", so maybe we can keep it
that way for now?

Toggle quote (14 lines)
>
>> +++ b/gnu/system/image.scm
>> @@ -235,6 +235,10 @@ used in the image."
>> #:references-graphs '#$graph
>> #:deduplicate? #f
>> #:system-directory #$os
>> + #:grub-efi #$(let-system (system target)
>> + (and (target-intel?
>> + (or target system))
>> + grub-efi))
>
> Some AArch64 systems such as the SoftIron OverDrive 1000 use EFI too.
> So I don’t think the above is correct.

Yes you're correct. Actually, I think that I was overthinking this
thing. Using "#+grub-efi" should work.

Toggle quote (7 lines)
>
>> +(define* (target-intel? #:optional (target (or (%current-target-system)
>> + (%current-system))))
>> + (any (cut string-prefix? <> target) '("x86_64" "i686")))
>
> Shouldn’t it include i[345]6 as well?

So I removed this bit.

Toggle quote (7 lines)
>
> Also, I think no 32-bit Intel systems use EFI.
>
> Actually, why do we need to guess, can’t we just use the bootloader
> specified in the <operating-system> record? (Naïve question, I haven’t
> checked…)

If we take the example of "installation-os", we use "grub-bootloader",
but we expect this image to be EFI ready. That's why "grub-efi" usage is
forced here.

As this is already the case without this serie, I think we can
proceed.

Thanks,

Mathieu
M
M
Mathieu Othacehe wrote on 29 May 2020 09:27
Re: [bug#41560] [PATCH 8/8] image: Do not use VM to create disk-images.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 41560-done@debbugs.gnu.org)
87v9kfxktr.fsf@gnu.org
Just pushed the serie, taking your remarks into account.

Thanks again for reviewing!

Mathieu

Toggle quote (20 lines)
> Mathieu Othacehe <m.othacehe@gmail.com> skribis:
>
>> Now that installing Grub on raw disk-images is supported, we do not need to
>> rely on (gnu system vm) module.
>>
>> * gnu/system/image.scm (make-system-image): Rename to ...
>> (system-image): ... this, and remove the compatibility wrapper.
>> (find-image): Turn to a monadic procedure. This will become useful when
>> introducing Hurd support, to be able to detect the target system.
>> * gnu/ci.scm (qemu-jobs): Use lower-object now that system-image returns a
>> file-like object.
>> * gnu/tests/install.scm (run-install): Ditto.
>> * guix/scripts/system.scm (system-derivation-for-action): Add a 'base-image'
>> argument,
>> (perform-action): adapt accordingly.
>
> Yay! \o/
>
> Thank you,
> Ludo’.
Closed
J
J
Jan Nieuwenhuizen wrote on 29 May 2020 10:45
Re: bug#41560: [PATCH 8/8] image: Do not use VM to create disk-images.
(address . 41560@debbugs.gnu.org)
87blm7ruy7.fsf@gnu.org
Mathieu Othacehe writes:

Toggle quote (4 lines)
> Just pushed the serie, taking your remarks into account.
>
> Thanks again for reviewing!

\o/

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
?
Your comment

This issue is archived.

To comment on this conversation send an email to 41560@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 41560
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch