[PATCH 0/2] ci: Build Guix System images.

  • Done
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Mathieu Othacehe
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Mathieu Othacehe
Severity
normal
M
M
Mathieu Othacehe wrote on 13 Jun 2020 21:40
(address . guix-patches@gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200613194001.9580-1-othacehe@gnu.org
Hello,

Now that we have a new shiny new "image" record that contains all the required
informations to build or cross-build a Guix System image, it would be nice if
Cuirass could provide associated substitutes.

This serie allows to build a list of Guix System images in the CI. For now
this list only contains the Hurd barebones Guix System image.

Thanks,

Mathieu

Mathieu Othacehe (2):
image: Move hurd image definition to a dedicated file.
ci: Build Guix System images.

gnu/ci.scm | 69 +++++++++++++++++-----------------
gnu/image.scm | 2 +
gnu/local.mk | 2 +
gnu/system/image.scm | 23 +++---------
gnu/system/images/hurd.scm | 76 ++++++++++++++++++++++++++++++++++++++
5 files changed, 121 insertions(+), 51 deletions(-)
create mode 100644 gnu/system/images/hurd.scm

--
2.26.2
M
M
Mathieu Othacehe wrote on 13 Jun 2020 21:41
[PATCH 2/2] ci: Build Guix System images.
(address . 41839@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200613194140.10367-2-othacehe@gnu.org
Build a list of Guix System images. For now, this list only contains the Hurd
barebones Guix System image.

* gnu/ci.scm (%guixsd-supported-systems): Remove "armhf-linux",
(%u-boot-systems): remove unused variable,
(%guix-system-images): new variable
(qemu-jobs): rename to "image-jobs" and build the Guix Systems listed in the
new "%guix-system-images" variable,
(system-test-jobs): adapt accordingly,
(hydra-jobs): ditto.
---
gnu/ci.scm | 69 +++++++++++++++++++++++++++---------------------------
1 file changed, 35 insertions(+), 34 deletions(-)

Toggle diff (117 lines)
diff --git a/gnu/ci.scm b/gnu/ci.scm
index fa67168e22..2d49009810 100644
--- a/gnu/ci.scm
+++ b/gnu/ci.scm
@@ -53,6 +53,7 @@
#:use-module (gnu system image)
#:use-module (gnu system vm)
#:use-module (gnu system install)
+ #:use-module (gnu system images hurd)
#:use-module (gnu tests)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -185,18 +186,21 @@ SYSTEM."
(remove (either from-32-to-64? same? pointless?)
%cross-targets)))
-(define %guixsd-supported-systems
- '("x86_64-linux" "i686-linux" "armhf-linux"))
+;; Architectures that are able to build or cross-build Guix System images.
+;; This does not mean that other architectures are not supported, only that
+;; they are often not fast enough to support Guix System images building.
+(define %guix-system-supported-systems
+ '("x86_64-linux" "i686-linux"))
-(define %u-boot-systems
- '("armhf-linux"))
+(define %guix-system-images
+ (list hurd-barebones-disk-image))
-(define (qemu-jobs store system)
- "Return a list of jobs that build QEMU images for SYSTEM."
+(define (image-jobs store system)
+ "Return a list of jobs that build images for SYSTEM."
(define (->alist drv)
`((derivation . ,(derivation-file-name drv))
- (description . "Stand-alone QEMU image of the GNU system")
- (long-description . "This is a demo stand-alone QEMU image of the GNU
+ (description . "Stand-alone image of the GNU system")
+ (long-description . "This is a demo stand-alone image of the GNU
system.")
(license . ,(license-name gpl3+))
(max-silent-time . 600)
@@ -211,29 +215,30 @@ system.")
(parameterize ((%graft? #f))
(->alist drv))))))
+ (define (build-image image)
+ (run-with-store store
+ (mbegin %store-monad
+ (set-guile-for-build (default-guile))
+ (lower-object (system-image image)))))
+
(define MiB
(expt 2 20))
- (if (member system %guixsd-supported-systems)
- (list (->job 'usb-image
- (run-with-store store
- (mbegin %store-monad
- (set-guile-for-build (default-guile))
- (lower-object
- (system-image
- (image
- (inherit efi-disk-image)
- (size (* 1500 MiB))
- (operating-system installation-os)))))))
- (->job 'iso9660-image
- (run-with-store store
- (mbegin %store-monad
- (set-guile-for-build (default-guile))
- (lower-object
- (system-image
- (image
- (inherit iso9660-image)
- (operating-system installation-os))))))))
+ (if (member system %guix-system-supported-systems)
+ `(,(->job 'usb-image
+ (build-image
+ (image
+ (inherit efi-disk-image)
+ (size (* 1500 MiB))
+ (operating-system installation-os))))
+ ,(->job 'iso9660-image
+ (build-image
+ (image
+ (inherit iso9660-image)
+ (operating-system installation-os))))
+ ,@(map (lambda (image)
+ (->job (image-name image) (build-image image)))
+ %guix-system-images))
'()))
(define channel-build-system
@@ -305,11 +310,7 @@ system.")
"." system))))
(cons name (test->thunk test))))
- (if (and (member system %guixsd-supported-systems)
-
- ;; XXX: Our build farm has too few ARMv7 machines and they are very
- ;; slow, so skip system tests there.
- (not (string=? system "armhf-linux")))
+ (if (member system %guix-system-supported-systems)
;; Override the value of 'current-guix' used by system tests. Using a
;; channel instance makes tests that rely on 'current-guix' less
;; expensive. It also makes sure we get a valid Guix package when this
@@ -486,7 +487,7 @@ Return #f if no such checkout is found."
(package->job store package
system))))
(append (filter-map job all)
- (qemu-jobs store system)
+ (image-jobs store system)
(system-test-jobs store system
#:source source
#:commit commit)
--
2.26.2
M
M
Mathieu Othacehe wrote on 13 Jun 2020 21:41
[PATCH 1/2] image: Move hurd image definition to a dedicated file.
(address . 41839@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200613194140.10367-1-othacehe@gnu.org
This moves hurd-disk-image to a dedicated file. It also defines a default
operating-system so that the image can be built standalone.

* gnu/system/images/hurd.scm: New file,
* gnu/local.mk (GNU_SYSTEM_MODULES): add it.
* gnu/image.scm (<image>)[name]: New field.
* gnu/system/image.scm (root-offset, root-label): Export it,
(hurd-disk-image): remove it as this is now defined in the new, Hurd dedicated
file above,
(find-image): adapt to avoid loop dependency.
---
gnu/image.scm | 2 +
gnu/local.mk | 2 +
gnu/system/image.scm | 23 +++---------
gnu/system/images/hurd.scm | 76 ++++++++++++++++++++++++++++++++++++++
4 files changed, 86 insertions(+), 17 deletions(-)
create mode 100644 gnu/system/images/hurd.scm

Toggle diff (160 lines)
diff --git a/gnu/image.scm b/gnu/image.scm
index 19b466527b..dc66f2c533 100644
--- a/gnu/image.scm
+++ b/gnu/image.scm
@@ -67,6 +67,8 @@
(define-record-type* <image>
image make-image
image?
+ (name image-name ;symbol
+ (default #f))
(format image-format) ;symbol
(target image-target
(default #f))
diff --git a/gnu/local.mk b/gnu/local.mk
index 583274235b..eeeb276478 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -629,6 +629,8 @@ GNU_SYSTEM_MODULES = \
%D%/system/uuid.scm \
%D%/system/vm.scm \
\
+ %D%/system/images/hurd.scm \
+ \
%D%/machine.scm \
%D%/machine/digital-ocean.scm \
%D%/machine/ssh.scm \
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 1bda25fd7f..cb8fe18bcc 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -53,10 +53,12 @@
#:use-module (srfi srfi-35)
#:use-module (rnrs bytevectors)
#:use-module (ice-9 match)
- #:export (esp-partition
+ #:export (root-offset
+ root-label
+
+ esp-partition
root-partition
- hurd-disk-image
efi-disk-image
iso9660-image
@@ -101,20 +103,6 @@
(list #:make-device-nodes
make-hurd-device-nodes)))))
-(define hurd-disk-image
- (image
- (format 'disk-image)
- (target "i586-pc-gnu")
- (partitions
- (list (partition
- (size 'guess)
- (offset root-offset)
- (label root-label)
- (file-system "ext2")
- (file-system-options '("-o" "hurd" "-O" "ext_attr"))
- (flags '(boot))
- (initializer hurd-initialize-root-partition))))))
-
(define efi-disk-image
(image
(format 'disk-image)
@@ -569,7 +557,8 @@ addition of the <image> record."
(_ (cond
((and target
(hurd-triplet? target))
- hurd-disk-image)
+ (module-ref (resolve-interface '(gnu system images hurd))
+ 'hurd-disk-image))
(else
efi-disk-image)))))
diff --git a/gnu/system/images/hurd.scm b/gnu/system/images/hurd.scm
new file mode 100644
index 0000000000..4417f03cc8
--- /dev/null
+++ b/gnu/system/images/hurd.scm
@@ -0,0 +1,76 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu system images hurd)
+ #:use-module (guix gexp)
+ #:use-module (gnu bootloader)
+ #:use-module (gnu bootloader grub)
+ #:use-module (gnu image)
+ #:use-module (gnu packages ssh)
+ #:use-module (gnu services)
+ #:use-module (gnu services ssh)
+ #:use-module (gnu system)
+ #:use-module (gnu system file-systems)
+ #:use-module (gnu system hurd)
+ #:use-module (gnu system image)
+ #:export (hurd-barebones-os
+ hurd-disk-image
+ hurd-barebones-disk-image))
+
+(define hurd-barebones-os
+ (operating-system
+ (inherit %hurd-default-operating-system)
+ (bootloader (bootloader-configuration
+ (bootloader grub-minimal-bootloader)
+ (target "/dev/sdX")))
+ (file-systems (cons (file-system
+ (device (file-system-label "my-root"))
+ (mount-point "/")
+ (type "ext2"))
+ %base-file-systems))
+ (host-name "guixygnu")
+ (timezone "Europe/Amsterdam")
+ (packages (cons openssh-sans-x %base-packages/hurd))
+ (services (cons (service openssh-service-type
+ (openssh-configuration
+ (openssh openssh-sans-x)
+ (use-pam? #f)
+ (port-number 2222)
+ (permit-root-login #t)
+ (allow-empty-passwords? #t)
+ (password-authentication? #t)))
+ %base-services/hurd))))
+
+(define hurd-disk-image
+ (image
+ (format 'disk-image)
+ (target "i586-pc-gnu")
+ (partitions
+ (list (partition
+ (size 'guess)
+ (offset root-offset)
+ (label root-label)
+ (file-system "ext2")
+ (flags '(boot))
+ (initializer (gexp initialize-root-partition)))))))
+
+(define hurd-barebones-disk-image
+ (image
+ (inherit hurd-disk-image)
+ (name 'hurd-barebones-disk-image)
+ (operating-system hurd-barebones-os)))
--
2.26.2
L
L
Ludovic Courtès wrote on 23 Jun 2020 23:47
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87sgelxxnd.fsf@gnu.org
Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (11 lines)
> This moves hurd-disk-image to a dedicated file. It also defines a default
> operating-system so that the image can be built standalone.
>
> * gnu/system/images/hurd.scm: New file,
> * gnu/local.mk (GNU_SYSTEM_MODULES): add it.
> * gnu/image.scm (<image>)[name]: New field.
> * gnu/system/image.scm (root-offset, root-label): Export it,
> (hurd-disk-image): remove it as this is now defined in the new, Hurd dedicated
> file above,
> (find-image): adapt to avoid loop dependency.

LGTM!

It would be good to check by running “make as-derivation” if gnu/image/*
gets included.

Ludo’.
L
L
Ludovic Courtès wrote on 23 Jun 2020 23:48
Re: [bug#41839] [PATCH 2/2] ci: Build Guix System images.
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87o8p9xxkn.fsf@gnu.org
Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (11 lines)
> Build a list of Guix System images. For now, this list only contains the Hurd
> barebones Guix System image.
>
> * gnu/ci.scm (%guixsd-supported-systems): Remove "armhf-linux",
> (%u-boot-systems): remove unused variable,
> (%guix-system-images): new variable
> (qemu-jobs): rename to "image-jobs" and build the Guix Systems listed in the
> new "%guix-system-images" variable,
> (system-test-jobs): adapt accordingly,
> (hydra-jobs): ditto.

Cool.

Toggle quote (17 lines)
> + (if (member system %guix-system-supported-systems)
> + `(,(->job 'usb-image
> + (build-image
> + (image
> + (inherit efi-disk-image)
> + (size (* 1500 MiB))
> + (operating-system installation-os))))
> + ,(->job 'iso9660-image
> + (build-image
> + (image
> + (inherit iso9660-image)
> + (operating-system installation-os))))
> + ,@(map (lambda (image)
> + (->job (image-name image) (build-image image)))
> + %guix-system-images))
> '()))

I think the effect is that we’ll also cross-build GNU/Hurd images from
i686-linux, right? It would be nice to avoid that and only cross-build
from x86_64.

Otherwise LGTM, thank you!

Ludo’.
M
M
Mathieu Othacehe wrote on 25 Jun 2020 11:54
Re: [bug#41839] [PATCH 1/2] image: Move hurd image definition to a dedicated file.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 41839@debbugs.gnu.org)
87y2obcvx5.fsf@gmail.com
Hey,

Toggle quote (3 lines)
> It would be good to check by running “make as-derivation” if gnu/image/*
> gets included.

Yep, its included. Thanks for reviewing!

Mathieu
M
M
Mathieu Othacehe wrote on 25 Jun 2020 12:02
Re: [bug#41839] [PATCH 2/2] ci: Build Guix System images.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 41839-done@debbugs.gnu.org)
87r1u3cvk9.fsf@gnu.org
Hey,

Toggle quote (4 lines)
> I think the effect is that we’ll also cross-build GNU/Hurd images from
> i686-linux, right? It would be nice to avoid that and only cross-build
> from x86_64.

Yes, I fixed that (in two tries actually :p). Now working on adding this
image to the website.

Thanks,

Mathieu
Closed
?