[PATCH 0/2] ci: Build Guix System images.

  • Done
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Mathieu Othacehe
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Mathieu Othacehe
Severity
normal
M
M
Mathieu Othacehe wrote on 13 Jun 2020 21:40
(address . guix-patches@gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200613194001.9580-1-othacehe@gnu.org
Hello,

Now that we have a new shiny new "image" record that contains all the required
informations to build or cross-build a Guix System image, it would be nice if
Cuirass could provide associated substitutes.

This serie allows to build a list of Guix System images in the CI. For now
this list only contains the Hurd barebones Guix System image.

Thanks,

Mathieu

Mathieu Othacehe (2):
image: Move hurd image definition to a dedicated file.
ci: Build Guix System images.

gnu/ci.scm | 69 +++++++++++++++++-----------------
gnu/image.scm | 2 +
gnu/local.mk | 2 +
gnu/system/image.scm | 23 +++---------
gnu/system/images/hurd.scm | 76 ++++++++++++++++++++++++++++++++++++++
5 files changed, 121 insertions(+), 51 deletions(-)
create mode 100644 gnu/system/images/hurd.scm

--
2.26.2
M
M
Mathieu Othacehe wrote on 13 Jun 2020 21:41
[PATCH 2/2] ci: Build Guix System images.
(address . 41839@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200613194140.10367-2-othacehe@gnu.org
Build a list of Guix System images. For now, this list only contains the Hurd
barebones Guix System image.

* gnu/ci.scm (%guixsd-supported-systems): Remove "armhf-linux",
(%u-boot-systems): remove unused variable,
(%guix-system-images): new variable
(qemu-jobs): rename to "image-jobs" and build the Guix Systems listed in the
new "%guix-system-images" variable,
(system-test-jobs): adapt accordingly,
(hydra-jobs): ditto.
---
gnu/ci.scm | 69 +++++++++++++++++++++++++++---------------------------
1 file changed, 35 insertions(+), 34 deletions(-)

Toggle diff (117 lines)
diff --git a/gnu/ci.scm b/gnu/ci.scm
index fa67168e22..2d49009810 100644
--- a/gnu/ci.scm
+++ b/gnu/ci.scm
@@ -53,6 +53,7 @@
#:use-module (gnu system image)
#:use-module (gnu system vm)
#:use-module (gnu system install)
+ #:use-module (gnu system images hurd)
#:use-module (gnu tests)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -185,18 +186,21 @@ SYSTEM."
(remove (either from-32-to-64? same? pointless?)
%cross-targets)))
-(define %guixsd-supported-systems
- '("x86_64-linux" "i686-linux" "armhf-linux"))
+;; Architectures that are able to build or cross-build Guix System images.
+;; This does not mean that other architectures are not supported, only that
+;; they are often not fast enough to support Guix System images building.
+(define %guix-system-supported-systems
+ '("x86_64-linux" "i686-linux"))
-(define %u-boot-systems
- '("armhf-linux"))
+(define %guix-system-images
+ (list hurd-barebones-disk-image))
-(define (qemu-jobs store system)
- "Return a list of jobs that build QEMU images for SYSTEM."
+(define (image-jobs store system)
+ "Return a list of jobs that build images for SYSTEM."
(define (->alist drv)
`((derivation . ,(derivation-file-name drv))
- (description . "Stand-alone QEMU image of the GNU system")
- (long-description . "This is a demo stand-alone QEMU image of the GNU
+ (description . "Stand-alone image of the GNU system")
+ (long-description . "This is a demo stand-alone image of the GNU
system.")
(license . ,(license-name gpl3+))
(max-silent-time . 600)
@@ -211,29 +215,30 @@ system.")
(parameterize ((%graft? #f))
(->alist drv))))))
+ (define (build-image image)
+ (run-with-store store
+ (mbegin %store-monad
+ (set-guile-for-build (default-guile))
+ (lower-object (system-image image)))))
+
(define MiB
(expt 2 20))
- (if (member system %guixsd-supported-systems)
- (list (->job 'usb-image
- (run-with-store store
- (mbegin %store-monad
- (set-guile-for-build (default-guile))
- (lower-object
- (system-image
- (image
- (inherit efi-disk-image)
- (size (* 1500 MiB))
- (operating-system installation-os)))))))
- (->job 'iso9660-image
- (run-with-store store
- (mbegin %store-monad
- (set-guile-for-build (default-guile))
- (lower-object
- (system-image
- (image
- (inherit iso9660-image)
- (operating-system installation-os))))))))
+ (if (member system %guix-system-supported-systems)
+ `(,(->job 'usb-image
+ (build-image
+ (image
+ (inherit efi-disk-image)
+ (size (* 1500 MiB))
+ (operating-system installation-os))))
+ ,(->job 'iso9660-image
+ (build-image
+ (image
+ (inherit iso9660-image)
+ (operating-system installation-os))))
+ ,@(map (lambda (image)
+ (->job (image-name image) (build-image image)))
+ %guix-system-images))
'()))
(define channel-build-system
@@ -305,11 +310,7 @@ system.")
"." system))))
(cons name (test->thunk test))))
- (if (and (member system %guixsd-supported-systems)
-
- ;; XXX: Our build farm has too few ARMv7 machines and they are very
- ;; slow, so skip system tests there.
- (not (string=? system "armhf-linux")))
+ (if (member system %guix-system-supported-systems)
;; Override the value of 'current-guix' used by system tests. Using a
;; channel instance makes tests that rely on 'current-guix' less
;; expensive. It also makes sure we get a valid Guix package when this
@@ -486,7 +487,7 @@ Return #f if no such checkout is found."
(package->job store package
system))))
(append (filter-map job all)
- (qemu-jobs store system)
+ (image-jobs store system)
(system-test-jobs store system
#:source source
#:commit commit)
--
2.26.2
M
M
Mathieu Othacehe wrote on 13 Jun 2020 21:41
[PATCH 1/2] image: Move hurd image definition to a dedicated file.
(address . 41839@debbugs.gnu.org)(name . Mathieu Othacehe)(address . othacehe@gnu.org)
20200613194140.10367-1-othacehe@gnu.org
This moves hurd-disk-image to a dedicated file. It also defines a default
operating-system so that the image can be built standalone.

* gnu/system/images/hurd.scm: New file,
* gnu/local.mk (GNU_SYSTEM_MODULES): add it.
* gnu/image.scm (<image>)[name]: New field.
* gnu/system/image.scm (root-offset, root-label): Export it,
(hurd-disk-image): remove it as this is now defined in the new, Hurd dedicated
file above,
(find-image): adapt to avoid loop dependency.
---
gnu/image.scm | 2 +
gnu/local.mk | 2 +
gnu/system/image.scm | 23 +++---------
gnu/system/images/hurd.scm | 76 ++++++++++++++++++++++++++++++++++++++
4 files changed, 86 insertions(+), 17 deletions(-)
create mode 100644 gnu/system/images/hurd.scm

Toggle diff (160 lines)
diff --git a/gnu/image.scm b/gnu/image.scm
index 19b466527b..dc66f2c533 100644
--- a/gnu/image.scm
+++ b/gnu/image.scm
@@ -67,6 +67,8 @@
(define-record-type* <image>
image make-image
image?
+ (name image-name ;symbol
+ (default #f))
(format image-format) ;symbol
(target image-target
(default #f))
diff --git a/gnu/local.mk b/gnu/local.mk
index 583274235b..eeeb276478 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -629,6 +629,8 @@ GNU_SYSTEM_MODULES = \
%D%/system/uuid.scm \
%D%/system/vm.scm \
\
+ %D%/system/images/hurd.scm \
+ \
%D%/machine.scm \
%D%/machine/digital-ocean.scm \
%D%/machine/ssh.scm \
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 1bda25fd7f..cb8fe18bcc 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -53,10 +53,12 @@
#:use-module (srfi srfi-35)
#:use-module (rnrs bytevectors)
#:use-module (ice-9 match)
- #:export (esp-partition
+ #:export (root-offset
+ root-label
+
+ esp-partition
root-partition
- hurd-disk-image
efi-disk-image
iso9660-image
@@ -101,20 +103,6 @@
(list #:make-device-nodes
make-hurd-device-nodes)))))
-(define hurd-disk-image
- (image
- (format 'disk-image)
- (target "i586-pc-gnu")
- (partitions
- (list (partition
- (size 'guess)
- (offset root-offset)
- (label root-label)
- (file-system "ext2")
- (file-system-options '("-o" "hurd" "-O" "ext_attr"))
- (flags '(boot))
- (initializer hurd-initialize-root-partition))))))
-
(define efi-disk-image
(image
(format 'disk-image)
@@ -569,7 +557,8 @@ addition of the <image> record."
(_ (cond
((and target
(hurd-triplet? target))
- hurd-disk-image)
+ (module-ref (resolve-interface '(gnu system images hurd))
+ 'hurd-disk-image))
(else
efi-disk-image)))))
diff --git a/gnu/system/images/hurd.scm b/gnu/system/images/hurd.scm
new file mode 100644
index 0000000000..4417f03cc8
--- /dev/null
+++ b/gnu/system/images/hurd.scm
@@ -0,0 +1,76 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu system images hurd)
+ #:use-module (guix gexp)
+ #:use-module (gnu bootloader)
+ #:use-module (gnu bootloader grub)
+ #:use-module (gnu image)
+ #:use-module (gnu packages ssh)
+ #:use-module (gnu services)
+ #:use-module (gnu services ssh)
+ #:use-module (gnu system)
+ #:use-module (gnu system file-systems)
+ #:use-module (gnu system hurd)
+ #:use-module (gnu system image)
+ #:export (hurd-barebones-os
+ hurd-disk-image
+ hurd-barebones-disk-image))
+
+(define hurd-barebones-os
+ (operating-system
+ (inherit %hurd-default-operating-system)
+ (bootloader (bootloader-configuration
+ (bootloader grub-minimal-bootloader)
+ (target "/dev/sdX")))
+ (file-systems (cons (file-system
+ (device (file-system-label "my-root"))
+ (mount-point "/")
+ (type "ext2"))
+ %base-file-systems))
+ (host-name "guixygnu")
+ (timezone "Europe/Amsterdam")
+ (packages (cons openssh-sans-x %base-packages/hurd))
+ (services (cons (service openssh-service-type
+ (openssh-configuration
+ (openssh openssh-sans-x)
+ (use-pam? #f)
+ (port-number 2222)
+ (permit-root-login #t)
+ (allow-empty-passwords? #t)
+ (password-authentication? #t)))
+ %base-services/hurd))))
+
+(define hurd-disk-image
+ (image
+ (format 'disk-image)
+ (target "i586-pc-gnu")
+ (partitions
+ (list (partition
+ (size 'guess)
+ (offset root-offset)
+ (label root-label)
+ (file-system "ext2")
+ (flags '(boot))
+ (initializer (gexp initialize-root-partition)))))))
+
+(define hurd-barebones-disk-image
+ (image
+ (inherit hurd-disk-image)
+ (name 'hurd-barebones-disk-image)
+ (operating-system hurd-barebones-os)))
--
2.26.2
L
L
Ludovic Courtès wrote on 23 Jun 2020 23:47
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87sgelxxnd.fsf@gnu.org
Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (11 lines)
> This moves hurd-disk-image to a dedicated file. It also defines a default
> operating-system so that the image can be built standalone.
>
> * gnu/system/images/hurd.scm: New file,
> * gnu/local.mk (GNU_SYSTEM_MODULES): add it.
> * gnu/image.scm (<image>)[name]: New field.
> * gnu/system/image.scm (root-offset, root-label): Export it,
> (hurd-disk-image): remove it as this is now defined in the new, Hurd dedicated
> file above,
> (find-image): adapt to avoid loop dependency.

LGTM!

It would be good to check by running “make as-derivation” if gnu/image/*
gets included.

Ludo’.
L
L
Ludovic Courtès wrote on 23 Jun 2020 23:48
Re: [bug#41839] [PATCH 2/2] ci: Build Guix System images.
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)
87o8p9xxkn.fsf@gnu.org
Mathieu Othacehe <m.othacehe@gmail.com> skribis:

Toggle quote (11 lines)
> Build a list of Guix System images. For now, this list only contains the Hurd
> barebones Guix System image.
>
> * gnu/ci.scm (%guixsd-supported-systems): Remove "armhf-linux",
> (%u-boot-systems): remove unused variable,
> (%guix-system-images): new variable
> (qemu-jobs): rename to "image-jobs" and build the Guix Systems listed in the
> new "%guix-system-images" variable,
> (system-test-jobs): adapt accordingly,
> (hydra-jobs): ditto.

Cool.

Toggle quote (17 lines)
> + (if (member system %guix-system-supported-systems)
> + `(,(->job 'usb-image
> + (build-image
> + (image
> + (inherit efi-disk-image)
> + (size (* 1500 MiB))
> + (operating-system installation-os))))
> + ,(->job 'iso9660-image
> + (build-image
> + (image
> + (inherit iso9660-image)
> + (operating-system installation-os))))
> + ,@(map (lambda (image)
> + (->job (image-name image) (build-image image)))
> + %guix-system-images))
> '()))

I think the effect is that we’ll also cross-build GNU/Hurd images from
i686-linux, right? It would be nice to avoid that and only cross-build
from x86_64.

Otherwise LGTM, thank you!

Ludo’.
M
M
Mathieu Othacehe wrote on 25 Jun 2020 11:54
Re: [bug#41839] [PATCH 1/2] image: Move hurd image definition to a dedicated file.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 41839@debbugs.gnu.org)
87y2obcvx5.fsf@gmail.com
Hey,

Toggle quote (3 lines)
> It would be good to check by running “make as-derivation” if gnu/image/*
> gets included.

Yep, its included. Thanks for reviewing!

Mathieu
M
M
Mathieu Othacehe wrote on 25 Jun 2020 12:02
Re: [bug#41839] [PATCH 2/2] ci: Build Guix System images.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 41839-done@debbugs.gnu.org)
87r1u3cvk9.fsf@gnu.org
Hey,

Toggle quote (4 lines)
> I think the effect is that we’ll also cross-build GNU/Hurd images from
> i686-linux, right? It would be nice to avoid that and only cross-build
> from x86_64.

Yes, I fixed that (in two tries actually :p). Now working on adding this
image to the website.

Thanks,

Mathieu
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 41839@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 41839
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch