[PATCH] DRAFT services: Add 'hurd-in-vm service-type'.

  • Done
  • quality assurance status badge
Details
5 participants
  • Diego Nicola Barbato
  • Jan (janneke) Nieuwenhuizen
  • Ludovic Courtès
  • Marius Bakke
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Jan (janneke) Nieuwenhuizen
Severity
normal
J
J
Jan (janneke) Nieuwenhuizen wrote on 10 Jun 2020 10:54
(address . guix-patches@gnu.org)
20200610085441.890-1-janneke@gnu.org
TODO: Figure-out how to run this hurd VM inside a VM.

Using

Toggle snippet (42 lines)
diff --git a/gnu/system/examples/bare-bones.tmpl b/gnu/system/examples/bare-bones.tmpl
index 1035ab1d60..40fb354ea4 100644
--- a/gnu/system/examples/bare-bones.tmpl
+++ b/gnu/system/examples/bare-bones.tmpl
@@ -5,6 +5,8 @@
(use-service-modules networking ssh)
(use-package-modules screen ssh)

+(use-service-modules hurd virtualization)
+
(operating-system
(host-name "komputilo")
(timezone "Europe/Berlin")
@@ -44,8 +46,12 @@
;; Add services to the baseline: a DHCP client and
;; an SSH server.
(services (append (list (service dhcp-client-service-type)
+ (service hurd-in-vm-service-type)
(service openssh-service-type
(openssh-configuration
(openssh openssh-sans-x)
- (port-number 2222))))
+ (port-number 2222)
+ (permit-root-login #t)
+ (allow-empty-passwords? #t)
+ (password-authentication? #t))))
%base-services)))
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 441b1eb7e0..e3560b80b7 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -878,7 +878,7 @@ functionality of the kernel Linux.")))
(define vm-command
#~(list
(string-append #$qemu "/bin/qemu-system-i386")
- #$@(if (file-exists? "/dev/kvm") '("-enable-kvm") '())
+ ;;huh? breaks hurd in bare-bones VM #$@(if (file-exists? "/dev/kvm") '("-enable-kvm") '())
"-m" (number->string #$memory-size)
#$@options
#+image))

and doing something like

./pre-inst-env guix system vm gnu/system/examples/bare-bones.tmpl --no-offload
/gnu/store/96wh3jwsla4p6d4s547mmqxsi4qbbc0r-run-vm.sh -m 2G \
--device rtl8139,netdev=net0 \
--netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:5900-:5900

nicely starts a bare-bones VM with the the hurd-in-vm service inside, but I
cannot seem to connect to the Hurd VM it in any way. Appending
",hostfwd=tcp:127.0.0.1:20022-:20022" (to directly ssh into the Hurd) even
blocks me from ssh'ing into the GNU/linux host VM.

hurd-in-vm works beautifully when added to my system configuration and
reconfiguring.

* gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
hurd-vm-disk-image): New procedures.
(%hurd-in-vm-operating-system, hurd-in-vm-service-type): New variable.
(<hurd-in-vm-configuration>): New record type.
* doc/guix.texi (Virtualization Services): Document it.
---
doc/guix.texi | 61 ++++++++++++++
gnu/services/virtualization.scm | 140 ++++++++++++++++++++++++++++++--
2 files changed, 194 insertions(+), 7 deletions(-)

Toggle diff (248 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 15e077a41c..cae77288f4 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24583,6 +24583,67 @@ Return true if @var{obj} is a platform object.
Return the name of @var{platform}---a string such as @code{"arm"}.
@end deffn
+
+@subsubheading The Hurd in a Virtual Machine
+
+@cindex @code{hurd}
+@cindex the Hurd
+
+Service @code{hurd-in-vm} provides support for running a Virtual Machine
+with the GNU@tie{}Hurd.
+
+@defvr {Scheme Variable} hurd-in-vm-service-type
+This is the type of the Hurd in a Virtual Machine service. Its value
+must be a @code{hurd-in-vm-configuration} object, which specifies the
+operating system (@pxref{operating-system Reference}) and the disk size
+for the Hurd Virtual Machine, the QEMU package to use as well as the
+options for running it.
+
+For example:
+
+@lisp
+(service hurd-in-vm-service-type
+ (hurd-in-vm-configuration
+ (disk-size (* 5000 (expt 2 20))) ;5G
+ (memory-size 1024))) ;1024MiB
+@end lisp
+
+would create a disk image big enough to build GNU@tie{}Hello, with some
+extra memory.
+@end defvr
+
+@deftp {Data Type} hurd-in-vm-configuration
+The data type representing the configuration for
+@code{hurd-in-vm-service-type}.
+
+@table @asis
+@item @code{os} (default: @var{%hurd-in-vm-operating-system})
+The operating system to instantiate. This default is bare-bones with a
+permissive OpenSSH secure shell daemon listening on port 2222
+(@pxref{Networking Services, @code{openssh-service-type}}).
+
+@item @code{qemu} (default: @code{qemu-minimal})
+The QEMU package to use.
+
+@item @code{image} (default: @var{hurd-vm-disk-image})
+The procedure used to build the disk-image built from this
+configuration.
+
+@item @code{disk-size} (default: @code{'guess})
+The size of the disk image.
+
+@item @code{memory-size} (default: @code{512})
+The memory size of the Virtual Machine in mebibytes.
+
+@item @code{options} (default: @code{'("--device"} @code{"rtl8139,netdev=net0"} @
+ @code{"--netdev"} @
+ @code{"user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222,hostfwd=tcp:127.0.0.1:25900-:5900"} @
+ @code{"--snapshot"} @
+ @code{"--hda")})
+The extra options for running QEMU.
+@end table
+@end deftp
+
@node Version Control Services
@subsection Version Control Services
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 989e439d5d..441b1eb7e0 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2017 Ryan Moe <ryan.moe@gmail.com>
;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -18,24 +19,41 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (gnu services virtualization)
- #:use-module (gnu services)
- #:use-module (gnu services configuration)
+ #:use-module (gnu bootloader)
+ #:use-module (gnu bootloader grub)
+ #:use-module (gnu image)
+ #:use-module (gnu packages admin)
+ #:use-module (gnu packages ssh)
+ #:use-module (gnu packages virtualization)
#:use-module (gnu services base)
+ #:use-module (gnu services configuration)
#:use-module (gnu services dbus)
#:use-module (gnu services shepherd)
- #:use-module (gnu system shadow)
+ #:use-module (gnu services ssh)
+ #:use-module (gnu services)
#:use-module (gnu system file-systems)
- #:use-module (gnu packages admin)
- #:use-module (gnu packages virtualization)
- #:use-module (guix records)
+ #:use-module (gnu system hurd)
+ #:use-module (gnu system image)
+ #:use-module (gnu system shadow)
+ #:use-module (gnu system)
+ #:use-module (guix derivations)
#:use-module (guix gexp)
+ #:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix records)
+ #:use-module (guix store)
+ #:use-module (guix utils)
+
#:use-module (srfi srfi-9)
#:use-module (srfi srfi-26)
#:use-module (rnrs bytevectors)
#:use-module (ice-9 match)
- #:export (libvirt-configuration
+ #:export (%hurd-in-vm-operating-system
+ hurd-in-vm-configuration
+ hurd-in-vm-service-type
+
+ libvirt-configuration
libvirt-service-type
virtlog-configuration
virtlog-service-type
@@ -773,3 +791,111 @@ given QEMU package."
"This service supports transparent emulation of binaries
compiled for other architectures using QEMU and the @code{binfmt_misc}
functionality of the kernel Linux.")))
+
+
+;;;
+;;; The Hurd in VM service.
+;;;
+
+(define* (disk-image os #:key (image-size 'guess) target)
+ "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
+ (with-store store
+ (run-with-store store
+ (let ((file-system-type "ext2"))
+ (mlet* %store-monad
+ ((base-image (find-image file-system-type))
+ (sys (lower-object
+ (system-image
+ (image
+ (inherit base-image)
+ (size image-size)
+ (operating-system os)))))
+ (drvs (mapm/accumulate-builds lower-object (list sys)))
+ (% (built-derivations drvs)))
+ (let ((output (derivation->output-path sys)))
+ (return output))))
+ #:target target)))
+
+(define %hurd-in-vm-operating-system
+ (operating-system
+ (inherit %hurd-default-operating-system)
+ (host-name "guixydevel")
+ (timezone "Europe/Amsterdam")
+ (bootloader (bootloader-configuration
+ (bootloader grub-minimal-bootloader)
+ (target "/dev/vda")
+ (timeout 0)))
+ (services (cons*
+ (service openssh-service-type
+ (openssh-configuration
+ (openssh openssh-sans-x)
+ (use-pam? #f)
+ (port-number 2222)
+ (permit-root-login #t)
+ (allow-empty-passwords? #t)
+ (password-authentication? #t)))
+ %base-services/hurd))))
+
+(define-record-type* <hurd-in-vm-configuration>
+ hurd-in-vm-configuration make-hurd-in-vm-configuration
+ hurd-in-vm-configuration?
+ (os hurd-in-vm-configuration-os ;<operating-system>
+ (default %hurd-in-vm-operating-system))
+ (qemu hurd-in-vm-configuration-qemu ;<package>
+ (default qemu-minimal))
+ (image hurd-in-vm-configuration-image ;string
+ (thunked)
+ (default (hurd-vm-disk-image this-record)))
+ (disk-size hurd-in-vm-configuration-disk-size ;number or 'guess
+ (default 'guess))
+ (memory-size hurd-in-vm-configuration-memory-size ;number
+ (default 512))
+ (options hurd-in-vm-configuration-options ;list of string
+ (default
+ `("--device" "rtl8139,netdev=net0"
+ "--netdev" (string-append
+ "user,id=net0"
+ ",hostfwd=tcp:127.0.0.1:20022-:2222"
+ ",hostfwd=tcp:127.0.0.1:25900-:5900")
+ "--snapshot"
+ "--hda"))))
+
+(define (hurd-vm-disk-image config)
+ "Return a disk-image for the Hurd according to CONFIG."
+ (let ((os (hurd-in-vm-configuration-os config))
+ (disk-size (hurd-in-vm-configuration-disk-size config))
+ (target (and (not (%current-target-system)) "i586-pc-gnu")))
+ (disk-image os #:target target #:image-size disk-size)))
+
+(define (hurd-in-vm-shepherd-service config)
+ "Return a <shepherd-service> for a Hurd in a Virtual Machine with CONFIG."
+
+ (let ((image (hurd-in-vm-configuration-image config))
+ (qemu (hurd-in-vm-configuration-qemu config))
+ (memory-size (hurd-in-vm-configuration-memory-size config))
+ (options (hurd-in-vm-configuration-options config)))
+
+ (define vm-command
+ #~(list
+ (string-append #$qemu "/bin/qemu-system-i386")
+ #$@(if (file-exists? "/dev/kvm") '("-enable-kvm") '())
+ "-m" (number->string #$memory-size)
+ #$@options
+ #+image))
+
+ (list
+ (shepherd-service
+ (documentation "Run the Hurd in a Virtual Machine.")
+ (provision '(hurd-in-vm))
+ (requirement '(networking))
+ (start #~(make-forkexec-constructor #$vm-command))
+ (stop #~(make-kill-destructor))))))
+
+(define hurd-in-vm-service-type
+ (service-type
+ (name 'hurd-in-vm)
+ (extensions (list (service-extension shepherd-root-service-type
+ hurd-in-vm-shepherd-service)))
+ (default-value (hurd-in-vm-configuration))
+ (description
+ "Provide a Virtual Machine running the GNU Hurd.")))
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
M
M
Mathieu Othacehe wrote on 10 Jun 2020 13:34
(name . Jan (janneke) Nieuwenhuizen)(address . janneke@gnu.org)(address . 41785@debbugs.gnu.org)
87r1unm9xy.fsf@gnu.org
Hello Jan,

Toggle quote (2 lines)
> TODO: Figure-out how to run this hurd VM inside a VM.

Just to clarify things here, we now have the "disk-image" command that
produces a Hurd bootable disk-image. The "vm-image" command is almost
working, and should produce a very similar result.

The "vm" command that produces a vm-image and spawns a VM isn't working
yet with the Hurd. This command is for now passing directly "kernel" and
"initrd" arguments to QEMU, so a few adaptations will be needed.

Now the goad of hurd-in-vm-service-type is to provide a service that
does almost the same thing as the "vm" command, but as a service, right?

So, I don't get why would we need to run a Hurd VM inside a VM. I've
been struggling a lot with running nested layers of virtualization (for
system generation before the recent patches), and the result is often
too slow to be really usable.

Thanks,

Mathieu
L
L
Ludovic Courtès wrote on 11 Jun 2020 21:43
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87tuzh9yn3.fsf@gnu.org
Hi,

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (11 lines)
> Just to clarify things here, we now have the "disk-image" command that
> produces a Hurd bootable disk-image. The "vm-image" command is almost
> working, and should produce a very similar result.
>
> The "vm" command that produces a vm-image and spawns a VM isn't working
> yet with the Hurd. This command is for now passing directly "kernel" and
> "initrd" arguments to QEMU, so a few adaptations will be needed.
>
> Now the goad of hurd-in-vm-service-type is to provide a service that
> does almost the same thing as the "vm" command, but as a service, right?

That’s my understanding.

Toggle quote (5 lines)
> So, I don't get why would we need to run a Hurd VM inside a VM. I've
> been struggling a lot with running nested layers of virtualization (for
> system generation before the recent patches), and the result is often
> too slow to be really usable.

I guess it’s useful if you want to test a GNU/Linux system that uses the
VM service, as in:

guix system vm my-gnu+linux.scm

where ‘my-gnu+linux.scm’ contains:

(services (cons (service hurd-vm-service-type)
…))

I suppose it should just work, though probably without ‘-enable-kvm’;
janneke?

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 11 Jun 2020 21:59
(name . Jan (janneke) Nieuwenhuizen)(address . janneke@gnu.org)
87eeql9xvt.fsf@gnu.org
Hey!

That was fast! :-)

"Jan (janneke) Nieuwenhuizen" <janneke@gnu.org> skribis:

Toggle quote (12 lines)
> and doing something like
>
> ./pre-inst-env guix system vm gnu/system/examples/bare-bones.tmpl --no-offload
> /gnu/store/96wh3jwsla4p6d4s547mmqxsi4qbbc0r-run-vm.sh -m 2G \
> --device rtl8139,netdev=net0 \
> --netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:5900-:5900
>
> nicely starts a bare-bones VM with the the hurd-in-vm service inside, but I
> cannot seem to connect to the Hurd VM it in any way. Appending
> ",hostfwd=tcp:127.0.0.1:20022-:20022" (to directly ssh into the Hurd) even
> blocks me from ssh'ing into the GNU/linux host VM.

Weird.

Toggle quote (9 lines)
> hurd-in-vm works beautifully when added to my system configuration and
> reconfiguring.
>
> * gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
> hurd-vm-disk-image): New procedures.
> (%hurd-in-vm-operating-system, hurd-in-vm-service-type): New variable.
> (<hurd-in-vm-configuration>): New record type.
> * doc/guix.texi (Virtualization Services): Document it.

[…]

Toggle quote (8 lines)
> +@subsubheading The Hurd in a Virtual Machine
> +
> +@cindex @code{hurd}
> +@cindex the Hurd
> +
> +Service @code{hurd-in-vm} provides support for running a Virtual Machine
> +with the GNU@tie{}Hurd.

“… support for running GNU/Hurd in a virtual machine (VM). The virtual
machine is a Shepherd service that can be controlled with commands such
as:

@example
herd stop hurd-vm
@end example

The given GNU/Hurd operating system configuration is cross-compiled.”

Nitpick: I’d call it “hurd-vm”, because it runs a Hurd VM. :-)

It’s a volatile VM, due to the use of ‘-snapshot’, right?

(The Hurd actually has “sub-Hurds”¹ and “neighborhurds”². I wonder if
it’s our duty to coin another term… a guesthurd? a visithurd?)


Toggle quote (3 lines)
> +(define* (disk-image os #:key (image-size 'guess) target)
> + "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
> + (with-store store
^
In general, procedures should talk to the user-provided store and never
open a new connection. They should also never call ‘build-derivations’
explicitly, the only exception so far being the graft implementation.

So you can drop ‘with-store’ here, and then:

Toggle quote (15 lines)
> + (run-with-store store
> + (let ((file-system-type "ext2"))
> + (mlet* %store-monad
> + ((base-image (find-image file-system-type))
> + (sys (lower-object
> + (system-image
> + (image
> + (inherit base-image)
> + (size image-size)
> + (operating-system os)))))
> + (drvs (mapm/accumulate-builds lower-object (list sys)))
> + (% (built-derivations drvs)))
> + (let ((output (derivation->output-path sys)))
> + (return output))))

Mathieu, can we make ‘find-image’ non-monadic? It really shouldn’t be
because it doesn’t interact with the store. It can take an optional
‘system’ parameter if we want.

So, assuming ‘find-image’ is non-monadic, the code above becomes
something like:

(system-image
(image (inherit base-image)
(size image-size)
(operating-system
(with-parameters ((%current-target-system "i586-pc-gnu"))
os))))

Toggle quote (20 lines)
> +(define %hurd-in-vm-operating-system
> + (operating-system
> + (inherit %hurd-default-operating-system)
> + (host-name "guixydevel")
> + (timezone "Europe/Amsterdam")
> + (bootloader (bootloader-configuration
> + (bootloader grub-minimal-bootloader)
> + (target "/dev/vda")
> + (timeout 0)))
> + (services (cons*
> + (service openssh-service-type
> + (openssh-configuration
> + (openssh openssh-sans-x)
> + (use-pam? #f)
> + (port-number 2222)
> + (permit-root-login #t)
> + (allow-empty-passwords? #t)
> + (password-authentication? #t)))
> + %base-services/hurd))))

I understand the need to factorize useful configs, but IMO it doesn’t
belong here. So I’d just leave it out. There’s already
‘%hurd-default-operating-system’ that does the heavy lifting anyway.

Toggle quote (9 lines)
> +(define hurd-in-vm-service-type
> + (service-type
> + (name 'hurd-in-vm)
> + (extensions (list (service-extension shepherd-root-service-type
> + hurd-in-vm-shepherd-service)))
> + (default-value (hurd-in-vm-configuration))
> + (description
> + "Provide a Virtual Machine running the GNU Hurd.")))

Being pedantic: s|the GNU Hurd|GNU/Hurd|. :-)

Otherwise looks great to me, thank you!

Ludo’.
J
J
Jan Nieuwenhuizen wrote on 11 Jun 2020 21:59
(name . Ludovic Courtès)(address . ludo@gnu.org)
87ftb15q68.fsf@gnu.org
Ludovic Courtès writes:

Hello,

Toggle quote (6 lines)
> Mathieu Othacehe <othacehe@gnu.org> skribis:
>
>> Just to clarify things here, we now have the "disk-image" command that
>> produces a Hurd bootable disk-image. The "vm-image" command is almost
>> working, and should produce a very similar result.

(yes)

Toggle quote (9 lines)
>> The "vm" command that produces a vm-image and spawns a VM isn't working
>> yet with the Hurd. This command is for now passing directly "kernel" and
>> "initrd" arguments to QEMU, so a few adaptations will be needed.
>>
>> Now the goad of hurd-in-vm-service-type is to provide a service that
>> does almost the same thing as the "vm" command, but as a service, right?
>
> That’s my understanding.

Indeed, mine too.

Toggle quote (18 lines)
>> So, I don't get why would we need to run a Hurd VM inside a VM. I've
>> been struggling a lot with running nested layers of virtualization (for
>> system generation before the recent patches), and the result is often
>> too slow to be really usable.
>
> I guess it’s useful if you want to test a GNU/Linux system that uses the
> VM service, as in:
>
> guix system vm my-gnu+linux.scm
>
> where ‘my-gnu+linux.scm’ contains:
>
> (services (cons (service hurd-vm-service-type)
> …))
>
> I suppose it should just work, though probably without ‘-enable-kvm’;
> janneke?

Well -- yes. That was my thinking.

We would not normally use such a VM-in-VM thing; when you want a Hurd VM
using the "vm" command would be great (or as now, create a disk image
and start QEMU).

However, for writing and debugging (testing?) this Hurd-in-VM service, I
thought adding it to a linux bare-bones.tmpl and using guix vm on that
was a nice and "trivial" way to test it. That proved so troublesome
that I added it to my system.

As I understand it the intended use for this service is to have a real
simple way to add a Hurd VM to some build machines, so that we can
easily create a herd of Hurd-VMs to aid the Hurd work. Ludo?

Greetings,
Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
M
M
Marius Bakke wrote on 11 Jun 2020 22:01
(address . 41785@debbugs.gnu.org)
87h7vhfk2y.fsf@gnu.org
Mathieu Othacehe <othacehe@gnu.org> writes:

Toggle quote (5 lines)
> So, I don't get why would we need to run a Hurd VM inside a VM. I've
> been struggling a lot with running nested layers of virtualization (for
> system generation before the recent patches), and the result is often
> too slow to be really usable.

Note that recent processors support nested layers of virtualization
natively with little overhead, but it's disabled by default.

For an Intel processor, it can be enabled by adding this to your system
configuration:

(kernel-arguments (cons "kvm_intel.nested=1" %default-kernel-arguments))

The corresponding AMD kernel module is called "kvm_amd".
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAl7ijZUACgkQoqBt8qM6
VPrZKggAw5jBvmJSnZkR+RCnKW/mSWf3bxteaO5RBGcXQuYHyZBRd0WviINXeGUr
mVoPrZ5yXLZdSQgaLj6Cfcgr78k9DLau0FYOwODc28rz8wsGj62IJB3XJe16Svps
TBsQHGluncnyBp5g0trO9acG2zKhaxF9tFX8e5ZT+hleNk0WcpXV7jY+ubrOjINQ
yleXOQpGcUnSD2LzJCwkj9cTjwGPIXSg1MH+G2rXsBnHoVuvLM/Xo2W4Kf2Hl+gl
+iMITdLfGDfHjbVOFtcJ7T7C7FerYA4sWcrIYwpOwfDXwnnVjf5zNrpbyjJpvelL
UHBNlFxVoKJlmnq6g5rTl39Ly109lQ==
=2aVn
-----END PGP SIGNATURE-----

J
J
Jan Nieuwenhuizen wrote on 11 Jun 2020 23:57
(name . Ludovic Courtès)(address . ludo@gnu.org)
87wo4d2rm5.fsf@gnu.org
Ludovic Courtès writes:

Hello,

Toggle quote (2 lines)
> That was fast! :-)

Yeah...we need this, right ;)

Toggle quote (45 lines)
> "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org> skribis:
>
>> and doing something like
>>
>> ./pre-inst-env guix system vm gnu/system/examples/bare-bones.tmpl --no-offload
>> /gnu/store/96wh3jwsla4p6d4s547mmqxsi4qbbc0r-run-vm.sh -m 2G \
>> --device rtl8139,netdev=net0 \
>> --netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:5900-:5900
>>
>> nicely starts a bare-bones VM with the the hurd-in-vm service inside, but I
>> cannot seem to connect to the Hurd VM it in any way. Appending
>> ",hostfwd=tcp:127.0.0.1:20022-:20022" (to directly ssh into the Hurd) even
>> blocks me from ssh'ing into the GNU/linux host VM.
>
> Weird.
>
>> hurd-in-vm works beautifully when added to my system configuration and
>> reconfiguring.
>>
>> * gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
>> hurd-vm-disk-image): New procedures.
>> (%hurd-in-vm-operating-system, hurd-in-vm-service-type): New variable.
>> (<hurd-in-vm-configuration>): New record type.
>> * doc/guix.texi (Virtualization Services): Document it.
>
> […]
>
>> +@subsubheading The Hurd in a Virtual Machine
>> +
>> +@cindex @code{hurd}
>> +@cindex the Hurd
>> +
>> +Service @code{hurd-in-vm} provides support for running a Virtual Machine
>> +with the GNU@tie{}Hurd.
>
> “… support for running GNU/Hurd in a virtual machine (VM). The virtual
> machine is a Shepherd service that can be controlled with commands such
> as:
>
> @example
> herd stop hurd-vm
> @end example
>
> The given GNU/Hurd operating system configuration is cross-compiled.”

Nice, thanks!

Toggle quote (2 lines)
> Nitpick: I’d call it “hurd-vm”, because it runs a Hurd VM. :-)

Done!

Toggle quote (2 lines)
> It’s a volatile VM, due to the use of ‘-snapshot’, right?

By default: Yes. That seemed more ready-to-use. A stateful VM image
would need to an out-of-store, writable copy. You can actually do that
and modify the hurd-vm-configuration.

Toggle quote (6 lines)
> (The Hurd actually has “sub-Hurds”¹ and “neighborhurds”². I wonder if
> it’s our duty to coin another term… a guesthurd? a visithurd?)
>
> ¹ https://www.gnu.org/software/hurd/hurd/subhurd.html
> ² https://www.gnu.org/software/hurd/hurd/neighborhurd.html

Oh, that's cool! Associating along from the neighborhurd pun, what
about a "childhurd" (as a pun on childhood -- only needed while the Hurd
is growing up)?

"herd start childhurd" -- hmm? In the updated patch, I still have
hurd-vm. If we do our duty and coin "childhurd", should I just
s/hurd-vm/childhurd/g ?

Toggle quote (29 lines)
>> +(define* (disk-image os #:key (image-size 'guess) target)
>> + "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
>> + (with-store store
> ^
> In general, procedures should talk to the user-provided store and never
> open a new connection. They should also never call ‘build-derivations’
> explicitly, the only exception so far being the graft implementation.
>
> So you can drop ‘with-store’ here, and then:
>
>> + (run-with-store store
>> + (let ((file-system-type "ext2"))
>> + (mlet* %store-monad
>> + ((base-image (find-image file-system-type))
>> + (sys (lower-object
>> + (system-image
>> + (image
>> + (inherit base-image)
>> + (size image-size)
>> + (operating-system os)))))
>> + (drvs (mapm/accumulate-builds lower-object (list sys)))
>> + (% (built-derivations drvs)))
>> + (let ((output (derivation->output-path sys)))
>> + (return output))))
>
> Mathieu, can we make ‘find-image’ non-monadic? It really shouldn’t be
> because it doesn’t interact with the store. It can take an optional
> ‘system’ parameter if we want.

It seems that "just works". I've made that change in a separate patch
(attached).

Toggle quote (10 lines)
> So, assuming ‘find-image’ is non-monadic, the code above becomes
> something like:
>
> (system-image
> (image (inherit base-image)
> (size image-size)
> (operating-system
> (with-parameters ((%current-target-system "i586-pc-gnu"))
> os))))

Hmm...I don't think that I understand. This

Toggle snippet (12 lines)
(define* (disk-image os #:key (image-size 'guess) target)
"Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
(let ((base-image (find-image "ext2")))
(system-image
(image (inherit base-image)
(size image-size)
(operating-system
(with-parameters ((%current-target-system target))
os))))))


gives

Toggle snippet (18 lines)
$ ~/src/guix/master/pre-inst-env guix system build dundal.scm
%default-substitute-urls:("https://ci.guix.gnu.org")
Backtrace:
In ice-9/boot-9.scm:
1736:10 4 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In unknown file:
3 (apply-smob/0 #<thunk 7f4ce92e3980>)
In ice-9/boot-9.scm:
718:2 2 (call-with-prompt _ _ #<procedure default-prompt-handler (k proc)>)
In ice-9/eval.scm:
619:8 1 (_ #(#(#<directory (guile-user) 7f4ce8f05f00>)))
In guix/ui.scm:
1945:12 0 (run-guix-command _ . _)

guix/ui.scm:1945:12: In procedure run-guix-command:
In procedure operating-system-file-systems: Wrong type argument: #<<parameterized> bindings: ((#<<parameter> 7f4ce7c23740 proc: #<procedure 7f4ce7c28200 at ice-9/boot-9.scm:1299:5 () | (x)>> #<procedure 7f4cd32f83c0 at gnu/services/virtualization.scm:806:14 ()>)) thunk: #<procedure 7f4cd32f8340 at gnu/services/virtualization.scm:806:14 ()>>

...I could do with some help here.

Toggle quote (1 lines)
>> +(define %hurd-in-vm-operating-system
[..]
Toggle quote (4 lines)
>> + (operating-system
>> + (service openssh-service-type
>> + (openssh-configuration
>> + (openssh openssh-sans-x)
[..]
Toggle quote (6 lines)
>> + %base-services/hurd))))
>
> I understand the need to factorize useful configs, but IMO it doesn’t
> belong here. So I’d just leave it out. There’s already
> ‘%hurd-default-operating-system’ that does the heavy lifting anyway.

Sure, removed! Users will most probably want to add an openssh server
using openssh-sans-x; but I guess that's something for a blog post or
cookbook then.

Toggle quote (13 lines)
>> +(define hurd-in-vm-service-type
>> + (service-type
>> + (name 'hurd-in-vm)
>> + (extensions (list (service-extension shepherd-root-service-type
>> + hurd-in-vm-shepherd-service)))
>> + (default-value (hurd-in-vm-configuration))
>> + (description
>> + "Provide a Virtual Machine running the GNU Hurd.")))
>
> Being pedantic: s|the GNU Hurd|GNU/Hurd|. :-)
>
> Otherwise looks great to me, thank you!

Great; thanks...find two new patches attached.

Janneke
From b01b8d2a46a6a04cb8f09d74c06cbbc82878f070 Mon Sep 17 00:00:00 2001
From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
Date: Thu, 11 Jun 2020 22:52:12 +0200
Subject: [PATCH v2 1/2] image: Make 'find-image' non-monadic.

* gnu/system/image.scm (find-image): Make non-monadic.
* gnu/tests/install.scm (run-install): Update caller.
* guix/scripts/system.scm (perform-action): Likewise.
---
gnu/system/image.scm | 21 ++++++++++-----------
gnu/tests/install.scm | 5 +++--
guix/scripts/system.scm | 3 ++-
3 files changed, 15 insertions(+), 14 deletions(-)

Toggle diff (84 lines)
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index a0e6bf31f1..66a9f6b335 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -1,5 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -551,16 +552,14 @@ image, depending on IMAGE format."
"Find and return an image that could match the given FILE-SYSTEM-TYPE. This
is useful to adapt to interfaces written before the addition of the <image>
record."
- (mlet %store-monad ((target (current-target-system)))
- (mbegin %store-monad
- (return
- (match file-system-type
- ("iso9660" iso9660-image)
- (_ (cond
- ((and target
- (hurd-triplet? target))
- hurd-disk-image)
- (else
- efi-disk-image))))))))
+ (let ((target (%current-target-system)))
+ (match file-system-type
+ ("iso9660" iso9660-image)
+ (_ (cond
+ ((and target
+ (hurd-triplet? target))
+ hurd-disk-image)
+ (else
+ efi-disk-image))))))
;;; image.scm ends here
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 6bd8c7d3d2..d18e33179f 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -3,6 +3,7 @@
;;; Copyright © 2017, 2019 Tobias Geerinckx-Rice <me@tobias.gr>
;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2020 Danny Milosavljevic <dannym@scratchpost.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -228,8 +229,8 @@ packages defined in installation-os."
(mlet* %store-monad ((_ (set-grafting #f))
(system (current-system))
(target (operating-system-derivation target-os))
- (base-image (find-image
- installation-disk-image-file-system-type))
+ (base-image -> (find-image
+ installation-disk-image-file-system-type))
;; Since the installation system has no network access,
;; we cheat a little bit by adding TARGET to its GC
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 3d7aa77cb7..fc92b9f07b 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2017, 2019 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2019 Christopher Baines <mail@cbaines.net>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -801,7 +802,7 @@ static checks."
(check-initrd-modules os)))
(mlet* %store-monad
- ((image (find-image file-system-type))
+ ((image -> (find-image file-system-type))
(sys (system-derivation-for-action os image action
#:file-system-type file-system-type
#:image-size image-size
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
J
J
Jan Nieuwenhuizen wrote on 12 Jun 2020 08:39
(name . Marius Bakke)(address . marius@gnu.org)
87tuzgyehe.fsf@gnu.org
Marius Bakke writes:

Hello,

Toggle quote (10 lines)
> Mathieu Othacehe <othacehe@gnu.org> writes:
>
>> So, I don't get why would we need to run a Hurd VM inside a VM. I've
>> been struggling a lot with running nested layers of virtualization (for
>> system generation before the recent patches), and the result is often
>> too slow to be really usable.
>
> Note that recent processors support nested layers of virtualization
> natively with little overhead, but it's disabled by default.

Ah!

Toggle quote (5 lines)
> For an Intel processor, it can be enabled by adding this to your system
> configuration:
>
> (kernel-arguments (cons "kvm_intel.nested=1" %default-kernel-arguments))

Is there an obvious downside to enabling this?

Great...So on the host I did

Toggle snippet (6 lines)
root@dundal ~# rmmod kvm_intel
root@dundal ~# modprobe kvm_intel kvm_intel.nested=1
root@dundal ~# cat /sys/module/kvm_intel/parameters/nested
Y

and the interwebs told me that to start the VM, you have to add "-cpu
host"; so I started it using

Toggle snippet (3 lines)
/gnu/store/k2b7nx34cwyi6yk49wgy4hg9mrwcmll5-run-vm.sh -cpu host -m 2G -device rtl8139,netdev=net0 -netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:25900-:25900

and trying to "ssh -p 20022 localhost" from inside the bare-bones VM now
prints

Toggle snippet (10 lines)
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
qemu-system-i386: Slirp: Failed to send package, ret: -1
key_exchange_identification: read: Connection reset by peer
Connection reset by 127.0.0.1 port 20022

...something networky with QEMU. Ideas?

Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
J
J
Jan Nieuwenhuizen wrote on 12 Jun 2020 08:46
(name . Ludovic Courtès)(address . ludo@gnu.org)
87a718ye59.fsf@gnu.org
Jan Nieuwenhuizen writes:

Toggle quote (8 lines)
> Ludovic Courtès writes:
>
> Hello,
>
>> That was fast! :-)
>
> Yeah...we need this, right ;)

Sometimes...too fast :-(

So...there's still the run-with-store/with-parameters puzzle; but just
wanted to say that

Toggle quote (5 lines)
>>From e5bdf050f628cc7ea1b6bc4ccdcfeb757429820f Mon Sep 17 00:00:00 2001
> From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
> Date: Wed, 10 Jun 2020 00:10:28 +0200
> Subject: [PATCH v2 2/2] services: Add 'hurd-vm service-type'.

[...]

Toggle quote (2 lines)
> gnu/system/examples/bare-bones.tmpl | 8 +-

Oops...


Toggle quote (5 lines)
> diff --git a/gnu/system/examples/bare-bones.tmpl b/gnu/system/examples/bare-bones.tmpl
> index 1035ab1d60..1d4f7743ab 100644
> --- a/gnu/system/examples/bare-bones.tmpl
> +++ b/gnu/system/examples/bare-bones.tmpl

...without this bits, of course!!

Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
D
D
Diego Nicola Barbato wrote on 12 Jun 2020 12:51
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
87v9jw8slq.fsf@GlaDOS.home
Hey,

Jan Nieuwenhuizen <janneke@gnu.org> writes:

[...]

Toggle quote (20 lines)
> and the interwebs told me that to start the VM, you have to add "-cpu
> host"; so I started it using
>
> /gnu/store/k2b7nx34cwyi6yk49wgy4hg9mrwcmll5-run-vm.sh -cpu host -m 2G -device rtl8139,netdev=net0 -netdev user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:25900-:25900
>
>
> and trying to "ssh -p 20022 localhost" from inside the bare-bones VM now
> prints
>
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> qemu-system-i386: Slirp: Failed to send package, ret: -1
> key_exchange_identification: read: Connection reset by peer
> Connection reset by 127.0.0.1 port 20022
>
> ...something networky with QEMU. Ideas?

I've recently had intermittent issues with 'guix system vm' and hostfwd.
Does it help if you remove the '-nic user,model=virtio-net-pci' option
from (a copy of) the run-vm.sh script so that there's only one netdev?

Regards,

Diego
M
M
Mathieu Othacehe wrote on 12 Jun 2020 16:42
(name . Ludovic Courtès)(address . ludo@gnu.org)
87a718l500.fsf@gnu.org
Hello Ludo & janneke,

Toggle quote (4 lines)
> Mathieu, can we make ‘find-image’ non-monadic? It really shouldn’t be
> because it doesn’t interact with the store. It can take an optional
> ‘system’ parameter if we want.

Yes, you're right, passing 'target' to 'find-image' should be enough to
make it non-monadic.

Toggle quote (10 lines)
> So, assuming ‘find-image’ is non-monadic, the code above becomes
> something like:
>
> (system-image
> (image (inherit base-image)
> (size image-size)
> (operating-system
> (with-parameters ((%current-target-system "i586-pc-gnu"))
> os))))

I would prefer 'target' to be part of the image itself, as I proposed

There's no way for now, that the image is built without cross-compiling
for "i586-pc-gnu", so I think it could be part of the "image" record
itself.

WDYT?

Toggle quote (13 lines)
>> +(define hurd-in-vm-service-type
>> + (service-type
>> + (name 'hurd-in-vm)
>> + (extensions (list (service-extension shepherd-root-service-type
>> + hurd-in-vm-shepherd-service)))
>> + (default-value (hurd-in-vm-configuration))
>> + (description
>> + "Provide a Virtual Machine running the GNU Hurd.")))
>
> Being pedantic: s|the GNU Hurd|GNU/Hurd|. :-)
>
> Otherwise looks great to me, thank you!

Looks really nice to me too :)

Thanks,

Mathieu
L
L
Ludovic Courtès wrote on 12 Jun 2020 16:45
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
87imfw737z.fsf@gnu.org
Hi!

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

Toggle quote (6 lines)
>> It’s a volatile VM, due to the use of ‘-snapshot’, right?
>
> By default: Yes. That seemed more ready-to-use. A stateful VM image
> would need to an out-of-store, writable copy. You can actually do that
> and modify the hurd-vm-configuration.

It’s maybe worth mentioning in the manual.

Toggle quote (10 lines)
>> (The Hurd actually has “sub-Hurds”¹ and “neighborhurds”². I wonder if
>> it’s our duty to coin another term… a guesthurd? a visithurd?)
>>
>> ¹ https://www.gnu.org/software/hurd/hurd/subhurd.html
>> ² https://www.gnu.org/software/hurd/hurd/neighborhurd.html
>
> Oh, that's cool! Associating along from the neighborhurd pun, what
> about a "childhurd" (as a pun on childhood -- only needed while the Hurd
> is growing up)?

“Childhurd”, LOVE IT!

Toggle quote (4 lines)
> "herd start childhurd" -- hmm? In the updated patch, I still have
> hurd-vm. If we do our duty and coin "childhurd", should I just
> s/hurd-vm/childhurd/g ?

Shepherd services can have more than one name, so I propose to have both!

Toggle quote (44 lines)
>> So, assuming ‘find-image’ is non-monadic, the code above becomes
>> something like:
>>
>> (system-image
>> (image (inherit base-image)
>> (size image-size)
>> (operating-system
>> (with-parameters ((%current-target-system "i586-pc-gnu"))
>> os))))
>
> Hmm...I don't think that I understand. This
>
> (define* (disk-image os #:key (image-size 'guess) target)
> "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
> (let ((base-image (find-image "ext2")))
> (system-image
> (image (inherit base-image)
> (size image-size)
> (operating-system
> (with-parameters ((%current-target-system target))
> os))))))
>
>
> gives
>
> $ ~/src/guix/master/pre-inst-env guix system build dundal.scm
> %default-substitute-urls:("https://ci.guix.gnu.org")
> Backtrace:
> In ice-9/boot-9.scm:
> 1736:10 4 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
> In unknown file:
> 3 (apply-smob/0 #<thunk 7f4ce92e3980>)
> In ice-9/boot-9.scm:
> 718:2 2 (call-with-prompt _ _ #<procedure default-prompt-handler (k proc)>)
> In ice-9/eval.scm:
> 619:8 1 (_ #(#(#<directory (guile-user) 7f4ce8f05f00>)))
> In guix/ui.scm:
> 1945:12 0 (run-guix-command _ . _)
>
> guix/ui.scm:1945:12: In procedure run-guix-command:
> In procedure operating-system-file-systems: Wrong type argument: #<<parameterized> bindings: ((#<<parameter> 7f4ce7c23740 proc: #<procedure 7f4ce7c28200 at ice-9/boot-9.scm:1299:5 () | (x)>> #<procedure 7f4cd32f83c0 at gnu/services/virtualization.scm:806:14 ()>)) thunk: #<procedure 7f4cd32f8340 at gnu/services/virtualization.scm:806:14 ()>>
>
> ...I could do with some help here.

Ooh, sadness. We can’t do that because the image machinery really
expects an <operating-system>.

What if you move ‘with-parameters’ around (system-image …) instead?

Toggle quote (17 lines)
>>> +(define %hurd-in-vm-operating-system
> [..]
>>> + (operating-system
>>> + (service openssh-service-type
>>> + (openssh-configuration
>>> + (openssh openssh-sans-x)
> [..]
>>> + %base-services/hurd))))
>>
>> I understand the need to factorize useful configs, but IMO it doesn’t
>> belong here. So I’d just leave it out. There’s already
>> ‘%hurd-default-operating-system’ that does the heavy lifting anyway.
>
> Sure, removed! Users will most probably want to add an openssh server
> using openssh-sans-x; but I guess that's something for a blog post or
> cookbook then.

Yeah.

Hmm, come to think about it, we need a default value here anyway, so
after all, we have a good reason to have it here. (Says the guy who
changes his mind.)

Toggle quote (9 lines)
> From b01b8d2a46a6a04cb8f09d74c06cbbc82878f070 Mon Sep 17 00:00:00 2001
> From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
> Date: Thu, 11 Jun 2020 22:52:12 +0200
> Subject: [PATCH v2 1/2] image: Make 'find-image' non-monadic.
>
> * gnu/system/image.scm (find-image): Make non-monadic.
> * gnu/tests/install.scm (run-install): Update caller.
> * guix/scripts/system.scm (perform-action): Likewise.

[...]

Toggle quote (24 lines)
> "Find and return an image that could match the given FILE-SYSTEM-TYPE. This
> is useful to adapt to interfaces written before the addition of the <image>
> record."
> - (mlet %store-monad ((target (current-target-system)))
> - (mbegin %store-monad
> - (return
> - (match file-system-type
> - ("iso9660" iso9660-image)
> - (_ (cond
> - ((and target
> - (hurd-triplet? target))
> - hurd-disk-image)
> - (else
> - efi-disk-image))))))))
> + (let ((target (%current-target-system)))
> + (match file-system-type
> + ("iso9660" iso9660-image)
> + (_ (cond
> + ((and target
> + (hurd-triplet? target))
> + hurd-disk-image)
> + (else
> + efi-disk-image))))))

I’d prefer:

(define* (find-image #:optional (system (%current-system)))
…)

In your case, you’d need to make this call:

(find-image "i586-gnu")

(Beware of the triplet/system type distinction!)

Perhaps the other call sites need to be adjusted.

Toggle quote (11 lines)
> From e5bdf050f628cc7ea1b6bc4ccdcfeb757429820f Mon Sep 17 00:00:00 2001
> From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
> Date: Wed, 10 Jun 2020 00:10:28 +0200
> Subject: [PATCH v2 2/2] services: Add 'hurd-vm service-type'.
>
> * gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
> hurd-vm-disk-image): New procedures.
> (hurd-in-vm-service-type): New variable.
> (<hurd-in-vm-configuration>): New record type.
> * doc/guix.texi (Virtualization Services): Document it.

s/hurd-in-vm/hurd-vm/ in the commit log.

Otherwise LGTM, thank you!

Ludo’.
M
M
Mathieu Othacehe wrote on 12 Jun 2020 17:04
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
873670l3z9.fsf@gnu.org
Hey janneke,

Toggle quote (10 lines)
> + (let ((target (%current-target-system)))
> + (match file-system-type
> + ("iso9660" iso9660-image)
> + (_ (cond
> + ((and target
> + (hurd-triplet? target))
> + hurd-disk-image)
> + (else
> + efi-disk-image))))))

I think it would be safe to pass a "target" argument. Then the two
actual callers could pass (current-target-system) as target
argument. This is guaranteed to return a correct value, whereas
%current-target-system is not here.

Toggle quote (6 lines)
> +@lisp
> +(service hurd-vm-service-type
> + (hurd-vm-configuration
> + (disk-size (* 5000 (expt 2 20))) ;5G
> + (memory-size 1024))) ;1024MiB

That's really nice! We could really use a (guix units) module or so
where we would put those definitions: "(define MiB (expt 2 20))". Then
we could use:

Toggle snippet (6 lines)
(service hurd-vm-service-type
(hurd-vm-configuration
(disk-size (* 5 GiB)
(memory-size (* 1024 MiB)))

Well, this is really not a blocking thing.

Toggle quote (10 lines)
> +(define* (disk-image os #:key (image-size 'guess) target)
> + "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
> + (let ((base-image (find-image "ext2")))
> + (system-image
> + (image (inherit base-image)
> + (size image-size)
> + (operating-system
> + (with-parameters ((%current-target-system target))
> + os))))))

Yeah then again, I think it would be nice to use something like the
"hurd-disk-image" as I proposed here:

This way, no need to call "find-image". You could just write:

Toggle snippet (7 lines)
(define (image os #:key (image-size 'guess))
(image
(inherit hurd-disk-image)
(size image-size)
(operating-system os)))

Toggle quote (3 lines)
> + (image hurd-vm-configuration-image ;string
> + (thunked)

Then the thunked field wouldn't be required.

Mostly comments related to the fact that (gnu image) needs some
polishing, your patches are really nice here :)

Thanks,

Mathieu
L
L
Ludovic Courtès wrote on 12 Jun 2020 17:39
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87sgf05m53.fsf@gnu.org
Hi Mathieu!

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (19 lines)
>> So, assuming ‘find-image’ is non-monadic, the code above becomes
>> something like:
>>
>> (system-image
>> (image (inherit base-image)
>> (size image-size)
>> (operating-system
>> (with-parameters ((%current-target-system "i586-pc-gnu"))
>> os))))
>
> I would prefer 'target' to be part of the image itself, as I proposed
> here: https://lists.gnu.org/archive/html/guix-devel/2020-05/msg00417.html.
>
> There's no way for now, that the image is built without cross-compiling
> for "i586-pc-gnu", so I think it could be part of the "image" record
> itself.
>
> WDYT?

Yes, why not, a ‘target’ field in <image> sounds fine.

In general, I think lowerable objects should not be parameterized by
target/system. However, <image> is an exception because it needs to
access the operating system record, and still use the desired target, if
any.

Thanks,
Ludo’.
J
J
Jan Nieuwenhuizen wrote on 12 Jun 2020 23:33
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87eeqknf4d.fsf@gnu.org
Mathieu Othacehe writes:

Hello Mathieu,

Toggle quote (15 lines)
>> + (let ((target (%current-target-system)))
>> + (match file-system-type
>> + ("iso9660" iso9660-image)
>> + (_ (cond
>> + ((and target
>> + (hurd-triplet? target))
>> + hurd-disk-image)
>> + (else
>> + efi-disk-image))))))
>
> I think it would be safe to pass a "target" argument. Then the two
> actual callers could pass (current-target-system) as target
> argument. This is guaranteed to return a correct value, whereas
> %current-target-system is not here.

Okay, sure. I've made that change for now.

Toggle quote (15 lines)
>> +@lisp
>> +(service hurd-vm-service-type
>> + (hurd-vm-configuration
>> + (disk-size (* 5000 (expt 2 20))) ;5G
>> + (memory-size 1024))) ;1024MiB
>
> That's really nice! We could really use a (guix units) module or so
> where we would put those definitions: "(define MiB (expt 2 20))". Then
> we could use:
>
> (service hurd-vm-service-type
> (hurd-vm-configuration
> (disk-size (* 5 GiB)
> (memory-size (* 1024 MiB)))

I like it!

Toggle quote (24 lines)
> Well, this is really not a blocking thing.

>> +(define* (disk-image os #:key (image-size 'guess) target)
>> + "Return a disk-image for OS with size IMAGE-SIZE, built for TARGET."
>> + (let ((base-image (find-image "ext2")))
>> + (system-image
>> + (image (inherit base-image)
>> + (size image-size)
>> + (operating-system
>> + (with-parameters ((%current-target-system target))
>> + os))))))
>
> Yeah then again, I think it would be nice to use something like the
> "hurd-disk-image" as I proposed here:
> https://lists.gnu.org/archive/html/guix-devel/2020-05/msg00417.html.
>
> This way, no need to call "find-image". You could just write:
>
> (define (image os #:key (image-size 'guess))
> (image
> (inherit hurd-disk-image)
> (size image-size)
> (operating-system os)))

Oh yes...that's great. It seems that until we have that, as Ludo
suggested with this:

Toggle snippet (7 lines)
(with-parameters ((%current-target-system target))
(system-image
(image (inherit base-image)
(size disk-size)
(operating-system os))))))

we have now something "that works" for the Hurd.

Toggle quote (8 lines)
>> + (image hurd-vm-configuration-image ;string
>> + (thunked)
>
> Then the thunked field wouldn't be required.
>
> Mostly comments related to the fact that (gnu image) needs some
> polishing, your patches are really nice here :)

Thank you. I'll be working to prepare a v3 series that works on master
right now; then we can see how to proceed.

Thanks!
Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
J
J
Jan Nieuwenhuizen wrote on 12 Jun 2020 23:33
(name . Ludovic Courtès)(address . ludo@gnu.org)
87d064nf40.fsf@gnu.org
Ludovic Courtès writes:

Hello,

Toggle quote (10 lines)
> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>>> It’s a volatile VM, due to the use of ‘-snapshot’, right?
>>
>> By default: Yes. That seemed more ready-to-use. A stateful VM image
>> would need to an out-of-store, writable copy. You can actually do that
>> and modify the hurd-vm-configuration.
>
> It’s maybe worth mentioning in the manual.

Ah right, I wanted to do that after your remark...but forgot. Could be
an unpleasant surprise to lose everything. I've now added

Toggle snippet (15 lines)
Note that by default the VM image is volatile, i.e., once stopped the
contents are lost. If you want a stateful image instead, override the
configuration's @code{image} and @code{options} without
the @code{--snapshot} flag using something along these lines:

@lisp
(service hurd-vm-service-type
(hurd-vm-configuration
(image (const "/out/of/store/writable/hurd.img"))
(options '("--device" "rtl8139,netdev=net0"
"--netdev"
"user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222"))))
@end lisp

to the patch.

Toggle quote (12 lines)
>>> (The Hurd actually has “sub-Hurds”¹ and “neighborhurds”². I wonder if
>>> it’s our duty to coin another term… a guesthurd? a visithurd?)
>>>
>>> ¹ https://www.gnu.org/software/hurd/hurd/subhurd.html
>>> ² https://www.gnu.org/software/hurd/hurd/neighborhurd.html
>>
>> Oh, that's cool! Associating along from the neighborhurd pun, what
>> about a "childhurd" (as a pun on childhood -- only needed while the Hurd
>> is growing up)?
>
> “Childhurd”, LOVE IT!

Heh; ...

Toggle quote (6 lines)
>> "herd start childhurd" -- hmm? In the updated patch, I still have
>> hurd-vm. If we do our duty and coin "childhurd", should I just
>> s/hurd-vm/childhurd/g ?
>
> Shepherd services can have more than one name, so I propose to have both!

... great, now lemme see where to mention this. I just put in in the
code

Toggle snippet (6 lines)
(list
(shepherd-service
(documentation "Run the Hurd in a Virtual Machine: a Childhurd.")
(provision '(hurd-vm childhurd))

and docs here

Toggle snippet (6 lines)
@example
herd start hurd-vm
herd stop childhurd
@end example

Toggle quote (5 lines)
> Ooh, sadness. We can’t do that because the image machinery really
> expects an <operating-system>.
>
> What if you move ‘with-parameters’ around (system-image …) instead?

=> \o/ that works! You're genius ;-)

Toggle quote (23 lines)
>>>> +(define %hurd-in-vm-operating-system
>> [..]
>>>> + (operating-system
>>>> + (service openssh-service-type
>>>> + (openssh-configuration
>>>> + (openssh openssh-sans-x)
>> [..]
>>>> + %base-services/hurd))))
>>>
>>> I understand the need to factorize useful configs, but IMO it doesn’t
>>> belong here. So I’d just leave it out. There’s already
>>> ‘%hurd-default-operating-system’ that does the heavy lifting anyway.
>>
>> Sure, removed! Users will most probably want to add an openssh server
>> using openssh-sans-x; but I guess that's something for a blog post or
>> cookbook then.
>
> Yeah.
>
> Hmm, come to think about it, we need a default value here anyway, so
> after all, we have a good reason to have it here. (Says the guy who
> changes his mind.)

Heh, it's such a pleasure working with you. I've put it back again,
without the -in- bit. (says the guy who was easily convinced we didn't
need it.)

Toggle quote (48 lines)
>> From b01b8d2a46a6a04cb8f09d74c06cbbc82878f070 Mon Sep 17 00:00:00 2001
>> From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
>> Date: Thu, 11 Jun 2020 22:52:12 +0200
>> Subject: [PATCH v2 1/2] image: Make 'find-image' non-monadic.
>>
>> * gnu/system/image.scm (find-image): Make non-monadic.
>> * gnu/tests/install.scm (run-install): Update caller.
>> * guix/scripts/system.scm (perform-action): Likewise.
>
> [...]
>
>> "Find and return an image that could match the given FILE-SYSTEM-TYPE. This
>> is useful to adapt to interfaces written before the addition of the <image>
>> record."
>> - (mlet %store-monad ((target (current-target-system)))
>> - (mbegin %store-monad
>> - (return
>> - (match file-system-type
>> - ("iso9660" iso9660-image)
>> - (_ (cond
>> - ((and target
>> - (hurd-triplet? target))
>> - hurd-disk-image)
>> - (else
>> - efi-disk-image))))))))
>> + (let ((target (%current-target-system)))
>> + (match file-system-type
>> + ("iso9660" iso9660-image)
>> + (_ (cond
>> + ((and target
>> + (hurd-triplet? target))
>> + hurd-disk-image)
>> + (else
>> + efi-disk-image))))))
>
> I’d prefer:
>
> (define* (find-image #:optional (system (%current-system)))
> …)
>
> In your case, you’d need to make this call:
>
> (find-image "i586-gnu")
>
> (Beware of the triplet/system type distinction!)
>
> Perhaps the other call sites need to be adjusted.

Then Mathieu writes

Toggle quote (9 lines)
> > I would prefer 'target' to be part of the image itself, as I proposed
> > here: https://lists.gnu.org/archive/html/guix-devel/2020-05/msg00417.html.
> >
> > There's no way for now, that the image is built without cross-compiling
> > for "i586-pc-gnu", so I think it could be part of the "image" record
> > itself.
> >
> > WDYT?

and Ludo writes

Toggle quote (2 lines)
> Yes, why not, a ‘target’ field in <image> sounds fine.

So...I'm sticking with a the 'target' parameter for now; as that was
Mathieu's suggestion and a closer match to the final solution of
moving TARGET into image.

Toggle quote (15 lines)
>> From e5bdf050f628cc7ea1b6bc4ccdcfeb757429820f Mon Sep 17 00:00:00 2001
>> From: "Jan (janneke) Nieuwenhuizen" <janneke@gnu.org>
>> Date: Wed, 10 Jun 2020 00:10:28 +0200
>> Subject: [PATCH v2 2/2] services: Add 'hurd-vm service-type'.
>>
>> * gnu/services/virtualization.scm (disk-image, hurd-in-vm-shepherd-service,
>> hurd-vm-disk-image): New procedures.
>> (hurd-in-vm-service-type): New variable.
>> (<hurd-in-vm-configuration>): New record type.
>> * doc/guix.texi (Virtualization Services): Document it.
>
> s/hurd-in-vm/hurd-vm/ in the commit log.
>
> Otherwise LGTM, thank you!

Thanks! I'll be sending a v3 series that should be OK (apart from the
image work that's not finished yet), and we can decide what to do.

Greetings,
Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
J
J
Jan (janneke) Nieuwenhuizen wrote on 12 Jun 2020 23:42
[PATCH v3 1/2] image: Make 'find-image' non-monadic.
20200612214214.14112-1-janneke@gnu.org
* gnu/system/image.scm (find-image): Make non-monadic. Add 'target'
parameter.
* gnu/tests/install.scm (run-install): Update caller,
passing (%current-target-system).
* guix/scripts/system.scm (perform-action): Likewise.
---
gnu/system/image.scm | 22 ++++++++++------------
gnu/tests/install.scm | 6 ++++--
guix/scripts/system.scm | 3 ++-
3 files changed, 16 insertions(+), 15 deletions(-)

Toggle diff (89 lines)
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index a0e6bf31f1..ac55301a60 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -1,5 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -547,20 +548,17 @@ image, depending on IMAGE format."
#:grub-mkrescue-environment
'(("MKRESCUE_SED_MODE" . "mbr_hfs")))))))
-(define (find-image file-system-type)
+(define (find-image file-system-type target)
"Find and return an image that could match the given FILE-SYSTEM-TYPE. This
is useful to adapt to interfaces written before the addition of the <image>
record."
- (mlet %store-monad ((target (current-target-system)))
- (mbegin %store-monad
- (return
- (match file-system-type
- ("iso9660" iso9660-image)
- (_ (cond
- ((and target
- (hurd-triplet? target))
- hurd-disk-image)
- (else
- efi-disk-image))))))))
+ (match file-system-type
+ ("iso9660" iso9660-image)
+ (_ (cond
+ ((and target
+ (hurd-triplet? target))
+ hurd-disk-image)
+ (else
+ efi-disk-image)))))
;;; image.scm ends here
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 6bd8c7d3d2..82098290f7 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -3,6 +3,7 @@
;;; Copyright © 2017, 2019 Tobias Geerinckx-Rice <me@tobias.gr>
;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2020 Danny Milosavljevic <dannym@scratchpost.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -228,8 +229,9 @@ packages defined in installation-os."
(mlet* %store-monad ((_ (set-grafting #f))
(system (current-system))
(target (operating-system-derivation target-os))
- (base-image (find-image
- installation-disk-image-file-system-type))
+ (base-image -> (find-image
+ installation-disk-image-file-system-type
+ (%current-target-system)))
;; Since the installation system has no network access,
;; we cheat a little bit by adding TARGET to its GC
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 3d7aa77cb7..6f3880cc34 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2017, 2019 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2019 Christopher Baines <mail@cbaines.net>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -801,7 +802,7 @@ static checks."
(check-initrd-modules os)))
(mlet* %store-monad
- ((image (find-image file-system-type))
+ ((image -> (find-image file-system-type (%current-target-system)))
(sys (system-derivation-for-action os image action
#:file-system-type file-system-type
#:image-size image-size
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
J
J
Jan (janneke) Nieuwenhuizen wrote on 12 Jun 2020 23:42
[PATCH v3 2/2] services: Add 'hurd-vm service-type'.
20200612214214.14112-2-janneke@gnu.org
* gnu/services/virtualization.scm (hurd-vm-shepherd-service,
hurd-vm-disk-image): New procedures.
(%hurd-vm-operating-system, hurd-vm-service-type): New variable.
(<hurd-vm-configuration>): New record type.
* doc/guix.texi (Virtualization Services): Document it.
---
doc/guix.texi | 83 +++++++++++++++++++++
gnu/services/virtualization.scm | 126 ++++++++++++++++++++++++++++++--
2 files changed, 202 insertions(+), 7 deletions(-)

Toggle diff (256 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 15e077a41c..ed36d5014b 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24583,6 +24583,89 @@ Return true if @var{obj} is a platform object.
Return the name of @var{platform}---a string such as @code{"arm"}.
@end deffn
+
+@subsubheading The Hurd in a Virtual Machine
+
+@cindex @code{hurd}
+@cindex the Hurd
+
+Service @code{hurd-vm} provides support for running GNU/Hurd in a
+virtual machine (VM), a so-called ``Childhurd''. The virtual machine is
+a Shepherd service that can be controlled with commands such as:
+
+@example
+herd start hurd-vm
+herd stop childhurd
+@end example
+
+The given GNU/Hurd operating system configuration is cross-compiled.
+
+@defvr {Scheme Variable} hurd-vm-service-type
+This is the type of the Hurd in a Virtual Machine service. Its value
+must be a @code{hurd-vm-configuration} object, which specifies the
+operating system (@pxref{operating-system Reference}) and the disk size
+for the Hurd Virtual Machine, the QEMU package to use as well as the
+options for running it.
+
+For example:
+
+@lisp
+(service hurd-vm-service-type
+ (hurd-vm-configuration
+ (disk-size (* 5000 (expt 2 20))) ;5G
+ (memory-size 1024))) ;1024MiB
+@end lisp
+
+would create a disk image big enough to build GNU@tie{}Hello, with some
+extra memory.
+@end defvr
+
+@deftp {Data Type} hurd-vm-configuration
+The data type representing the configuration for
+@code{hurd-vm-service-type}.
+
+@table @asis
+@item @code{os} (default: @var{%hurd-vm-operating-system})
+The operating system to instantiate. This default is bare-bones with a
+permissive OpenSSH secure shell daemon listening on port 2222
+(@pxref{Networking Services, @code{openssh-service-type}}).
+
+@item @code{qemu} (default: @code{qemu-minimal})
+The QEMU package to use.
+
+@item @code{image} (default: @var{hurd-vm-disk-image})
+The procedure used to build the disk-image built from this
+configuration.
+
+@item @code{disk-size} (default: @code{'guess})
+The size of the disk image.
+
+@item @code{memory-size} (default: @code{512})
+The memory size of the Virtual Machine in mebibytes.
+
+@item @code{options} (default: @code{'("--device"} @code{"rtl8139,netdev=net0"} @
+ @code{"--netdev"} @
+ @code{"user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222,hostfwd=tcp:127.0.0.1:25900-:5900"} @
+ @code{"--snapshot"} @
+ @code{"--hda")})
+The extra options for running QEMU.
+@end table
+@end deftp
+
+Note that by default the VM image is volatile, i.e., once stopped the
+contents are lost. If you want a stateful image instead, override the
+configuration's @code{image} and @code{options} without
+the @code{--snapshot} flag using something along these lines:
+
+@lisp
+(service hurd-vm-service-type
+ (hurd-vm-configuration
+ (image (const "/out/of/store/writable/hurd.img"))
+ (options '("--device" "rtl8139,netdev=net0"
+ "--netdev"
+ "user,id=net0,hostfwd=tcp:127.0.0.1:20022-:2222"))))
+@end lisp
+
@node Version Control Services
@subsection Version Control Services
diff --git a/gnu/services/virtualization.scm b/gnu/services/virtualization.scm
index 989e439d5d..dc1e91b0bd 100644
--- a/gnu/services/virtualization.scm
+++ b/gnu/services/virtualization.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2017 Ryan Moe <ryan.moe@gmail.com>
;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -18,24 +19,41 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (gnu services virtualization)
- #:use-module (gnu services)
- #:use-module (gnu services configuration)
+ #:use-module (gnu bootloader)
+ #:use-module (gnu bootloader grub)
+ #:use-module (gnu image)
+ #:use-module (gnu packages admin)
+ #:use-module (gnu packages ssh)
+ #:use-module (gnu packages virtualization)
#:use-module (gnu services base)
+ #:use-module (gnu services configuration)
#:use-module (gnu services dbus)
#:use-module (gnu services shepherd)
- #:use-module (gnu system shadow)
+ #:use-module (gnu services ssh)
+ #:use-module (gnu services)
#:use-module (gnu system file-systems)
- #:use-module (gnu packages admin)
- #:use-module (gnu packages virtualization)
- #:use-module (guix records)
+ #:use-module (gnu system hurd)
+ #:use-module (gnu system image)
+ #:use-module (gnu system shadow)
+ #:use-module (gnu system)
+ #:use-module (guix derivations)
#:use-module (guix gexp)
+ #:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix records)
+ #:use-module (guix store)
+ #:use-module (guix utils)
+
#:use-module (srfi srfi-9)
#:use-module (srfi srfi-26)
#:use-module (rnrs bytevectors)
#:use-module (ice-9 match)
- #:export (libvirt-configuration
+ #:export (%hurd-vm-operating-system
+ hurd-vm-configuration
+ hurd-vm-service-type
+
+ libvirt-configuration
libvirt-service-type
virtlog-configuration
virtlog-service-type
@@ -773,3 +791,97 @@ given QEMU package."
"This service supports transparent emulation of binaries
compiled for other architectures using QEMU and the @code{binfmt_misc}
functionality of the kernel Linux.")))
+
+
+;;;
+;;; The Hurd in VM service.
+;;;
+
+(define %hurd-vm-operating-system
+ (operating-system
+ (inherit %hurd-default-operating-system)
+ (host-name "guixydevel")
+ (timezone "Europe/Amsterdam")
+ (bootloader (bootloader-configuration
+ (bootloader grub-minimal-bootloader)
+ (target "/dev/vda")
+ (timeout 0)))
+ (services (cons*
+ (service openssh-service-type
+ (openssh-configuration
+ (openssh openssh-sans-x)
+ (use-pam? #f)
+ (port-number 2222)
+ (permit-root-login #t)
+ (allow-empty-passwords? #t)
+ (password-authentication? #t)))
+ %base-services/hurd))))
+
+(define-record-type* <hurd-vm-configuration>
+ hurd-vm-configuration make-hurd-vm-configuration
+ hurd-vm-configuration?
+ (os hurd-vm-configuration-os ;<operating-system>
+ (default %hurd-vm-operating-system))
+ (qemu hurd-vm-configuration-qemu ;<package>
+ (default qemu-minimal))
+ (image hurd-vm-configuration-image ;string
+ (thunked)
+ (default (hurd-vm-disk-image this-record)))
+ (disk-size hurd-vm-configuration-disk-size ;number or 'guess
+ (default 'guess))
+ (memory-size hurd-vm-configuration-memory-size ;number
+ (default 512))
+ (options hurd-vm-configuration-options ;list of string
+ (default
+ `("--device" "rtl8139,netdev=net0"
+ "--netdev" ,(string-append
+ "user,id=net0"
+ ",hostfwd=tcp:127.0.0.1:20022-:2222"
+ ",hostfwd=tcp:127.0.0.1:25900-:5900")
+ "--snapshot"
+ "--hda"))))
+
+(define (hurd-vm-disk-image config)
+ "Return a disk-image for the Hurd according to CONFIG."
+ (let ((os (hurd-vm-configuration-os config))
+ (disk-size (hurd-vm-configuration-disk-size config))
+ (target (and (not (%current-target-system)) "i586-pc-gnu"))
+ (base-image (find-image "ext2" (%current-target-system))))
+ (with-parameters ((%current-target-system target))
+ (system-image
+ (image (inherit base-image)
+ (size disk-size)
+ (operating-system os))))))
+
+(define (hurd-vm-shepherd-service config)
+ "Return a <shepherd-service> for a Hurd in a Virtual Machine with CONFIG."
+
+ (let ((image (hurd-vm-configuration-image config))
+ (qemu (hurd-vm-configuration-qemu config))
+ (memory-size (hurd-vm-configuration-memory-size config))
+ (options (hurd-vm-configuration-options config)))
+
+ (define vm-command
+ #~(list
+ (string-append #$qemu "/bin/qemu-system-i386")
+ #$@(if (file-exists? "/dev/kvm") '("-enable-kvm") '())
+ "-m" (number->string #$memory-size)
+ #$@options
+ #+image))
+
+ (list
+ (shepherd-service
+ (documentation "Run the Hurd in a Virtual Machine: a Childhurd.")
+ (provision '(hurd-vm childhurd))
+ (requirement '(networking))
+ (start #~(make-forkexec-constructor #$vm-command))
+ (stop #~(make-kill-destructor))))))
+
+(define hurd-vm-service-type
+ (service-type
+ (name 'hurd-vm)
+ (extensions (list (service-extension shepherd-root-service-type
+ hurd-vm-shepherd-service)))
+ (default-value (hurd-vm-configuration))
+ (description
+ "Provide a Virtual Machine running the GNU/Hurd.")))
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
J
J
Jan Nieuwenhuizen wrote on 13 Jun 2020 09:30
Re: [bug#41785] [PATCH] DRAFT services: Add 'hurd-in-vm service-type'.
(name . Diego Nicola Barbato)(address . dnbarbato@posteo.de)
87d063h17k.fsf@gnu.org
Diego Nicola Barbato writes:

Hi,

Toggle quote (30 lines)
> Jan Nieuwenhuizen <janneke@gnu.org> writes:
>
> [...]
>
>> and the interwebs told me that to start the VM, you have to add "-cpu
>> host"; so I started it using
>>
>> /gnu/store/k2b7nx34cwyi6yk49wgy4hg9mrwcmll5-run-vm.sh -cpu host -m
>> 2G -device rtl8139,netdev=net0 -netdev
>> user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:25900-:25900
>>
>>
>> and trying to "ssh -p 20022 localhost" from inside the bare-bones VM now
>> prints
>>
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> qemu-system-i386: Slirp: Failed to send package, ret: -1
>> key_exchange_identification: read: Connection reset by peer
>> Connection reset by 127.0.0.1 port 20022
>>
>> ...something networky with QEMU. Ideas?
>
> I've recently had intermittent issues with 'guix system vm' and hostfwd.
> Does it help if you remove the '-nic user,model=virtio-net-pci' option
> from (a copy of) the run-vm.sh script so that there's only one netdev?

That's helpful! I now noticed that the QEMU command above I used was
the one I have been using to start the Hurd, networking wise.

After running

Toggle snippet (3 lines)
./run-vm.sh -cpu host -m 2G -net nic -net user,id=net0,hostfwd=tcp:127.0.0.1:10022-:2222,hostfwd=tcp:127.0.0.1:5900-:25900,hostfwd=tcp:127.0.0.1:20022-:20022

I noticed that instead of two network interfaces, there is now just one,
as expected; much better.

No luck ssh'ing into the second VM, directly or indirectly. I'm not
exploring this further as we don't have an actually need for this
vm-in-vm thing.

Greetings,
Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
M
M
Mathieu Othacehe wrote on 13 Jun 2020 12:56
Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
(name . Jan (janneke) Nieuwenhuizen)(address . janneke@gnu.org)
87r1ujjkst.fsf@gnu.org
Hey Jan,

Toggle quote (8 lines)
> (system (current-system))
> (target (operating-system-derivation target-os))
> - (base-image (find-image
> - installation-disk-image-file-system-type))
> + (base-image -> (find-image
> + installation-disk-image-file-system-type
> + (%current-target-system)))

This should be:

Toggle snippet (7 lines)
(target (current-target-system))
...
(base-image -> (find-image
installation-disk-image-file-system-type
target))

Toggle quote (5 lines)
> + ((image -> (find-image file-system-type (%current-target-system)))
> (sys (system-derivation-for-action os image action
> #:file-system-type file-system-type
> #:image-size image-size

Same here, otherwise this look good to me.

Thanks,

Mathieu
M
M
Mathieu Othacehe wrote on 13 Jun 2020 14:49
Re: [bug#41785] [PATCH v3 2/2] services: Add 'hurd-vm service-type'.
(name . Jan (janneke) Nieuwenhuizen)(address . janneke@gnu.org)
87imfvjfl8.fsf@gnu.org
Hey!

Toggle quote (12 lines)
> +(define (hurd-vm-disk-image config)
> + "Return a disk-image for the Hurd according to CONFIG."
> + (let ((os (hurd-vm-configuration-os config))
> + (disk-size (hurd-vm-configuration-disk-size config))
> + (target (and (not (%current-target-system)) "i586-pc-gnu"))
> + (base-image (find-image "ext2" (%current-target-system))))
> + (with-parameters ((%current-target-system target))
> + (system-image
> + (image (inherit base-image)
> + (size disk-size)
> + (operating-system os))))))

With the attached patch, you could write:

Toggle snippet (11 lines)
(define (hurd-vm-disk-image config)
"Return a disk-image for the Hurd according to CONFIG."
(let ((os (hurd-vm-configuration-os config))
(disk-size (hurd-vm-configuration-disk-size config)))
(system-image
(image
(inherit hurd-disk-image)
(size disk-size)
(operating-system os)))))

WDYT?

Mathieu
From dbcfd86a74903cb0fe77843518625436d749ed09 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@gnu.org>
Date: Sat, 13 Jun 2020 14:01:18 +0200
Subject: [PATCH] image: Add 'target' support.

* gnu/image.scm (<image>)[target]: New field,
(image-target): new public method.
* gnu/system/image.scm (hurd-disk-image): Set "i586-pc-gnu" as image 'target'
field,
(maybe-with-target): new procedure,
(system-image): honor image 'target' field using the above procedure.
---
gnu/image.scm | 3 ++
gnu/system/image.scm | 66 +++++++++++++++++++++++++++-----------------
2 files changed, 43 insertions(+), 26 deletions(-)

Toggle diff (119 lines)
diff --git a/gnu/image.scm b/gnu/image.scm
index 0a92d168e9..19b466527b 100644
--- a/gnu/image.scm
+++ b/gnu/image.scm
@@ -33,6 +33,7 @@
image
image-name
image-format
+ image-target
image-size
image-operating-system
image-partitions
@@ -67,6 +68,8 @@
image make-image
image?
(format image-format) ;symbol
+ (target image-target
+ (default #f))
(size image-size ;size in bytes as integer
(default 'guess))
(operating-system image-operating-system ;<operating-system>
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 6c4573509d..7b45fdfea7 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -103,6 +103,7 @@
(define hurd-disk-image
(image
(format 'disk-image)
+ (target "i586-pc-gnu")
(partitions
(list (partition
(size 'guess)
@@ -518,6 +519,14 @@ it can be used for bootloading."
(type root-file-system-type))
file-systems-to-keep)))))
+(define-syntax-rule (maybe-with-target image exp ...)
+ (let ((target (image-target image)))
+ (if target
+ (with-parameters ((%current-target-system target))
+ exp ...)
+ (begin
+ exp ...))))
+
(define* (system-image image)
"Return the derivation of IMAGE. It can be a raw disk-image or an ISO9660
image, depending on IMAGE format."
@@ -529,32 +538,33 @@ image, depending on IMAGE format."
(bootcfg (operating-system-bootcfg os))
(bootloader (bootloader-configuration-bootloader
(operating-system-bootloader os))))
- (case (image-format image)
- ((disk-image)
- (system-disk-image image*
- #:bootcfg bootcfg
- #:bootloader bootloader
- #:register-closures? register-closures?
- #:inputs `(("system" ,os)
- ("bootcfg" ,bootcfg))))
- ((iso9660)
- (system-iso9660-image
- image*
- #:bootcfg bootcfg
- #:bootloader bootloader
- #:register-closures? register-closures?
- #:inputs `(("system" ,os)
- ("bootcfg" ,bootcfg))
- ;; Make sure to use a mode that does no imply
- ;; HFS+ tree creation that may fail with:
- ;;
- ;; "libisofs: FAILURE : Too much files to mangle,
- ;; cannot guarantee unique file names"
- ;;
- ;; This happens if some limits are exceeded, see:
- ;; https://lists.gnu.org/archive/html/grub-devel/2020-06/msg00048.html
- #:grub-mkrescue-environment
- '(("MKRESCUE_SED_MODE" . "mbr_only")))))))
+ (maybe-with-target image
+ (case (image-format image)
+ ((disk-image)
+ (system-disk-image image*
+ #:bootcfg bootcfg
+ #:bootloader bootloader
+ #:register-closures? register-closures?
+ #:inputs `(("system" ,os)
+ ("bootcfg" ,bootcfg))))
+ ((iso9660)
+ (system-iso9660-image
+ image*
+ #:bootcfg bootcfg
+ #:bootloader bootloader
+ #:register-closures? register-closures?
+ #:inputs `(("system" ,os)
+ ("bootcfg" ,bootcfg))
+ ;; Make sure to use a mode that does no imply
+ ;; HFS+ tree creation that may fail with:
+ ;;
+ ;; "libisofs: FAILURE : Too much files to mangle,
+ ;; cannot guarantee unique file names"
+ ;;
+ ;; This happens if some limits are exceeded, see:
+ ;; https://lists.gnu.org/archive/html/grub-devel/2020-06/msg00048.html
+ #:grub-mkrescue-environment
+ '(("MKRESCUE_SED_MODE" . "mbr_only"))))))))
(define (find-image file-system-type)
"Find and return an image that could match the given FILE-SYSTEM-TYPE. This
@@ -572,4 +582,8 @@ record."
(else
efi-disk-image))))))))
+;;; Local Variables:
+;;; eval: (put 'maybe-with-target 'scheme-indent-function 1)
+;;; End:
+
;;; image.scm ends here
--
2.24.0
J
J
Jan Nieuwenhuizen wrote on 13 Jun 2020 15:05
Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
878sgrazg8.fsf@gnu.org
Mathieu Othacehe writes:

Hey Mathieu,

Toggle quote (16 lines)
>> (system (current-system))
>> (target (operating-system-derivation target-os))
>> - (base-image (find-image
>> - installation-disk-image-file-system-type))
>> + (base-image -> (find-image
>> + installation-disk-image-file-system-type
>> + (%current-target-system)))
>
> This should be:
>
> (target (current-target-system))
> ...
> (base-image -> (find-image
> installation-disk-image-file-system-type
> target))

Ah, right. Done and moved this

Toggle quote (2 lines)
>> (target (operating-system-derivation target-os))

down.

Toggle quote (7 lines)
>> + ((image -> (find-image file-system-type (%current-target-system)))
>> (sys (system-derivation-for-action os image action
>> #:file-system-type file-system-type
>> #:image-size image-size
>
> Same here, otherwise this look good to me.

Thanks! Pushed to master as 7ca533c7237622d70b423033c4506217d9ce4014

(probably/hopefully soon to be revisited by your image-catalog feature :-)

Greetings,
Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
J
J
Jan Nieuwenhuizen wrote on 13 Jun 2020 15:10
Re: [bug#41785] [PATCH v3 2/2] services: Add 'hurd-vm service-type'.
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
875zbvaz7p.fsf@gnu.org
Mathieu Othacehe writes:

Hey Mathieu!

Toggle quote (26 lines)
>> +(define (hurd-vm-disk-image config)
>> + "Return a disk-image for the Hurd according to CONFIG."
>> + (let ((os (hurd-vm-configuration-os config))
>> + (disk-size (hurd-vm-configuration-disk-size config))
>> + (target (and (not (%current-target-system)) "i586-pc-gnu"))
>> + (base-image (find-image "ext2" (%current-target-system))))
>> + (with-parameters ((%current-target-system target))
>> + (system-image
>> + (image (inherit base-image)
>> + (size disk-size)
>> + (operating-system os))))))
>
> With the attached patch, you could write:
>
> (define (hurd-vm-disk-image config)
> "Return a disk-image for the Hurd according to CONFIG."
> (let ((os (hurd-vm-configuration-os config))
> (disk-size (hurd-vm-configuration-disk-size config)))
> (system-image
> (image
> (inherit hurd-disk-image)
> (size disk-size)
> (operating-system os)))))
>
> WDYT?

I like it a lot! Ludo already said adding target to image would be a
good idea. Go for it!

Toggle quote (5 lines)
> From dbcfd86a74903cb0fe77843518625436d749ed09 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe@gnu.org>
> Date: Sat, 13 Jun 2020 14:01:18 +0200
> Subject: [PATCH] image: Add 'target' support.

This is already one step towards the image catalog, looks good to me!

Note that I found https://bugs.gnu.org/41835so I had to revert that
patch in order to test...less than great; but I didn't want to push
a revert without communicating about it first.

Greetings,
Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 13 Jun 2020 16:35
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87pna311bb.fsf@gnu.org
Hi,

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (8 lines)
> +(define-syntax-rule (maybe-with-target image exp ...)
> + (let ((target (image-target image)))
> + (if target
> + (with-parameters ((%current-target-system target))
> + exp ...)
> + (begin
> + exp ...))))

I think you don’t need the ‘if’ here. But then maybe you don’t need the
macro either since you can always write:

(with-parameters ((%current-target-system (image-target image)))
…)

Anyway, all this looks great to me! :-)

Ludo’.
M
M
Mathieu Othacehe wrote on 13 Jun 2020 17:01
(name . Ludovic Courtès)(address . ludo@gnu.org)
871rmjj9gl.fsf@gnu.org
Hey Ludo,

Toggle quote (3 lines)
> I think you don’t need the ‘if’ here. But then maybe you don’t need the
> macro either since you can always write:

Thanks for the quick review :)

You're right it's not needed, I just removed it.

Mathieu
J
J
Jan Nieuwenhuizen wrote on 14 Jun 2020 14:10
[PATCH v4] services: Add 'hurd-in-vm service-type'.
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
878sgpg856.fsf_-_@gnu.org
Jan Nieuwenhuizen writes:

Hello,

Toggle quote (2 lines)
> we have now something "that works" for the Hurd.

Hmm...that was too fast; apparently I fooled myself last Friday.

Anyway, I found it! Using this

Toggle snippet (12 lines)
(define (hurd-vm-disk-image config)
"Return a disk-image for the Hurd according to CONFIG."
(with-parameters ((%current-target-system "i586-pc-gnu"))
(let ((os (hurd-vm-configuration-os config))
(disk-size (hurd-vm-configuration-disk-size config)))
(system-image
(image
(inherit hurd-disk-image)
(size disk-size)
(operating-system os))))))

almost worked...but it goes wrong cross-building the services.

In gnu/services/shepherd.scm's scm->go, we were bitten by

Toggle snippet (6 lines)
(define (scm->go file)
;; FIXME: %current-target-system may not be bound <https://bugs.gnu.org/29296>
(let ((target (%current-target-system)))
(with-extensions (list shepherd)

...but we now have let-system; so using

Toggle snippet (16 lines)
diff --git a/gnu/services/shepherd.scm b/gnu/services/shepherd.scm
index 77c4d0a8be..e14ceca231 100644
--- a/gnu/services/shepherd.scm
+++ b/gnu/services/shepherd.scm
@@ -266,8 +266,7 @@ stored."
(define (scm->go file)
"Compile FILE, which contains code to be loaded by shepherd's config file,
and return the resulting '.go' file."
- ;; FIXME: %current-target-system may not be bound <https://bugs.gnu.org/29296>
- (let ((target (%current-target-system)))
+ (let-system (system target)
(with-extensions (list shepherd)
(computed-file (string-append (basename (scheme-file-name file) ".scm")
".go")

produces a working hurd-vm service!

New patch attached, I intend to push to master once rebased, all the
tests pass, etc. ;-)

Phew, thank you!

Greetings,
Janneke
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
M
M
Mathieu Othacehe wrote on 14 Jun 2020 14:37
Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
87imftkel5.fsf@gnu.org
Hey!

Toggle quote (7 lines)
>>> + ((image -> (find-image file-system-type (%current-target-system)))
>>> (sys (system-derivation-for-action os image action
>>> #:file-system-type file-system-type
>>> #:image-size image-size
>>
>> Same here, otherwise this look good to me.

I missed something, "target" variable is shadowing the "target"
argument. The attached patch should fix the issue, I'm testing it.

Thanks,

Mathieu
From 760e46d2ff57fab2b67656b85d800572bbc56f4b Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@gnu.org>
Date: Sun, 14 Jun 2020 14:16:37 +0200
Subject: [PATCH] scripts: system: Fix "init" command.

This is a follow-up of 7ca533c7237622d70b423033c4506217d9ce4014. The
introduced "target" variable is shadowing the target argument.

* guix/scripts/system.scm (perform-action): Rename "target" variable to
"target*".
---
guix/scripts/system.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (17 lines)
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 6769a602b1..212b49f008 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -802,8 +802,8 @@ static checks."
(check-initrd-modules os)))
(mlet* %store-monad
- ((target (current-target-system))
- (image -> (find-image file-system-type target))
+ ((target* (current-target-system))
+ (image -> (find-image file-system-type target*))
(sys (system-derivation-for-action os image action
#:file-system-type file-system-type
#:image-size image-size
--
2.26.2
M
M
Mathieu Othacehe wrote on 14 Jun 2020 14:44
Re: [bug#41785] [PATCH v4] services: Add 'hurd-in-vm service-type'.
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
87eeqhkea1.fsf@gnu.org
Hey janneke!

Toggle quote (11 lines)
> (define (hurd-vm-disk-image config)
> "Return a disk-image for the Hurd according to CONFIG."
> (with-parameters ((%current-target-system "i586-pc-gnu"))
> (let ((os (hurd-vm-configuration-os config))
> (disk-size (hurd-vm-configuration-disk-size config)))
> (system-image
> (image
> (inherit hurd-disk-image)
> (size disk-size)
> (operating-system os))))))

Nice! I'm not sure why we still need the "with-parameters" call because,
it's done first thing in "system-image". I'd like to understand this
before proceeding, do you have a branch where I could test it?

Toggle quote (9 lines)
> - ;; FIXME: %current-target-system may not be bound https://bugs.gnu.org/29296
> - (let ((target (%current-target-system)))
> + (let-system (system target)
> (with-extensions (list shepherd)
> (computed-file (string-append (basename (scheme-file-name file) ".scm")
> ".go")
>
> produces a working hurd-vm service!

Good catch!

Thanks,

Mathieu
J
J
Jan Nieuwenhuizen wrote on 14 Jun 2020 15:12
Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87tuzdeqqc.fsf@gnu.org
Mathieu Othacehe writes:

Hi Mathieu,

Toggle quote (10 lines)
>>>> + ((image -> (find-image file-system-type (%current-target-system)))
>>>> (sys (system-derivation-for-action os image action
>>>> #:file-system-type file-system-type
>>>> #:image-size image-size
>>>
>>> Same here, otherwise this look good to me.
>
> I missed something, "target" variable is shadowing the "target"
> argument. The attached patch should fix the issue, I'm testing it.

Oops!

Toggle quote (30 lines)
> From 760e46d2ff57fab2b67656b85d800572bbc56f4b Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe@gnu.org>
> Date: Sun, 14 Jun 2020 14:16:37 +0200
> Subject: [PATCH] scripts: system: Fix "init" command.
>
> This is a follow-up of 7ca533c7237622d70b423033c4506217d9ce4014. The
> introduced "target" variable is shadowing the target argument.
>
> * guix/scripts/system.scm (perform-action): Rename "target" variable to
> "target*".
> ---
> guix/scripts/system.scm | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
> index 6769a602b1..212b49f008 100644
> --- a/guix/scripts/system.scm
> +++ b/guix/scripts/system.scm
> @@ -802,8 +802,8 @@ static checks."
> (check-initrd-modules os)))
>
> (mlet* %store-monad
> - ((target (current-target-system))
> - (image -> (find-image file-system-type target))
> + ((target* (current-target-system))
> + (image -> (find-image file-system-type target*))
> (sys (system-derivation-for-action os image action
> #:file-system-type file-system-type
> #:image-size image-size

Yeah, that looks right! I didn't notice either. I could have been more
careful/suspicious, as in the other, similar case in
gnu/tests/install.scm I moved

(target (operating-system-derivation target-os))

down to have mlet* do the shadowing properly.

Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
J
J
Jan Nieuwenhuizen wrote on 14 Jun 2020 15:18
Re: [bug#41785] [PATCH v4] services: Add 'hurd-in-vm service-type'.
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87o8pleqg5.fsf@gnu.org
Mathieu Othacehe writes:

Hey Mathieu,

Toggle quote (15 lines)
>> (define (hurd-vm-disk-image config)
>> "Return a disk-image for the Hurd according to CONFIG."
>> (with-parameters ((%current-target-system "i586-pc-gnu"))
>> (let ((os (hurd-vm-configuration-os config))
>> (disk-size (hurd-vm-configuration-disk-size config)))
>> (system-image
>> (image
>> (inherit hurd-disk-image)
>> (size disk-size)
>> (operating-system os))))))
>
> Nice! I'm not sure why we still need the "with-parameters" call because,
> it's done first thing in "system-image". I'd like to understand this
> before proceeding, do you have a branch where I could test it?

It's on my gitlab master...almost ready to push ;-)

Initially I did not have this "with-parameters", but then in
gnu/system.scm:

Toggle snippet (10 lines)
(define (hurd-multiboot-modules os)
...
(libc (if target
(with-parameters ((%current-target-system #f))
;; TODO: cross-libc has extra patches for the Hurd;
;; remove in next rebuild cycle
(cross-libc target))
glibc))

we take the ELSE branch here -- and that does not work. AIUI, this is
really a temporary kludge until the next rebuild cycle we can move some
hurd-specific glibc patches from cross-libc to glibc-proper.

...but it's tricky to see how this all plays together, not entirely
clear to me anyway...

Toggle quote (11 lines)
>> - ;; FIXME: %current-target-system may not be bound https://bugs.gnu.org/29296
>> - (let ((target (%current-target-system)))
>> + (let-system (system target)
>> (with-extensions (list shepherd)
>> (computed-file (string-append (basename (scheme-file-name file) ".scm")
>> ".go")
>>
>> produces a working hurd-vm service!
>
> Good catch!

:-)
Almost there...

Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
J
J
Jan Nieuwenhuizen wrote on 14 Jun 2020 15:32
Re: [bug#41785] [PATCH v3 1/2] image: Make 'find-image' non-monadic.
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87k109ept2.fsf@gnu.org
Jan Nieuwenhuizen writes:

Hello again,

Toggle quote (17 lines)
>>>>> + ((image -> (find-image file-system-type (%current-target-system)))
>>>>> (sys (system-derivation-for-action os image action
>>>>> #:file-system-type file-system-type
>>>>> #:image-size image-size
>>>>
>>>> Same here, otherwise this look good to me.
>>
>> I missed something, "target" variable is shadowing the "target"
>> argument. The attached patch should fix the issue, I'm testing it.
>
> Oops!
>
>> From 760e46d2ff57fab2b67656b85d800572bbc56f4b Mon Sep 17 00:00:00 2001
>> From: Mathieu Othacehe <othacehe@gnu.org>
>> Date: Sun, 14 Jun 2020 14:16:37 +0200
>> Subject: [PATCH] scripts: system: Fix "init" command.

Just to confirm that this fixes

make check-system TESTS="installed-extlinux-os"

failing early, where it now possibly fails for lack of disk
space...retrying...

Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
M
M
Mathieu Othacehe wrote on 14 Jun 2020 17:44
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
87a715k5yx.fsf@gnu.org
Toggle quote (4 lines)
> Just to confirm that this fixes
>
> make check-system TESTS="installed-extlinux-os"

Ok, thanks for checking. I pushed this one.

Toggle quote (3 lines)
> failing early, where it now possibly fails for lack of disk
> space...retrying...

Yup, that's another issue I'm working on!

Thanks,

Mathieu
M
M
Mathieu Othacehe wrote on 14 Jun 2020 17:52
Re: [bug#41785] [PATCH v4] services: Add 'hurd-in-vm service-type'.
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
875zbtk5l8.fsf@gnu.org
Toggle quote (13 lines)
> (define (hurd-multiboot-modules os)
> ...
> (libc (if target
> (with-parameters ((%current-target-system #f))
> ;; TODO: cross-libc has extra patches for the Hurd;
> ;; remove in next rebuild cycle
> (cross-libc target))
> glibc))
>
> we take the ELSE branch here -- and that does not work. AIUI, this is
> really a temporary kludge until the next rebuild cycle we can move some
> hurd-specific glibc patches from cross-libc to glibc-proper.

Oh, I know why! In "system-image", there are a few calls, in the first
let, that are outside the "with-parameters", in particular "bootcfg"
which triggers the problem you are describing above.

Maybe something as naive as:

Toggle snippet (26 lines)
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 1bda25fd7f..8a3c4e22d9 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -525,14 +525,14 @@ it can be used for bootloading."
image, depending on IMAGE format."
(define substitutable? (image-substitutable? image))
- (let* ((os (operating-system-for-image image))
- (image* (image-with-os image os))
- (target (image-target image))
- (register-closures? (has-guix-service-type? os))
- (bootcfg (operating-system-bootcfg os))
- (bootloader (bootloader-configuration-bootloader
- (operating-system-bootloader os))))
- (with-parameters ((%current-target-system target))
+ (with-parameters ((%current-target-system target))
+ (let* ((os (operating-system-for-image image))
+ (image* (image-with-os image os))
+ (target (image-target image))
+ (register-closures? (has-guix-service-type? os))
+ (bootcfg (operating-system-bootcfg os))
+ (bootloader (bootloader-configuration-bootloader
+ (operating-system-bootloader os))))

would do the trick.

Thanks,

Mathieu
M
M
Mathieu Othacehe wrote on 14 Jun 2020 18:22
(name . Jan Nieuwenhuizen)(address . janneke@gnu.org)
87sgexiplv.fsf@gnu.org
Toggle quote (11 lines)
> + (with-parameters ((%current-target-system target))
> + (let* ((os (operating-system-for-image image))
> + (image* (image-with-os image os))
> + (target (image-target image))
> + (register-closures? (has-guix-service-type? os))
> + (bootcfg (operating-system-bootcfg os))
> + (bootloader (bootloader-configuration-bootloader
> + (operating-system-bootloader os))))
>
> would do the trick.

I went ahead and pushed a variant of this as
c9f6e2e5bdff186583bdc360832b57f4c56e3427.

Thanks,

Mathieu
J
J
Jan Nieuwenhuizen wrote on 14 Jun 2020 18:42
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87mu558uql.fsf@gnu.org
Mathieu Othacehe writes:

Toggle quote (17 lines)
>> (define (hurd-multiboot-modules os)
>> ...
>> (libc (if target
>> (with-parameters ((%current-target-system #f))
>> ;; TODO: cross-libc has extra patches for the Hurd;
>> ;; remove in next rebuild cycle
>> (cross-libc target))
>> glibc))
>>
>> we take the ELSE branch here -- and that does not work. AIUI, this is
>> really a temporary kludge until the next rebuild cycle we can move some
>> hurd-specific glibc patches from cross-libc to glibc-proper.
>
> Oh, I know why! In "system-image", there are a few calls, in the first
> let, that are outside the "with-parameters", in particular "bootcfg"
> which triggers the problem you are describing above.

Ah, yes that makes sense.

Toggle quote (4 lines)
> Maybe something as naive as:
>
> diff --git a/gnu/system/image.scm b/gnu/system/image.scm

[..]
Toggle quote (1 lines)
> - (let* ((os (operating-system-for-image image))
[..]
Toggle quote (6 lines)
> - (with-parameters ((%current-target-system target))
> + (with-parameters ((%current-target-system target))
> + (let* ((os (operating-system-for-image image))
>
> would do the trick.

It does!

Toggle quote (3 lines)
> I went ahead and pushed a variant of this as
> c9f6e2e5bdff186583bdc360832b57f4c56e3427.

Woohoo! I remove the with-parameters in hurd-vm-disk-image and pushed
to master 5e9cf93364d87c70f8bfad915417cd75d21c0fed

Greetings,
Janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
Closed
?