hidden derivation build with inferiors

OpenSubmitted by Julien Lepiller.
Details
2 participants
  • Julien Lepiller
  • Ludovic Courtès
Owner
unassigned
Severity
important
J
J
Julien Lepiller wrote on 26 Jun 01:47 +0200
(address . bug-guix@gnu.org)
20200626014712.5620ac06@tachikoma.lepiller.eu
Hi,
someone (kernel-help) on IRC today was having issues using an inferiorkernel with their os configuration and reported that guix was"hanging". It turns out that guix was actually building the kernel inthe background, but did not report anything to the user. kernel-helpwas kind enough to share their config, and I can confirm that guix isbuilding the kernel, but doesn't tell anything.
I was able to track this done to a call to operating-system-derivation,which is supposed to return the path to a .drv file. I did not expectit to have to build anything, since it's only computing a derivationname.
Using the attached operating system declaration, defining it as "os",you can see that calling ((operating-system-derivation os)(open-connection)) "hangs" with no message, and a build user isbuilding the kernel (unless you already have the kernel built).
Note that the build process continues even if you interrupt thatprocedure, and only stops after you exit guix repl entirely.
Going one step further, operating-system-derivation calls fold-services(which is fine), then service-value (which is also fine) and calls thatvalue with the store, which builds the kernel without saying anything.That value comes from the root service type, system-service-type and itssystem-derivation procedure.
With some pk, I was able to find that mapm/accumulate-builds was theculprit, but I don't know how to investigate further.
(use-modules (gnu) (guix channels) (srfi srfi-1) (guix inferior) (gnu packages linux) (srfi srfi-1) (ice-9 match) (guix packages) (gnu packages gnome)) (use-service-modules desktop networking ssh xorg) (define kernel-inferior-channel ;; This is the old revision from which ;; we will obtain our desired kernel version. (list (channel (name 'guix) (url "https://git.savannah.gnu.org/git/guix.git") (commit "4f33bf5709a77a98f3d0330e45753d54675239ff")))) (define linux-libre-4.19-inferior (first (lookup-inferior-packages (inferior-for-channels kernel-inferior-channel) "linux-libre" "4.19.129"))) (define-public gnome-minimal (package (inherit gnome) (name "gnome-minimal") (propagated-inputs (remove (match-lambda ((name _) (member name '("baobab" "cheese" "eog" "epiphany" "evince" "file-roller" "gedit" "gnome-boxes" "gnome-calculator" "gnome-calendar" "gnome-characters" "gnome-clocks" "gnome-contacts" "gnome-disk-utility" "gnome-font-viewer" "gnome-maps" "gnome-weather" "simple-scan" "totem" "gnome-default-applications")))) (package-propagated-inputs gnome))))) (operating-system (locale "en_US.utf8") (kernel linux-libre-4.19-inferior) (timezone "America/Chicago") (keyboard-layout (keyboard-layout "us")) (host-name "t430") (users (cons* (user-account (name "brettg") (comment "Redacted") (group "users") (home-directory "/home/redacted") (supplementary-groups '("wheel" "netdev" "audio" "video"))) %base-user-accounts)) (packages (append (list (specification->package "nss-certs")) %base-packages)) (services (append (list (service gnome-desktop-service-type (gnome-desktop-configuration (inherit config) (gnome gnome-minimal))) (service openssh-service-type) (set-xorg-configuration (xorg-configuration (keyboard-layout keyboard-layout)))) %desktop-services)) (bootloader (bootloader-configuration (bootloader grub-efi-bootloader) (target "/boot/efi") (keyboard-layout keyboard-layout))) (swap-devices (list "/dev/sda2")) (file-systems (cons* (file-system (mount-point "/boot/efi") (device (uuid "0721-45AE" 'fat32)) (type "vfat")) (file-system (mount-point "/") (device (uuid "ce05e959-56a7-4520-ad73-556c82861537" 'ext4)) (type "ext4")) %base-file-systems)))
L
L
Ludovic Courtès wrote on 26 Jun 12:13 +0200
control message for bug #42053
(address . control@debbugs.gnu.org)
87wo3unnib.fsf@gnu.org
severity 42053 importantquit
L
L
Ludovic Courtès wrote on 26 Jun 12:17 +0200
Re: bug#42053: hidden derivation build with inferiors
(name . Julien Lepiller)(address . julien@lepiller.eu)(address . 42053@debbugs.gnu.org)
87mu4qnnbb.fsf@gnu.org
Hi,
Julien Lepiller <julien@lepiller.eu> skribis:
Toggle quote (7 lines)> someone (kernel-help) on IRC today was having issues using an inferior> kernel with their os configuration and reported that guix was> "hanging". It turns out that guix was actually building the kernel in> the background, but did not report anything to the user. kernel-help> was kind enough to share their config, and I can confirm that guix is> building the kernel, but doesn't tell anything.
Aaah right. What happens here is that grafts lead the inferior to startbuilding the kernel (this is expected). However, ‘proxy’ in (guixinferior) doesn’t forward build output to ‘current-build-output-port’.That’s why there’s no message saying what’s being built, no spinner, etc.
I’ll see what can be done.
The problem has always been there but it was perhaps less visible before710854304b1ab29332edcb76f3de532e0724c197, at least when substitutes wereavailable, because then the inferior wouldn’t start building things.
Ludo’.
J
J
Julien Lepiller wrote on 26 Jun 12:19 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 42053@debbugs.gnu.org)
DE1BF95D-B772-4AAD-A423-4FBA7C979F43@lepiller.eu
Le 26 juin 2020 06:17:28 GMT-04:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
Toggle quote (29 lines)>Hi,>>Julien Lepiller <julien@lepiller.eu> skribis:>>> someone (kernel-help) on IRC today was having issues using an>inferior>> kernel with their os configuration and reported that guix was>> "hanging". It turns out that guix was actually building the kernel in>> the background, but did not report anything to the user. kernel-help>> was kind enough to share their config, and I can confirm that guix is>> building the kernel, but doesn't tell anything.>>Aaah right. What happens here is that grafts lead the inferior to>start>building the kernel (this is expected). However, ‘proxy’ in (guix>inferior) doesn’t forward build output to ‘current-build-output-port’.>That’s why there’s no message saying what’s being built, no spinner,>etc.>>I’ll see what can be done.>>The problem has always been there but it was perhaps less visible>before>710854304b1ab29332edcb76f3de532e0724c197, at least when substitutes>were>available, because then the inferior wouldn’t start building things.>>Ludo’.
Thanks for the answer. The build also starts with -n, which I didn't expect either. I suppose this is also because the inferior doesn't report anything?
L
L
Ludovic Courtès wrote on 26 Jun 14:08 +0200
(name . Julien Lepiller)(address . julien@lepiller.eu)(address . 42053@debbugs.gnu.org)
87bll6ni73.fsf@gnu.org
Hi,
Julien Lepiller <julien@lepiller.eu> skribis:
Toggle quote (2 lines)> Le 26 juin 2020 06:17:28 GMT-04:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
[...]
Toggle quote (10 lines)>>The problem has always been there but it was perhaps less visible>>before>>710854304b1ab29332edcb76f3de532e0724c197, at least when substitutes>>were>>available, because then the inferior wouldn’t start building things.>>>>Ludo’.>> Thanks for the answer. The build also starts with -n, which I didn't expect either. I suppose this is also because the inferior doesn't report anything?
Yes, that and the fact that ‘-n’ no longer implies ‘--no-grafts’.
Ludo’.
L
L
Ludovic Courtès wrote on 28 Jun 22:43 +0200
(name . Julien Lepiller)(address . julien@lepiller.eu)(address . 42053@debbugs.gnu.org)
87pn9igbvj.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:
Toggle quote (16 lines)> Julien Lepiller <julien@lepiller.eu> skribis:>>> someone (kernel-help) on IRC today was having issues using an inferior>> kernel with their os configuration and reported that guix was>> "hanging". It turns out that guix was actually building the kernel in>> the background, but did not report anything to the user. kernel-help>> was kind enough to share their config, and I can confirm that guix is>> building the kernel, but doesn't tell anything.>> Aaah right. What happens here is that grafts lead the inferior to start> building the kernel (this is expected). However, ‘proxy’ in (guix> inferior) doesn’t forward build output to ‘current-build-output-port’.> That’s why there’s no message saying what’s being built, no spinner, etc.>> I’ll see what can be done.
Not surprisingly, to fix it, we need to implement a proxy thatinterprets all the messages exchanged so that it can write to‘current-build-output-port’ when appropriate (that’s the usualall-or-nothing problem with proxies.)
For the daemon rewrite in Scheme, we’ll need to have server stubsanyway, so perhaps that’s an opportunity to fiddle with that. Caleb?
Ludo’.
?