'guix system switch-generation' does not reload Shepherd services

Open

Details

7 participants

Brice Waegeneire
Chris Marusich
Christopher Lemmer Webber
Ludovic Courtès
Mark H Weaver
Robert Vollmert
Jakob L. Kreuze

Owner: unassigned

Submitted by: Robert Vollmert

Severity: important

Merged with

37596

Robert Vollmert wrote on 30 Jul 2019 12:00

guix system switch-generation doesn't

Recipients:(address . bug-guix@gnu.org)

Message-ID:7BE8190F-A8E9-454E-8F37-FBFE42FBDE10@vllmrt.net

What I see:

1. edit ~/pzprnode/pzprnode

rob@garp ~/pzprnode$ git diff

Toggle diff (57 lines)diff --git a/pzprnode b/pzprnode
index 612e6a8..d8ef0ea 100755
--- a/pzprnode
+++ b/pzprnode
@@ -190,5 +190,6 @@ const server = http.createServer((req, res) => {
 });
 
 server.listen(port, hostname, () => {
+       console.log("updated version");
        console.log(`Server running at http://${hostname}:${port}/`);
 });

2. sudo guix system reconfigure -L ~/garp-config ~/garp-config/config.scm
3. sudo herd restart pzprnode
4. less /var/log/messages

Jul 30 11:46:57 localhost shepherd[1]: Service pzprnode has been stopped. 
Jul 30 11:46:57 localhost shepherd[1]: Service pzprnode has been started. 
Jul 30 11:46:58 localhost pzprnode[4954]: updated version 
Jul 30 11:46:58 localhost pzprnode[4954]: Server running at http://127.0.0.1:3456/ 

5. sudo guix system list-generations

Generation 151	Jul 30 2019 10:37:06
  file name: /var/guix/profiles/system-151-link
  canonical file name: /gnu/store/jis33accsfpa068aps0a9mrycmjzfm4m-system
  label: GNU with Linux-Libre 5.2.1
  bootloader: grub
  root device: label: "guix-root"
  kernel: /gnu/store/fp6wsvn10h1is0wkz8l2sbzjmjbzi7vr-linux-libre-5.2.1/bzImage
Generation 152	Jul 30 2019 11:43:13	(current)
  file name: /var/guix/profiles/system-152-link
  canonical file name: /gnu/store/3z3wmaj0399kihqc372y91nzcjxc1myl-system
  label: GNU with Linux-Libre 5.2.1
  bootloader: grub
  root device: label: "guix-root"
  kernel: /gnu/store/fp6wsvn10h1is0wkz8l2sbzjmjbzi7vr-linux-libre-5.2.1/bzImage

6. sudo guix system switch-generation 151

substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivation will be built:
   /gnu/store/qvxbl3gm6406dbbkm8jigmpc8zi42lfw-grub.cfg.drv
building /gnu/store/qvxbl3gm6406dbbkm8jigmpc8zi42lfw-grub.cfg.drv...
switched from generation 152 to 151

7. sudo herd restart pzprnode
8. less /var/log/messages

Jul 30 11:48:02 localhost shepherd[1]: Service pzprnode has been stopped. 
Jul 30 11:48:02 localhost shepherd[1]: Service pzprnode has been started. 
Jul 30 11:48:03 localhost pzprnode[4994]: updated version 
Jul 30 11:48:03 localhost pzprnode[4994]: Server running at http://127.0.0.1:3456/ 

The line with “updated version” should not be there.

Presumably, this is due to switch-generations not calling upgrade-shepherd-services.

Jakob L. Kreuze wrote on 30 Jul 2019 18:16

Recipients:(name . Robert Vollmert)(address . rob@vllmrt.net)(address . 36855@debbugs.gnu.org)

Message-ID:87zhkvv6pm.fsf@sdf.lonestar.org

Hi Robert,

Robert Vollmert <rob@vllmrt.net> writes:

Toggle quote (5 lines)

> The line with “updated version” should not be there.

> Presumably, this is due to switch-generations not calling

> upgrade-shepherd-services.

I can confirm that 'switch-to-system-generation', the procedure that
carries out 'guix system switch-generation', neither invokes
'upgrade-shepherd-services' nor runs that system's activation script.

I'd be interested to hear from others who have worked on 'guix system'
as to whether or not it would be a good idea to perform the effectful
parts of a system activation for 'switch-generation'. I haven't yet
pondered upon the implications.

Regards,
Jakob

Robert Vollmert wrote on 6 Aug 2019 23:21

Re: guix system switch-generation doesn't

Recipients:(address . bug-guix@gnu.org)(name . GitHub Developer Support)(address . developer@githubsupport.com)

Message-ID:3F889811-B4FA-415F-B4C0-7994DD00A968@vllmrt.net

Could we get some input on this bug?

Maybe I’m misunderstanding something, but it seems that a core guix

feature (atomic rollbacks) doesn’t work…

Toggle quote (65 lines)> On 30. Jul 2019, at 12:00, Robert Vollmert <rob@vllmrt.net> wrote:
> 
> What I see:
> 
> 1. edit ~/pzprnode/pzprnode
> 
> rob@garp ~/pzprnode$ git diff
> diff --git a/pzprnode b/pzprnode
> index 612e6a8..d8ef0ea 100755
> --- a/pzprnode
> +++ b/pzprnode
> @@ -190,5 +190,6 @@ const server = http.createServer((req, res) => {
> });
> 
> server.listen(port, hostname, () => {
> +       console.log("updated version");
>        console.log(`Server running at http://${hostname}:${port}/`);
> });
> 
> 2. sudo guix system reconfigure -L ~/garp-config ~/garp-config/config.scm
> 3. sudo herd restart pzprnode
> 4. less /var/log/messages
> 
> Jul 30 11:46:57 localhost shepherd[1]: Service pzprnode has been stopped. 
> Jul 30 11:46:57 localhost shepherd[1]: Service pzprnode has been started. 
> Jul 30 11:46:58 localhost pzprnode[4954]: updated version 
> Jul 30 11:46:58 localhost pzprnode[4954]: Server running at http://127.0.0.1:3456/ 
> 
> 5. sudo guix system list-generations
> 
> Generation 151	Jul 30 2019 10:37:06
>  file name: /var/guix/profiles/system-151-link
>  canonical file name: /gnu/store/jis33accsfpa068aps0a9mrycmjzfm4m-system
>  label: GNU with Linux-Libre 5.2.1
>  bootloader: grub
>  root device: label: "guix-root"
>  kernel: /gnu/store/fp6wsvn10h1is0wkz8l2sbzjmjbzi7vr-linux-libre-5.2.1/bzImage
> Generation 152	Jul 30 2019 11:43:13	(current)
>  file name: /var/guix/profiles/system-152-link
>  canonical file name: /gnu/store/3z3wmaj0399kihqc372y91nzcjxc1myl-system
>  label: GNU with Linux-Libre 5.2.1
>  bootloader: grub
>  root device: label: "guix-root"
>  kernel: /gnu/store/fp6wsvn10h1is0wkz8l2sbzjmjbzi7vr-linux-libre-5.2.1/bzImage
> 
> 6. sudo guix system switch-generation 151
> 
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> The following derivation will be built:
>   /gnu/store/qvxbl3gm6406dbbkm8jigmpc8zi42lfw-grub.cfg.drv
> building /gnu/store/qvxbl3gm6406dbbkm8jigmpc8zi42lfw-grub.cfg.drv...
> switched from generation 152 to 151
> 
> 7. sudo herd restart pzprnode
> 8. less /var/log/messages
> 
> Jul 30 11:48:02 localhost shepherd[1]: Service pzprnode has been stopped. 
> Jul 30 11:48:02 localhost shepherd[1]: Service pzprnode has been started. 
> Jul 30 11:48:03 localhost pzprnode[4994]: updated version 
> Jul 30 11:48:03 localhost pzprnode[4994]: Server running at http://127.0.0.1:3456/ 
> 
> The line with “updated version” should not be there.
> 
> Presumably, this is due to switch-generations not calling upgrade-shepherd-services.
>

Robert Vollmert wrote on 6 Aug 2019 23:25

Recipients:(address . 36855@debbugs.gnu.org)(address . guix-devel@gnu.org)

Message-ID:A3C3B87A-1B74-4983-A8EA-7281E3103567@vllmrt.net

Could we get some input on this bug?

Maybe I’m misunderstanding something, but it seems that a core guix

feature (atomic rollbacks) doesn’t work…

Toggle quote (65 lines)> On 30. Jul 2019, at 12:00, Robert Vollmert <rob@vllmrt.net> wrote:
> 
> What I see:
> 
> 1. edit ~/pzprnode/pzprnode
> 
> rob@garp ~/pzprnode$ git diff
> diff --git a/pzprnode b/pzprnode
> index 612e6a8..d8ef0ea 100755
> --- a/pzprnode
> +++ b/pzprnode
> @@ -190,5 +190,6 @@ const server = http.createServer((req, res) => {
> });
> 
> server.listen(port, hostname, () => {
> +       console.log("updated version");
>       console.log(`Server running at http://${hostname}:${port}/`);
> });
> 
> 2. sudo guix system reconfigure -L ~/garp-config ~/garp-config/config.scm
> 3. sudo herd restart pzprnode
> 4. less /var/log/messages
> 
> Jul 30 11:46:57 localhost shepherd[1]: Service pzprnode has been stopped. 
> Jul 30 11:46:57 localhost shepherd[1]: Service pzprnode has been started. 
> Jul 30 11:46:58 localhost pzprnode[4954]: updated version 
> Jul 30 11:46:58 localhost pzprnode[4954]: Server running at http://127.0.0.1:3456/ 
> 
> 5. sudo guix system list-generations
> 
> Generation 151	Jul 30 2019 10:37:06
> file name: /var/guix/profiles/system-151-link
> canonical file name: /gnu/store/jis33accsfpa068aps0a9mrycmjzfm4m-system
> label: GNU with Linux-Libre 5.2.1
> bootloader: grub
> root device: label: "guix-root"
> kernel: /gnu/store/fp6wsvn10h1is0wkz8l2sbzjmjbzi7vr-linux-libre-5.2.1/bzImage
> Generation 152	Jul 30 2019 11:43:13	(current)
> file name: /var/guix/profiles/system-152-link
> canonical file name: /gnu/store/3z3wmaj0399kihqc372y91nzcjxc1myl-system
> label: GNU with Linux-Libre 5.2.1
> bootloader: grub
> root device: label: "guix-root"
> kernel: /gnu/store/fp6wsvn10h1is0wkz8l2sbzjmjbzi7vr-linux-libre-5.2.1/bzImage
> 
> 6. sudo guix system switch-generation 151
> 
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> The following derivation will be built:
>  /gnu/store/qvxbl3gm6406dbbkm8jigmpc8zi42lfw-grub.cfg.drv
> building /gnu/store/qvxbl3gm6406dbbkm8jigmpc8zi42lfw-grub.cfg.drv...
> switched from generation 152 to 151
> 
> 7. sudo herd restart pzprnode
> 8. less /var/log/messages
> 
> Jul 30 11:48:02 localhost shepherd[1]: Service pzprnode has been stopped. 
> Jul 30 11:48:02 localhost shepherd[1]: Service pzprnode has been started. 
> Jul 30 11:48:03 localhost pzprnode[4994]: updated version 
> Jul 30 11:48:03 localhost pzprnode[4994]: Server running at http://127.0.0.1:3456/ 
> 
> The line with “updated version” should not be there.
> 
> Presumably, this is due to switch-generations not calling upgrade-shepherd-services.
>

Christopher Lemmer Webber wrote on 7 Aug 2019 21:37

Re: bug#36855: guix system switch-generation doesn't

Recipients:(address . bug-guix@gnu.org)

Message-ID:87zhkkojfv.fsf@dustycloud.org

Could you look at bug #36878 and commit 1db6f137d... as of latest

master, is this fixed?

Robert Vollmert writes:

Toggle quote (70 lines)> Could we get some input on this bug?
>
> Maybe I’m misunderstanding something, but it seems that a core guix
> feature (atomic rollbacks) doesn’t work…
>
>> On 30. Jul 2019, at 12:00, Robert Vollmert <rob@vllmrt.net> wrote:
>> 
>> What I see:
>> 
>> 1. edit ~/pzprnode/pzprnode
>> 
>> rob@garp ~/pzprnode$ git diff
>> diff --git a/pzprnode b/pzprnode
>> index 612e6a8..d8ef0ea 100755
>> --- a/pzprnode
>> +++ b/pzprnode
>> @@ -190,5 +190,6 @@ const server = http.createServer((req, res) => {
>> });
>> 
>> server.listen(port, hostname, () => {
>> +       console.log("updated version");
>>       console.log(`Server running at http://${hostname}:${port}/`);
>> });
>> 
>> 2. sudo guix system reconfigure -L ~/garp-config ~/garp-config/config.scm
>> 3. sudo herd restart pzprnode
>> 4. less /var/log/messages
>> 
>> Jul 30 11:46:57 localhost shepherd[1]: Service pzprnode has been stopped. 
>> Jul 30 11:46:57 localhost shepherd[1]: Service pzprnode has been started. 
>> Jul 30 11:46:58 localhost pzprnode[4954]: updated version 
>> Jul 30 11:46:58 localhost pzprnode[4954]: Server running at http://127.0.0.1:3456/ 
>> 
>> 5. sudo guix system list-generations
>> 
>> Generation 151	Jul 30 2019 10:37:06
>> file name: /var/guix/profiles/system-151-link
>> canonical file name: /gnu/store/jis33accsfpa068aps0a9mrycmjzfm4m-system
>> label: GNU with Linux-Libre 5.2.1
>> bootloader: grub
>> root device: label: "guix-root"
>> kernel: /gnu/store/fp6wsvn10h1is0wkz8l2sbzjmjbzi7vr-linux-libre-5.2.1/bzImage
>> Generation 152	Jul 30 2019 11:43:13	(current)
>> file name: /var/guix/profiles/system-152-link
>> canonical file name: /gnu/store/3z3wmaj0399kihqc372y91nzcjxc1myl-system
>> label: GNU with Linux-Libre 5.2.1
>> bootloader: grub
>> root device: label: "guix-root"
>> kernel: /gnu/store/fp6wsvn10h1is0wkz8l2sbzjmjbzi7vr-linux-libre-5.2.1/bzImage
>> 
>> 6. sudo guix system switch-generation 151
>> 
>> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
>> The following derivation will be built:
>>  /gnu/store/qvxbl3gm6406dbbkm8jigmpc8zi42lfw-grub.cfg.drv
>> building /gnu/store/qvxbl3gm6406dbbkm8jigmpc8zi42lfw-grub.cfg.drv...
>> switched from generation 152 to 151
>> 
>> 7. sudo herd restart pzprnode
>> 8. less /var/log/messages
>> 
>> Jul 30 11:48:02 localhost shepherd[1]: Service pzprnode has been stopped. 
>> Jul 30 11:48:02 localhost shepherd[1]: Service pzprnode has been started. 
>> Jul 30 11:48:03 localhost pzprnode[4994]: updated version 
>> Jul 30 11:48:03 localhost pzprnode[4994]: Server running at http://127.0.0.1:3456/ 
>> 
>> The line with “updated version” should not be there.
>> 
>> Presumably, this is due to switch-generations not calling upgrade-shepherd-services.
>>

Jakob L. Kreuze wrote on 7 Aug 2019 21:57

Recipients:(name . Christopher Lemmer Webber)(address . cwebber@dustycloud.org)

Message-ID:877e7on3zd.fsf@sdf.lonestar.org

Hi Chris,

Christopher Lemmer Webber <cwebber@dustycloud.org> writes:

Toggle quote (3 lines)

> Could you look at bug #36878 and commit 1db6f137d... as of latest

> master, is this fixed?

Unfortunately, I don't think that 1db6f137d fixes this. The issue is a
bit more structural as 'switch-to-system-generation' doesn't call out to
'upgrade-shepherd-services'. I'm not sure if this was an intentional
decision or not (perhaps we can ask Ludo when he returns).

Regards,
Jakob

Chris Marusich wrote on 8 Aug 2019 18:40

Recipients:(name . Jakob L. Kreuze)(address . zerodaysfordays@sdf.lonestar.org)

Message-ID:87h86ry5j5.fsf@gmail.com

Hi Jakob,

zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:

Toggle quote (4 lines)

> 'switch-to-system-generation' doesn't call out to

> 'upgrade-shepherd-services'. I'm not sure if this was an intentional

> decision or not

It is intentional, but only because there is currently no way to call
upgrade-shepherd-services when switching system generations.

Consider the procedure upgrade-shepherd-services: you must pass it an
<operating-system> record.  When you are flipping from one generation to
another, how do you get the <operating-system> record that was used for
the generation you're switching to?  Guix doesn't currently store the
operating system configuration file or its <operating-system> record
anywhere, so we can't call upgrade-shepherd-services.

This was discussed in 2016 and we agreed we need to persist some
information to enable us to handle Shepherd services correctly.  This is
what Ludo suggested at the time:

https://lists.gnu.org/archive/html/guix-devel/2016-06/msg00173.html

"Maybe we could store in the system output (result of ‘guix system
build’) an sexp representation of (part of) our <shepherd-service>
records:

  (shepherd-service
    (provisions (x y z))
    (requirements (a b c))
    (start-script "/gnu/store/…-start-foo.scm")
    (stop-script "/gnu/store/…-stop-foo.scm")
    …)

Then ‘upgrade-shepherd-services’ could start from this simplified
representation instead of using the full-blown <shepherd-service>
objects, and thus could work both when instantiating a new generation
and when rolling back."

Until that happens, you'll always have to reboot to complete the
switch.

FYI, last I checked (about 3 years ago), in NixOS they took a slightly
different approach: instead of storing state describing the previous
system generation and relying on the current system's logic to correctly
parse it and use it to revert the system to a prior configuration, they
just dump everything into a self-contained script that knows how to
update the entire system to one specific configuration.  That approach
is nice in some ways because switching generations is dead simple - you
just run the switching script belonging to the generation you want to
switch to - but it also has downsides.

For example, if the target generation is old enough compared to the
current system, then the target generation's old switching script might
not understand how to deal with the current system correctly.  Instead,
if you only store the bare minimum of state required to take the right
actions, and you implement the meat of the logic in the current Guix
installation, you are more likely to be able to switch generations even
to very old ones where the world was very different, since the current
Guix can be taught how to deal gracefully with the old world.  But it
seems more complicated.  It's all about trade-offs.

That said, I've never gone back to implement this.  If you want to give
it a try, you're more than welcome to do so!

-- 
Chris

Robert Vollmert wrote on 8 Aug 2019 19:03

Recipients:(name . Chris Marusich)(address . cmmarusich@gmail.com)

Message-ID:51A8B412-1AE3-4464-8146-DE8B19B7C4DF@vllmrt.net

On 8. Aug 2019, at 18:40, Chris Marusich <cmmarusich@gmail.com> wrote:

Toggle quote (9 lines)

> zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:

>> 'switch-to-system-generation' doesn't call out to

>> 'upgrade-shepherd-services'. I'm not sure if this was an intentional

>> decision or not

> It is intentional, but only because there is currently no way to call

> upgrade-shepherd-services when switching system generations.

How does shepherd work on a non-guix system? Can’t be it be configured

like other daemons to read its configuration from a file, e.g. from

/run/current-system/etc/shepherd.conf

and be told via signal to reload its configuration from disk?

…

(I feel a bit cheated right now. This behaviour makes Guix System entirely

unsuitable for server use. It shouldn’t be advertised as supporting

transactional upgrades and rollbacks if those require a reboot.)

Chris Marusich wrote on 9 Aug 2019 09:35

Recipients:(name . Robert Vollmert)(address . rob@vllmrt.net)

Message-ID:87o90yvlin.fsf@gmail.com

Hi Robert,

Robert Vollmert <rob@vllmrt.net> writes:

Toggle quote (17 lines)> On 8. Aug 2019, at 18:40, Chris Marusich <cmmarusich@gmail.com> wrote:
>> zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:
>> 
>>> 'switch-to-system-generation' doesn't call out to
>>> 'upgrade-shepherd-services'. I'm not sure if this was an intentional
>>> decision or not
>> 
>> It is intentional, but only because there is currently no way to call
>> upgrade-shepherd-services when switching system generations.
>
> How does shepherd work on a non-guix system? Can’t be it be configured
> like other daemons to read its configuration from a file, e.g. from
>
>    /run/current-system/etc/shepherd.conf
>
> and be told via signal to reload its configuration from disk?

Maybe!  In the email thread I linked, Ludo talked about storing a
description of the Shepherd services in the system generation for future
reference.  Maybe we could store it in a place like this, and maybe
Shepherd already has mechanisms for reloading configurations like this.
I don't intend to work on this because I need to focus on other things
right now, but I would be happy if someone took up this work!

Toggle quote (4 lines)

> (I feel a bit cheated right now. This behaviour makes Guix System entirely

> unsuitable for server use. It shouldn’t be advertised as supporting

> transactional upgrades and rollbacks if those require a reboot.)

I agree that Guix should update as many Shepherd services as it can when
switching generations.  However, I don't think it's inaccurate to say
that Guix supports transactional upgrades and rollbacks.  When you
invoke "guix system switch-generation", the system profile symlink is
flipped atomically, so you get an atomic update from one version of the
system to another.  Software running in the system never sees an
inconsistent view of the system.  Contrast this with nearly any other
mutable GNU/Linux system, in which files are more or less sprayed into
the existing file system with no guarantee of consistency or atomicity.

-- 
Chris

Ludovic Courtès wrote on 26 Aug 2019 12:07

Recipients:(name . Chris Marusich)(address . cmmarusich@gmail.com)

Message-ID:874l241bq6.fsf@gnu.org

Hey Chris & Jakob,

Chris Marusich <cmmarusich@gmail.com> skribis:

Toggle quote (9 lines)

> zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:

>> 'switch-to-system-generation' doesn't call out to

>> 'upgrade-shepherd-services'. I'm not sure if this was an intentional

>> decision or not

> It is intentional, but only because there is currently no way to call

> upgrade-shepherd-services when switching system generations.

[...]

Toggle quote (10 lines)> FYI, last I checked (about 3 years ago), in NixOS they took a slightly
> different approach: instead of storing state describing the previous
> system generation and relying on the current system's logic to correctly
> parse it and use it to revert the system to a prior configuration, they
> just dump everything into a self-contained script that knows how to
> update the entire system to one specific configuration.  That approach
> is nice in some ways because switching generations is dead simple - you
> just run the switching script belonging to the generation you want to
> switch to - but it also has downsides.

Jakob, now that we generate scripts for the effectful bits of system
reconfiguration (one of these bits being service upgrades), couldn’t we
take it one step further and store those scripts in the “system”
derivation so we can run them eventually, notably upon
‘switch-generation’?

Toggle quote (10 lines)> For example, if the target generation is old enough compared to the
> current system, then the target generation's old switching script might
> not understand how to deal with the current system correctly.  Instead,
> if you only store the bare minimum of state required to take the right
> actions, and you implement the meat of the logic in the current Guix
> installation, you are more likely to be able to switch generations even
> to very old ones where the world was very different, since the current
> Guix can be taught how to deal gracefully with the old world.  But it
> seems more complicated.  It's all about trade-offs.

Indeed.  The important thing to me is that from the GRUB menu you can
really switch to any generation.  I’ve actually never used
‘switch-generations’ on my laptop, but technically, I feel like storing
the “switch-to-system” script would be the easiest way.

Thanks,
Ludo’.

Mark H Weaver wrote on 26 Aug 2019 20:51

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87woezoj3p.fsf@netris.org

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (6 lines)

> Jakob, now that we generate scripts for the effectful bits of system

> reconfiguration (one of these bits being service upgrades), couldn’t we

> take it one step further and store those scripts in the “system”

> derivation so we can run them eventually, notably upon

> ‘switch-generation’?

As a bonus, this approach might solve another issue I've observed: on my
Guix system, where I build everything locally, several derivations are
built *during* activation.  Based on the terminal output, I get the
impression that the system is compiling things while the system in an
intermediate state, when some of the activation steps have been done,
but not all of them.

As I recall, the derivations built during activation are limited to
compiled modules for Guile, but it still sometimes takes on the order of
a minute or two on my laptop to complete the "activating system" steps.
This seems suboptimal.

The next time I update my system, I'll try to remember to keep a
transcript of this, so that I can be more specific.

      Best,
       Mark

Mark H Weaver wrote on 28 Aug 2019 02:34

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87tva2m8ki.fsf@netris.org

Hello again,

Mark H Weaver <mhw@netris.org> writes:

Toggle quote (23 lines)> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Jakob, now that we generate scripts for the effectful bits of system
>> reconfiguration (one of these bits being service upgrades), couldn’t we
>> take it one step further and store those scripts in the “system”
>> derivation so we can run them eventually, notably upon
>> ‘switch-generation’?
>
> As a bonus, this approach might solve another issue I've observed: on my
> Guix system, where I build everything locally, several derivations are
> built *during* activation.  Based on the terminal output, I get the
> impression that the system is compiling things while the system in an
> intermediate state, when some of the activation steps have been done,
> but not all of them.
>
> As I recall, the derivations built during activation are limited to
> compiled modules for Guile, but it still sometimes takes on the order of
> a minute or two on my laptop to complete the "activating system" steps.
> This seems suboptimal.
>
> The next time I update my system, I'll try to remember to keep a
> transcript of this, so that I can be more specific.

Here's a transcript:

Toggle snippet (19 lines)

activating system...

building /gnu/store/fbp6bbxw9cf617fmk57sddrz7zfsfw5p-module-import-compiled.drv...

building /gnu/store/wfi6hnr9ggal0s1d32xx5wbl5k5wqlvx-switch-to-system.scm.drv...

making '/gnu/store/mjzk53ia3bajn08lscpyzz5apcw3r70g-system' the current system...

setting up setuid programs in '/run/setuid-programs'...

populating /etc from /gnu/store/l7r1has973n26hfqrs6vxbi94xzgh360-etc...

building /gnu/store/h2fqcxv3xx14lkdhyphm3lawkayw7sdl-module-import-compiled.drv...

building /gnu/store/dar9smjyxmri6v6cchnmp5mpyiimyx64-install-bootloader.scm.drv...

guix system: bootloader successfully installed on '/dev/sda'

building /gnu/store/vkk3h5p799lfpmf6msdhrzlq0wqvk3zq-module-import-compiled.drv...

building /gnu/store/hn8sr8p13gg2mf379xawscabckp03fkb-upgrade-shepherd-services.scm.drv...

shepherd: Evaluating user expression (let* ((services (map primitive-load (?))) # ?) ?).

guix system: warning: only 3.9% of free space available on /gnu/store

hint: Consider deleting old profile generations and collecting garbage, along these lines:

guix gc --delete-generations=1m

Mark

Jakob L. Kreuze wrote on 28 Aug 2019 20:28

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:8736hlp2js.fsf@sdf.lonestar.org

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (6 lines)

> Jakob, now that we generate scripts for the effectful bits of system

> reconfiguration (one of these bits being service upgrades), couldn’t

> we take it one step further and store those scripts in the “system”

> derivation so we can run them eventually, notably upon

> ‘switch-generation’?

We'd need to find a way of serializing at least the relationships
between services, but I think it's possible (albeit quite involved). I
do really like the idea, though. That way, the system generation would
fully encompass the desired state of the system.

Regards,
Jakob

Jakob L. Kreuze wrote on 28 Aug 2019 20:33

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)

Message-ID:87y2zdnnrl.fsf@sdf.lonestar.org

Hi Mark,

Mark H Weaver <mhw@netris.org> writes:

Toggle quote (40 lines)> Hello again,
>
> Mark H Weaver <mhw@netris.org> writes:
>
>> As a bonus, this approach might solve another issue I've observed: on my
>> Guix system, where I build everything locally, several derivations are
>> built *during* activation.  Based on the terminal output, I get the
>> impression that the system is compiling things while the system in an
>> intermediate state, when some of the activation steps have been done,
>> but not all of them.
>>
>> As I recall, the derivations built during activation are limited to
>> compiled modules for Guile, but it still sometimes takes on the order of
>> a minute or two on my laptop to complete the "activating system" steps.
>> This seems suboptimal.
>>
>> The next time I update my system, I'll try to remember to keep a
>> transcript of this, so that I can be more specific.
>
> Here's a transcript:
>
> activating system...
> building /gnu/store/fbp6bbxw9cf617fmk57sddrz7zfsfw5p-module-import-compiled.drv...
> building /gnu/store/wfi6hnr9ggal0s1d32xx5wbl5k5wqlvx-switch-to-system.scm.drv...
> making '/gnu/store/mjzk53ia3bajn08lscpyzz5apcw3r70g-system' the current system...
> setting up setuid programs in '/run/setuid-programs'...
> populating /etc from /gnu/store/l7r1has973n26hfqrs6vxbi94xzgh360-etc...
> building /gnu/store/h2fqcxv3xx14lkdhyphm3lawkayw7sdl-module-import-compiled.drv...
> building /gnu/store/dar9smjyxmri6v6cchnmp5mpyiimyx64-install-bootloader.scm.drv...
> guix system: bootloader successfully installed on '/dev/sda'
> building /gnu/store/vkk3h5p799lfpmf6msdhrzlq0wqvk3zq-module-import-compiled.drv...
> building /gnu/store/hn8sr8p13gg2mf379xawscabckp03fkb-upgrade-shepherd-services.scm.drv...
> shepherd: Evaluating user expression (let* ((services (map primitive-load (?))) # ?) ?).
> guix system: warning: only 3.9% of free space available on /gnu/store
> hint: Consider deleting old profile generations and collecting garbage, along these lines:
>
>      guix gc --delete-generations=1m
>
>        Mark

Thanks for the input; I wasn't aware that the activation process was
taking so long for some people. One of Ludovic's suggestions was to
create a single derivation, rather than three, to speed up system
activation. I'll look into this further.

Regards,
Jakob

Mark H Weaver wrote on 28 Aug 2019 20:46

Recipients:(name . Jakob L. Kreuze)(address . zerodaysfordays@sdf.lonestar.org)

Message-ID:875zmhm8jo.fsf@netris.org

Hi Jakob,

zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:

Toggle quote (5 lines)

> Thanks for the input; I wasn't aware that the activation process was

> taking so long for some people. One of Ludovic's suggestions was to

> create a single derivation, rather than three, to speed up system

> activation. I'll look into this further.

To be clear, I don't care how long it takes to build these derivations.

However, I think they should all be built before starting to activate

the system. Does that make sense?

On a side note, what would happen if one of those builds failed? Would

the system activation be left half-done?

Thanks,

Mark

Jakob L. Kreuze wrote on 29 Aug 2019 02:08

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)

Message-ID:87k1awomto.fsf@sdf.lonestar.org

Hi Mark,

Mark H Weaver <mhw@netris.org> writes:

Toggle quote (13 lines)> Hi Jakob,
>
> zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:
>
>> Thanks for the input; I wasn't aware that the activation process was
>> taking so long for some people. One of Ludovic's suggestions was to
>> create a single derivation, rather than three, to speed up system
>> activation. I'll look into this further.
>
> To be clear, I don't care how long it takes to build these
> derivations. However, I think they should all be built before starting
> to activate the system. Does that make sense?

Yes, and I agree that it would be ideal to have the derivations built

prior to system activation -- that way, the activation scripts could be

included in 'show-what-to-build*'.

Toggle quote (3 lines)

> On a side note, what would happen if one of those builds failed? Would

> the system activation be left half-done?

Yes. You raise another very good reason for why system activation should

be carried out by a single, atomic activation script :)

Regards,

Jakob

Ludovic Courtès wrote on 6 Nov 2019 18:46

control message for bug #36855

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87woccvqxx.fsf@gnu.org

merge 36855 37596

quit

Ludovic Courtès wrote on 6 Nov 2019 18:46

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87v9rwvqxf.fsf@gnu.org

retitle 36855 'guix system switch-generation' does not reload Shepherd services

quit

Ludovic Courtès wrote on 6 Nov 2019 18:46

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87tv7gvqxa.fsf@gnu.org

severity 36855 important

quit

Brice Waegeneire wrote on 9 Mar 2021 08:01

Re: bug#37596: 'guix system switch-generation' does not reload Shepherd services

Recipients:(name . Robert Vollmert)(address . rob@vllmrt.net)

Message-ID:877dmgzu7l.fsf@waegenei.re

Hello Guix,

To summarise the whole thread, in addition to the main issue explained

in the title “'guix system switch-generation' does not reload Shepherd

services“ there are 2 other issues talked about.

1. The activation script isn't executed by 'guix system

switch-generation', which is fixed in

df138dc20858725b90ed77be85f3318cbe1be73a and later.

2. The activation process isn't exclusivly build during the system

building process, whihc may the leave the system in half-hazard state

if the building taking place during the activation fail.

The second side issue should proablby have it's own bug report since

it's still present and we should keep that bug report only for the lack

of reloading of shepherd service when using 'guix system

switch-generation' (or 'roll-back').

Cheers,

- Brice

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 36855@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 36855

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date