guix system reconfigure after kernel panic user or group not created

Hello Guix,

During 'guix system reconfigure' I got a kernel panic.

Getting it quite often while 'guix system reconfigure' at Linux magnolia

4.13.4-gnu #1 SMP 1 x86_64 GNU/Linux. But it's not a subject of current

report.

After reboot I tried another attempt to make a 'guix system

reconfigure', but because of /etc/group.lock and /etc/passwd.lock

operating-system groups and users fields didn't applied.

I tried to get a creation of adbusers group and add my user to it, but

because of those files I cannot do it, until manually removed them.

So, do we need to check if *.lock files are exist and remove them at the

start of 'guix system reconfigure' OR abort reconfigure and creation of

Grub boot entry OR something else?

Thanks,

Oleg.

Oleg Pykhalov <go.wigust@gmail.com> writes:

[...]

Kernel panic was caused by my miss configuring. I used a Guile module

on reconfigure via GUILE_LOAD_PATH and (use-modles (my-module) in

config.scm. I guess it's also required after reconfigure, according to

some times getting errors after reconfigure: cannot find my-module. So

I need to add my-module globally on system.

[...]

Hello,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Can you show the exact command and its output?

A user-land program is not supposed to be able to cause a kernel panic;

if it does, that’s a kernel bug.

But perhaps you got the kernel panic *after* rebooting in the

reconfigured system? That could well be a GuixSD bug, indeed.

I’m not sure I understand what happened. Could you copy/paste commands

and outputs, or transcribe what happened?

TIA,

Ludo’.

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Without seeing the config, I cannot tell whether it’s a bug on our side

or not.

However, if you’re confident that the bug is in your code, that’s good.

;-) Should we close this bug?

Thanks,

Ludo’.

Hello Ludovic,

apologies for not adding logs before. It's hard to do when I do guix

commands from Xterm and not from Emacs. Emacs *shell* or *compilation*

buffers will eat all memory if they get too much text.

Probably need to redirect STDOUT STDERR in file when Xterm do guix.

I heard Guix folks work on implementing tiny log output to console and

redirect everything else to a log file. This will be my life saver.

ludo@gnu.org (Ludovic Courtès) writes:

Sorry, as I said this is not a topic and I don't want to do it again and

I caution to make it on my current system.

I will setup a specific Guix VM for this, where I could make a 'system

reconfigure'. Then I'll create a new bug report with full log.

Neverless I'll leave a how-to reproduce it below for at least for myself

TODO list.

The problem

===========

The bigger problem from my view are files like /etc/group.lock and

/etc/passwd.lock. For example:

sudo touch /etc/group.lock

/etc/config.scm

(operating-system

;; …

(groups (cons

(user-group (name "test"))

%base-groups)))

reconfigure log

$ guix system reconfigure $HOME/dotfiles/guix/system-magnolia.scm

substitute: updating list of substitutes from 'https://berlin.guixsd.org'... 100.0%

The following derivations will be built:

/gnu/store/v9dp6193rpxrx1rqfdw59s5ss4wlrfdh-system.drv

/gnu/store/carkycnf6zcarbmnk5745pgsx1nv478y-grub.cfg.drv

/gnu/store/r5p953fx3dl18aav1ggwmiy2bqnv991s-activate-service.drv

/gnu/store/pjjm6595562ysk40zjrznhmsfsid1k8r-activate.drv

/gnu/store/l41adszqk24sb200dwm8sj6ky71ivwpi-boot.drv

/gnu/store/qqhzapsv5w8mrbz3s8hgy7w42r3dbyv9-system

/gnu/store/b4i4drp7lpxmgpcfkbvgmrig2hlszl3j-grub.cfg

/gnu/store/0b459jxdmyz5vf22avav9sm8ig03173k-grub-efi-2.02

/gnu/store/ijw065yljn1np4x0p5l1qkx9w4z9ikcl-bootloader-installer

activating system...

making '/gnu/store/qqhzapsv5w8mrbz3s8hgy7w42r3dbyv9-system' the current system...

setting up setuid programs in '/run/setuid-programs'...

populating /etc from /gnu/store/iyr9ji3idg3iphi3fskh2hqjlmg4h5w0-etc...

usermod: no changes

adding group 'test'...

groupadd: existing lock file /etc/group.lock without a PID

groupadd: cannot lock /etc/group; try again later.

usermod: no changes

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: no changes

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: existing lock file /etc/group.lock without a PID

usermod: cannot lock /etc/group; try again later.

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

usermod: no changes

creating nginx log directory '/var/log/nginx'

creating nginx run directory '/var/run/nginx'

creating nginx temp directories '/var/run/nginx/{client_body,proxy,fastcgi,uwsgi,scgi}_temp'

nginx: [alert] could not open error log file: open() "/gnu/store/vyj2vkmdmlpxn3mnj71vz8zc8j30ahkf-nginx-1.12.1/logs/error.log" failed (2: No such file or directory)

nginx: the configuration file /gnu/store/xms1g2z62rcj2h9i9d6fbqyl65a4yycm-nginx.conf syntax is ok

nginx: configuration file /gnu/store/xms1g2z62rcj2h9i9d6fbqyl65a4yycm-nginx.conf test is successful

guix system: unloading service 'user-homes'...

shepherd: Removing service 'user-homes'...

shepherd: Done.

guix system: loading new services: user-homes...

shepherd: Evaluating user expression (register-services (primitive-load "/gnu/sto?")).

shepherd: Service user-homes could not be started.

Installing for x86_64-efi platform.

Installation finished. No error reported.

The new system generation was produced without "test" group, so you

could reboot into it. And it could lead to problems if we will have a

tiny output to console and big output to a log file, I guess.

How to make a kernel panic

The problem will be No defined variable IPTABLES-SSH after 'guix system

reconfigure' and kernel crash after.

$HOME/src/iptables/iptables/ru.scm

(define-module (iptables ru)

;; …

)

(define %iptables-ssh

"-A INPUT -p tcp --dport 22 \

-m state --state NEW -m recent --set --name SSH -j ACCEPT")

/etc/config.scm

(use-modules ;; …

(iptables ru))

(define start-firewall

#~(let ((iptables

(lambda (str)

(zero? (system (string-join `(,#$(file-append iptables

"/sbin/iptables")

,str) " "))))))

(format #t "Install iptables rules.~%")

(and

;; …

(iptables %iptables-ssh))))

(define firewall-service

(simple-service 'firewall shepherd-root-service-type

(list

(shepherd-service

(provision '(firewall))

(requirement '())

(start #~(lambda _

#$start-firewall))

(respawn? #f)

(stop #~(lambda _

(zero?

(system* #$(file-append iptables

"/sbin/iptables")

"-F"))))))))

(operating-system

;; …

(services (cons* ;; …

firewall-service)))

Make a kernel panic

sudo GUILE_LOAD_PATH=\"$HOME/src/iptables\

:$GUILE_LOAD_PATH\" guix system reconfigure \

$HOME/dotfiles/guix/system-magnolia.scm

# Run above again and kernel will panic.

No, it happens after second 'guix system reconfigure' with howto above.

[...]

Thanks,

Oleg.

Hi Oleg,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

I sympathize…

I think we can avoid the problem by forcefully removing these two lock

files at boot time:

[...]

I tried to reproduce it with my user’s shepherd, but that didn’t work:

IOW, shepherd caught the exceptions and didn’t die.

What am I missing?

Ludo’.

Hello Ludovic,

ludo@gnu.org (Ludovic Courtès) writes:

[...]

There is also a '/etc/.pwd.lock'. Info about this file

I'm not sure if any files are exist. Days past after reconfigure

failure.

$ sudo find /etc -name '*.lock' # Shows nothing.

[...]

I'll try to make a reproducible thing later.

Thanks,

Oleg.

Hi Oleg,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

I’ve pushed it as aad8a143000600abec5c8ebfadec4c09f34f1b73.

Awesome.

Thanks,

Ludo’.

Hi!

rekado@elephly.net (Ricardo Wurmus) skribis:

Good idea.

https://bugs.gnu.org/28647 is somewhat related.

Fundamentally though, to simplify this use case, we should have:

(define (additional-udev-rules . rules)

"Add RULES, a list of file-like object, as a udev rules."

(simple-service 'udev-rule udev-service-type rules))

so one can write:

(operating-system

;; …

(services (cons (additional-udev-rules %example-udev-rule)

%desktop-services)))

Thoughts?

Ludo’.

Hi,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Did you eventually gather more info?

Thanks,

Ludo’.

tags 28772 moreinfo

Hello Ludovic,

Apologies for the late reply.

ludo@gnu.org (Ludovic Courtès) writes:

[...]

Yes, I got an undefined %iptables-rst variable in reconfigure output

at first run.

But at second run substitutions didn't work. GuixSD rebuilds the world.

I didn't wait for this.

I also didn't make guix pull.

/tmp/guixsd

??? bootstrap.sh

??? iptables

?   ??? iptables.scm

??? panic.sh

??? vm-image.scm

1 directory, 4 files

ludo@gnu.org (Ludovic Courtès) writes:

[...]

In addition to previous message.

--- /home/natsu/r/bootstrap.sh 2017-10-22 04:48:42.992394510 +0300

+++ ./bootstrap.sh 2017-10-22 23:33:54.113495658 +0300

@@ -11,7 +11,7 @@

SIG=guixsd-vm-image-$VERSION.$KERNEL.xz.sig

pull_release () {

- if [ ! -f $RELEASE ] || [ ! -f $RELEASE.xz ] || ! gpg --verify $SIG

+ if [ ! -f $RELEASE ] || [ ! -f $RELEASE.xz ]

then

gpg --keyserver pgp.mit.edu --recv-keys $GPG_KEY

wget --output-document=$RELEASE.xz https://alpha.gnu.org/gnu/guix/$RELEASE.xz

@@ -23,6 +23,7 @@

if pull_release

then qemu-system-x86_64 \

+ -enable-kvm \

-daemonize \

-m 1024 \

-virtfs local,path=$PWD,security_model=none,mount_tag=TAG_pwd \

Also need to increase a size of qemu image. I don't know how I

reconfigured successfully in previous message.

Hi Oleg,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

In my previous message I showed a way to (attempt to) reproduce the

problem by directly using ‘herd load’.

Could you try to reproduce the problem in this way?

You should even be able to check with a shepherd instance not running as

PID 1, which is more convenient.

Thanks in advance,

Ludo’.

Hi,

On Tue, 10 Oct 2017 at 08:51, Oleg Pykhalov <go.wigust@gmail.com> wrote:

The last message for bug 28772 [1] is 22 Oct 2017 and it is marked

’moreinfo’ since 2017. I propose to close or provide more details.

1: http://issues.guix.gnu.org/issue/28772

Cheers,

simon