udevd error with lvm-raid array leading to race condition with luks

Hi there.

My setup is the following (LVM array containing a LUKS partition):

(mapped-devices

(list

(mapped-device

(source "HOMERAID")

(target "HOMERAID-HOMERAID_data")

(type lvm-device-mapping))

(mapped-device

(source "/dev/mapper/HOMERAID-HOMERAID_data")

(target "luks-f0a72a6c-499b-4445-8d13-21dc99337752")

(type luks-device-mapping))))

(file-systems

(cons*

(file-system

(mount-point "/")

(device (uuid "2e44f3f7-bb6b-43ac-933a-e8992bf10d29" 'ext4))

(type "ext4"))

(file-system

(mount-point "/home")

(device "/dev/mapper/luks-f0a72a6c-499b-4445-8d13-21dc99337752")

(type "ext4")

(dependencies mapped-devices))

(file-system

(mount-point "/boot/efi")

(device (uuid "DC58-946E" 'fat32))

(type "vfat"))

%base-file-systems)))

I use Guix System since 2022, and never had any problem booting with

this configuration. But I did update my system (and reconfigure it) last

week and now I can't boot. I don't have any older generation to restore

(yes I'm dumb, I executed a delete-generations to show how it works to a

friend), but I think the last working one was from July, 25th or just a

bit older.

I have in my /var/log/messages :

Aug 9 11:40:27 localhost vmunix: [ 7.525877] udevd[515]: failed

to execute '/usr/bin/systemd-run' '/usr/bin/systemd-run --no-block

--property DefaultDependencies=no --unit lvm-activate-HOMERAID

/gnu/store/hffkn63zx2zjadawrkxpnr486frc9n74-lvm2-2.03.21/sbin/lvm

vgchange -aay --autoactivation event HOMERAID': No such file or directory

On the screen, during boot, I obtain a wall of messages like (not the

exact message, sorry, can't find on syslog) "Device

HOMERAID-HOMERAID_data could not be found: does not exist or access

denied" and after that the system hangs.

However, I can boot successfully if and only if I press Scroll Lock just

after modesetting and before this message ("Device ... not be found")

appears and when I press it again (to deactivate its effect) after some

seconds, the LUKS passphrase prompt appears and allows me to boot properly.

I can use my system but it requires multiple tries each time x) I have

to be really synchronized with the modesetting and the message xD

Thank you very much.

Freely,

--

Adrien Bourmault

Co-maintainer, GNU Boot project

Elected member, XMPP Standards Foundation

Associate member, Free Software Foundation

Trésorier, Association Libre en Communs (https://www.a-lec.org)

GPG : 1DF1132CF1658A8559025C98AAD6B069819E6979

Hi Adrien,

Adrien 'neox' Bourmault <neox@gnu.org> writes:

Since lvm2 2.03.14, the included udev rules use systemd-run to run

vgchange and activate the volume group. lvm2 was updated recently from

2.03.11 to 2.03.21, then 2.03.22 (cc'ing tobias), and probably started

exhibiting this behavior then. I think we can probably remove the

indirection through systemd-run and directly run vgchange.

Best,

--

Josselin Poiret

hello everyone, we're also victims of this bug, in an even simpler use case.

[…]

(file-system

                                (device "/dev/mapper/VG0-DATA")

                                (mount-point "/VG0-DATA")

                                (type "ext4"))

[…]

The culprit seems to be 69-dm-lvm.rules

[ 18.226226] udevd[115]: failed to execute '/usr/bin/systemd-run'

'/usr/bin/systemd-run --no-block --property DefaultDependencies=no

--unit lvm-activate-VG0

/gnu/store/0hndg947ywdl5izvy63ny38hyywci66k-lvm2-2.03.22/sbin/lvm vy

I can confirm that when using time-machine to revert to lvm2 2-03.11

versions, the VM boots.

cheers,

Hi, as suggested by Josselin, I tested the following patch and it seems

to do the job here.

Be careful, I'm not an udev or lvm2 specialist at all and basically, I

don't really know if what I did is the right way to do it.

All I can say is that the VMs now boot.

Cheers,

Hi,

On Thu, 14 Sep 2023 at 18:23, Yann Dupont <Yann.Dupont@univ-nantes.fr> wrote:

WDYT about this patch?

Josselin, is it what you had in mind?

Cheers,

simon

Hi,

On Mon, 25 Sep 2023 at 09:35, Simon Tournier <zimon.toutoune@gmail.com> wrote:

For easing the discussion, I extracted Yann’s diff and prepared a patch

ready to merge. See:

https://issues.guix.gnu.org/issue/66586

Cheers,

simon

Hi,

Patch pushed as c0895371c5759c7d9edb330774e90f192cc4cf2c.

Closing.

Feel free to reopen if the patch does not fix the issue.

Cheers,

simon

Hi,

I think the patch is wrong. No comment regarding the udev rule itself (I do not know much about it), it does spam the logs with:

Oct 31 18:31:35 localhost vmunix: [ 60.013538] udevd[327]: invalid key/value pair in file /gnu/store/a3xvv1zxzyb53r5r752q8nh9v9vd150h-udev-rules/69-dm-lvm.rules on line 78, starting at character 1 ('I')

Oct 31 18:31:35 localhost vmunix: [ 60.013544] udevd[327]: invalid key/value pair in file /gnu/store/a3xvv1zxzyb53r5r752q8nh9v9vd150h-udev-rules/69-dm-lvm.rules on line 79, starting at character 32 (',')

The fix[0] for that is fairly trivial.

0: https://git.sr.ht/~graywolf/guix/commit/207794679281c1e7e5f2051719fe5c705506eb23

Have a nice day,

W.

--

There are only two hard things in Computer Science:

cache invalidation, naming things and off-by-one errors.

Hi,

On Tue, 31 Oct 2023 at 19:38, Tomas Volf <wolf@wolfsden.cz> wrote:

Is patch v3 from

https://issues.guix.gnu.org/issue/66586#10

fixing the issue?

Cheers,

simon