an installer dump was sent
(address . bug-guix@gnu.org)
CAMLcmSwJ4g9OeCxMAJL3nxj4yOWRvKNbkszCTtBKVBvOYar2bw@mail.gmail.com
Hello,
I just sent an installer dump to you. The name is "installer-dump-2a14e489"
best regards
[1.4.0rc1] Installer fails to identify installation device on Ventoy-made images
- Done
Details
- 4 participants
- Ludovic Courtès
- Mathieu Othacehe
- Mathieu Othacehe
- So'n Typ im Internet
- Owner
- unassigned
- Submitted by
- So'n Typ im Internet
- Severity
- normal
S
M
(name . So'n Typ im Internet)(address . sontypiminternet@gmail.com)(address . 59823@debbugs.gnu.org)
87tu2a8drd.fsf@gnu.org
Hello,
Thanks for reporting! So the error is:
Toggle snippet (34 lines)
mathieu@meije ~$ cat dump.2022-12-04.10.54.06/installer-backtrace
In ./gnu/installer/steps.scm:
150:13 19 (run ((locale . "de_DE.utf8")) #:todo-steps _ #:done-steps _)
150:13 18 (run ((welcome . #t) (locale . "de_DE.utf8")) #:todo-steps _ #:done-steps _)
150:13 17 (run ((timezone . "Europe/Berlin") (welcome . #t) (locale . "de_DE.utf8")) #:todo-steps _ #:done-steps _)
150:13 16 (run ((keymap "de" #f #f) (timezone . "Europe/Berlin") (welcome . #t) (locale . "de_DE.utf8")) #:todo-steps _ #:done-steps _)
150:13 15 (run ((hostname . "guix-hp") (keymap "de" #f #f) (timezone . "Europe/Berlin") (welcome . #t) (locale . "de_DE.utf8")) #:todo-steps _ #:done-steps _)
150:13 14 (run ((network (select-technology . #<<technology> name: "Wired" type: "ethernet" powered?: #t connected?: #t>) (power-technology . #<unspecified>) (connect-service . #<<servi…>) …) …) …)
150:13 13 (run ((substitutes #t) (network (select-technology . #<<technology> name: "Wired" type: "ethernet" powered?: #t connected?: #t>) (power-technology . #<unspecified>) (# . #<<se…>) …) …) …)
150:13 12 (run ((user #<<user> name: "root" real-name: "" group: "users" password: <secret> home-directory: "/root"> #<<user> name: "typ" real-name: "Typ" group: "users" password: <secret…>) …) …)
148:23 11 (run ((services #<<system-service> name: "GNOME" type: desktop recommended?: #f snippet: ((service gnome-desktop-service-type)) packages: ()> #<<system-service> name: "OpenSSH …> …) …) …)
In ./gnu/installer/newt/partition.scm:
814:4 10 (run-partitioning-page)
In srfi/srfi-1.scm:
634:9 9 (for-each #<procedure 7fc5c4397698 at ./gnu/installer/parted.scm:1524:14 (file-name)> _)
In ./gnu/installer/parted.scm:
1528:22 8 (_ #<time type: time-duration nanosecond: 261179000 second: 4> #t)
In ice-9/boot-9.scm:
1685:16 7 (raise-exception _ #:continuable? _)
1780:13 6 (_ #<&compound-exception components: (#<&error> #<&origin origin: #f> #<&message message: "~A"> #<&irritants irritants: ("Gerät /dev/sdd wird noch verwendet.")> #<&exception-with-kind…>)
In ice-9/eval.scm:
619:8 5 (_ #(#(#(#<directory (guile-user) 7fc5d77d0c80>) misc-error (#f "~A" ("Gerät /dev/sdd wird noch verwendet.") #f)) #<variable 7fc5c0c7d150 value: #<unspecified>> #<variable 7fc5c0c7…> …))
626:19 4 (_ #(#(#(#<directory (guile-user) 7fc5d77d0c80>) misc-error (#f "~A" ("Gerät /dev/sdd wird noch verwendet.") #f)) #<variable 7fc5c0c7d150 value: #<unspecified>> #<variable 7fc5c0c7…> …))
In ./gnu/installer/dump.scm:
58:4 3 (prepare-dump misc-error (#f "~A" ("Gerät /dev/sdd wird noch verwendet.") #f) #:result _)
In ice-9/ports.scm:
433:17 2 (call-with-output-file _ _ #:binary _ #:encoding _)
In ./gnu/installer/dump.scm:
60:27 1 (_ #<output: installer-backtrace 20>)
In unknown file:
0 (make-stack #t)
./gnu/installer/dump.scm:62:36: Gerät /dev/sdd wird noch verwendet.
Which means that the delay in the "with-delay-device-in-use?" procedure
is probably not high enough.
Here is an attached patch to bump it from 16 to 96 seconds. I also
uploaded an image built on top of of the version-1.4.0 with this patch
if you are up for a retry :), you can download it this way:
Toggle snippet (3 lines)
wget https://othacehe.org/files/installer.iso
Thanks,
Mathieu
From b53d7f0c930f029d6b17be92dfa408b74615c1a5 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@gnu.org>
Date: Mon, 5 Dec 2022 08:56:43 +0100
Subject: [PATCH 1/1] installer: Dump the device in use retry count to 96.
* gnu/installer/parted.scm (with-delay-device-in-use): Bump it.
---
gnu/installer/parted.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Toggle diff (15 lines)
diff --git a/gnu/installer/parted.scm b/gnu/installer/parted.scm
index 82375d29e3..518b40a7ea 100644
--- a/gnu/installer/parted.scm
+++ b/gnu/installer/parted.scm
@@ -348,7 +348,7 @@ (define (with-delay-device-in-use? file-name)
fail. See rereadpt function in wipefs.c of util-linux for an explanation."
;; Kernel always return EINVAL for BLKRRPART on loopdevices.
(and (not (string-match "/dev/loop*" file-name))
- (let loop ((try 16))
+ (let loop ((try 96))
(usleep 250000)
(let ((in-use? (device-in-use? file-name)))
(if (and in-use? (> try 0))
--
2.38.1
S
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 59823@debbugs.gnu.org)
CAMLcmSxV6igZd_LGLbb9Acj2fzKz0Yd-UuVpjsbSxiW-YdCimg@mail.gmail.com
Hi Mathieu,
Thanks so much dir the help.
Best regards,
Typ
Mathieu Othacehe <othacehe@gnu.org> schrieb am Mo. 5. Dez. 2022 um 09:18:
Toggle quote (77 lines)
>
> Hello,
>
> Thanks for reporting! So the error is:
>
> --8<---------------cut here---------------start------------->8---
> mathieu@meije ~$ cat dump.2022-12-04.10.54.06/installer-backtrace
> In ./gnu/installer/steps.scm:
> 150:13 19 (run ((locale . "de_DE.utf8")) #:todo-steps _ #:done-steps _)
> 150:13 18 (run ((welcome . #t) (locale . "de_DE.utf8")) #:todo-steps _
> #:done-steps _)
> 150:13 17 (run ((timezone . "Europe/Berlin") (welcome . #t) (locale .
> "de_DE.utf8")) #:todo-steps _ #:done-steps _)
> 150:13 16 (run ((keymap "de" #f #f) (timezone . "Europe/Berlin")
> (welcome . #t) (locale . "de_DE.utf8")) #:todo-steps _ #:done-steps _)
> 150:13 15 (run ((hostname . "guix-hp") (keymap "de" #f #f) (timezone .
> "Europe/Berlin") (welcome . #t) (locale . "de_DE.utf8")) #:todo-steps _
> #:done-steps _)
> 150:13 14 (run ((network (select-technology . #<<technology> name:
> "Wired" type: "ethernet" powered?: #t connected?: #t>) (power-technology .
> #<unspecified>) (connect-service . #<<servi…>) …) …) …)
> 150:13 13 (run ((substitutes #t) (network (select-technology .
> #<<technology> name: "Wired" type: "ethernet" powered?: #t connected?: #t>)
> (power-technology . #<unspecified>) (# . #<<se…>) …) …) …)
> 150:13 12 (run ((user #<<user> name: "root" real-name: "" group:
> "users" password: <secret> home-directory: "/root"> #<<user> name: "typ"
> real-name: "Typ" group: "users" password: <secret…>) …) …)
> 148:23 11 (run ((services #<<system-service> name: "GNOME" type:
> desktop recommended?: #f snippet: ((service gnome-desktop-service-type))
> packages: ()> #<<system-service> name: "OpenSSH …> …) …) …)
> In ./gnu/installer/newt/partition.scm:
> 814:4 10 (run-partitioning-page)
> In srfi/srfi-1.scm:
> 634:9 9 (for-each #<procedure 7fc5c4397698 at
> ./gnu/installer/parted.scm:1524:14 (file-name)> _)
> In ./gnu/installer/parted.scm:
> 1528:22 8 (_ #<time type: time-duration nanosecond: 261179000 second:
> 4> #t)
> In ice-9/boot-9.scm:
> 1685:16 7 (raise-exception _ #:continuable? _)
> 1780:13 6 (_ #<&compound-exception components: (#<&error> #<&origin
> origin: #f> #<&message message: "~A"> #<&irritants irritants: ("Gerät
> /dev/sdd wird noch verwendet.")> #<&exception-with-kind…>)
> In ice-9/eval.scm:
> 619:8 5 (_ #(#(#(#<directory (guile-user) 7fc5d77d0c80>) misc-error
> (#f "~A" ("Gerät /dev/sdd wird noch verwendet.") #f)) #<variable
> 7fc5c0c7d150 value: #<unspecified>> #<variable 7fc5c0c7…> …))
> 626:19 4 (_ #(#(#(#<directory (guile-user) 7fc5d77d0c80>) misc-error
> (#f "~A" ("Gerät /dev/sdd wird noch verwendet.") #f)) #<variable
> 7fc5c0c7d150 value: #<unspecified>> #<variable 7fc5c0c7…> …))
> In ./gnu/installer/dump.scm:
> 58:4 3 (prepare-dump misc-error (#f "~A" ("Gerät /dev/sdd wird noch
> verwendet.") #f) #:result _)
> In ice-9/ports.scm:
> 433:17 2 (call-with-output-file _ _ #:binary _ #:encoding _)
> In ./gnu/installer/dump.scm:
> 60:27 1 (_ #<output: installer-backtrace 20>)
> In unknown file:
> 0 (make-stack #t)
> ./gnu/installer/dump.scm:62:36: Gerät /dev/sdd wird noch verwendet.
> --8<---------------cut here---------------end--------------->8---
>
> Which means that the delay in the "with-delay-device-in-use?" procedure
> is probably not high enough.
>
> Here is an attached patch to bump it from 16 to 96 seconds. I also
> uploaded an image built on top of of the version-1.4.0 with this patch
> if you are up for a retry :), you can download it this way:
>
> --8<---------------cut here---------------start------------->8---
> wget https://othacehe.org/files/installer.iso
> --8<---------------cut here---------------end--------------->8---
>
> Thanks,
>
> Mathieu
>
M
(name . So'n Typ im Internet)(address . sontypiminternet@gmail.com)(address . 59823@debbugs.gnu.org)
87lenj8srr.fsf@gnu.org
Hello,
So I had a closer look and we do have this strange kernel warning:
Toggle snippet (3 lines)
Dec 4 10:49:05 localhost vmunix: [ 1351.610773] device-mapper: ioctl: remove_all left 1 open device(s)
I also noticed that we do not have the following trace:
Toggle snippet (3 lines)
/dev/sdX is not eligible because it is the installation device.
which indicates that the "eligible-devices" procedure failed to identify
the installation device. So /dev/sdd could be the installation device
and the "with-delay-device-in-use?" always reports true because it is in
fact always in use.
My proposed patch probably won't help and the question is, why is the
installation device not detected?
Would it be possible for you to try the installation with an
instrumented installer that I would provide you with?
Thanks,
Mathieu
S
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 59823@debbugs.gnu.org)
CAMLcmSzLeQH_h0dppwrYVzo0Dp1yv+-r-sogmLsiYoCSEsLo9Q@mail.gmail.com
Hallo Mathieu,
Thank you so much for your effort.
I already managed the installation by burning the iso directly to my usb
drive and not via Ventoy.
Everything works fine.
I should have try this in the first place. Sometimes the ways of the lord
gnu are inscrutable.
Big thanks again and a warm regards to you and your team.
Typ.
Mathieu Othacehe <othacehe@gnu.org> schrieb am Mi. 7. Dez. 2022 um 10:31:
Toggle quote (31 lines)
>
> Hello,
>
> So I had a closer look and we do have this strange kernel warning:
>
> --8<---------------cut here---------------start------------->8---
> Dec 4 10:49:05 localhost vmunix: [ 1351.610773] device-mapper: ioctl:
> remove_all left 1 open device(s)
> --8<---------------cut here---------------end--------------->8---
>
> I also noticed that we do not have the following trace:
>
> --8<---------------cut here---------------start------------->8---
> /dev/sdX is not eligible because it is the installation device.
> --8<---------------cut here---------------end--------------->8---
>
> which indicates that the "eligible-devices" procedure failed to identify
> the installation device. So /dev/sdd could be the installation device
> and the "with-delay-device-in-use?" always reports true because it is in
> fact always in use.
>
> My proposed patch probably won't help and the question is, why is the
> installation device not detected?
>
> Would it be possible for you to try the installation with an
> instrumented installer that I would provide you with?
>
> Thanks,
>
> Mathieu
>
M
(name . So'n Typ im Internet)(address . sontypiminternet@gmail.com)
87y1rj6xb0.fsf@gnu.org
Hello,
It's really good that you managed to install it anyway. Thanks for
persevering :) Nevertheless we need to fix the problem.
Toggle quote (3 lines)
> I already managed the installation by burning the iso directly to my
> usb drive and not via Ventoy.
Turns out Ventoy was the crux of the issue here. Ventox allows to
install multiple iso images just by copying them in a directory of the
drive. It then creates a device mapping which look like that:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 238.5G 0 disk
??sda1 8:1 0 512M 0 part
??sda2 8:2 0 238G 0 part
sdb 8:16 0 14.6G 0 disk
??sdb1 8:17 0 14.6G 0 part
? ??ventoy 253:0 0 842.9M 1 dm
??sdb2 8:18 0 32M 0 part
This device mapping defeats our "eligible-devices" procedure. It's
because the UUID that is passed as the root=xxx argument of the Linux
command line is in fact related to /dev/dm-0 which is different from
/dev/sdb.
I had a look to the parted sources and it should detect mapped devices
but for some reason it doesn't. Figuring it out and fixing it is maybe a
bit risky before the release.
So I'm trying to figure out a (cheap) way to make the correlation
between /dev/dm-0 and /dev/sdb to exclude the latter in the eligible
devices procedure.
Any idea is welcomed!
Thanks,
Mathieu
L
control message for bug #59823
(address . control@debbugs.gnu.org)
87o7sfi04p.fsf@gnu.org
retitle 59823 [1.4.0rc1] Installer fails to identify installation device on Ventoy-made images
quit
M
Re: bug#59823: [1.4.0rc1] Installer fails to identify installation device on Ventoy-made images
(name . So'n Typ im Internet)(address . sontypiminternet@gmail.com)
877cz213mk.fsf_-_@gnu.org
Hello,
The attached patch fixes it for me. We could maybe use libdevmapper
instead of the plain "dmsetup" call but that's not critical in my
opinion.
Thanks,
Mathieu
From 0afda5b3ed32e73bece9db96ab970d83f9f2e74b Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@gnu.org>
Date: Thu, 8 Dec 2022 13:24:02 +0100
Subject: [PATCH 1/1] installer: Detect mapped installation devices.
* gnu/installer/parted.scm (mapped-device?,
mapped-device->parent-partition-path): New procedures.
(eligible-devices): Detect mapped installation devices using the new
procedures.
---
gnu/installer/parted.scm | 35 ++++++++++++++++++++++++++++++++++-
1 file changed, 34 insertions(+), 1 deletion(-)
Toggle diff (53 lines)
diff --git a/gnu/installer/parted.scm b/gnu/installer/parted.scm
index 82375d29e3..058f2a8dab 100644
--- a/gnu/installer/parted.scm
+++ b/gnu/installer/parted.scm
@@ -379,12 +379,45 @@ (define (installer-root-partition-path)
(define %min-device-size
(* 2 GIBIBYTE-SIZE)) ;2GiB
+(define (mapped-device? device)
+ "Return #true if DEVICE is a mapped device, false otherwise."
+ (string-prefix? "/dev/dm-" device))
+
+(define (mapped-device->parent-partition-path device)
+ "Return the parent partition path of the mapped DEVICE."
+ (let* ((command `("dmsetup" "deps" ,device "-o" "devname"))
+ (parent #f)
+ (handler
+ (lambda (input)
+ (let ((result
+ (string-match "\\(([^\\)]+)\\)"
+ (get-string-all input))))
+ (and result
+ (set! parent
+ (format #f "/dev/~a"
+ (match:substring result 1))))))))
+ (run-external-command-with-handler handler command)
+ parent))
+
(define (eligible-devices)
"Return all the available devices except the install device and the devices
which are smaller than %MIN-DEVICE-SIZE."
(define the-installer-root-partition-path
- (installer-root-partition-path))
+ (let ((root-path
+ (installer-root-partition-path)))
+ (cond
+ ((mapped-device? root-path)
+ ;; If the partition is a mapped device (/dev/dm-X), locate the parent
+ ;; partition. It is the case when Ventoy is used to host the
+ ;; installation image.
+ (let ((parent-path
+ (mapped-device->parent-partition-path root-path)))
+ (installer-log-line "mapped device ~a -> ~a"
+ parent-path root-path)
+ parent-path))
+ (else
+ root-path))))
(define (small-device? device)
(let ((length (device-length device))
--
2.38.1
L
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87y1rg5il4.fsf@gnu.org
Hi!
Mathieu Othacehe <othacehe@gnu.org> skribis:
Toggle quote (12 lines)
>>From 0afda5b3ed32e73bece9db96ab970d83f9f2e74b Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe@gnu.org>
> Date: Thu, 8 Dec 2022 13:24:02 +0100
> Subject: [PATCH 1/1] installer: Detect mapped installation devices.
>
> Fixes: <https://issues.guix.gnu.org/59823>
>
> * gnu/installer/parted.scm (mapped-device?,
> mapped-device->parent-partition-path): New procedures.
> (eligible-devices): Detect mapped installation devices using the new
> procedures.
Well done!
Toggle quote (3 lines)
> +(define (mapped-device->parent-partition-path device)
> + "Return the parent partition path of the mapped DEVICE."
Maybe ‘mapped-device-parent-partition’? (“Path” being used for search
paths, not for file names.)
Toggle quote (14 lines)
> + (let* ((command `("dmsetup" "deps" ,device "-o" "devname"))
> + (parent #f)
> + (handler
> + (lambda (input)
> + (let ((result
> + (string-match "\\(([^\\)]+)\\)"
> + (get-string-all input))))
> + (and result
> + (set! parent
> + (format #f "/dev/~a"
> + (match:substring result 1))))))))
> + (run-external-command-with-handler handler command)
> + parent))
The advantage of ‘run-external-command-with-handler’ is logging I guess,
but that leads to an awkward style compared to using ‘open-input-pipe’,
which I would favor.
Then we want to parse a line that looks like this:
Toggle snippet (3 lines)
1 dependencies : (sda2)
I’m not super confident relying on this format.
‘dmsetup deps …’ does this (here /dev/dm-0 is the same as
/dev/mapper/root):
Toggle snippet (14 lines)
ioctl(3, DM_VERSION, {version=[4, 0, 0], data_size=16384, flags=DM_EXISTS_FLAG} => {version=[4, 47, 0], data_size=16384, flags=DM_EXISTS_FLAG}) = 0
newfstatat(AT_FDCWD, "/dev/dm-0", {st_mode=S_IFBLK|0660, st_rdev=makedev(0xfd, 0), ...}, 0) = 0
newfstatat(AT_FDCWD, "/dev/mapper/dm-0", 0x7ffe78f99020, 0) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/dev/mapper", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 4
newfstatat(4, "", {st_mode=S_IFDIR|0755, st_size=80, ...}, AT_EMPTY_PATH) = 0
getdents64(4, 0x563c1d72f8b0 /* 4 entries */, 32768) = 104
newfstatat(AT_FDCWD, "/dev/mapper/root", {st_mode=S_IFBLK|0600, st_rdev=makedev(0xfd, 0), ...}, 0) = 0
close(4) = 0
ioctl(3, DM_TABLE_DEPS, {version=[4, 0, 0], data_size=16384, data_start=312, name="root", flags=DM_EXISTS_FLAG} => {version=[4, 47, 0], data_size=328, data_start=312, dev=makedev(0xfd, 0), name="root", uuid="CRYPT-LUKS1-xxxxxx-root", target_count=1, open_count=1, event_nr=0, flags=DM_EXISTS_FLAG|DM_ACTIVE_PRESENT_FLAG, ...}) = 0
newfstatat(1, "", {st_mode=S_IFCHR|0622, st_rdev=makedev(0x88, 0), ...}, AT_EMPTY_PATH) = 0
readlink("/sys/dev/block/8:2", "../../devices/pci0000:00/0000:00:17.0/ata3/host2/target2:0:0/2:0:0:0/block/sda/sda2", 4095) = 83
write(1, "1 dependencies\t: (sda2)\n", 24) = 24
We could implement those ioctls, but not in the time frame we’re talking
about…
The libdevmapper interface doesn’t look more pleasant than the raw
ioctl.
Anyway, all in all, calling out to dmsetup looks reasonable for now; I
have a slight preference for using ‘open-pipe* OPEN_READ’, but no big
deal. Perhaps add a comment showing what the line we’re parsing should
look like.
Toggle quote (9 lines)
> (define (eligible-devices)
> "Return all the available devices except the install device and the devices
> which are smaller than %MIN-DEVICE-SIZE."
>
> (define the-installer-root-partition-path
> - (installer-root-partition-path))
> + (let ((root-path
> + (installer-root-partition-path)))
Just ‘root’. :-)
Thanks a lot for fixing this! You can push to ‘version-1.4.0’.
Hopefully I’ll prepare RC2 tomorrow evening.
Ludo’.
M
(name . Ludovic Courtès)(address . ludo@gnu.org)
87bkobo8ps.fsf@gnu.org
Hola,
Toggle quote (5 lines)
> Anyway, all in all, calling out to dmsetup looks reasonable for now; I
> have a slight preference for using ‘open-pipe* OPEN_READ’, but no big
> deal. Perhaps add a comment showing what the line we’re parsing should
> look like.
Yeah, I agree that open-pipe would be a bit clearer, but as this is
already tested as-is plus we'll switch to the ioctl after the release, I
think we can proceed.
Toggle quote (11 lines)
>> (define (eligible-devices)
>> "Return all the available devices except the install device and the devices
>> which are smaller than %MIN-DEVICE-SIZE."
>>
>> (define the-installer-root-partition-path
>> - (installer-root-partition-path))
>> + (let ((root-path
>> + (installer-root-partition-path)))
>
> Just ‘root’. :-)
Fixed, added a few comments and pushed.
Thanks for having a look!
Mathieu
M
control message for bug #59823
(address . control@debbugs.gnu.org)
877cyzo8ov.fsf@meije.mail-host-address-is-not-set
close 59823
quit
L
Re: bug#59823: [1.4.0rc1] Installer fails to identify installation device on Ventoy-made images
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
874ju35xqo.fsf@gnu.org
¡Hola!
Mathieu Othacehe <othacehe@gnu.org> skribis:
Toggle quote (4 lines)
> Yeah, I agree that open-pipe would be a bit clearer, but as this is
> already tested as-is plus we'll switch to the ioctl after the release, I
> think we can proceed.
Yup, that makes sense to me.
Toggle quote (2 lines)
> Fixed, added a few comments and pushed.
Thank you!
Ludo’.
?