D-Bus system service breaks reconfiguration when /var/run/dbus is present + /run and /var/run are on separate file systems.

  • Open
  • quality assurance status badge
Details
4 participants
  • Collin J. Doering
  • Hilton Chain
  • Liliana Marie Prikler
  • Ludovic Courtès
Owner
unassigned
Submitted by
Hilton Chain
Severity
normal
H
H
Hilton Chain wrote on 3 Apr 18:36 +0200
(address . bug-guix@gnu.org)
87frw2mmix.wl-hako@ultrarare.space
Hi,

I have /var/run and /run on separate file systems, recently I noticed system
reconfiguration stopped with "guix system: error: rename-file: Invalid
cross-device link":

Toggle snippet (19 lines)
newfstatat(AT_FDCWD, "/run", {st_mode=S_IFDIR|0755, st_size=440, ...}, AT_SYMLINK_NOFOLLOW) = 0
newfstatat(AT_FDCWD, "/run/dbus", {st_mode=S_IFDIR|0700, st_size=40, ...}, AT_SYMLINK_NOFOLLOW) = 0
mkdir("/run", 0777) = -1 EEXIST (File exists)
mkdir("/run/dbus", 0777) = -1 EEXIST (File exists)
chown("/run/dbus", 988, 983) = 0
chmod("/run/dbus", 0755) = 0
symlink("/run/dbus", "/var/run/dbus") = -1 EEXIST (File exists)
readlink("/var/run/dbus", 0x1634730, 100) = -1 EINVAL (Invalid argument)
openat(AT_FDCWD, "/var/run/dbus", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 17
newfstatat(17, "", {st_mode=S_IFDIR|0755, st_size=80, ...}, AT_EMPTY_PATH) = 0
getdents64(17, 0x16dfe10 /* 4 entries */, 32768) = 112
rename("/var/run/dbus/system_bus_socket", "/run/dbus/system_bus_socket") = -1 EXDEV (Invalid cross-device link)
close(13) = 0
write(2, "\33[1m\33[0mguix system: error: rena"..., 67guix system: error: rename-file: Invalid cross-device link
) = 67
exit_group(1) = ?
+++ exited with 1 +++

It's because /var/run/dbus was used for dbus service before, and now migration
to /run/dbus is done with ‘rename-file’:

Toggle snippet (4 lines)
(rename-file (string-append "/var/run/dbus/" next)
(string-append "/run/dbus/" next))

I think the logic can be improved for this case, but not sure how at the moment.
What do you think?

Thanks
L
L
Liliana Marie Prikler wrote on 4 Apr 20:52 +0200
(name . Vivien Kraus)(address . vivien@planete-kraus.eu)
4420d6a7c4845c60cec8884066bf15b0d5b4c5fc.camel@gmail.com
Am Donnerstag, dem 04.04.2024 um 00:36 +0800 schrieb Hilton Chain:
Toggle quote (44 lines)
> Hi,
>
> I have /var/run and /run on separate file systems, recently I noticed
> system reconfiguration stopped with "guix system: error: rename-file:
> Invalid cross-device link":
>
> --8<---------------cut here---------------start------------->8---
> newfstatat(AT_FDCWD, "/run", {st_mode=S_IFDIR|0755, st_size=440,
> ...}, AT_SYMLINK_NOFOLLOW) = 0
> newfstatat(AT_FDCWD, "/run/dbus", {st_mode=S_IFDIR|0700, st_size=40,
> ...}, AT_SYMLINK_NOFOLLOW) = 0
> mkdir("/run", 0777)                     = -1 EEXIST (File exists)
> mkdir("/run/dbus", 0777)                = -1 EEXIST (File exists)
> chown("/run/dbus", 988, 983)            = 0
> chmod("/run/dbus", 0755)                = 0
> symlink("/run/dbus", "/var/run/dbus")   = -1 EEXIST (File exists)
> readlink("/var/run/dbus", 0x1634730, 100) = -1 EINVAL (Invalid
> argument)
> openat(AT_FDCWD, "/var/run/dbus",
> O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 17
> newfstatat(17, "", {st_mode=S_IFDIR|0755, st_size=80, ...},
> AT_EMPTY_PATH) = 0
> getdents64(17, 0x16dfe10 /* 4 entries */, 32768) = 112
> rename("/var/run/dbus/system_bus_socket",
> "/run/dbus/system_bus_socket") = -1 EXDEV (Invalid cross-device link)
> close(13)                               = 0
> write(2, "\33[1m\33[0mguix system: error: rena"..., 67guix
> system: error: rename-file: Invalid cross-device link
> ) = 67
> exit_group(1)                           = ?
> +++ exited with 1 +++
> --8<---------------cut here---------------end--------------->8---
>
> It's because /var/run/dbus was used for dbus service before, and now
> migration to /run/dbus is done with ‘rename-file’:
>
> --8<---------------cut here---------------start------------->8---
> (rename-file (string-append "/var/run/dbus/" next)
>              (string-append "/run/dbus/" next))
> --8<---------------cut here---------------end--------------->8---
>
> I think the logic can be improved for this case, but not sure how at
> the moment.
> What do you think?
Is there a *good* reason to have those be separate file systems? :D
What does the FHS say?

Anyhow, cross-fs symlinks and rename-file ought to work assuming that
both file-systems are already mounted. Is one of them not needed for
boot?

Cheers
H
H
Hilton Chain wrote on 5 Apr 03:53 +0200
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
8734s0wp62.wl-hako@ultrarare.space
On Fri, 05 Apr 2024 02:52:09 +0800,
Liliana Marie Prikler wrote:
Toggle quote (53 lines)
>
> Am Donnerstag, dem 04.04.2024 um 00:36 +0800 schrieb Hilton Chain:
> > Hi,
> >
> > I have /var/run and /run on separate file systems, recently I noticed
> > system reconfiguration stopped with "guix system: error: rename-file:
> > Invalid cross-device link":
> >
> > --8<---------------cut here---------------start------------->8---
> > newfstatat(AT_FDCWD, "/run", {st_mode=S_IFDIR|0755, st_size=440,
> > ...}, AT_SYMLINK_NOFOLLOW) = 0
> > newfstatat(AT_FDCWD, "/run/dbus", {st_mode=S_IFDIR|0700, st_size=40,
> > ...}, AT_SYMLINK_NOFOLLOW) = 0
> > mkdir("/run", 0777)                     = -1 EEXIST (File exists)
> > mkdir("/run/dbus", 0777)                = -1 EEXIST (File exists)
> > chown("/run/dbus", 988, 983)            = 0
> > chmod("/run/dbus", 0755)                = 0
> > symlink("/run/dbus", "/var/run/dbus")   = -1 EEXIST (File exists)
> > readlink("/var/run/dbus", 0x1634730, 100) = -1 EINVAL (Invalid
> > argument)
> > openat(AT_FDCWD, "/var/run/dbus",
> > O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 17
> > newfstatat(17, "", {st_mode=S_IFDIR|0755, st_size=80, ...},
> > AT_EMPTY_PATH) = 0
> > getdents64(17, 0x16dfe10 /* 4 entries */, 32768) = 112
> > rename("/var/run/dbus/system_bus_socket",
> > "/run/dbus/system_bus_socket") = -1 EXDEV (Invalid cross-device link)
> > close(13)                               = 0
> > write(2, "\33[1m\33[0mguix system: error: rena"..., 67guix
> > system: error: rename-file: Invalid cross-device link
> > ) = 67
> > exit_group(1)                           = ?
> > +++ exited with 1 +++
> > --8<---------------cut here---------------end--------------->8---
> >
> > It's because /var/run/dbus was used for dbus service before, and now
> > migration to /run/dbus is done with ‘rename-file’:
> >
> > --8<---------------cut here---------------start------------->8---
> > (rename-file (string-append "/var/run/dbus/" next)
> >              (string-append "/run/dbus/" next))
> > --8<---------------cut here---------------end--------------->8---
> >
> > I think the logic can be improved for this case, but not sure how at
> > the moment.
> > What do you think?
> Is there a *good* reason to have those be separate file systems? :D
> What does the FHS say?
>
> Anyhow, cross-fs symlinks and rename-file ought to work assuming that
> both file-systems are already mounted. Is one of them not needed for
> boot?

Specifically, I'm using tmpfs for them:

Toggle snippet (20 lines)
(file-system
(type "tmpfs")
(mount-point "/run")
(device "tmpfs")
(flags '(no-dev strict-atime))
(options "mode=0755,nr_inodes=800k,size=20%")
(needed-for-boot? #t)
(check? #f))

(file-system
(type "tmpfs")
(mount-point "/var/run")
(device "tmpfs")
(flags '(no-suid no-dev strict-atime))
(options "mode=0755,nr_inodes=800k,size=20%")
(needed-for-boot? #t)
(check? #f))


And here's a reproducer:

Toggle snippet (11 lines)
test_dir="$(mktemp --directory)"

sudo mount --type tmpfs none "$test_dir"
touch "$test_dir/test"

guile -c "(rename-file \"$test_dir/test\" \"/tmp/test\")"

sudo umount "$test_dir"
rmdir "$test_dir"

Toggle snippet (18 lines)
Backtrace:
In ice-9/boot-9.scm:
1752:10 6 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
5 (apply-smob/0 #<thunk 7061ede8c300>)
In ice-9/boot-9.scm:
724:2 4 (call-with-prompt ("prompt") #<procedure 7061ede99320 …> …)
In ice-9/eval.scm:
619:8 3 (_ #(#(#<directory (guile-user) 7061ede8fc80>)))
In ice-9/command-line.scm:
185:19 2 (_ #<input: string 7061ede89850>)
In unknown file:
1 (eval (rename-file "/tmp/tmp.9wyzRfQ28l/test" "/tmp/…") #)
0 (rename-file "/tmp/tmp.9wyzRfQ28l/test" "/tmp/test")

ERROR: In procedure rename-file:
In procedure rename-file: Invalid cross-device link
L
L
Liliana Marie Prikler wrote on 6 Apr 08:43 +0200
(name . Hilton Chain)(address . hako@ultrarare.space)
359765bed179300f3ed4da2b3d1bf8032bb8bc7d.camel@gmail.com
Am Freitag, dem 05.04.2024 um 09:53 +0800 schrieb Hilton Chain:
Toggle quote (35 lines)
> [H]ere's a reproducer:
>
> --8<---------------cut here---------------start------------->8---
> test_dir="$(mktemp --directory)"
>
> sudo mount --type tmpfs none "$test_dir"
> touch "$test_dir/test"
>
> guile -c "(rename-file \"$test_dir/test\" \"/tmp/test\")"
>
> sudo umount "$test_dir"
> rmdir "$test_dir"
> --8<---------------cut here---------------end--------------->8---
>
> --8<---------------cut here---------------start------------->8---
> Backtrace:
> In ice-9/boot-9.scm:
>   1752:10  6 (with-exception-handler _ _ #:unwind? _ # _)
> In unknown file:
>            5 (apply-smob/0 #<thunk 7061ede8c300>)
> In ice-9/boot-9.scm:
>     724:2  4 (call-with-prompt ("prompt") #<procedure 7061ede99320 …>
> …)
> In ice-9/eval.scm:
>     619:8  3 (_ #(#(#<directory (guile-user) 7061ede8fc80>)))
> In ice-9/command-line.scm:
>    185:19  2 (_ #<input: string 7061ede89850>)
> In unknown file:
>            1 (eval (rename-file "/tmp/tmp.9wyzRfQ28l/test" "/tmp/…")
> #)
>            0 (rename-file "/tmp/tmp.9wyzRfQ28l/test" "/tmp/test")
>
> ERROR: In procedure rename-file:
> In procedure rename-file: Invalid cross-device link
> --8<---------------cut here---------------end--------------->8---
If I understand this reproducer correctly, there aren't even symbolic
links involved, are there?

Adding Ludo to CC, because this looks like a Guile bug to me.

Cheers
C
C
Collin J. Doering wrote on 10 May 16:22 +0200
D-Bus system service breaks reconfiguration when /var/run/dbus is present + /run and /var/run are on separate file systems.
(address . 70165@debbugs.gnu.org)
87bk5dlpb8.fsf@rekahsoft.ca
Hi there,

I recently hit this issue, and was able work around it by:

1. Get root shell (via sudo or otherwise)
2. Disable dbus-system: 'herd stop guix-system'
3. Update guix: 'guix system reconfigure ...'
4. Reboot

Thought I would note it just in case its useful to someone else.

Kind regards,

--
Collin J. Doering

-----BEGIN PGP SIGNATURE-----

iIoEARMIADIWIQRDtw5R/qhdokLw/KlsfZrKhJK2jQUCZj4tmxQcY29sbGluQHJl
a2Foc29mdC5jYQAKCRBsfZrKhJK2jVMvAP46E2NisVE7lpVtpNkWo/geYLmmy8gf
2UPTO9O+8eOKCAD+MqY6r1KdILQSIVHNrQkPANADWYxKhrwJuULShEPp+6w=
=qCTm
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote 6 days ago
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
87sey8c2s1.fsf@gnu.org
Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:

Toggle quote (2 lines)
> Am Freitag, dem 05.04.2024 um 09:53 +0800 schrieb Hilton Chain:

[...]

Toggle quote (10 lines)
>>            0 (rename-file "/tmp/tmp.9wyzRfQ28l/test" "/tmp/test")
>>
>> ERROR: In procedure rename-file:
>> In procedure rename-file: Invalid cross-device link
>> --8<---------------cut here---------------end--------------->8---
> If I understand this reproducer correctly, there aren't even symbolic
> links involved, are there?
>
> Adding Ludo to CC, because this looks like a Guile bug to me.

‘rename-file’ merely wraps rename(2), which errors out with EXDEV
(“Invalid cross-link device”) when the source and targets are on
different file systems.

So the case above is behaving as expected.

What that means is that probably we shouldn’t be using ‘rename-file’
directly in cases where the source and target might be on different file
systems, and instead do something like:

(define (rename-file* old new)
;; Like rename-file, but handle the case when OLD and NEW are on
;; different file systems.
(catch 'system-error
(lambda ()
(rename-file old new))
(lambda args
(if (= EXDEV (system-error-errno args))
(begin
(copy-file old new)
(delete-file old))
(apply throw args)))))

(Untested.)

HTH!

Ludo’.
?