Bubblewrap hates Guix containers 😞

OpenSubmitted by Leo Prikler.
Details
5 participants
  • Bengt Richter
  • Leo Prikler
  • Leo Famulari
  • Ludovic Courtès
  • zimoun
Owner
unassigned
Severity
normal
L
L
Leo Prikler wrote on 12 Mar 2021 18:43
(address . bug-guix@gnu.org)
fbb3401a61ae78f092b33b7a36428f8520a7a6bd.camel@student.tugraz.at
Hi Guix,

both Epiphany and Eolie (post fixing #47097; will submit patch shortly)
fail inside Guix containers with the suggested incantation. After
getting the environment to no longer complain about $DISPLAY by adding
`--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly outputs
lines like
bwrap: Can't find source path /sys/class: No such file or directory
before closing the process altogether.

Regards,
Leo
L
L
Ludovic Courtès wrote on 13 Mar 2021 11:48
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)(address . 47106@debbugs.gnu.org)
87r1kjpbvx.fsf@gnu.org
Hi!

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

Toggle quote (8 lines)
> both Epiphany and Eolie (post fixing #47097; will submit patch shortly)
> fail inside Guix containers with the suggested incantation. After
> getting the environment to no longer complain about $DISPLAY by adding
> `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly outputs
> lines like
> bwrap: Can't find source path /sys/class: No such file or directory
> before closing the process altogether.

What is ‘bwrap’ looking for? /sys is mounted inside ‘guix environment -C’,
but perhaps it needs something special?

I suggest running these things (or ‘bwrap’ directly) in ‘strace -f -o
log’ inside the container to see.

Thanks,
Ludo’.
L
L
Leo Prikler wrote on 13 Mar 2021 12:07
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 47106@debbugs.gnu.org)
2922127e61435e64f95d3d398ef6932a02336188.camel@student.tugraz.at
Hi!
Am Samstag, den 13.03.2021, 11:48 +0100 schrieb Ludovic Courtès:
Toggle quote (21 lines)
> Hi!
>
> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
>
> > both Epiphany and Eolie (post fixing #47097; will submit patch
> > shortly)
> > fail inside Guix containers with the suggested incantation. After
> > getting the environment to no longer complain about $DISPLAY by
> > adding
> > `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly
> > outputs
> > lines like
> > bwrap: Can't find source path /sys/class: No such file or directory
> > before closing the process altogether.
>
> What is ‘bwrap’ looking for? /sys is mounted inside ‘guix
> environment -C’,
> but perhaps it needs something special?
>
> I suggest running these things (or ‘bwrap’ directly) in ‘strace -f -o
> log’ inside the container to see.
It seems to be
Toggle quote (5 lines)
> openat(AT_FDCWD, "/sys/class/dmi/id/chassis_type", O_RDONLY) = -1
> ENOENT (No such file or directory)
> openat(AT_FDCWD, "/sys/firmware/acpi/pm_profile", O_RDONLY) = -1
> ENOENT (No such file or directory)

I haven't repeated that for all warnings of similar kind, but if I add
`--expose=/sys/block --expose=/sys/class --expose=/sys/bus --
expose=/sys/dev --expose=/sys/devices` to the invocation, I instead get
a warning, that the WebKitWebProcess can't open $DISPLAY. I'm not sure
how to resolve that one, given that I already had to sneak DISPLAY and
XAUTHORITY into the container, but it's a start.

Regards,
Leo
B
B
Bengt Richter wrote on 13 Mar 2021 13:27
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
20210313122718.GA11708@LionPure
Hi,

On +2021-03-13 12:07:51 +0100, Leo Prikler wrote:
Toggle quote (39 lines)
> Hi!
> Am Samstag, den 13.03.2021, 11:48 +0100 schrieb Ludovic Courtès:
> > Hi!
> >
> > Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> >
> > > both Epiphany and Eolie (post fixing #47097; will submit patch
> > > shortly)
> > > fail inside Guix containers with the suggested incantation. After
> > > getting the environment to no longer complain about $DISPLAY by
> > > adding
> > > `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly
> > > outputsn
> > > lines like
> > > bwrap: Can't find source path /sys/class: No such file or directory
> > > before closing the process altogether.
> >
> > What is ‘bwrap’ looking for? /sys is mounted inside ‘guix
> > environment -C’,
> > but perhaps it needs something special?
> >
> > I suggest running these things (or ‘bwrap’ directly) in ‘strace -f -o
> > log’ inside the container to see.
> It seems to be
> > openat(AT_FDCWD, "/sys/class/dmi/id/chassis_type", O_RDONLY) = -1
> > ENOENT (No such file or directory)
> > openat(AT_FDCWD, "/sys/firmware/acpi/pm_profile", O_RDONLY) = -1
> > ENOENT (No such file or directory)
>
> I haven't repeated that for all warnings of similar kind, but if I add
> `--expose=/sys/block --expose=/sys/class --expose=/sys/bus --
> expose=/sys/dev --expose=/sys/devices` to the invocation, I instead get
> a warning, that the WebKitWebProcess can't open $DISPLAY. I'm not sure
> how to resolve that one, given that I already had to sneak DISPLAY and
> XAUTHORITY into the container, but it's a start.
>
> Regards,
> Leo
>
Does $DISPLAY mean ":0" and does the Wayland server answer that with
its XWayland X-interface?

I am wonderering how that is resolved inside a container.

HTH with the detective work ;)

--
Regards,
Bengt Richter
L
L
Leo Prikler wrote on 13 Mar 2021 15:43
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Bengt Richter)(address . bokr@bokr.com)
fa11fb1fb6dfb6e2c048d4fe8dec005e3b2b114a.camel@student.tugraz.at
Am Samstag, den 13.03.2021, 13:27 +0100 schrieb Bengt Richter:
Toggle quote (51 lines)
> Hi,
>
> On +2021-03-13 12:07:51 +0100, Leo Prikler wrote:
> > Hi!
> > Am Samstag, den 13.03.2021, 11:48 +0100 schrieb Ludovic Courtès:
> > > Hi!
> > >
> > > Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> > >
> > > > both Epiphany and Eolie (post fixing #47097; will submit patch
> > > > shortly)
> > > > fail inside Guix containers with the suggested
> > > > incantation. After
> > > > getting the environment to no longer complain about $DISPLAY by
> > > > adding
> > > > `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly
> > > > outputsn
> > > > lines like
> > > > bwrap: Can't find source path /sys/class: No such file or
> > > > directory
> > > > before closing the process altogether.
> > >
> > > What is ‘bwrap’ looking for? /sys is mounted inside ‘guix
> > > environment -C’,
> > > but perhaps it needs something special?
> > >
> > > I suggest running these things (or ‘bwrap’ directly) in ‘strace
> > > -f -o
> > > log’ inside the container to see.
> > It seems to be
> > > openat(AT_FDCWD, "/sys/class/dmi/id/chassis_type", O_RDONLY) = -1
> > > ENOENT (No such file or directory)
> > > openat(AT_FDCWD, "/sys/firmware/acpi/pm_profile", O_RDONLY) = -1
> > > ENOENT (No such file or directory)
> >
> > I haven't repeated that for all warnings of similar kind, but if I
> > add
> > `--expose=/sys/block --expose=/sys/class --expose=/sys/bus --
> > expose=/sys/dev --expose=/sys/devices` to the invocation, I instead
> > get
> > a warning, that the WebKitWebProcess can't open $DISPLAY. I'm not
> > sure
> > how to resolve that one, given that I already had to sneak DISPLAY
> > and
> > XAUTHORITY into the container, but it's a start.
> >
> > Regards,
> > Leo
> >
> Does $DISPLAY mean ":0" and does the Wayland server answer that with
> its XWayland X-interface?
In my setup $DISPLAY=:1, but obviously the exact value depends on other
circumstances (i.e. if there's already an open session belonging to
another user it'd be :2, :3, ...). I'm not sure how X vs. Wayland
plays out here, but I'm still using Gnome on X, so that should
hopefully not be an issue here.

Toggle quote (1 lines)
> I am wonderering how that is resolved inside a container.
Well, for X you'd usually preserve DISPLAY and XAUTHORITY and also
expose $XAUTHORITY or something along those lines. Not sure how you
Wayland folk do that.

Regards,
Leo
B
B
Bengt Richter wrote on 13 Mar 2021 18:07
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
20210313170704.GA3712@LionPure
Hi again,

On +2021-03-13 15:43:15 +0100, Leo Prikler wrote:
Toggle quote (64 lines)
> Am Samstag, den 13.03.2021, 13:27 +0100 schrieb Bengt Richter:
> > Hi,
> >
> > On +2021-03-13 12:07:51 +0100, Leo Prikler wrote:
> > > Hi!
> > > Am Samstag, den 13.03.2021, 11:48 +0100 schrieb Ludovic Courtès:
> > > > Hi!
> > > >
> > > > Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> > > >
> > > > > both Epiphany and Eolie (post fixing #47097; will submit patch
> > > > > shortly)
> > > > > fail inside Guix containers with the suggested
> > > > > incantation. After
> > > > > getting the environment to no longer complain about $DISPLAY by
> > > > > adding
> > > > > `--preserve="XAUTHORITY" --expose=$XAUTHORITY', it repeatedly
> > > > > outputsn
> > > > > lines like
> > > > > bwrap: Can't find source path /sys/class: No such file or
> > > > > directory
> > > > > before closing the process altogether.
> > > >
> > > > What is ‘bwrap’ looking for? /sys is mounted inside ‘guix
> > > > environment -C’,
> > > > but perhaps it needs something special?
> > > >
> > > > I suggest running these things (or ‘bwrap’ directly) in ‘strace
> > > > -f -o
> > > > log’ inside the container to see.
> > > It seems to be
> > > > openat(AT_FDCWD, "/sys/class/dmi/id/chassis_type", O_RDONLY) = -1
> > > > ENOENT (No such file or directory)
> > > > openat(AT_FDCWD, "/sys/firmware/acpi/pm_profile", O_RDONLY) = -1
> > > > ENOENT (No such file or directory)
> > >
> > > I haven't repeated that for all warnings of similar kind, but if I
> > > add
> > > `--expose=/sys/block --expose=/sys/class --expose=/sys/bus --
> > > expose=/sys/dev --expose=/sys/devices` to the invocation, I instead
> > > get
> > > a warning, that the WebKitWebProcess can't open $DISPLAY. I'm not
> > > sure
> > > how to resolve that one, given that I already had to sneak DISPLAY
> > > and
> > > XAUTHORITY into the container, but it's a start.
> > >
> > > Regards,
> > > Leo
> > >
> > Does $DISPLAY mean ":0" and does the Wayland server answer that with
> > its XWayland X-interface?
> In my setup $DISPLAY=:1, but obviously the exact value depends on other
> circumstances (i.e. if there's already an open session belonging to
> another user it'd be :2, :3, ...). I'm not sure how X vs. Wayland
> plays out here, but I'm still using Gnome on X, so that should
> hopefully not be an issue here.
>
> > I am wonderering how that is resolved inside a container.
> Well, for X you'd usually preserve DISPLAY and XAUTHORITY and also
> expose $XAUTHORITY or something along those lines. Not sure how you
> Wayland folk do that.
>

I am not a Wayland developer, if that's what you mean by "Wayland folk" :)
But I have been experimenting with writing my own text and graphics widget,
poking 32-bit pixels into buffers for display by the Wayland compositor,
so I've learned a little :) I am using the linux kernel's sun12x22 bitmap
font to do text at a low level.

Idk much about containers yet, but I imagine using lowlevel stuff to
make images for a trivial web server running in a container could be interesting.

Toggle quote (4 lines)
> Regards,
> Leo
>

I am curious what the commands below would show inside your container.
"pidparents" [1] is a little script I find handy, which would have to be
accessible in your container of course. Idk how you put local bash scripts
in your container. I assume it's possible :)

I did these commands in a debian gnome terminal window, where pidparents[1]
showed (timetagged later, since I just went back to do that) this context:
Toggle snippet (8 lines)
[17:29 ~/bs]$ pidparents
pidparents pts/1 5800 S+ /usr/bin/bash /home/bokr/bin/pidparents
bash pts/1 5711 Ss /bin/bash
tilix ? 2007 Sl /usr/bin/tilix --gapplication-service
systemd ? 1308 Ss /lib/systemd/systemd --user
systemd ? 1 Ss /sbin/init splash

;;;; First I just look for processes with X11 or way in their names,
;;;; then I use pidparents to see how they are started.

[17:18 ~/bs]$ ps af|egrep -i 'x11|way'
5741 pts/1 S+ 0:00 \_ grep -E -i x11|way
1329 tty2 Ssl+ 0:00 /usr/lib/gdm3/gdm-wayland-session /usr/bin/gnome-session
1433 tty2 Sl+ 0:13 | \_ /usr/bin/Xwayland :0 -rootless -terminate -accessx -core -listen 4 -listen 5 -displayfd 6
1468 tty2 Sl 0:00 /usr/lib/ibus/ibus-x11 --kill-daemon

[17:21 ~/bs]$ pidparents 1329
gdm-wayland-ses tty2 1329 Ssl+ /usr/lib/gdm3/gdm-wayland-session /usr/bin/gnome-session
gdm-session-wor ? 1304 Sl gdm-session-worker [pam/gdm-password]
gdm3 ? 711 Ssl /usr/sbin/gdm3
systemd ? 1 Ss /sbin/init splash

;;;; this one might be the most interesting in your container
;;;; can you SSH into it to do these things?
[17:22 ~/bs]$ pidparents 1433
Xwayland tty2 1433 Sl+ /usr/bin/Xwayland :0 -rootless -terminate -accessx -core -listen 4 -listen 5 -displayfd 6
gnome-shell tty2 1408 Rl+ /usr/bin/gnome-shell
gnome-session-b tty2 1333 Sl+ /usr/lib/gnome-session/gnome-session-binary
gdm-wayland-ses tty2 1329 Ssl+ /usr/lib/gdm3/gdm-wayland-session /usr/bin/gnome-session
gdm-session-wor ? 1304 Sl gdm-session-worker [pam/gdm-password]
gdm3 ? 711 Ssl /usr/sbin/gdm3
systemd ? 1 Ss /sbin/init splash

;;;; not really sure what this one does
[17:22 ~/bs]$ pidparents 1468
ibus-x11 tty2 1468 Sl /usr/lib/ibus/ibus-x11 --kill-daemon
systemd ? 1 Ss /sbin/init splash

;;;; [1] here is pidparents -- pretty short, so you could manually
;;;; enter it if necessary :)

[17:22 ~/bs]$ cat $(which pidparents)|gxsnip
Toggle snippet (11 lines)
#!/usr/bin/bash
# ~/bin/pidparents

pid=${1:-$$} #this process if no pid specified as $1

while [ $(($pid)) -gt 0 ]; do
ps h -p $pid -o comm,tt,pid,stat,args
pid=$(ps -q $pid -o ppid=)
done

[17:24 ~/bs]$

Hopefully this would reveal a little more about what $DISPLAY means in your container.

WDYT?

--
Regards,
Bengt Richter
L
L
Leo Prikler wrote on 13 Mar 2021 19:01
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Bengt Richter)(address . bokr@bokr.com)
a4efcc5c7928de5d89596500803dee510d85b7c0.camel@student.tugraz.at
Am Samstag, den 13.03.2021, 18:07 +0100 schrieb Bengt Richter:
Toggle quote (2 lines)
> I am not a Wayland developer, if that's what you mean by "Wayland
> folk" :)
I meant it as "folk using Wayland in their display manager".

Toggle quote (7 lines)
> I am curious what the commands below would show inside your
> container.
> "pidparents" [1] is a little script I find handy, which would have to
> be
> accessible in your container of course. Idk how you put local bash
> scripts
> in your container. I assume it's possible :)
Far from getting a script into my container, I can't even really get
into my container through means like `guix container exec`, so I simply
bloated it with screen and pstree. The result:

Toggle snippet (10 lines)
sh-+-dbus-daemon
|-dbus-launch
`-screen---screen-+-sh---.epiphany-real-+-WebKitNetworkPr---
11*[{WebKitNetworkPr}]
| |-bwrap---bwrap---
WebKitWebProces
| `-18*[{.epiphany-real}]
`-sh---pstree

I think these processes are created and die too quickly for me to
reliably extract PIDs.

Regards,
Leo
B
B
Bengt Richter wrote on 14 Mar 2021 18:45
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
20210314174539.GA10548@LionPure
Hi again^2,

On +2021-03-13 19:01:29 +0100, Leo Prikler wrote:
Toggle quote (34 lines)
> Am Samstag, den 13.03.2021, 18:07 +0100 schrieb Bengt Richter:
> > I am not a Wayland developer, if that's what you mean by "Wayland
> > folk" :)
> I meant it as "folk using Wayland in their display manager".
>
> > I am curious what the commands below would show inside your
> > container.
> > "pidparents" [1] is a little script I find handy, which would have to
> > be
> > accessible in your container of course. Idk how you put local bash
> > scripts
> > in your container. I assume it's possible :)
> Far from getting a script into my container, I can't even really get
> into my container through means like `guix container exec`, so I simply
> bloated it with screen and pstree. The result:
>
> --8<---------------cut here---------------start------------->8---
> sh-+-dbus-daemon
> |-dbus-launch
> `-screen---screen-+-sh---.epiphany-real-+-WebKitNetworkPr---
> 11*[{WebKitNetworkPr}]
> | |-bwrap---bwrap---
> WebKitWebProces
> | `-18*[{.epiphany-real}]
> `-sh---pstree
> --8<---------------cut here---------------end--------------->8---
>
> I think these processes are created and die too quickly for me to
> reliably extract PIDs.
>
> Regards,
> Leo
>

Maybe
pstree -at
would show a little more?
Also,
ls -lr /sys/class/drm
if that's accessible -- I'm wondering if the version of screen
in the container is built with libdrm and is bypassing X or ??

Do you have a makefile or a guix something.scm defining
what's built/packed into your container?

Sorry if my curiosity is making work for you, but I'd like to
try containers down the road -- tho right now I'm taking a break
from events IRL, so I may disappear for a while...

--
Regards,
Bengt Richter
L
L
Leo Prikler wrote on 14 Mar 2021 19:05
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Bengt Richter)(address . bokr@bokr.com)
d0638eba7e63c71edd4267c1675e0ea7f5b7b4ae.camel@student.tugraz.at
Hi again³

Am Sonntag, den 14.03.2021, 18:45 +0100 schrieb Bengt Richter:
Toggle quote (5 lines)
> Hi again^2,
>
> Maybe
> pstree -at
> would show a little more?
sh
|-dbus-daemon --syslog-only --fork --print-pid 5 --print-address 7
--sess
|-dbus-launch --autolaunch=fa7a4d52637958ddd37547bb5d8bd9d2--binary-
synt
`-screen
`-screen
|-sh
| `-.epiphany-real
| |-WebKitNetworkPr 3 21
| | |-{BMScavenger}
| | |-{ReceiveQueue}
| | |-{StorageTask}
| | |-{Storage}
| | |-{WebStorage}
| | |-{background}
| | |-{dconf worker}
| | |-{erialBackground}
| | |-{gdbus}
| | `-{gmain}
| |-bwrap --args 37 --
/gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
| | `-bwrap --args 37 --
/gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
| | `-WebKitWebProces 1277 28
| |-{.epiphany-real}
| |-{BMScavenger}
| |-{HashSaltStorage}
| |-{IconDatabase}
| |-{PressureMonitor}
| |-2*[{ReceiveQueue}]
| |-{dconf worker}
| |-{e Compile Queue}
| |-{ebsiteDataStore}
| |-{gdbus}
| |-{gmain}
| |-{re Remove Queue}
| `-{tore Read Queue}
`-sh
`-pstree -at
Toggle quote (2 lines)
> Also,
> ls -lr /sys/class/drm
total 0
-r--r--r-- 1 65534 overflow 4096 Mar 14 17:59 version
lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:58 ttm ->
../../devices/virtual/drm/ttm
lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:59 renderD128 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/renderD128
lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:59 card0-VGA-1 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-VGA-
1
lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:59 card0-HDMI-A-1 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
HDMI-A-1
lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:58 card0-DVI-D-1 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-DVI-
D-1
lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:58 card0 ->
../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0
Toggle quote (2 lines)
> if that's accessible -- I'm wondering if the version of screen
> in the container is built with libdrm and is bypassing X or ??
I doubt it is being built differently than screen normally is.

Toggle quote (2 lines)
> Do you have a makefile or a guix something.scm defining
> what's built/packed into your container?
Nah, it's a rather ad-hoc definition grown from what should be an Eolie
container from the cookbook (also refer to #47097).

guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
--preserve=TERM \
--expose=$XAUTHORITY \
--expose=/etc/machine-id \
--expose=/etc/ssl/certs/ \
--expose=/sys/block --expose=/sys/class --expose=/sys/bus \
--expose=/sys/dev --expose=/sys/devices \
--ad-hoc epiphany nss-certs dbus procps coreutils psmisc screen

Given that I expose most of /sys explicitly, you should take the above
with a grain of salt.

Toggle quote (3 lines)
> Sorry if my curiosity is making work for you, but I'd like to
> try containers down the road -- tho right now I'm taking a break
> from events IRL, so I may disappear for a while...
I'm not personally impacted by this bug or anything, it's much rather a
follow-up to my attempted fix of #47097. I think there might be some
flaw in trying to run a sandbox inside a sandbox (like bubblewrap
inside `guix container`), that doesn't actually improve security in any
meaningful way.

Regards,
Leo
L
L
Ludovic Courtès wrote on 14 Mar 2021 21:32
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
87sg4xlbn0.fsf@gnu.org
Hi Leo,

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

Toggle quote (12 lines)
> Nah, it's a rather ad-hoc definition grown from what should be an Eolie
> container from the cookbook (also refer to #47097).
>
> guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
> --preserve=TERM \
> --expose=$XAUTHORITY \
> --expose=/etc/machine-id \
> --expose=/etc/ssl/certs/ \
> --expose=/sys/block --expose=/sys/class --expose=/sys/bus \
> --expose=/sys/dev --expose=/sys/devices \
> --ad-hoc epiphany nss-certs dbus procps coreutils psmisc screen

I’m not sure I follow; does it work when you do this?

/sys is already mounted inside ‘guix environment -C’ containers so I
don’t see what difference it would make.

But wait, the example above lacks ‘-C’; a mistake?

Thanks,
Ludo’.
L
L
Leo Prikler wrote on 14 Mar 2021 21:43
(name . Ludovic Courtès)(address . ludo@gnu.org)
6c6b39f495962ec906255cac212b66962d549eab.camel@student.tugraz.at
Am Sonntag, den 14.03.2021, 21:32 +0100 schrieb Ludovic Courtès:
Toggle quote (19 lines)
> Hi Leo,
>
> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
>
> > Nah, it's a rather ad-hoc definition grown from what should be an
> > Eolie
> > container from the cookbook (also refer to #47097).
> >
> > guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
> > --preserve=TERM \
> > --expose=$XAUTHORITY \
> > --expose=/etc/machine-id \
> > --expose=/etc/ssl/certs/ \
> > --expose=/sys/block --expose=/sys/class --expose=/sys/bus \
> > --expose=/sys/dev --expose=/sys/devices \
> > --ad-hoc epiphany nss-certs dbus procps coreutils psmisc
> > screen
>
> I’m not sure I follow; does it work when you do this?
It does work insofar as I don't get any warnings about resources
missing from /sys, but the bubblewrapped WebKit processes don't have
access to $DISPLAY even though epiphany itself has. While they don't
crash the browser itself and just infinitely respawn, that's still far
from usable.

Toggle quote (2 lines)
> /sys is already mounted inside ‘guix environment -C’ containers so I
> don’t see what difference it would make.
I think I've been told this several times, but I don't believe it. Not
adding all these expose=/sys lines triggers the "warnings" in the
original post. (Okay, perhaps one of /sys/dev and /sys/devices is
superfluous, I would need to check.)

Toggle quote (1 lines)
> But wait, the example above lacks ‘-C’; a mistake?
Indeed, -CN should also be given, but I hastily edited the command line
inside the email to make it appear more beautiful than it actually is,
thereby deleting it. I'm sorry. The preserves and exposes should be
the same list as I'm actually using however.

Regards,
Leo
L
L
Ludovic Courtès wrote on 15 Mar 2021 10:52
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
87pn00iw1f.fsf@gnu.org
Hi Leo,

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

Toggle quote (7 lines)
>> /sys is already mounted inside ‘guix environment -C’ containers so I
>> don’t see what difference it would make.
> I think I've been told this several times, but I don't believe it. Not
> adding all these expose=/sys lines triggers the "warnings" in the
> original post. (Okay, perhaps one of /sys/dev and /sys/devices is
> superfluous, I would need to check.)

It would be great if you could pinpoint which of these --expose=/sys/xyz
makes a difference. From there we could compare the output of ‘find
/sys/xyz’ inside and outside the container, without --expose.

Thanks,
Ludo’.
L
L
Leo Prikler wrote on 15 Mar 2021 11:14
(name . Ludovic Courtès)(address . ludo@gnu.org)
b7e184f6a0ceece097f9fc3f25cb47a0ccdf6d34.camel@student.tugraz.at
Am Montag, den 15.03.2021, 10:52 +0100 schrieb Ludovic Courtès:
Toggle quote (17 lines)
> Hi Leo,
>
> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
>
> > > /sys is already mounted inside ‘guix environment -C’ containers
> > > so I
> > > don’t see what difference it would make.
> > I think I've been told this several times, but I don't believe
> > it. Not
> > adding all these expose=/sys lines triggers the "warnings" in the
> > original post. (Okay, perhaps one of /sys/dev and /sys/devices is
> > superfluous, I would need to check.)
>
> It would be great if you could pinpoint which of these --
> expose=/sys/xyz
> makes a difference. From there we could compare the output of ‘find
> /sys/xyz’ inside and outside the container, without --expose.
Okay, so here's my basic workflow: Starting with an empty set of --
expose:
bwrap: Can't find source path /sys/block: No such file or directory
repeated a few times along with warnings, that the web process crashed
until finally Epiphany itself crashes.
I add /sys/block, and Epiphany miraculously doesn't crash, but bwrap
still complains and the web processes still crash, so we march on.
I'll abbreviate it a little and only show the error messages.
bwrap: Can't find source path /sys/bus: No such file or directory
bwrap: Can't find source path /sys/class: No such file or directory
bwrap: Can't find source path /sys/dev: No such file or directory
bwrap: Can't find source path /sys/devices: No such file or directory

After exposing all of the above, I get
Unable to init server: Could not connect: Connection refused

(WebKitWebProcess:2): Gtk-WARNING **: 10:09:01.497: cannot open
display: :1
Using --share instead of --expose for the final set does not seem to
change anything.

For /sys/block, the find inside the container is empty before exposing
it and non-empty on the host. I assume the same holds for the others.

Regards,
Leo
L
L
Ludovic Courtès wrote on 15 Mar 2021 14:29
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
87sg4wh7f6.fsf@gnu.org
Hi,

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

Toggle quote (13 lines)
> Okay, so here's my basic workflow: Starting with an empty set of --
> expose:
> bwrap: Can't find source path /sys/block: No such file or directory
> repeated a few times along with warnings, that the web process crashed
> until finally Epiphany itself crashes.
> I add /sys/block, and Epiphany miraculously doesn't crash, but bwrap
> still complains and the web processes still crash, so we march on.
> I'll abbreviate it a little and only show the error messages.
> bwrap: Can't find source path /sys/bus: No such file or directory
> bwrap: Can't find source path /sys/class: No such file or directory
> bwrap: Can't find source path /sys/dev: No such file or directory
> bwrap: Can't find source path /sys/devices: No such file or directory

OK. Additional data points from my Guix System laptop:

Toggle snippet (11 lines)
$ guix environment -C --ad-hoc findutils -- find /sys |wc -l
37575
$ find /sys | wc -l
find: ‘/sys/kernel/debug’: Mankas permeso
38026
$ guix environment -C --ad-hoc findutils -- find /sys/block |wc -l
27
$ find /sys/block | wc -l
27

The offending Bubblewrap code is:

Toggle snippet (31 lines)
static void
resolve_symlinks_in_ops (void)
{
SetupOp *op;

for (op = ops; op != NULL; op = op->next)
{
const char *old_source;

switch (op->type)
{
case SETUP_RO_BIND_MOUNT:
case SETUP_DEV_BIND_MOUNT:
case SETUP_BIND_MOUNT:
old_source = op->source;
op->source = realpath (old_source, NULL);
if (op->source == NULL)
{
if (op->flags & ALLOW_NOTEXIST && errno == ENOENT)
op->source = old_source;
else
die_with_error("Can't find source path %s", old_source);
}
break;
default:
break;
}
}
}

I wonder how ‘realpath’ can fail here. In fact, it Works For Me:

Toggle snippet (21 lines)
$ cat realpath.scm
(use-modules (system foreign))

(define realpath
(let ((proc (pointer->procedure '* (dynamic-func "realpath" (dynamic-link))
'(* *))))
(lambda (path)
(let ((result (proc (string->pointer path) %null-pointer)))
(and (not (null-pointer? result))
(pointer->string result))))))

(pk 'realpath-> (realpath "/sys/block"))
$ guix environment -C --ad-hoc guile -- guile -s realpath.scm
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;; or pass the --no-auto-compile argument to disable.
;;; compiling /home/ludo/src/guix-debugging/realpath.scm
;;; compiled /home/ludo/.cache/guile/ccache/3.0-LE-8-4.4/home/ludo/src/guix-debugging/realpath.scm.go

;;; (realpath-> "/sys/block")

So I presume bwrap does something else before it reaches that warning.
We should really strace it.

Thanks,
Ludo’.
B
B
Bengt Richter wrote on 16 Mar 2021 11:54
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)
20210316105442.GA3903@LionPure
Hi Leo,
One more favor? ;)

On +2021-03-14 19:05:24 +0100, Leo Prikler wrote:
Toggle quote (101 lines)
> Hi again³
>
> Am Sonntag, den 14.03.2021, 18:45 +0100 schrieb Bengt Richter:
> > Hi again^2,
> >
> > Maybe
> > pstree -at
> > would show a little more?
> sh
> |-dbus-daemon --syslog-only --fork --print-pid 5 --print-address 7
> --sess
> |-dbus-launch --autolaunch=fa7a4d52637958ddd37547bb5d8bd9d2--binary-
> synt
> `-screen
> `-screen
> |-sh
> | `-.epiphany-real
> | |-WebKitNetworkPr 3 21
> | | |-{BMScavenger}
> | | |-{ReceiveQueue}
> | | |-{StorageTask}
> | | |-{Storage}
> | | |-{WebStorage}
> | | |-{background}
> | | |-{dconf worker}
> | | |-{erialBackground}
> | | |-{gdbus}
> | | `-{gmain}
> | |-bwrap --args 37 --
> /gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
> | | `-bwrap --args 37 --
> /gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
> | | `-WebKitWebProces 1277 28
> | |-{.epiphany-real}
> | |-{BMScavenger}
> | |-{HashSaltStorage}
> | |-{IconDatabase}
> | |-{PressureMonitor}
> | |-2*[{ReceiveQueue}]
> | |-{dconf worker}
> | |-{e Compile Queue}
> | |-{ebsiteDataStore}
> | |-{gdbus}
> | |-{gmain}
> | |-{re Remove Queue}
> | `-{tore Read Queue}
> `-sh
> `-pstree -at
> > Also,
> > ls -lr /sys/class/drm
> total 0
> -r--r--r-- 1 65534 overflow 4096 Mar 14 17:59 version
> lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:58 ttm ->
> ../../devices/virtual/drm/ttm
> lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:59 renderD128 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/renderD128
> lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:59 card0-VGA-1 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-VGA-
> 1
> lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:59 card0-HDMI-A-1 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
> HDMI-A-1
> lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:58 card0-DVI-D-1 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-DVI-
> D-1
> lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:58 card0 ->
> ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0
> > if that's accessible -- I'm wondering if the version of screen
> > in the container is built with libdrm and is bypassing X or ??
> I doubt it is being built differently than screen normally is.
>
> > Do you have a makefile or a guix something.scm defining
> > what's built/packed into your container?
> Nah, it's a rather ad-hoc definition grown from what should be an Eolie
> container from the cookbook (also refer to #47097).
>
> guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
> --preserve=TERM \
> --expose=$XAUTHORITY \
> --expose=/etc/machine-id \
> --expose=/etc/ssl/certs/ \
> --expose=/sys/block --expose=/sys/class --expose=/sys/bus \
> --expose=/sys/dev --expose=/sys/devices \
> --ad-hoc epiphany nss-certs dbus procps coreutils psmisc screen
>
> Given that I expose most of /sys explicitly, you should take the above
> with a grain of salt.
>
> > Sorry if my curiosity is making work for you, but I'd like to
> > try containers down the road -- tho right now I'm taking a break
> > from events IRL, so I may disappear for a while...
> I'm not personally impacted by this bug or anything, it's much rather a
> follow-up to my attempted fix of #47097. I think there might be some
> flaw in trying to run a sandbox inside a sandbox (like bubblewrap
> inside `guix container`), that doesn't actually improve security in any
> meaningful way.
>
> Regards,
> Leo
>

If you can run this inside your container, I think it will be interesting:
lsof -U|grep -i wayland

The above ought to show quickly if wayland is running.

lsof -U shows the open sockets.

If the above shows nothing, try
lsof -U|grep -i x11
or
lsof -U|grep X

finally, it is interesting to see
lsof -U|less

but on my laptop I just got
lsof -U|wc
403 3760 34643

so its a lot to look at.
Hopefully less in a container ;)

--
Regards,
Bengt Richter
L
L
Leo Prikler wrote on 16 Mar 2021 12:13
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Bengt Richter)(address . bokr@bokr.com)
65e3ddcc4b625ed496222f6072542cd250b08a76.camel@student.tugraz.at
Hi,
Am Dienstag, den 16.03.2021, 11:54 +0100 schrieb Bengt Richter:
Toggle quote (127 lines)
> Hi Leo,
> One more favor? ;)
>
> On +2021-03-14 19:05:24 +0100, Leo Prikler wrote:
> > Hi again³
> >
> > Am Sonntag, den 14.03.2021, 18:45 +0100 schrieb Bengt Richter:
> > > Hi again^2,
> > >
> > > Maybe
> > > pstree -at
> > > would show a little more?
> > sh
> > |-dbus-daemon --syslog-only --fork --print-pid 5 --print-address
> > 7
> > --sess
> > |-dbus-launch --autolaunch=fa7a4d52637958ddd37547bb5d8bd9d2
> > --binary-
> > synt
> > `-screen
> > `-screen
> > |-sh
> > | `-.epiphany-real
> > | |-WebKitNetworkPr 3 21
> > | | |-{BMScavenger}
> > | | |-{ReceiveQueue}
> > | | |-{StorageTask}
> > | | |-{Storage}
> > | | |-{WebStorage}
> > | | |-{background}
> > | | |-{dconf worker}
> > | | |-{erialBackground}
> > | | |-{gdbus}
> > | | `-{gmain}
> > | |-bwrap --args 37 --
> > /gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
> > | | `-bwrap --args 37 --
> > /gnu/store/hqhxgw0i8xh38h6kwmyrkywcd24q5f1z-webk
> > | | `-WebKitWebProces 1277 28
> > | |-{.epiphany-real}
> > | |-{BMScavenger}
> > | |-{HashSaltStorage}
> > | |-{IconDatabase}
> > | |-{PressureMonitor}
> > | |-2*[{ReceiveQueue}]
> > | |-{dconf worker}
> > | |-{e Compile Queue}
> > | |-{ebsiteDataStore}
> > | |-{gdbus}
> > | |-{gmain}
> > | |-{re Remove Queue}
> > | `-{tore Read Queue}
> > `-sh
> > `-pstree -at
> > > Also,
> > > ls -lr /sys/class/drm
> > total 0
> > -r--r--r-- 1 65534 overflow 4096 Mar 14 17:59 version
> > lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:58 ttm ->
> > ../../devices/virtual/drm/ttm
> > lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:59 renderD128 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/renderD128
> > lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:59 card0-VGA-1 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
> > VGA-
> > 1
> > lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:59 card0-HDMI-A-1 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
> > HDMI-A-1
> > lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:58 card0-DVI-D-1 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0/card0-
> > DVI-
> > D-1
> > lrwxrwxrwx 1 65534 overflow 0 Mar 14 17:58 card0 ->
> > ../../devices/pci0000:00/0000:00:02.0/0000:01:00.0/drm/card0
> > > if that's accessible -- I'm wondering if the version of screen
> > > in the container is built with libdrm and is bypassing X or ??
> > I doubt it is being built differently than screen normally is.
> >
> > > Do you have a makefile or a guix something.scm defining
> > > what's built/packed into your container?
> > Nah, it's a rather ad-hoc definition grown from what should be an
> > Eolie
> > container from the cookbook (also refer to #47097).
> >
> > guix environment --preserve='^DISPLAY$' --preserve=XAUTHORITY \
> > --preserve=TERM \
> > --expose=$XAUTHORITY \
> > --expose=/etc/machine-id \
> > --expose=/etc/ssl/certs/ \
> > --expose=/sys/block --expose=/sys/class --expose=/sys/bus \
> > --expose=/sys/dev --expose=/sys/devices \
> > --ad-hoc epiphany nss-certs dbus procps coreutils psmisc
> > screen
> >
> > Given that I expose most of /sys explicitly, you should take the
> > above
> > with a grain of salt.
> >
> > > Sorry if my curiosity is making work for you, but I'd like to
> > > try containers down the road -- tho right now I'm taking a break
> > > from events IRL, so I may disappear for a while...
> > I'm not personally impacted by this bug or anything, it's much
> > rather a
> > follow-up to my attempted fix of #47097. I think there might be
> > some
> > flaw in trying to run a sandbox inside a sandbox (like bubblewrap
> > inside `guix container`), that doesn't actually improve security in
> > any
> > meaningful way.
> >
> > Regards,
> > Leo
> >
>
> If you can run this inside your container, I think it will be
> interesting:
> lsof -U|grep -i wayland
>
> The above ought to show quickly if wayland is running.
>
> lsof -U shows the open sockets.
>
> If the above shows nothing, try
> lsof -U|grep -i x11
> or
> lsof -U|grep X
Nothing showed up for either, but this got me thinking. Exposing
/tmp/.X11-unix/X1 did do away with the warning, now it's unexposed
dbus, missing icons, etc. etc. Exposing all of /tmp instead yields

** (epiphany:2): ERROR **: 11:11:28.855: Failed to start embed shell D-
Bus server on unix:dir=(null): Error binding to address: No such file
or directory

I still think that exposing all of that is perhaps not the wisest idea,
but eh…

Regards,
Leo
Z
Z
zimoun wrote on 24 Mar 2021 10:23
block 47297 by 47097 and 47106
(address . control@debbugs.gnu.org)
86mtushpm3.fsf@gmail.com
block 47297 by 47097
block 47297 by 47106
thanks
L
L
Leo Famulari wrote on 14 Apr 2021 22:07
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)(address . 47106@debbugs.gnu.org)
YHdLeYOvZKHh/3v0@jasmine.lan
This bug is registered as "blocking" the upcoming release.

Is there any chance it will be closed in the next 3 days? Or should I
"unblock"?
L
L
Leo Prikler wrote on 14 Apr 2021 23:23
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Leo Famulari)(address . leo@famulari.name)(address . 47106@debbugs.gnu.org)
d53a3d55f49f7f858eda39862d4b23753b850cb4.camel@student.tugraz.at
Am Mittwoch, den 14.04.2021, 16:07 -0400 schrieb Leo Famulari:
Toggle quote (4 lines)
> This bug is registered as "blocking" the upcoming release.
>
> Is there any chance it will be closed in the next 3 days? Or should I
> "unblock"?
There is a slim chance, that someone might fix it "by accident", but
I'm personally not diving that deep into bubblewrap, so I don't even
have an estimate of how much we would need to change. I think
unblocking is the wiser choice here; we can update the manual post
string freeze.

Regards,
Leo
L
L
Leo Famulari wrote on 14 Apr 2021 23:59
(no subject)
(address . control@debbugs.gnu.org)
YHdlyi8rBGZGkLsA@jasmine.lan
unblock 47297 with 47106
L
L
Leo Famulari wrote on 15 Apr 2021 00:00
Re: bug#47106: Bubblewrap hates Guix containers 😞
(name . Leo Prikler)(address . leo.prikler@student.tugraz.at)(address . 47106@debbugs.gnu.org)
YHdl4re+gltgRrL5@jasmine.lan
On Wed, Apr 14, 2021 at 11:23:28PM +0200, Leo Prikler wrote:
Toggle quote (11 lines)
> Am Mittwoch, den 14.04.2021, 16:07 -0400 schrieb Leo Famulari:
> > This bug is registered as "blocking" the upcoming release.
> >
> > Is there any chance it will be closed in the next 3 days? Or should I
> > "unblock"?
> There is a slim chance, that someone might fix it "by accident", but
> I'm personally not diving that deep into bubblewrap, so I don't even
> have an estimate of how much we would need to change. I think
> unblocking is the wiser choice here; we can update the manual post
> string freeze.

Alright, thanks for explaining. I've done the "unblocking".
?