Fakechroot execution engine doesn’t outlive ‘exec’ calls

  • Done
  • quality assurance status badge
Details
3 participants
  • Josselin Poiret
  • Ludovic Courtès
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 18 Mar 2023 12:48
(address . bug-guix@gnu.org)
877cveuvov.fsf@inria.fr
For ‘guix pack -RR’ packs, unlike the “userns” and “proot” execution
engines, the “fakechroot” execution engine doesn’t survive ‘exec’ calls:

Toggle snippet (11 lines)
$ mkdir -p /tmp/fakechroot-test && cd /tmp/fakechroot-test/ && tar xf $(guix pack -RR openmpi intel-mpi-benchmarks bash-minimal -S /bin=bin)
$ unshare -m -U -r -f sh -c 'mount -t tmpfs none /gnu; GUIX_EXECUTION_ENGINE=fakechroot /tmp/fakechroot-test/bin/bash'
bash-5.1# echo /gnu/store/*coreutils*/bin/ls
/gnu/store/d251rfgc9nm2clzffzhgiipdvfvzkvwi-coreutils-8.32/bin/ls /gnu/store/vqdsrvs9jbn0ix2a58s99jwkh74124y5-coreutils-minimal-8.32/bin/ls
bash-5.1# test -f /gnu/store/*coreutils-8*/bin/ls
bash-5.1# echo $?
0
bash-5.1# /gnu/store/*coreutils-8*/bin/ls
bash: /gnu/store/d251rfgc9nm2clzffzhgiipdvfvzkvwi-coreutils-8.32/bin/ls: No such file or directory

This is because the ELF interpreter of the unwrapped ‘ls’ binary remains
/gnu/store/…-glibc-2.33/lib/ld-linux-x86-64-so.2 and no LD_PRELOAD
interposition can address that.

In this case, adding ‘coreutils’ to the profile (on the ‘guix pack’
command line) would give us wrapped binaries, and the problem is solved.
But in other cases, it’s not that simple. For instance, libmpi.so from
Open MPI tries to exec one its programs, using its absolute file name:

Toggle snippet (15 lines)
$ unshare -m -U -r -f sh -c 'mount -t tmpfs none /gnu; GUIX_EXECUTION_ENGINE=fakechroot /tmp/fakechroot-test/bin/IMB-MPI1'
--------------------------------------------------------------------------
The singleton application was not able to find the executable "orted" in
your PATH or in the directory where Open MPI/OpenRTE was initially installed,
and therefore cannot continue.

For reference, we tried the following command:

/gnu/store/c7g9qalmbz4a94hwzk1v1cbq7n5m8plq-openmpi-4.1.4/bin/orted

and got the error No such file or directory.

[…]

I can think of several ways to address that:

1. Change the exec* wrappers in libfakechroot such that, on ENOENT,
they try a direct ld.so invocation to run program, like
‘run-in-namespace.c’ does.

Problem is that for this to work correctly, it would need to
compute the ‘--library-path’ argument at run time, by computing the
equivalent of (map dirname (file-needed/recursive program)).
Impractical at best.

2. Wrap/graft every package in the closure (as opposed to generating
wrappers for just those packages that appear in the profile, which
is what ‘guix pack’ currently does).

The downside is that the “userns” and “proot” execution engines
don’t need something this heavyweight: they just need a leaf
package to be wrapped.

3. Ignore the problem. After all, we’re talking about a corner case
of the “fakechroot” engine, which is a niche within a niche.

Food for thought…

Ludo’.
J
J
Josselin Poiret wrote on 18 Mar 2023 22:47
Re: bug#62253: Fakechroot execution engine doesn ’t outlive ‘exec’ calls
87jzzd68af.fsf@jpoiret.xyz
Hi Ludo,

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

Toggle quote (24 lines)
> I can think of several ways to address that:
>
> 1. Change the exec* wrappers in libfakechroot such that, on ENOENT,
> they try a direct ld.so invocation to run program, like
> ‘run-in-namespace.c’ does.
>
> Problem is that for this to work correctly, it would need to
> compute the ‘--library-path’ argument at run time, by computing the
> equivalent of (map dirname (file-needed/recursive program)).
> Impractical at best.
>
> 2. Wrap/graft every package in the closure (as opposed to generating
> wrappers for just those packages that appear in the profile, which
> is what ‘guix pack’ currently does).
>
> The downside is that the “userns” and “proot” execution engines
> don’t need something this heavyweight: they just need a leaf
> package to be wrapped.
>
> 3. Ignore the problem. After all, we’re talking about a corner case
> of the “fakechroot” engine, which is a niche within a niche.
>
> Food for thought…

I would like to be proven wrong, but I don't think anyone has run into
this, and there are other possible engines (that do require more
privileges, sure). It seems quite non-trivial to fix, so this can
probably go on the back-burner until someone actually complains (please
do so).

Best,
--
Josselin Poiret
-----BEGIN PGP SIGNATURE-----

iQHEBAEBCAAuFiEEOSSM2EHGPMM23K8vUF5AuRYXGooFAmQWMYgQHGRldkBqcG9p
cmV0Lnh5egAKCRBQXkC5FhcaiqmIC/sGGKV8E/eh+V686YljPOpCFofNfD8zG8f4
SmAm1ejzeHZ2zpC75226tZqxNLEcu/H34O6hZuiBEJKlBpVdy4mcKR7yy9/NqILR
gW6KrQvPKo2IJe52oOXFi6HJLYKVDx5V5+EdDk9vdDK6X6TihKfj9q8lPIzyrIxh
A4+mEbRBMBQOqghErpWHPRwsBVSxsSILE/cpjrKArcMktIGsyKQxFKr+4s4AggjV
UNQ27WiQ5LQNf+TfuUiyDWdw7RLz5fsVe7fItjLT5AGSBYcG/zXPvTqZB/if0wXs
WEV7kfRdT6tNSZV9SKgfj5KaHYXXLh9gdj5iTB+4Mo9OrcacXyjazxGkEbOkNYeD
rZ+bYsXY04JBb4FkvaWze+TmuJ/m0rvkxMhXtIvURGmH+Ki9YUFwb0nj7qQPuVPz
tZ+/mAQSYaI9CwuyGyQigX+WIpwhM8hAHHo9/NjRln0Wn25syrO0RYFidIv8c2Kc
mvMbjhLIMwYxgmpd/y4c4xZM1X6xFnk=
=1JeR
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 20 Mar 2023 10:17
(name . Josselin Poiret)(address . dev@jpoiret.xyz)(address . 62253@debbugs.gnu.org)
87o7onok83.fsf@inria.fr
Hi,

Josselin Poiret <dev@jpoiret.xyz> skribis:

Toggle quote (6 lines)
> I would like to be proven wrong, but I don't think anyone has run into
> this, and there are other possible engines (that do require more
> privileges, sure). It seems quite non-trivial to fix, so this can
> probably go on the back-burner until someone actually complains (please
> do so).

The Open MPI example I gave earlier is one that I’m interested in for
work (running packs on HPC clusters that have neither unprivileged user
namespaces nor Singularity¹). It might be that we can work around the
problem though, we’ll see…

Ludo’.

L
L
Ludovic Courtès wrote on 18 Apr 2023 14:09
(address . 62253@debbugs.gnu.org)
87wn29wfwo.fsf@gnu.org
Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

Toggle quote (17 lines)
> In this case, adding ‘coreutils’ to the profile (on the ‘guix pack’
> command line) would give us wrapped binaries, and the problem is solved.
> But in other cases, it’s not that simple. For instance, libmpi.so from
> Open MPI tries to exec one its programs, using its absolute file name:
>
> $ unshare -m -U -r -f sh -c 'mount -t tmpfs none /gnu; GUIX_EXECUTION_ENGINE=fakechroot /tmp/fakechroot-test/bin/IMB-MPI1'
> --------------------------------------------------------------------------
> The singleton application was not able to find the executable "orted" in
> your PATH or in the directory where Open MPI/OpenRTE was initially installed,
> and therefore cannot continue.
>
> For reference, we tried the following command:
>
> /gnu/store/c7g9qalmbz4a94hwzk1v1cbq7n5m8plq-openmpi-4.1.4/bin/orted
>
> and got the error No such file or directory.

In the case of Open MPI as shown above, there’s actually an easy fix:
telling Open MPI where to look for ‘orted’.

Assuming you created a pack with:

guix pack -RR openmpi intel-mpi-benchmarks bash-minimal -S /bin=bin

You can run code extracted from the tarball like this:

Toggle snippet (6 lines)
export GUIX_EXECUTION_ENGINE=performance
salloc -N2 ./bin/mpirun -np 2 --launch-agent ./bin/orted \
--map-by node -x GUIX_EXECUTION_ENGINE=performance -- \
./bin/IMB-MPI1 PingPong

The ‘--launch-agent’ trick allows us to work around the fact that exec’d
code escapes the fakechroot environment.

I’m closing this bug as “wontfix”.

Ludo’.
L
L
Ludovic Courtès wrote on 18 Apr 2023 14:09
control message for bug #62253
(address . control@debbugs.gnu.org)
87v8htwfwg.fsf@gnu.org
tags 62253 wontfix
close 62253
quit
?