Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
Toggle quote (12 lines)
> $ guix describe
> Generacio 159 Sep 18 2020 08:55:27 (nuna)
> guix 98b89f4
> repository URL: https://git.savannah.gnu.org/git/guix.git
> branch: master
> commit: 98b89f432103b66efacee0bcba41a94148b8e870
> $ mkdir /tmp/fakechroot-test
> $ (cd /tmp/fakechroot-test; tar xf $(guix pack -RR openmpi -S /bin=bin -v0))
> $ unshare -m -U -r -f sh -c 'mount -t tmpfs none /gnu; echo /gnu/*; GUIX_EXECUTION_ENGINE=fakechroot /tmp/fakechroot-test/bin/orterun'
> /gnu/*
> /tmp/fakechroot-test/gnu/store/rm1d9z3z3iwsv779vqlg7q5z7pdk5k12-openmpi-4.0.5/bin//orterun: error while loading shared libraries: libevent_core-2.1.so.7: cannot open shared object file: No such file or directory
If we look at the LD_DEBUG=all output, we see that
/gnu/store/…libevent-2.11.1/lib has disappeared from the search path at
the point where ld.so looks for libevent_core.so:
Toggle snippet (25 lines)
15220: file=libevent_core-2.1.so.7 [0]; needed by /tmp/openmpi/gnu/store/rm1d9z3z3iwsv779vqlg7q5z7pdk5k12-openmpi-4.0.5/bin//orterun [0]
15220: find library=libevent_core-2.1.so.7 [0]; searching
15220: search path=/tmp/openmpi/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib:/tmp/openmpi/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib:/tmp/openmpi/gnu/store:/tmp/openmpi/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../.. (LD_LIBRARY_PATH)
15220: trying file=/tmp/openmpi/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/libevent_core-2.1.so.7
15220: trying file=/tmp/openmpi/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/libevent_core-2.1.so.7
15220: trying file=/tmp/openmpi/gnu/store/libevent_core-2.1.so.7
15220: trying file=/tmp/openmpi/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../libevent_core-2.1.so.7
15220: search path=/gnu/store/rm1d9z3z3iwsv779vqlg7q5z7pdk5k12-openmpi-4.0.5/lib:/gnu/store/xk22b03jb0c0b6vkadqbkr9d4556rwxh-hwloc-2.2.0-lib/lib:/gnu/store/i1smdd4r44c5wwi03g40dvc6wfnqgdwq-libpciaccess-0.16/lib (RUNPATH from file /tmp/openmpi/gnu/store/rm1d9z3z3iwsv779vqlg7q5z7pdk5k12-openmpi-4.0.5/bin//orterun)
15220: trying file=/gnu/store/rm1d9z3z3iwsv779vqlg7q5z7pdk5k12-openmpi-4.0.5/lib/libevent_core-2.1.so.7
15220: trying file=/gnu/store/xk22b03jb0c0b6vkadqbkr9d4556rwxh-hwloc-2.2.0-lib/lib/libevent_core-2.1.so.7
15220: trying file=/gnu/store/i1smdd4r44c5wwi03g40dvc6wfnqgdwq-libpciaccess-0.16/lib/libevent_core-2.1.so.7
15220: search path= (system search path)
15220: search path=/gnu/store/rykm237xkmq7rl1p0nwass01p090p88x-zlib-1.2.11/lib:/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/x86_64:/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell:/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/x86_64:/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls:/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/x86_64:/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell:/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../x86_64:/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../.. (RUNPATH from file /tmp/openmpi/gnu/store/rm1d9z3z3iwsv779vqlg7q5z7pdk5k12-openmpi-4.0.5/bin//orterun)
15220: trying file=/gnu/store/rykm237xkmq7rl1p0nwass01p090p88x-zlib-1.2.11/lib/libevent_core-2.1.so.7
15220: trying file=/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/x86_64/libevent_core-2.1.so.7
15220: trying file=/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/libevent_core-2.1.so.7
15220: trying file=/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/x86_64/libevent_core-2.1.so.7
15220: trying file=/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/libevent_core-2.1.so.7
15220: trying file=/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/x86_64/libevent_core-2.1.so.7
15220: trying file=/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/libevent_core-2.1.so.7
15220: trying file=/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../x86_64/libevent_core-2.1.so.7
15220: trying file=/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../libevent_core-2.1.so.7
15220: search path= (system search path)
IIUC, the only way a search path entry can “disappear” is if ld.so marks
it as “nonexisting”, as per dl-load.c:
Toggle snippet (31 lines)
if (__glibc_unlikely (GLRO(dl_debug_mask) & DL_DEBUG_LIBS))
_dl_debug_printf (" trying file=%s\n", buf);
fd = open_verify (buf, -1, fbp, loader, whatcode, mode,
found_other_class, false);
if (this_dir->status[cnt] == unknown)
{
if (fd != -1)
this_dir->status[cnt] = existing;
/* Do not update the directory information when loading
auditing code. We must try to disturb the program as
little as possible. */
else if (loader == NULL
|| GL(dl_ns)[loader->l_ns]._ns_loaded->l_auditing == 0)
{
/* We failed to open machine dependent library. Let's
test whether there is any directory at all. */
struct stat64 st;
buf[buflen - namelen - 1] = '\0';
if (__xstat64 (_STAT_VER, buf, &st) != 0
|| ! S_ISDIR (st.st_mode))
/* The directory does not exist or it is no directory. */
this_dir->status[cnt] = nonexisting;
else
this_dir->status[cnt] = existing;
}
}
Indeed, we can see ‘stat’ calls passed raw /gnu/store file names from
RUNPATH entries (instead of /tmp/fakechroot-test/gnu/store), suggesting
that ‘la_objsearch’ didn’t have a chance to rewrite them:
Toggle snippet (20 lines)
18952 openat(AT_FDCWD, "/tmp/fakechroot-test/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/rm1d9z3z3iwsv779vqlg7q5z7pdk5k12-openmpi-4.0.5/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/xk22b03jb0c0b6vkadqbkr9d4556rwxh-hwloc-2.2.0-lib/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/i1smdd4r44c5wwi03g40dvc6wfnqgdwq-libpciaccess-0.16/lib/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/tls/haswell/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 stat("/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/tls/haswell/x86_64", 0x7ffc99e48700) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/tls/haswell/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 stat("/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/tls/haswell", 0x7ffc99e48700) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/tls/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 stat("/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/tls/x86_64", 0x7ffc99e48700) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/tls/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 stat("/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/tls", 0x7ffc99e48700) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/haswell/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 stat("/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/haswell/x86_64", 0x7ffc99e48700) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/haswell/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 stat("/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/haswell", 0x7ffc99e48700) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 openat(AT_FDCWD, "/tmp/guix-exec-3Y7bac/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
18952 stat("/gnu/store/c7wscymmk379v16invi8m68f6v5c8gsv-libevent-2.1.11/lib/x86_64", 0x7ffc99e48700) = -1 ENOENT (Dosiero a? dosierujo ne ekzistas)
My guess is that after these ‘stat’ calls, the libevent directory is
marked as non-existing and is skipped from then on, hence the failure.
Does that make sense?
The patch below fixes the issue for this particular example by simply
passing the whole RUNPATH of the wrapped executable as ‘--library-path’,
as was the case in v1 of the patch set¹. (This assumes that the RUNPATH
of pack-audit.so is a subset of that of the program, which is usually
the case.)
Thanks,
Ludo’.
Toggle diff (14 lines)
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 379e6a3ac6..6355b99252 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -820,8 +820,7 @@ last resort for relocation."
(string-append "-DLOADER_AUDIT_RUNPATH={ "
(string-join
(map object->string
- (runpath
- #$(audit-module)))
+ (runpath program))
", " 'suffix)
"NULL }")
(if gconv