[PATCH 0/3] Using 'ld.so.cache' to speed up application startup

DoneSubmitted by Ludovic Courtès.
Details
One participant
  • Ludovic Courtès
Owner
unassigned
Severity
normal
L
L
Ludovic Courtès wrote on 27 Nov 2020 09:33
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201127083302.2578-1-ludo@gnu.org
Hello Guix!
The other day on IRC Ricardo had the brilliant idea of using theld.so cache to avoid the “stat storm” stemming from our long RUNPATHs,and thus to speed up application startup. As an example, Guile has9 entries in its RUNPATH and Inkscape has 44 entries.
The first patch changes the loader (1) to look for the cache in$ORIGIN/../etc/ld.so.cache, and (2) to look for the cache beforelooking at RUNPATH entries.
The second patch adds a build phase that creates ‘etc/ld.so.cache’.It passes ‘ldconfig’ a config file that contains the union of allthe RUNPATH entries of all the executables found in the output athand. (It cannot be done in a profile hook because $ORIGIN isdetermined by looking at /proc/self/exe, which is the canonical filename in the store.)
You can see it in action with LD_DEBUG=libs:
Toggle snippet (12 lines)$ LD_DEBUG=libs /gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/bin/guile --version 11150: find library=libguile-3.0.so.1 [0]; searching 11150: search cache=/gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/bin/../etc/ld.so.cache 11150: trying file=/gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/lib/libguile-3.0.so.1 11150: 11150: find library=libgc.so.1 [0]; searching 11150: search cache=/gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/bin/../etc/ld.so.cache 11150: trying file=/gnu/store/hy88vf2ynlica0wj0ppi0d3b11gi2b2h-libgc-8.0.4/lib/libgc.so.1
[...]
Here’s the after/before for Guile:
Toggle snippet (29 lines)$ strace -c /gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/bin/guile --version
[...]
2.70 0.000259 5 46 6 openat 1.87 0.000180 1 130 88 stat 1.66 0.000159 4 36 rt_sigprocmask 0.74 0.000071 1 40 close 0.64 0.000061 3 18 fstat
[...]
100.00 0.009604 600 105 total$ strace -c guile --version
[...]
13.82 0.000723 4 165 114 openat 13.46 0.000704 3 190 144 stat
[...]
1.43 0.000075 2 29 fstat
[...]
100.00 0.005232 773 268 total
Erroneous syscalls are divided by 2.5; total syscalls reduced by 22%.
For Bash:
Toggle snippet (59 lines)$ strace -c /gnu/store/qs33sf58502v1wx77va092y14sbspv4f-bash-minimal-5.0.16/bin/bash --versionGNU bash, version 5.0.16(1)-release (x86_64-unknown-linux-gnu)Copyright (C) 2019 Free Software Foundation, Inc.License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.There is NO WARRANTY, to the extent permitted by law.% time seconds usecs/call calls errors syscall------ ----------- ----------- --------- --------- ---------------- 0.00 0.000000 0 4 read 0.00 0.000000 0 6 write 0.00 0.000000 0 5 close 0.00 0.000000 0 5 fstat 0.00 0.000000 0 12 mmap 0.00 0.000000 0 5 mprotect 0.00 0.000000 0 1 munmap 0.00 0.000000 0 3 brk 0.00 0.000000 0 1 rt_sigprocmask 0.00 0.000000 0 1 1 access 0.00 0.000000 0 1 execve 0.00 0.000000 0 1 readlink 0.00 0.000000 0 1 getuid 0.00 0.000000 0 1 getgid 0.00 0.000000 0 1 geteuid 0.00 0.000000 0 1 getegid 0.00 0.000000 0 1 arch_prctl 0.00 0.000000 0 9 4 openat------ ----------- ----------- --------- --------- ----------------100.00 0.000000 59 5 total$ strace -c /gnu/store/fvhj74pghapbjvsvj27skvkra1by1965-bash-minimal-5.0.16/bin/bash --versionGNU bash, version 5.0.16(1)-release (x86_64-unknown-linux-gnu)Copyright (C) 2019 Free Software Foundation, Inc.License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.There is NO WARRANTY, to the extent permitted by law.% time seconds usecs/call calls errors syscall------ ----------- ----------- --------- --------- ---------------- 34.15 0.000420 12 35 16 openat 18.29 0.000225 8 27 mmap 8.13 0.000100 25 4 brk 7.72 0.000095 6 14 14 stat 7.24 0.000089 89 1 1 access 6.18 0.000076 4 19 fstat 5.45 0.000067 3 19 close 4.47 0.000055 2 20 read 3.82 0.000047 9 5 mprotect 2.36 0.000029 4 6 write 0.65 0.000008 8 1 execve 0.41 0.000005 5 1 arch_prctl 0.33 0.000004 4 1 getuid 0.24 0.000003 3 1 rt_sigprocmask 0.24 0.000003 3 1 getegid 0.16 0.000002 2 1 getgid 0.16 0.000002 2 1 geteuid------ ----------- ----------- --------- --------- ----------------100.00 0.001230 157 31 total
Erroneous syscalls are divided by 6; total syscalls divided by 2.7.
As always, this is probably not that big a deal on warm-cache SSD,but it probably makes a difference on a cold cache, on spinningdisks, and on network file systems.
* Possible improvements
The hard-coded ‘../etc/ld.so.cache’ means that it can only be usedwith first-level sub-directories like bin/ and sbin/; it won’t beused for libexec/guix/guile, for instance, which is a bummer.Perhaps we should compute the ‘ld.so.cache’ file name “lexically”instead.
We should also think hard about ways users could be tricked intoloading a malicious ‘ld.so.cache’. That’s also another reason why“lexical dot-dot” would be safer: we could ensure that onlypre-computed ‘ld.so.cache’ that live in the store are ever loaded.
The ‘ld.so.conf’ file passed to ‘ldconfig’ should ideally containsthe RUNPATH entries _recursively_, such that even indirectdependencies can be found in cache.
Thoughts?
Ludo’.
Ludovic Courtès (3): gnu: glibc: Load ${ORIGIN}/../etc/ld.so.cache when available. gremlin: Fix typo in docstring. build-system/gnu: Add 'make-dynamic-linker-cache' phase.
gnu/local.mk | 1 + gnu/packages/base.scm | 11 +- gnu/packages/patches/glibc-dl-cache.patch | 122 ++++++++++++++++++++++ guix/build-system/gnu.scm | 4 + guix/build/gnu-build-system.scm | 73 +++++++++++++ guix/build/gremlin.scm | 2 +- 6 files changed, 202 insertions(+), 11 deletions(-) create mode 100644 gnu/packages/patches/glibc-dl-cache.patch
-- 2.29.2
L
L
Ludovic Courtès wrote on 27 Nov 2020 10:05
[PATCH 1/3] gnu: glibc: Load ${ORIGIN}/../etc/ld.so.cache when available.
(address . 44899@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201127090523.3853-1-ludo@gnu.org
* gnu/packages/patches/glibc-dl-cache.patch: New file.* gnu/local.mk (dist_patch_DATA): Add it.* gnu/packages/base.scm (glibc)[source]: Remove 'snippet' and 'modules'.--- gnu/local.mk | 1 + gnu/packages/base.scm | 11 +- gnu/packages/patches/glibc-dl-cache.patch | 122 ++++++++++++++++++++++ 3 files changed, 124 insertions(+), 10 deletions(-) create mode 100644 gnu/packages/patches/glibc-dl-cache.patch
Toggle diff (165 lines)diff --git a/gnu/local.mk b/gnu/local.mkindex f9fed30a3f..82c3c608c6 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -1051,6 +1051,7 @@ dist_patch_DATA = \ %D%/packages/patches/glibc-bootstrap-system-2.2.5.patch \ %D%/packages/patches/glibc-bootstrap-system-2.16.0.patch \ %D%/packages/patches/glibc-bootstrap-system.patch \+ %D%/packages/patches/glibc-dl-cache.patch \ %D%/packages/patches/glibc-hidden-visibility-ldconfig.patch \ %D%/packages/patches/glibc-hurd-clock_gettime_monotonic.patch \ %D%/packages/patches/glibc-hurd-clock_t_centiseconds.patch \diff --git a/gnu/packages/base.scm b/gnu/packages/base.scmindex bd352319a1..f8f4ae37fd 100644--- a/gnu/packages/base.scm+++ b/gnu/packages/base.scm@@ -675,17 +675,8 @@ the store.") (sha256 (base32 "0di848ibffrnwq7g2dvgqrnn4xqhj3h96csn69q4da51ymafl9qn"))- (snippet- ;; Disable 'ldconfig' and /etc/ld.so.cache. The latter is- ;; required on LFS distros to avoid loading the distro's libc.so- ;; instead of ours.- '(begin- (substitute* "sysdeps/unix/sysv/linux/configure"- (("use_ldconfig=yes")- "use_ldconfig=no"))- #t))- (modules '((guix build utils))) (patches (search-patches "glibc-ldd-x86_64.patch"+ "glibc-dl-cache.patch" "glibc-hidden-visibility-ldconfig.patch" "glibc-versioned-locpath.patch" "glibc-allow-kernel-2.6.32.patch"diff --git a/gnu/packages/patches/glibc-dl-cache.patch b/gnu/packages/patches/glibc-dl-cache.patchnew file mode 100644index 0000000000..9d578bb3d9--- /dev/null+++ b/gnu/packages/patches/glibc-dl-cache.patch@@ -0,0 +1,122 @@+Read the shared library cache relative to $ORIGIN instead of reading+from /etc/ld.so.cache. Also arrange so that this cache takes+precedence over RUNPATH.++diff --git a/elf/dl-cache.c b/elf/dl-cache.c+index 93d185e788..6a2989bd4c 100644+--- a/elf/dl-cache.c++++ b/elf/dl-cache.c+@@ -171,6 +171,33 @@ _dl_cache_libcmp (const char *p1, const char *p2)+ return *p1 - *p2;+ }+ ++/* Special value representing the lack of an ld.so cache. */++static const char ld_so_cache_lacking[] = "/ld.so cache is lacking";++++/* Return the per-application ld.so cache, relative to $ORIGIN, or NULL if++ that fails for some reason. Do not return the system-wide LD_SO_CACHE++ since on a foreign distro it would contain invalid information. */++static const char *++ld_so_cache (void)++{++ static const char *loader_cache;++++ if (loader_cache == NULL)++ {++ const char suffix[] = "/../etc/ld.so.cache";++ const char *origin = _dl_get_origin ();++++ /* Note: We can't use 'malloc' because it can be interposed. */++ char *cache = alloca (strlen (origin) + sizeof suffix);++++ strcpy (cache, origin);++ strcat (cache, suffix);++++ loader_cache = __strdup (cache) ?: ld_so_cache_lacking;++ }++++ return loader_cache;++}+ + /* Look up NAME in ld.so.cache and return the file name stored there, or null+ if none is found. The cache is loaded if it was not already. If loading+@@ -190,12 +217,15 @@ _dl_load_cache_lookup (const char *name)+ + /* Print a message if the loading of libs is traced. */+ if (__glibc_unlikely (GLRO(dl_debug_mask) & DL_DEBUG_LIBS))+- _dl_debug_printf (" search cache=%s\n", LD_SO_CACHE);++ _dl_debug_printf (" search cache=%s\n", ld_so_cache ());++++ if (__glibc_unlikely (ld_so_cache () == ld_so_cache_lacking))++ return NULL;+ + if (cache == NULL)+ {+ /* Read the contents of the file. */+- void *file = _dl_sysdep_read_whole_file (LD_SO_CACHE, &cachesize,++ void *file = _dl_sysdep_read_whole_file (ld_so_cache (), &cachesize,+ PROT_READ);+ + /* We can handle three different cache file formats here:+diff --git a/elf/dl-load.c b/elf/dl-load.c+index f3201e7c14..a69aec3428 100644+--- a/elf/dl-load.c++++ b/elf/dl-load.c+@@ -2152,28 +2152,6 @@ _dl_map_object (struct link_map *loader, const char *name,+ loader ?: GL(dl_ns)[LM_ID_BASE]._ns_loaded,+ LA_SER_LIBPATH, &found_other_class);+ +- /* Look at the RUNPATH information for this binary. */+- if (fd == -1 && loader != NULL+- && cache_rpath (loader, &loader->l_runpath_dirs,+- DT_RUNPATH, "RUNPATH"))+- fd = open_path (name, namelen, mode,+- &loader->l_runpath_dirs, &realname, &fb, loader,+- LA_SER_RUNPATH, &found_other_class);+-+- if (fd == -1)+- {+- realname = _dl_sysdep_open_object (name, namelen, &fd);+- if (realname != NULL)+- {+- fd = open_verify (realname, fd,+- &fb, loader ?: GL(dl_ns)[nsid]._ns_loaded,+- LA_SER_CONFIG, mode, &found_other_class,+- false);+- if (fd == -1)+- free (realname);+- }+- }+-+ #ifdef USE_LDCONFIG+ if (fd == -1+ && (__glibc_likely ((mode & __RTLD_SECURE) == 0)+@@ -2232,6 +2210,28 @@ _dl_map_object (struct link_map *loader, const char *name,+ }+ #endif+ ++ /* Look at the RUNPATH information for this binary. */++ if (fd == -1 && loader != NULL++ && cache_rpath (loader, &loader->l_runpath_dirs,++ DT_RUNPATH, "RUNPATH"))++ fd = open_path (name, namelen, mode,++ &loader->l_runpath_dirs, &realname, &fb, loader,++ LA_SER_RUNPATH, &found_other_class);++++ if (fd == -1)++ {++ realname = _dl_sysdep_open_object (name, namelen, &fd);++ if (realname != NULL)++ {++ fd = open_verify (realname, fd,++ &fb, loader ?: GL(dl_ns)[nsid]._ns_loaded,++ LA_SER_CONFIG, mode, &found_other_class,++ false);++ if (fd == -1)++ free (realname);++ }++ }+++ /* Finally, try the default path. */+ if (fd == -1+ && ((l = loader ?: GL(dl_ns)[nsid]._ns_loaded) == NULL-- 2.29.2
L
L
Ludovic Courtès wrote on 27 Nov 2020 10:05
[PATCH 2/3] gremlin: Fix typo in docstring.
(address . 44899@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201127090523.3853-2-ludo@gnu.org
* guix/build/gremlin.scm (file-runpath): Fix typo.--- guix/build/gremlin.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Toggle diff (15 lines)diff --git a/guix/build/gremlin.scm b/guix/build/gremlin.scmindex 6857e47b99..44604827a9 100644--- a/guix/build/gremlin.scm+++ b/guix/build/gremlin.scm@@ -250,7 +250,7 @@ info." (elf-dynamic-info (parse-elf (get-bytevector-all port)))))) (define (file-runpath file)- "Return the DT_RUNPATH dynamic entry of FILE as a list of string, or #f if+ "Return the DT_RUNPATH dynamic entry of FILE as a list of strings, or #f if FILE lacks dynamic info." (and=> (file-dynamic-info file) elf-dynamic-info-runpath)) -- 2.29.2
L
L
Ludovic Courtès wrote on 27 Nov 2020 10:05
[PATCH 3/3] build-system/gnu: Add 'make-dynamic-linker-cache' phase.
(address . 44899@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201127090523.3853-3-ludo@gnu.org
* guix/build/gnu-build-system.scm (make-dynamic-linker-cache): Newprocedure.(%standard-phases): Add it.* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): Add #:make-dynamic-linker-cache? and honor it.--- guix/build-system/gnu.scm | 4 ++ guix/build/gnu-build-system.scm | 73 +++++++++++++++++++++++++++++++++ 2 files changed, 77 insertions(+)
Toggle diff (129 lines)diff --git a/guix/build-system/gnu.scm b/guix/build-system/gnu.scmindex 2c23197e77..d6c4dc9bbc 100644--- a/guix/build-system/gnu.scm+++ b/guix/build-system/gnu.scm@@ -342,6 +342,7 @@ standard packages used as implicit inputs of the GNU build system." (strip-directories ''("lib" "lib64" "libexec" "bin" "sbin")) (validate-runpath? #t)+ (make-dynamic-linker-cache? #t) (license-file-regexp %license-file-regexp) (phases '%standard-phases) (locale "en_US.utf8")@@ -410,6 +411,7 @@ packages that must not be referenced." #:patch-shebangs? ,patch-shebangs? #:strip-binaries? ,strip-binaries? #:validate-runpath? ,validate-runpath?+ #:make-dynamic-linker-cache? ,make-dynamic-linker-cache? #:license-file-regexp ,license-file-regexp #:strip-flags ,strip-flags #:strip-directories ,strip-directories)))@@ -497,6 +499,7 @@ is one of `host' or `target'." (strip-directories ''("lib" "lib64" "libexec" "bin" "sbin")) (validate-runpath? #t)+ (make-dynamic-linker-cache? #t) (license-file-regexp %license-file-regexp) (phases '%standard-phases) (locale "en_US.utf8")@@ -577,6 +580,7 @@ platform." #:patch-shebangs? ,patch-shebangs? #:strip-binaries? ,strip-binaries? #:validate-runpath? ,validate-runpath?+ #:make-dynamic-linker-cache? ,make-dynamic-linker-cache? #:license-file-regexp ,license-file-regexp #:strip-flags ,strip-flags #:strip-directories ,strip-directories))))diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scmindex 8fa11f4ea9..194cddc047 100644--- a/guix/build/gnu-build-system.scm+++ b/guix/build/gnu-build-system.scm@@ -712,6 +712,78 @@ which cannot be found~%" (which binary) rest))))))))) outputs)) +(define* (make-dynamic-linker-cache #:key outputs+ (make-dynamic-linker-cache? #t)+ #:allow-other-keys)+ "Create a dynamic linker cache under 'etc/ld.so.cache' in each of the+OUTPUTS. This reduces application startup time by avoiding the 'stat' storm+that traversing all the RUNPATH entries entails."+ (define (make-cache-for-output directory)+ (define bin-directories+ (filter-map (lambda (sub-directory)+ (let ((directory (string-append directory "/"+ sub-directory)))+ (and (directory-exists? directory)+ directory)))+ '("bin" "sbin")))++ (define programs+ ;; Programs that can benefit from the ld.so cache. These programs must+ ;; be in a directory such that:+ ;;+ ;; (string-append (dirname PROGRAM) "../etc/ld.so.cache")+ ;;+ ;; potentially exists since that's what ld.so will look for. Thus,+ ;; something like 'libexec/foo/PROGRAM' is not a valid candidate.+ (append-map (lambda (directory)+ (if (directory-exists? directory)+ (filter-map (lambda (file)+ (let ((file (string-append+ directory "/" file)))+ (and (executable-file? file)+ (not (file-is-directory? file))+ (elf-file? file)+ file)))+ (scandir directory))+ '()))+ bin-directories))++ (define runpaths+ ;; The union of RUNPATH entries.+ (delete-duplicates+ (append-map (lambda (program)+ (or (file-runpath program) '()))+ programs)))++ (define cache-file+ (string-append directory "/etc/ld.so.cache"))++ (unless (null? runpaths)+ (mkdir-p (dirname cache-file))+ (guard (c ((invoke-error? c)+ ;; Do not treat 'ldconfig' failure as an error.+ (format (current-error-port)+ "warning: 'ldconfig' failed:~%")+ (report-invoke-error c (current-error-port))))+ ;; Create a config file to tell 'ldconfig' where to look for the+ ;; libraries that PROGRAMS need.+ (call-with-output-file "/tmp/ld.so.conf"+ (lambda (port)+ (for-each (lambda (directory)+ (display directory port)+ (newline port))+ runpaths)))++ (invoke "ldconfig" "-f" "/tmp/ld.so.conf" "-C" cache-file)+ (format #t "created '~a' from ~a library search path entries~%"+ cache-file (length runpaths)))))++ (if make-dynamic-linker-cache?+ (match outputs+ (((_ . directories) ...)+ (for-each make-cache-for-output directories)))+ (format #t "ld.so cache not built~%")))+ (define %license-file-regexp ;; Regexp matching license files. "^(COPYING.*|LICEN[CS]E.*|[Ll]icen[cs]e.*|Copy[Rr]ight(\\.(txt|md))?)$")@@ -791,6 +863,7 @@ which cannot be found~%" validate-documentation-location delete-info-dir-file patch-dot-desktop-files+ make-dynamic-linker-cache install-license-files reset-gzip-timestamps compress-documentation)))-- 2.29.2
L
L
Ludovic Courtès wrote on 28 Nov 2020 11:24
[PATCH v2 1/4] gremlin: Fix typo in docstring.
(address . 44899@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201128102409.3157-2-ludo@gnu.org
* guix/build/gremlin.scm (file-runpath): Fix typo.--- guix/build/gremlin.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Toggle diff (15 lines)diff --git a/guix/build/gremlin.scm b/guix/build/gremlin.scmindex 6857e47b99..44604827a9 100644--- a/guix/build/gremlin.scm+++ b/guix/build/gremlin.scm@@ -250,7 +250,7 @@ info." (elf-dynamic-info (parse-elf (get-bytevector-all port)))))) (define (file-runpath file)- "Return the DT_RUNPATH dynamic entry of FILE as a list of string, or #f if+ "Return the DT_RUNPATH dynamic entry of FILE as a list of strings, or #f if FILE lacks dynamic info." (and=> (file-dynamic-info file) elf-dynamic-info-runpath)) -- 2.29.2
L
L
Ludovic Courtès wrote on 28 Nov 2020 11:24
[PATCH v2 2/4] gremlin: Add 'file-needed/recursive'.
(address . 44899@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201128102409.3157-3-ludo@gnu.org
* guix/build/gremlin.scm (file-needed/recursive): New procedure.* tests/gremlin.scm ("file-needed/recursive"): New test.--- guix/build/gremlin.scm | 41 +++++++++++++++++++++++++++++++++++++++++ tests/gremlin.scm | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 77 insertions(+)
Toggle diff (115 lines)diff --git a/guix/build/gremlin.scm b/guix/build/gremlin.scmindex 44604827a9..d56984b85e 100644--- a/guix/build/gremlin.scm+++ b/guix/build/gremlin.scm@@ -44,6 +44,7 @@ file-dynamic-info file-runpath file-needed+ file-needed/recursive missing-runpath-error? missing-runpath-error-file@@ -259,6 +260,46 @@ FILE lacks dynamic info." dynamic info." (and=> (file-dynamic-info file) elf-dynamic-info-needed)) +(define (file-needed/recursive file)+ "Return two values: the list of absolute .so file names FILE depends on,+recursively, and the list of .so file names that could not be found. File+names are resolved by searching the RUNPATH of the file that NEEDs them.++This is similar to the info returned by the 'ldd' command."+ (let loop ((files (list file))+ (result '())+ (not-found '()))+ (match files+ (()+ (values (reverse result)+ (reverse (delete-duplicates not-found))))+ ((file . rest)+ (match (file-dynamic-info (pk 'file file))+ (#f+ (loop rest result not-found))+ (info+ (let ((runpath (elf-dynamic-info-runpath info))+ (needed (elf-dynamic-info-needed info)))+ (if (and runpath needed)+ (let* ((runpath (map (cute expand-origin <> (dirname file))+ runpath))+ (resolved (map (cut search-path runpath <>)+ needed))+ (failed (filter-map (lambda (needed resolved)+ (and (not resolved)+ (not (libc-library? needed))+ needed))+ needed resolved))+ (needed (remove (lambda (value)+ (or (not value)+ ;; XXX: quadratic+ (member value result)))+ resolved)))+ (loop (append rest needed)+ (append needed result)+ (append failed not-found)))+ (loop rest result not-found)))))))))+ (define %libc-libraries ;; List of libraries as of glibc 2.21 (there are more but those are ;; typically mean to be LD_PRELOADed and thus do not appear as NEEDED.)diff --git a/tests/gremlin.scm b/tests/gremlin.scmindex f191adb8b3..9ddac14265 100644--- a/tests/gremlin.scm+++ b/tests/gremlin.scm@@ -27,6 +27,8 @@ #:use-module (srfi srfi-64) #:use-module (rnrs io ports) #:use-module (ice-9 popen)+ #:use-module (ice-9 rdelim)+ #:use-module (ice-9 regex) #:use-module (ice-9 match)) (define %guile-executable@@ -58,6 +60,40 @@ (string-take lib (string-contains lib ".so"))) (elf-dynamic-info-needed dyninfo)))))) +(unless (and %guile-executable (not (getenv "LD_LIBRARY_PATH"))+ (file-needed %guile-executable)) ;statically linked?+ (test-skip 1))+(test-assert "file-needed/recursive"+ (let* ((needed (file-needed/recursive %guile-executable))+ (pipe (dynamic-wind+ (lambda ()+ ;; Tell ld.so to list loaded objects, like 'ldd' does.+ (setenv "LD_TRACE_LOADED_OBJECTS" "yup"))+ (lambda ()+ (open-pipe* OPEN_READ %guile-executable))+ (lambda ()+ (unsetenv "LD_TRACE_LOADED_OBJECTS")))))+ (define ldd-rx+ (make-regexp "^[[:blank:]]+([[:graph:]]+ => )?([[:graph:]]+) .*$"))++ (define (read-ldd-output port)+ ;; Read from PORT output in GNU ldd format.+ (let loop ((result '()))+ (match (read-line port)+ ((? eof-object?)+ (reverse result))+ ((= (cut regexp-exec ldd-rx <>) m)+ (if m+ (loop (cons (match:substring m 2) result))+ (loop result))))))++ (define ground-truth+ (remove (cut string-prefix? "linux-vdso.so" <>)+ (read-ldd-output pipe)))++ (and (zero? (close-pipe pipe))+ (lset= string=? (pk 'truth ground-truth) (pk 'needed needed)))))+ (test-equal "expand-origin" '("OOO/../lib" "OOO"-- 2.29.2
L
L
Ludovic Courtès wrote on 28 Nov 2020 11:24
[PATCH v2 4/4] build-system/gnu: Add 'make-dynamic-linker-cache' phase.
(address . 44899@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201128102409.3157-6-ludo@gnu.org
* guix/build/gnu-build-system.scm (make-dynamic-linker-cache): Newprocedure.(%standard-phases): Add it.* guix/build-system/gnu.scm (gnu-build, gnu-cross-build): Add #:make-dynamic-linker-cache? and honor it.--- guix/build-system/gnu.scm | 4 ++ guix/build/gnu-build-system.scm | 68 +++++++++++++++++++++++++++++++++ 2 files changed, 72 insertions(+)
Toggle diff (124 lines)diff --git a/guix/build-system/gnu.scm b/guix/build-system/gnu.scmindex 2c23197e77..d6c4dc9bbc 100644--- a/guix/build-system/gnu.scm+++ b/guix/build-system/gnu.scm@@ -342,6 +342,7 @@ standard packages used as implicit inputs of the GNU build system." (strip-directories ''("lib" "lib64" "libexec" "bin" "sbin")) (validate-runpath? #t)+ (make-dynamic-linker-cache? #t) (license-file-regexp %license-file-regexp) (phases '%standard-phases) (locale "en_US.utf8")@@ -410,6 +411,7 @@ packages that must not be referenced." #:patch-shebangs? ,patch-shebangs? #:strip-binaries? ,strip-binaries? #:validate-runpath? ,validate-runpath?+ #:make-dynamic-linker-cache? ,make-dynamic-linker-cache? #:license-file-regexp ,license-file-regexp #:strip-flags ,strip-flags #:strip-directories ,strip-directories)))@@ -497,6 +499,7 @@ is one of `host' or `target'." (strip-directories ''("lib" "lib64" "libexec" "bin" "sbin")) (validate-runpath? #t)+ (make-dynamic-linker-cache? #t) (license-file-regexp %license-file-regexp) (phases '%standard-phases) (locale "en_US.utf8")@@ -577,6 +580,7 @@ platform." #:patch-shebangs? ,patch-shebangs? #:strip-binaries? ,strip-binaries? #:validate-runpath? ,validate-runpath?+ #:make-dynamic-linker-cache? ,make-dynamic-linker-cache? #:license-file-regexp ,license-file-regexp #:strip-flags ,strip-flags #:strip-directories ,strip-directories))))diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scmindex 8fa11f4ea9..5f08b9d6ac 100644--- a/guix/build/gnu-build-system.scm+++ b/guix/build/gnu-build-system.scm@@ -712,6 +712,73 @@ which cannot be found~%" (which binary) rest))))))))) outputs)) +(define* (make-dynamic-linker-cache #:key outputs+ (make-dynamic-linker-cache? #t)+ #:allow-other-keys)+ "Create a dynamic linker cache under 'etc/ld.so.cache' in each of the+OUTPUTS. This reduces application startup time by avoiding the 'stat' storm+that traversing all the RUNPATH entries entails."+ (define (make-cache-for-output directory)+ (define bin-directories+ (filter-map (lambda (sub-directory)+ (let ((directory (string-append directory "/"+ sub-directory)))+ (and (directory-exists? directory)+ directory)))+ '("bin" "sbin" "libexec")))++ (define programs+ ;; Programs that can benefit from the ld.so cache.+ (append-map (lambda (directory)+ (if (directory-exists? directory)+ (find-files directory+ (lambda (file stat)+ (and (executable-file? file)+ (elf-file? file))))+ '()))+ bin-directories))++ (define library-path+ ;; Directories containing libraries that PROGRAMS depend on,+ ;; recursively.+ (delete-duplicates+ (append-map (lambda (program)+ (map dirname (file-needed/recursive program)))+ programs)))++ (define cache-file+ (string-append directory "/etc/ld.so.cache"))++ (define ld.so.conf+ (string-append (or (getenv "TMPDIR") "/tmp")+ "/ld.so.conf"))++ (unless (null? library-path)+ (mkdir-p (dirname cache-file))+ (guard (c ((invoke-error? c)+ ;; Do not treat 'ldconfig' failure as an error.+ (format (current-error-port)+ "warning: 'ldconfig' failed:~%")+ (report-invoke-error c (current-error-port))))+ ;; Create a config file to tell 'ldconfig' where to look for the+ ;; libraries that PROGRAMS need.+ (call-with-output-file ld.so.conf+ (lambda (port)+ (for-each (lambda (directory)+ (display directory port)+ (newline port))+ library-path)))++ (invoke "ldconfig" "-f" ld.so.conf "-C" cache-file)+ (format #t "created '~a' from ~a library search path entries~%"+ cache-file (length library-path)))))++ (if make-dynamic-linker-cache?+ (match outputs+ (((_ . directories) ...)+ (for-each make-cache-for-output directories)))+ (format #t "ld.so cache not built~%")))+ (define %license-file-regexp ;; Regexp matching license files. "^(COPYING.*|LICEN[CS]E.*|[Ll]icen[cs]e.*|Copy[Rr]ight(\\.(txt|md))?)$")@@ -791,6 +858,7 @@ which cannot be found~%" validate-documentation-location delete-info-dir-file patch-dot-desktop-files+ make-dynamic-linker-cache install-license-files reset-gzip-timestamps compress-documentation)))-- 2.29.2
L
L
Ludovic Courtès wrote on 28 Nov 2020 11:24
[PATCH v2 0/4] Using 'ld.so.cache' to speed up application startup
(address . 44899@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201128102409.3157-1-ludo@gnu.org
Hi!
This new version addresses the shortcomings I mentioned earlier andother issues reported on IRC:
• ld.so no longer uses “../etc/ld.so.cache”. Instead, it (1) ensures $ORIGIN is in the store, (2) extracts the store file name, and (3) appends “/etc/ld.so.cache”. IOW, the ‘ld.so.cache’ is always resolved relative to the store directory $ORIGIN belongs to, not relative to $ORIGIN itself.
Thinking about it, it’s a direct translation of the ld.so.cache model from FHS to the functional model where a system-wide /etc/ld.so.cache makes no sense.
• ‘make-dynamic-linker-cache’ now creates an ‘ld.so.conf’ that contains all the dependencies, recursively, as would be returned by ‘ldd’. The new ‘file-needed/recursive’ procedure returns the list of shared objects depended on like ‘ldd’.
• ‘make-dynamic-linker-cache’ creates ‘ld.so.conf’ in $TMPDIR rather than hard-code /tmp, so as to be friendlier to ‘--disable-chroot’ builds (as is currently used on GNU/Hurd).
I’m rather happy and confident with that version. :-)
Feedback welcome!
Ludo’.
Ludovic Courtès (4): gremlin: Fix typo in docstring. gremlin: Add 'file-needed/recursive'. gnu: glibc: Load 'etc/ld.so.cache' in $ORIGIN's store item when available. build-system/gnu: Add 'make-dynamic-linker-cache' phase.
gnu/local.mk | 1 + gnu/packages/base.scm | 16 +-- gnu/packages/patches/glibc-dl-cache.patch | 140 ++++++++++++++++++++++ guix/build-system/gnu.scm | 4 + guix/build/gnu-build-system.scm | 68 +++++++++++ guix/build/gremlin.scm | 43 ++++++- tests/gremlin.scm | 36 ++++++ 7 files changed, 297 insertions(+), 11 deletions(-) create mode 100644 gnu/packages/patches/glibc-dl-cache.patch
-- 2.29.2
L
L
Ludovic Courtès wrote on 28 Nov 2020 11:24
[PATCH v2 3/4] gnu: glibc: Load 'etc/ld.so.cache' in $ORIGIN's store item when available.
(address . 44899@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201128102409.3157-4-ludo@gnu.org
* gnu/packages/patches/glibc-dl-cache.patch: New file.* gnu/local.mk (dist_patch_DATA): Add it.* gnu/packages/base.scm (glibc)[source]: Remove 'snippet' and 'modules'.[arguments]: In 'pre-configure' phase, substitute @STORE_DIRECTORY@ in'elf/dl-cache.c'.--- gnu/local.mk | 1 + gnu/packages/base.scm | 16 +-- gnu/packages/patches/glibc-dl-cache.patch | 140 ++++++++++++++++++++++ 3 files changed, 147 insertions(+), 10 deletions(-) create mode 100644 gnu/packages/patches/glibc-dl-cache.patch
Toggle diff (195 lines)diff --git a/gnu/local.mk b/gnu/local.mkindex f9fed30a3f..82c3c608c6 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -1051,6 +1051,7 @@ dist_patch_DATA = \ %D%/packages/patches/glibc-bootstrap-system-2.2.5.patch \ %D%/packages/patches/glibc-bootstrap-system-2.16.0.patch \ %D%/packages/patches/glibc-bootstrap-system.patch \+ %D%/packages/patches/glibc-dl-cache.patch \ %D%/packages/patches/glibc-hidden-visibility-ldconfig.patch \ %D%/packages/patches/glibc-hurd-clock_gettime_monotonic.patch \ %D%/packages/patches/glibc-hurd-clock_t_centiseconds.patch \diff --git a/gnu/packages/base.scm b/gnu/packages/base.scmindex bd352319a1..ad4415f226 100644--- a/gnu/packages/base.scm+++ b/gnu/packages/base.scm@@ -675,17 +675,8 @@ the store.") (sha256 (base32 "0di848ibffrnwq7g2dvgqrnn4xqhj3h96csn69q4da51ymafl9qn"))- (snippet- ;; Disable 'ldconfig' and /etc/ld.so.cache. The latter is- ;; required on LFS distros to avoid loading the distro's libc.so- ;; instead of ours.- '(begin- (substitute* "sysdeps/unix/sysv/linux/configure"- (("use_ldconfig=yes")- "use_ldconfig=no"))- #t))- (modules '((guix build utils))) (patches (search-patches "glibc-ldd-x86_64.patch"+ "glibc-dl-cache.patch" "glibc-hidden-visibility-ldconfig.patch" "glibc-versioned-locpath.patch" "glibc-allow-kernel-2.6.32.patch"@@ -800,6 +791,11 @@ the store.") ;; 4.7.1. ((" -lgcc_s") "")) + ;; Tell the ld.so cache code where the store is.+ (substitute* "elf/dl-cache.c"+ (("@STORE_DIRECTORY@")+ (string-append "\"" (%store-directory) "\"")))+ ;; Have `system' use that Bash. (substitute* "sysdeps/posix/system.c" (("#define[[:blank:]]+SHELL_PATH.*$")diff --git a/gnu/packages/patches/glibc-dl-cache.patch b/gnu/packages/patches/glibc-dl-cache.patchnew file mode 100644index 0000000000..0f23b12add--- /dev/null+++ b/gnu/packages/patches/glibc-dl-cache.patch@@ -0,0 +1,140 @@+Read the shared library cache relative to $ORIGIN instead of reading+from /etc/ld.so.cache. Also arrange so that this cache takes+precedence over RUNPATH.++diff --git a/elf/dl-cache.c b/elf/dl-cache.c+index 93d185e788..e0760a1f40 100644+--- a/elf/dl-cache.c++++ b/elf/dl-cache.c+@@ -171,6 +171,51 @@ _dl_cache_libcmp (const char *p1, const char *p2)+ return *p1 - *p2;+ }+ ++/* Special value representing the lack of an ld.so cache. */++static const char ld_so_cache_lacking[] = "/ld.so cache is lacking";++++/* Return the per-application ld.so cache, relative to $ORIGIN, or NULL if++ that fails for some reason. Do not return the system-wide LD_SO_CACHE++ since on a foreign distro it would contain invalid information. */++static const char *++ld_so_cache (void)++{++ static const char *loader_cache;++++ if (loader_cache == NULL)++ {++ static const char store[] = "/gnu/store";++ const char *origin = _dl_get_origin ();++++ /* Check whether ORIGIN is something like "/gnu/store/…-foo/bin". */++ if (strncmp (store, origin, strlen (store)) == 0++ && origin[sizeof store - 1] == '/')++ {++ char *store_item_end = strchr (origin + sizeof store, '/');++++ if (store_item_end != NULL)++ {++ static const char suffix[] = "/etc/ld.so.cache";++ size_t store_item_len = store_item_end - origin;++++ /* Note: We can't use 'malloc' because it can be interposed.++ Likewise, 'strncpy' is not available. */++ char *cache = alloca (strlen (origin) + sizeof suffix);++++ strcpy (cache, origin);++ strcpy (cache + store_item_len, suffix);++++ loader_cache = __strdup (cache) ?: ld_so_cache_lacking;++ }++ else++ loader_cache = ld_so_cache_lacking;++ }++ else++ loader_cache = ld_so_cache_lacking;++ }++++ return loader_cache;++}+ + /* Look up NAME in ld.so.cache and return the file name stored there, or null+ if none is found. The cache is loaded if it was not already. If loading+@@ -190,12 +235,15 @@ _dl_load_cache_lookup (const char *name)+ + /* Print a message if the loading of libs is traced. */+ if (__glibc_unlikely (GLRO(dl_debug_mask) & DL_DEBUG_LIBS))+- _dl_debug_printf (" search cache=%s\n", LD_SO_CACHE);++ _dl_debug_printf (" search cache=%s\n", ld_so_cache ());++++ if (__glibc_unlikely (ld_so_cache () == ld_so_cache_lacking))++ return NULL;+ + if (cache == NULL)+ {+ /* Read the contents of the file. */+- void *file = _dl_sysdep_read_whole_file (LD_SO_CACHE, &cachesize,++ void *file = _dl_sysdep_read_whole_file (ld_so_cache (), &cachesize,+ PROT_READ);+ + /* We can handle three different cache file formats here:+diff --git a/elf/dl-load.c b/elf/dl-load.c+index f3201e7c14..a69aec3428 100644+--- a/elf/dl-load.c++++ b/elf/dl-load.c+@@ -2152,28 +2152,6 @@ _dl_map_object (struct link_map *loader, const char *name,+ loader ?: GL(dl_ns)[LM_ID_BASE]._ns_loaded,+ LA_SER_LIBPATH, &found_other_class);+ +- /* Look at the RUNPATH information for this binary. */+- if (fd == -1 && loader != NULL+- && cache_rpath (loader, &loader->l_runpath_dirs,+- DT_RUNPATH, "RUNPATH"))+- fd = open_path (name, namelen, mode,+- &loader->l_runpath_dirs, &realname, &fb, loader,+- LA_SER_RUNPATH, &found_other_class);+-+- if (fd == -1)+- {+- realname = _dl_sysdep_open_object (name, namelen, &fd);+- if (realname != NULL)+- {+- fd = open_verify (realname, fd,+- &fb, loader ?: GL(dl_ns)[nsid]._ns_loaded,+- LA_SER_CONFIG, mode, &found_other_class,+- false);+- if (fd == -1)+- free (realname);+- }+- }+-+ #ifdef USE_LDCONFIG+ if (fd == -1+ && (__glibc_likely ((mode & __RTLD_SECURE) == 0)+@@ -2232,6 +2210,28 @@ _dl_map_object (struct link_map *loader, const char *name,+ }+ #endif+ ++ /* Look at the RUNPATH information for this binary. */++ if (fd == -1 && loader != NULL++ && cache_rpath (loader, &loader->l_runpath_dirs,++ DT_RUNPATH, "RUNPATH"))++ fd = open_path (name, namelen, mode,++ &loader->l_runpath_dirs, &realname, &fb, loader,++ LA_SER_RUNPATH, &found_other_class);++++ if (fd == -1)++ {++ realname = _dl_sysdep_open_object (name, namelen, &fd);++ if (realname != NULL)++ {++ fd = open_verify (realname, fd,++ &fb, loader ?: GL(dl_ns)[nsid]._ns_loaded,++ LA_SER_CONFIG, mode, &found_other_class,++ false);++ if (fd == -1)++ free (realname);++ }++ }+++ /* Finally, try the default path. */+ if (fd == -1+ && ((l = loader ?: GL(dl_ns)[nsid]._ns_loaded) == NULL-- 2.29.2
L
L
Ludovic Courtès wrote on 28 Nov 2020 11:24
[PATCH v2 3/4] gnu: glibc: Load ${ORIGIN}/../etc/ld.so.cache when available.
(address . 44899@debbugs.gnu.org)(name . Ludovic Courtès)(address . ludo@gnu.org)
20201128102409.3157-5-ludo@gnu.org
* gnu/packages/patches/glibc-dl-cache.patch: New file.* gnu/local.mk (dist_patch_DATA): Add it.* gnu/packages/base.scm (glibc)[source]: Remove 'snippet' and 'modules'.[arguments]: In 'pre-configure' phase, substitute @STORE_DIRECTORY@ in'elf/dl-cache.c'.--- gnu/local.mk | 1 + gnu/packages/base.scm | 16 +-- gnu/packages/patches/glibc-dl-cache.patch | 140 ++++++++++++++++++++++ 3 files changed, 147 insertions(+), 10 deletions(-) create mode 100644 gnu/packages/patches/glibc-dl-cache.patch
Toggle diff (195 lines)diff --git a/gnu/local.mk b/gnu/local.mkindex f9fed30a3f..82c3c608c6 100644--- a/gnu/local.mk+++ b/gnu/local.mk@@ -1051,6 +1051,7 @@ dist_patch_DATA = \ %D%/packages/patches/glibc-bootstrap-system-2.2.5.patch \ %D%/packages/patches/glibc-bootstrap-system-2.16.0.patch \ %D%/packages/patches/glibc-bootstrap-system.patch \+ %D%/packages/patches/glibc-dl-cache.patch \ %D%/packages/patches/glibc-hidden-visibility-ldconfig.patch \ %D%/packages/patches/glibc-hurd-clock_gettime_monotonic.patch \ %D%/packages/patches/glibc-hurd-clock_t_centiseconds.patch \diff --git a/gnu/packages/base.scm b/gnu/packages/base.scmindex bd352319a1..ad4415f226 100644--- a/gnu/packages/base.scm+++ b/gnu/packages/base.scm@@ -675,17 +675,8 @@ the store.") (sha256 (base32 "0di848ibffrnwq7g2dvgqrnn4xqhj3h96csn69q4da51ymafl9qn"))- (snippet- ;; Disable 'ldconfig' and /etc/ld.so.cache. The latter is- ;; required on LFS distros to avoid loading the distro's libc.so- ;; instead of ours.- '(begin- (substitute* "sysdeps/unix/sysv/linux/configure"- (("use_ldconfig=yes")- "use_ldconfig=no"))- #t))- (modules '((guix build utils))) (patches (search-patches "glibc-ldd-x86_64.patch"+ "glibc-dl-cache.patch" "glibc-hidden-visibility-ldconfig.patch" "glibc-versioned-locpath.patch" "glibc-allow-kernel-2.6.32.patch"@@ -800,6 +791,11 @@ the store.") ;; 4.7.1. ((" -lgcc_s") "")) + ;; Tell the ld.so cache code where the store is.+ (substitute* "elf/dl-cache.c"+ (("@STORE_DIRECTORY@")+ (string-append "\"" (%store-directory) "\"")))+ ;; Have `system' use that Bash. (substitute* "sysdeps/posix/system.c" (("#define[[:blank:]]+SHELL_PATH.*$")diff --git a/gnu/packages/patches/glibc-dl-cache.patch b/gnu/packages/patches/glibc-dl-cache.patchnew file mode 100644index 0000000000..0f23b12add--- /dev/null+++ b/gnu/packages/patches/glibc-dl-cache.patch@@ -0,0 +1,140 @@+Read the shared library cache relative to $ORIGIN instead of reading+from /etc/ld.so.cache. Also arrange so that this cache takes+precedence over RUNPATH.++diff --git a/elf/dl-cache.c b/elf/dl-cache.c+index 93d185e788..e0760a1f40 100644+--- a/elf/dl-cache.c++++ b/elf/dl-cache.c+@@ -171,6 +171,51 @@ _dl_cache_libcmp (const char *p1, const char *p2)+ return *p1 - *p2;+ }+ ++/* Special value representing the lack of an ld.so cache. */++static const char ld_so_cache_lacking[] = "/ld.so cache is lacking";++++/* Return the per-application ld.so cache, relative to $ORIGIN, or NULL if++ that fails for some reason. Do not return the system-wide LD_SO_CACHE++ since on a foreign distro it would contain invalid information. */++static const char *++ld_so_cache (void)++{++ static const char *loader_cache;++++ if (loader_cache == NULL)++ {++ static const char store[] = "/gnu/store";++ const char *origin = _dl_get_origin ();++++ /* Check whether ORIGIN is something like "/gnu/store/…-foo/bin". */++ if (strncmp (store, origin, strlen (store)) == 0++ && origin[sizeof store - 1] == '/')++ {++ char *store_item_end = strchr (origin + sizeof store, '/');++++ if (store_item_end != NULL)++ {++ static const char suffix[] = "/etc/ld.so.cache";++ size_t store_item_len = store_item_end - origin;++++ /* Note: We can't use 'malloc' because it can be interposed.++ Likewise, 'strncpy' is not available. */++ char *cache = alloca (strlen (origin) + sizeof suffix);++++ strcpy (cache, origin);++ strcpy (cache + store_item_len, suffix);++++ loader_cache = __strdup (cache) ?: ld_so_cache_lacking;++ }++ else++ loader_cache = ld_so_cache_lacking;++ }++ else++ loader_cache = ld_so_cache_lacking;++ }++++ return loader_cache;++}+ + /* Look up NAME in ld.so.cache and return the file name stored there, or null+ if none is found. The cache is loaded if it was not already. If loading+@@ -190,12 +235,15 @@ _dl_load_cache_lookup (const char *name)+ + /* Print a message if the loading of libs is traced. */+ if (__glibc_unlikely (GLRO(dl_debug_mask) & DL_DEBUG_LIBS))+- _dl_debug_printf (" search cache=%s\n", LD_SO_CACHE);++ _dl_debug_printf (" search cache=%s\n", ld_so_cache ());++++ if (__glibc_unlikely (ld_so_cache () == ld_so_cache_lacking))++ return NULL;+ + if (cache == NULL)+ {+ /* Read the contents of the file. */+- void *file = _dl_sysdep_read_whole_file (LD_SO_CACHE, &cachesize,++ void *file = _dl_sysdep_read_whole_file (ld_so_cache (), &cachesize,+ PROT_READ);+ + /* We can handle three different cache file formats here:+diff --git a/elf/dl-load.c b/elf/dl-load.c+index f3201e7c14..a69aec3428 100644+--- a/elf/dl-load.c++++ b/elf/dl-load.c+@@ -2152,28 +2152,6 @@ _dl_map_object (struct link_map *loader, const char *name,+ loader ?: GL(dl_ns)[LM_ID_BASE]._ns_loaded,+ LA_SER_LIBPATH, &found_other_class);+ +- /* Look at the RUNPATH information for this binary. */+- if (fd == -1 && loader != NULL+- && cache_rpath (loader, &loader->l_runpath_dirs,+- DT_RUNPATH, "RUNPATH"))+- fd = open_path (name, namelen, mode,+- &loader->l_runpath_dirs, &realname, &fb, loader,+- LA_SER_RUNPATH, &found_other_class);+-+- if (fd == -1)+- {+- realname = _dl_sysdep_open_object (name, namelen, &fd);+- if (realname != NULL)+- {+- fd = open_verify (realname, fd,+- &fb, loader ?: GL(dl_ns)[nsid]._ns_loaded,+- LA_SER_CONFIG, mode, &found_other_class,+- false);+- if (fd == -1)+- free (realname);+- }+- }+-+ #ifdef USE_LDCONFIG+ if (fd == -1+ && (__glibc_likely ((mode & __RTLD_SECURE) == 0)+@@ -2232,6 +2210,28 @@ _dl_map_object (struct link_map *loader, const char *name,+ }+ #endif+ ++ /* Look at the RUNPATH information for this binary. */++ if (fd == -1 && loader != NULL++ && cache_rpath (loader, &loader->l_runpath_dirs,++ DT_RUNPATH, "RUNPATH"))++ fd = open_path (name, namelen, mode,++ &loader->l_runpath_dirs, &realname, &fb, loader,++ LA_SER_RUNPATH, &found_other_class);++++ if (fd == -1)++ {++ realname = _dl_sysdep_open_object (name, namelen, &fd);++ if (realname != NULL)++ {++ fd = open_verify (realname, fd,++ &fb, loader ?: GL(dl_ns)[nsid]._ns_loaded,++ LA_SER_CONFIG, mode, &found_other_class,++ false);++ if (fd == -1)++ free (realname);++ }++ }+++ /* Finally, try the default path. */+ if (fd == -1+ && ((l = loader ?: GL(dl_ns)[nsid]._ns_loaded) == NULL-- 2.29.2
L
L
Ludovic Courtès wrote on 1 Dec 2020 21:45
Re: [bug#44899] [PATCH v2 0/4] Using 'ld.so.cache' to speed up application startup
(address . 44899-done@debbugs.gnu.org)
874kl5i7bv.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:
Toggle quote (6 lines)> gremlin: Fix typo in docstring.> gremlin: Add 'file-needed/recursive'.> gnu: glibc: Load 'etc/ld.so.cache' in $ORIGIN's store item when> available.> build-system/gnu: Add 'make-dynamic-linker-cache' phase.
Pushed as f85efa86e7690d9181946351631e02b1c20958c9, sans hard-coded/gnu/store in the patch as reported by Ricardo on IRC and debuggingleftover in gremlin.scm.
Enjoy… and report any issues you may find! :-)
Ludo’.
Closed
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 44899@debbugs.gnu.org