[PATCH] derivations: Avoid readlink syscalls in read-derivation-from-file.

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Christopher Baines
Owner
unassigned
Submitted by
Christopher Baines
Severity
normal

Debbugs page

Christopher Baines wrote 1 years ago
(address . guix-patches@gnu.org)
a0061470c9d839f1503fea238c5e3b3ed92fd660.1700220934.git.mail@cbaines.net
strace -c reports over 10,000 readlink syscalls when reading the derivation
for the hello package. By just setting the %file-port-name-canonicalization
fluid, this drops to less than 10.

I'm not sure if this actually improves performance, but doing less is surely
better.

* guix/derivations.scm (read-derivation-from-file): Set
%file-port-name-canonicalization to 'none when calling call-with-input-file.

Change-Id: I1ff16a059160576a576f2e9ed881379596e66af3
---
guix/derivations.scm | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)

Toggle diff (22 lines)
diff --git a/guix/derivations.scm b/guix/derivations.scm
index 9fec7f4f0b..e6ecb570c4 100644
--- a/guix/derivations.scm
+++ b/guix/derivations.scm
@@ -556,7 +556,12 @@ (define (read-derivation-from-file file)
;; and because the same argument is read more than 15 times on average
;; during something like (package-derivation s gdb).
(or (and file (hash-ref %derivation-cache file))
- (let ((drv (call-with-input-file file read-derivation)))
+ (let ((drv
+ ;; Avoid calling scm_i_relativize_path in
+ ;; fport_canonicalize_filename since this leads to lots of
+ ;; readlink calls
+ (with-fluids ((%file-port-name-canonicalization 'none))
+ (call-with-input-file file read-derivation))))
(hash-set! %derivation-cache file drv)
drv)))

base-commit: e35b7c5386c1bfacf47ed31bac9b503373dd26fc
--
2.41.0
Ludovic Courtès wrote 1 years ago
(name . Christopher Baines)(address . mail@cbaines.net)
87wmu775lq.fsf@gnu.org
Hi,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (12 lines)
> strace -c reports over 10,000 readlink syscalls when reading the derivation
> for the hello package. By just setting the %file-port-name-canonicalization
> fluid, this drops to less than 10.
>
> I'm not sure if this actually improves performance, but doing less is surely
> better.
>
> * guix/derivations.scm (read-derivation-from-file): Set
> %file-port-name-canonicalization to 'none when calling call-with-input-file.
>
> Change-Id: I1ff16a059160576a576f2e9ed881379596e66af3

[...]

Toggle quote (7 lines)
> + (let ((drv
> + ;; Avoid calling scm_i_relativize_path in
> + ;; fport_canonicalize_filename since this leads to lots of
> + ;; readlink calls
> + (with-fluids ((%file-port-name-canonicalization 'none))
> + (call-with-input-file file read-derivation))))

This is already done in ‘run-guix’ in (guix ui), for all the ‘guix’
commands (so this patch would be a slight performance regression for
Guix itself).

I’d suggest setting this fluid globally in applications that use Guix
(the Build Coordinator, etc.), as is done in Guix itself.

WDYT?

Ludo’.
Christopher Baines wrote 1 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 67238-close@debbugs.gnu.org)
87edgdmtew.fsf@cbaines.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (34 lines)
> Hi,
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> strace -c reports over 10,000 readlink syscalls when reading the derivation
>> for the hello package. By just setting the %file-port-name-canonicalization
>> fluid, this drops to less than 10.
>>
>> I'm not sure if this actually improves performance, but doing less is surely
>> better.
>>
>> * guix/derivations.scm (read-derivation-from-file): Set
>> %file-port-name-canonicalization to 'none when calling call-with-input-file.
>>
>> Change-Id: I1ff16a059160576a576f2e9ed881379596e66af3
>
> [...]
>
>> + (let ((drv
>> + ;; Avoid calling scm_i_relativize_path in
>> + ;; fport_canonicalize_filename since this leads to lots of
>> + ;; readlink calls
>> + (with-fluids ((%file-port-name-canonicalization 'none))
>> + (call-with-input-file file read-derivation))))
>
> This is already done in ‘run-guix’ in (guix ui), for all the ‘guix’
> commands (so this patch would be a slight performance regression for
> Guix itself).
>
> I’d suggest setting this fluid globally in applications that use Guix
> (the Build Coordinator, etc.), as is done in Guix itself.
>
> WDYT?

Ah, I didn't realise it was already set for Guix scripts. But yeah,
setting it in other places that read derivations makes sense.
-----BEGIN PGP SIGNATURE-----

iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmViYpdfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XfcDhAAhpy+TsA6QPr+tmBAZrLtMocrvkU7DHyi
HTfdD9lTL082HX9S3R7O7cAcMADiUOUNJfeIMA7NWfgZDBcksgEYlKKM6vff2HFP
uylS4tiwOiURPl70HUm3wlLbPrjXO3yVfpNuSgXjxy8ZFVqFUyhFxT5En96+yZj2
pD4yUxbff2J13kO3H4dvgnDM3lQpuQkxP//Scn20/cJRT85ADniDTUXTilRpP0JZ
dG0/1w2fDgvfnsOBltPAyRgRZG0chvWD6amNVx19t0rIdDZ2JYhlMJTFpnSZsi5H
InxpJcuFEaiHkwOGGe+XJghqckiJHuKBbiXa8PJ8tN1UbwioFLuotcHz1cEct9VQ
Zm3mZi0+aZBfrAml+bOz7gFC/mU3uscvZEOjHw6pj4lPtdxAmOWc+/jG794SD+rf
oO/SE7V0yWib/EIZX+saUlWPMeAlooLhKHyG/7nhdHqDExYRajmCZ7x9gf740CV7
PGwFXaHTC8yJllZYvszS+VM19KfL18M+koNruE8Ta0deFDPW12VHAm+s955UDkIA
1zFPpc777g1xzKt+EYwMUzpIegb3JBDDlMukUMiD2v4Wn28mNY2gKv3BdJz5Wld7
qBLRBWwbcC6Z2PuS2HFTvlGXNFP2OD6I/gmoF2nK38A1fbVfkGQTDDrHBvB3Wobv
uR3xDz01X2E=
=Ct4r
-----END PGP SIGNATURE-----

?
Your comment

This issue is archived.

To comment on this conversation send an email to 67238@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 67238
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help