Poor 'guix substitute' performance when receiving Zstd-compressed substitutes

OpenSubmitted by Maxim Cournoyer.
Details
2 participants
  • Ludovic Courtès
  • Maxim Cournoyer
Owner
unassigned
Severity
normal
M
M
Maxim Cournoyer wrote on 22 Jun 20:50 +0200
(name . bug-guix)(address . bug-guix@gnu.org)
87fsx93f99.fsf@gmail.com
Hello,
It's something I've been observing for a while, but substitutes are veryIO intensive (as can be seen in iotop, the substitute process is waitingon IO > 99% of the time) and is much slower than expected (3 minutes totransfer 100 MiB uncompressed over a 50 mbps downstream link):
Toggle snippet (4 lines)TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND13934 be/4 root 1033.09 K/s 1485.06 K/s 0.00 % 93.36 % guile \ /gnu/store/vphx2839xv0qj9xwcwrb95592lzrrnx7-guix-1.3.0-3.50dfbbf/bin/guix substitute --substitute
The publisher (remote machine) is has its guix-daemon configured via:
Toggle snippet (7 lines) (service guix-publish-service-type (guix-publish-configuration (advertise? #t) (compression '(("zstd" 3))) (host "0.0.0.0"))) ;listen on all interfaces
CPU is idling during on both sides during the transfer (all the workappears to be in the IO on the receiving end).
The above example was observed downloading/gnu/store/4fcwwlv9bzfrraxiz41b4vcv131930fx-libstdc++-doc-9.4.0, but Ido not think it is substitute-specific.
Maxim
L
L
Ludovic Courtès wrote on 23 Jun 15:26 +0200
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 49174@debbugs.gnu.org)
87eecsyang.fsf@gnu.org
Hi,
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
Toggle quote (17 lines)> It's something I've been observing for a while, but substitutes are very> IO intensive (as can be seen in iotop, the substitute process is waiting> on IO > 99% of the time) and is much slower than expected (3 minutes to> transfer 100 MiB uncompressed over a 50 mbps downstream link):>> TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND> 13934 be/4 root 1033.09 K/s 1485.06 K/s 0.00 % 93.36 % guile \ /gnu/store/vphx2839xv0qj9xwcwrb95592lzrrnx7-guix-1.3.0-3.50dfbbf/bin/guix substitute --substitute>>> The publisher (remote machine) is has its guix-daemon configured via:>> (service guix-publish-service-type> (guix-publish-configuration> (advertise? #t)> (compression '(("zstd" 3)))> (host "0.0.0.0"))) ;listen on all interfaces
Note that in this case nars are built and compressed on the fly on theserver side, which puts an upper bound on the bandwidth you can achieve.
I showed earlier how I profiled these things:
https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more-quickly/
If the client is I/O-bound, that’s good: it means we can’t do any better(unless we skip unpacking as demonstrated by distri).
If you can provide detailed profiles of either the server side or theclient side (but in that case, make sure the server is caching things),that’d be great!
Otherwise I’m afraid this is not actionable. :-)
Thanks,Ludo’.
L
L
Ludovic Courtès wrote on 3 Jul 18:10 +0200
control message for bug #49174
(address . control@debbugs.gnu.org)
87czrzz8d6.fsf@gnu.org
tags 49174 + moreinfoquit
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 49174@debbugs.gnu.org