[PATCH] Adaptive substitute decompression selection

  • Done
  • quality assurance status badge
Details
One participant
  • Ludovic Courtès
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 14 Mar 2021 15:38
(address . guix-patches@gnu.org)
87wnu9ls08.fsf@inria.fr
Hi!

The patch below is a followup to the thread started in December:


It provides a naïve but apparently good enough way for ‘guix substitute’
to choose the compression method that yields the best speed given the
CPU and current networking conditions.

On a recent x86_64 laptop with fast networking, using ci.guix.gnu.org,
the effect so far is to choose gzip substitutes, which indeed provides
slightly faster substitute installation. When ci.guix provides zstd
substitutes, the speedup will be higher.

I have yet to check that it sticks to lzip when bandwidth is low.

Thoughts?

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 21 Mar 2021 23:46
(address . 47137-done@debbugs.gnu.org)
874kh415xr.fsf@gnu.org
Hi!

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (15 lines)
> The patch below is a followup to the thread started in December:
>
> https://lists.gnu.org/archive/html/guix-devel/2020-12/msg00177.html
>
> It provides a naïve but apparently good enough way for ‘guix substitute’
> to choose the compression method that yields the best speed given the
> CPU and current networking conditions.
>
> On a recent x86_64 laptop with fast networking, using ci.guix.gnu.org,
> the effect so far is to choose gzip substitutes, which indeed provides
> slightly faster substitute installation. When ci.guix provides zstd
> substitutes, the speedup will be higher.
>
> I have yet to check that it sticks to lzip when bandwidth is low.

I did that, using ‘tc’, and it works as expected, staying on lzip.

Pushed as 9da5ec7099b992a8969a17627548cd341c01bd90 with two minor
tweaks: lowered the low hysteresis threshold, and added a comment on how
to use ‘tc’ to test the behavior on “slow” networks.

Rather than running ‘guix build’ followed by ‘guix gc’, I found that
manually invoking ‘guix substitute’ was nicer (long line ahead!):

( echo substitute /gnu/store/svv4826f8zfj8grl2qa17xnxk3acsppc-elixir-1.11.4 /tmp/t1; echo substitute /gnu/store/d9dk53m7pwx1dc1p97zm0q323gpk70f9-poezio-0.13.1 /tmp/t4; echo substitute /gnu/store/mra8i18y9gjavhmdlkbb10m4miinirgz-ocaml-4.11.1 /tmp/t2; echo substitute /gnu/store/ay2j5mp20j9vbhibcwp5lmmcmhqkdnga-vim-full-8.2.2632 /tmp/t3; echo substitute /gnu/store/svv4826f8zfj8grl2qa17xnxk3acsppc-elixir-1.11.4 /tmp/t5; echo substitute /gnu/store/ay2j5mp20j9vbhibcwp5lmmcmhqkdnga-vim-full-8.2.2632 /tmp/t6) | GUIX_ALLOW_UNAUTHENTICATED_SUBSTITUTES=yes ./pre-inst-env guix substitute --substitute 4>&2

Note that this change won’t take effect until we update the ‘guix’
package.

Ludo’.
Closed
?