[PATCH] store: Use buffered I/O for all protocol writes

> I did some digging to investigate why SSH-based guix operations are currently

> so slow. One of the reasons seems to be that some operations write single words

> to the socket and due to NOWAIT these result in a lot of small packets.

> The attached patch modifies store.scm to buffer all writes, so that won’t happen

> any more. I’m seeing about ~40% speedup (6.194s vs 10.075s for the best out of

> five runs) for `GUIX_DAEMON_SOCKET=ssh://localhost guix environment guix --

> true`, but it also seems to have a negative impact on UNIX domain socket

> communication by ~10% (1.561s vs 1.385s). For the notorious r-learnr package

> it’s still ~20% better (ssh, 16.620/21.418) and only ~3% worse (unix,

> 11.489/11.199). Keep in mind localhost has a much larger MTU and much lower

> latency than usual networks though. It might benefit from having an even bigger

> write buffer (64k vs. currently 8k).

> From eb38dc246e5dd33f8a48a99aded8b8dc0c378376 Mon Sep 17 00:00:00 2001

> From: Lars-Dominik Braun <ldb@leibniz-psychology.org>

> Date: Fri, 5 Jun 2020 10:38:32 +0200

> Subject: [PATCH] store: Use buffered I/O for all protocol writes

>

> * guix/store.scm (run-gc) Use buffered output port.

> (export-path) Same.

> (add-file-tree-to-store) Same.

> (set-build-options): Same. Add explicit flush.

> Nice, though the slowdown on Unix-domain sockets is annoying given that

> it’s the primary transport. Are the 10% significant and stable over

> multiple runs?

> Otherwise the patch LGTM as long as it doesn’t introduce any test suite

> regressions.

>> Nice, though the slowdown on Unix-domain sockets is annoying given that

>> it’s the primary transport. Are the 10% significant and stable over

>> multiple runs?

> I don’t want to blind your eyes with meaningless statistics here, but let’s do

> this properly then. So for `guix environment guix -- true` with 10 trials these

> are the results:

>

> ---snip---

> $ ministat -A -c 99 before_ssh after_ssh

> x before_ssh

> + after_ssh

> N Min Max Median Avg Stddev

> x 10 7.63 10.635 10.1585 9.9382 0.85517585

> + 10 5.986 6.424 6.1755 6.1917 0.15398416

> Difference at 99.0% confidence

> -3.7465 +/- 0.790815

> -37.698% +/- 5.0788%

> (Student's t, pooled s = 0.614425)

>

> $ ministat -A -c 99 before_unix after_unix

> x before_unix

> + after_unix

> N Min Max Median Avg Stddev

> x 10 1.528 1.624 1.5445 1.5636 0.035842867

> + 10 1.576 1.701 1.6145 1.6229 0.037869513

> Difference at 99.0% confidence

> 0.0593 +/- 0.0474548

> 3.79253% +/- 3.08989%

> (Student's t, pooled s = 0.0368701)

> ---snap---

>

> And the same for `guix environment --ad-hoc r-learnr`:

>

> ---snip---

> $ ministat -A -c 99 before_ssh after_ssh

> x before_ssh

> + after_ssh

> N Min Max Median Avg Stddev

> x 10 20.104 21.629 21.082 21.0115 0.45142472

> + 10 16.082 16.713 16.292 16.3416 0.25018224

> Difference at 99.0% confidence

> -4.6699 +/- 0.469719

> -22.2254% +/- 1.86734%

> (Student's t, pooled s = 0.364949)

>

> $ ministat -A -c 99 before_unix after_unix

> x before_unix

> + after_unix

> N Min Max Median Avg Stddev

> x 10 11.322 11.529 11.401 11.4127 0.062467858

> + 10 11.249 11.469 11.3515 11.368 0.078689262

> No difference proven at 99.0% confidence

> ---snap---

> What package provides ‘ministat’?