From debbugs-submit-bounces@debbugs.gnu.org Sat Oct 24 10:49:44 2020 Received: (at submit) by debbugs.gnu.org; 24 Oct 2020 14:49:44 +0000 Received: from localhost ([127.0.0.1]:34253 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWKrY-0001E7-6W for submit@debbugs.gnu.org; Sat, 24 Oct 2020 10:49:44 -0400 Received: from lists.gnu.org ([209.51.188.17]:43150) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWKrV-0001Dx-Ba for submit@debbugs.gnu.org; Sat, 24 Oct 2020 10:49:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39644) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWKrV-0006Gn-12 for guix-patches@gnu.org; Sat, 24 Oct 2020 10:49:41 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47624) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWKrS-0006RB-73; Sat, 24 Oct 2020 10:49:39 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51756 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kWKrQ-0004dT-Hx; Sat, 24 Oct 2020 10:49:37 -0400 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH 0/1] 'guix publish --cache' can publish items not yet cached Date: Sat, 24 Oct 2020 16:49:29 +0200 Message-Id: <20201024144929.4529-1-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: submit Cc: =?UTF-8?q?Ludovic=20Court=C3=A8s?= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hello! The ‘--cache’ mode of ‘guix publish’ is nice for many reasons. Until now though, it would only return 200 to narinfo and nar requests corresponding to items already in cache—already “baked”. Thus, the first narinfo request for an item would always return 404; one would have to wait until the item is baked to get 200 and download the substitute. If you’re fetching substitutes for popular packages on a popular instance, the extra delay is most likely invisible. However, if you’re using an instance with few users, or if you’re interesting in substitutes for an package that’s not popular, the behavior described above is a showstopper. (Many people here have criticized it in the past. :-)) This patch changes the behavior of ‘--cache’: if a store item is not yet in cache, and if it’s “small enough”, then narinfo/nar requests for it immediately return 200, as in the no-cache mode. You’re trading possibly increased server resource usage for reduced publication delay. To put an upper bound on the extra resource usage, the ‘--cache-bypass-threshold’ allows users to control the maximum size of a store item that can receive this treatment. An interesting use case that can benefit from this change is .drv substitutes: .drv themselves can be substitutes, so you can run: guix build /gnu/store/….drv and the daemon will fetch the .drv and its closure and start building it (since commit 9c9982dc0c8c38ce3821b154b7e92509c1564317). It’s not something we use much so far, but maybe we could put it to good use in the future (I know Chris has been playing with it in the context of the Data Service). Thoughts? Ludo’. Ludovic Courtès (1): publish: Add '--cache-bypass-threshold'. doc/guix.texi | 24 +++++++++++- guix/scripts/publish.scm | 85 ++++++++++++++++++++++++++++++++-------- tests/publish.scm | 43 ++++++++++++++++++-- 3 files changed, 130 insertions(+), 22 deletions(-) -- 2.28.0