From debbugs-submit-bounces@debbugs.gnu.org Fri Dec 28 18:13:01 2018 Received: (at submit) by debbugs.gnu.org; 28 Dec 2018 23:13:01 +0000 Received: from localhost ([127.0.0.1]:40703 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gd1JM-0001Jx-Do for submit@debbugs.gnu.org; Fri, 28 Dec 2018 18:13:01 -0500 Received: from eggsout.gnu.org ([209.51.188.92]:39980) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gd1JK-0001Jh-T1 for submit@debbugs.gnu.org; Fri, 28 Dec 2018 18:12:59 -0500 Received: from lists.gnu.org ([208.118.235.17]:58459) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gd1JF-0005rK-EO for submit@debbugs.gnu.org; Fri, 28 Dec 2018 18:12:53 -0500 Received: from eggs.gnu.org ([208.118.235.92]:33661) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gd1J9-0000oR-VG for guix-patches@gnu.org; Fri, 28 Dec 2018 18:12:53 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gd1J3-0005bq-Kh for guix-patches@gnu.org; Fri, 28 Dec 2018 18:12:47 -0500 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:50661) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gd1Ib-00052a-Gm; Fri, 28 Dec 2018 18:12:13 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51292 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gd1Ib-0006mA-74; Fri, 28 Dec 2018 18:12:13 -0500 From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= To: guix-patches@gnu.org Subject: [PATCH 0/5] Distributing substitutes over IPFS Date: Sat, 29 Dec 2018 00:12:05 +0100 Message-Id: <20181228231205.8068-1-ludo@gnu.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: submit Cc: Hector Sanjuan , =?UTF-8?q?Ludovic=20Court=C3=A8s?= , Pierre Neidhardt X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello Guix! Here is a first draft adding support to distribute and retrieve substitutes over IPFS. This builds on discussions at the R-B Summit with Héctor Sanjuan of IPFS, lewo of Nix, Pierre Neidhardt, and also on the work Florian Paul Schmidt posted on guix-devel last month. The IPFS daemon exposes an HTTP API and the (guix ipfs) module provides bindings to a subset of that API. This module also implements a custom “directory” format to store directory trees in IPFS (IPFS already provides “UnixFS” and “tar” but they store too many or too few file attributes.) ‘guix publish’ and ‘guix substitute’ use (guix ipfs) to store and retrieve store items. Complete directory trees are stored in IPFS “as is”, rather than as compressed archives (nars). This allows for deduplication in IPFS. ‘guix publish’ adds a new “IPFS” field in narinfos and ‘guix substitute’ can then query those objects over IPFS. So the idea is that you still get narinfos over HTTP(S), and then you have the option of downloading substitutes over IPFS. I’ve pushed these patches in ‘wip-ipfs-substitutes’. This is rough on the edges and probably buggy, but the adventurous among us might want to give it a spin. :-) Thanks, Ludo’. Ludovic Courtès (5): Add (guix json). tests: 'file=?' now recurses on directories. Add (guix ipfs). publish: Add IPFS support. DRAFT substitute: Add IPFS support. Makefile.am | 3 + doc/guix.texi | 33 +++++ guix/ipfs.scm | 250 ++++++++++++++++++++++++++++++++++++ guix/json.scm | 63 +++++++++ guix/scripts/publish.scm | 67 +++++++--- guix/scripts/substitute.scm | 106 ++++++++------- guix/swh.scm | 35 +---- guix/tests.scm | 26 +++- tests/ipfs.scm | 55 ++++++++ 9 files changed, 535 insertions(+), 103 deletions(-) create mode 100644 guix/ipfs.scm create mode 100644 guix/json.scm create mode 100644 tests/ipfs.scm -- 2.20.1