Hello Guix, This is an initial patch and proposal towards decentralizing substitute distribution with ERIS. ERIS (Encoding for Robust Immutable Storage) [1] is an encoding of content into uniformly sized, encryped and content-addressed blocks. The original content can be reconstructed only with access to a read capability, which can be encoded as an URN. One key advantage of ERIS is that the encoding is protocol agnostic. Any protocol that can transfer small (32KiB) sized blocks referenced by the hash of their content will do. This can be done with things such as GNUNet, IPFS, OpenDHT, HTTP or a USB stick on a bicycle. The following patch allows substitutes to be published over IPFS using ERIS. This is inspired and very similar to previous work on distributing substitutes over IPFS [2]. The narinfos served by `guix publish` look like this: --8<---------------cut here---------------start------------->8--- StorePath: /gnu/store/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 URL: nar/gzip/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 Compression: gzip FileSize: 67363 ERIS: urn:erisx2:BIBC2LUTIQH43S2KRIAV7TBXNUUVPZTMV6KFA2M7AL5V6FNE77VNUDDVDAGJUEEAFATVO2QQT67SMOPTO3LGWCJFU7BZVCF5VXEQQW25BE URL: nar/zstd/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 Compression: zstd FileSize: 64917 ERIS: urn:erisx2:BIBO7KS7SAWHDNC43DVILOSQ3F3SRRHEV6YPLDCSZ7MMD6LZVCHQMEQ6FUBTJAPSNFF7XR5XPTP4OQ72OPABNEO7UYBUN42O46ARKHBTGM NarHash: sha256:1sagsz1mnlqkr8r8s6gwkzvvhq619rlzhpbxl3h0b111n5hn2w9w NarSize: 220704 References: 2fk1gz2s7ppdicynscra9b19byrrr866-glibc-2.33 81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 90lbavffg0csrf208nw0ayj1bz5knl47-gcc-10.3.0-lib Deriver: 260bk0ch4np4h2yz5yqhf8hjbsyhwpmr-hello-2.10.drv Signature: 1;strawberry;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjNDk4ODkwODZDNTY4MzQyRENFQzk3QzA3NDE4NEQ1RkRCOTNCNDA2MUNCRDM4MUExRjVBQzVDODI0MTgwMTU3OSMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMEU2NDlFODE4QzRFNjNGNEY2OUQ5QTAwRjUwNjRDMzQ3QjY3RDM0RTM0NTg2MkI4NTc3RTg5MUY5Q0Q3NDhBQiMpCiAgIChzICMwMTZGRjA1MDdCQjZGMzA2NUEzMjYzRDA2MTAyRDc5MTBEOEZGODc5RTdENjREODRFODBENDBGMTJFMTBBOTQ1IykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjMDRDMkY4ODk1QTU0NDNGNTlCODk2NDEwMEI1MDY0NzU4RjQ1N0YzMENEREE1MTQyQzE0MDc0NjExNTA1NTc5MCMpCiAgICkKICApCiApCg== --8<---------------cut here---------------end--------------->8--- For every compressed nar the ERIS URN is computed and added. If the `--ipfs` is used for `guix publish` then the encoded blocks are also uploaded to the IPFS daemon. The nar could then be retrieved from anywhere like this: --8<---------------cut here---------------start------------->8--- (use-modules (eris) (eris blocks ipfs)) (eris-decode->bytevector "urn:erisx2:BIBC2LUTIQH43S2KRIAV7TBXNUUVPZTMV6KFA2M7AL5V6FNE77VNUDDVDAGJUEEAFATVO2QQT67SMOPTO3LGWCJFU7BZVCF5VXEQQW25BE" eris-blocks-ipfs-ref) --8<---------------cut here---------------end--------------->8--- These patches do not yet retrieve content from IPFS (TODO). But in principle, anybody connected to IPFS can get the nar with the ERIS URN. This could be used to reduce load on substitute server as they would only need to publish the ERIS URN directly - substitutes could be delivered much more peer-to-peer. Other transports that I have been looking in to and am pretty sure will work include: HTTP (with RFC 2169 [3]), GNUNet, OpenDHT. This is, imho, the advantage of ERIS over IPFS directly or GNUNet directly. The encoding and identifiers (URN) are abstracted away from specific transports (and also applications). ERIS is almost exactly the same encoding as used in GNUNet (ECRS). Blocks can be stored in any kind of databases (see for example the GDBM bindings [4]). A tricky things is figuring out how to multiplex all these different transports and storages... The ERIS specification is still considered "experimental". However we feel confident to stabilize it and intend to do so around February/March 2022 with a release 1.0.0 of the specification. This will ensure that the identifiers remain stable for the forseeable future (until the crypto breaks). Before that there is also a small external security audit of the specification planned (thanks to NGI0/NLnet!). This is just a little demo of the idea and some food for thought and discussion. Give it a try and let me know what you think! I've also pushed the patches to my personal Guix mirror if you want to check it out from there: https://inqlab.net/git/guix.git/log/?h=wip-eris Also CCing ~pukkamustard/eris@lists.sr.ht where there is some general ERIS related discussion. Thanks, -pukkamustard [1] http://purl.org/eris [2] https://issues.guix.gnu.org/33899 [3] https://www.ietf.org/rfc/rfc2169.txt [4] https://inqlab.net/git/guile-eris.git/tree/eris/blocks/gdbm.scm pukkamustard (3): publish: Add ERIS URN to narinfo WIP: gnu: guile-eris: Update to unreleased git version. publish: Add IPFS support. configure.ac | 5 ++ gnu/packages/guile-xyz.scm | 10 ++-- gnu/packages/package-management.scm | 1 + guix/narinfo.scm | 10 ++-- guix/scripts/publish.scm | 79 ++++++++++++++++++++++------- 5 files changed, 79 insertions(+), 26 deletions(-) -- 2.34.0