Hi, On ven., 10 sept. 2021 at 16:34, Ludovic Courtès wrote: > Finally we can enjoy content-addressability and brittle URLs > are becoming a thing of the past!* Yeah, it is awesome! The original URL of the channel was: . And this channel defines a package where the upstream has also disappeared . Note the URL in the package definition is not bogus… but using one was already working. :-) All is saved on SWH, so now all is transparent! From my point of view, this is a killer feature for scientific folks. :-) --8<---------------cut here---------------start------------->8--- $ cat /tmp/channels.scm (list (channel (name 'guix) (url "/home/sitour/src/guix/guix") (branch "fix-44187") (commit "cdea76a2fdaf7705583a02081a6468d436b8df05")) (channel (name 'example) (url "https://example.org/foo.git") (commit "67c9f2143aa6f545419ae913b4ae02af4cd3effc"))) $ ./pre-inst-env guix time-machine -C /tmp/channels.scm --disable-authentication -- build hi Updating channel 'guix' from Git repository at '/home/sitour/src/guix/guix'... guix time-machine: warning: channel authentication disabled Updating channel 'example' from Git repository at 'https://example.org/foo.git'... SWH: found revision 67c9f2143aa6f545419ae913b4ae02af4cd3effc with directory at 'https://archive.softwareheritage.org/api/1/directory/fe423e88ce277d3fc230c88d408e42b14a3a458c/' SWH vault: requested bundle cooking, waiting for completion... swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/HEAD swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/branches/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/config swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/description swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/hooks/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/info/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/info/exclude swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/info/refs swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/info/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/info/packs swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/pack/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/pack/pack-4e9279a1b64e4dda7bd9d84bb6b50bb1f80def08.idx swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/objects/pack/pack-4e9279a1b64e4dda7bd9d84bb6b50bb1f80def08.pack swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/refs/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/refs/heads/ swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/refs/heads/master swh:1:rev:67c9f2143aa6f545419ae913b4ae02af4cd3effc.git/refs/tags/ guix time-machine: warning: channel authentication disabled [...] Computing Guix derivation for 'x86_64-linux'... - [...] construction de /gnu/store/6g9qlysbbk7p4609xrv82j0wzbib1y4r-git-checkout.drv... guile: warning: failed to install locale environment variable `PATH' set to `/gnu/store/378zjf2kgajcfd7mfr98jn5xyc5wa3qv-gzip-1.10/bin:/gnu/store/sf3rbvb6iqcphgm1afbplcs72hsywg25-tar-1.32/bin' hint: Using 'master' as the name for the initial branch. This default branch name hint: is subject to change. To configure the initial branch name to use in all hint: of your new repositories, which will suppress this warning, call: hint: hint: git config --global init.defaultBranch hint: hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and hint: 'development'. The just-created branch can be renamed via this command: hint: hint: git branch -m Initialized empty Git repository in /gnu/store/884nsva9r8wkp40kbqyvpj1ad57jc5dd-git-checkout/.git/ fatal: could not read Username for 'https://github.com': No such device or address Failed to do a shallow fetch; retrying a full fetch... fatal: could not read Username for 'https://github.com': No such device or address git-fetch: '/gnu/store/5vai7bfrfkzv22dx13bxpszjrqyi78x6-git-minimal-2.33.0/bin/git fetch origin' failed with exit code 128 Trying content-addressed mirror at berlin.guix.gnu.org... Trying content-addressed mirror at berlin.guix.gnu.org... Trying to download from Software Heritage... SWH: found revision e1eefd033b8a2c4c81babc6fde08ebb116c6abb8 with directory at 'https://archive.softwareheritage.org/api/1/directory/c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/' swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/ swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/ABOUT-NLS swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/AUTHORS swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/COPYING [...] swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/tests/hello-1 swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/tests/last-1 swh:1:dir:c3e538ed2de412d54c567ed7c8cfc46cbbc35d07/tests/traditional-1 construction de /gnu/store/6g9qlysbbk7p4609xrv82j0wzbib1y4r-git-checkout.drv réussie construction de /gnu/store/jx1r7w8xaw768176pjl0j0q1l1529w75-hi-2.10.drv... starting phase `set-SOURCE-DATE-EPOCH' phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds [...] construction de /gnu/store/jx1r7w8xaw768176pjl0j0q1l1529w75-hi-2.10.drv réussie /gnu/store/jn8d031zx4znxy7s5zhj4dbr6xjsfq9v-hi-2.10 --8<---------------cut here---------------end--------------->8--- Well, it still misses the tarball and non-Git fetch method fallback and the story will be more than awesome! :-) > Limitations > ~~~~~~~~~~~~ > > Yes, there’s a couple of them. Well, yes some limitations but not so much. ;-) > First, fallback is implemented only for fresh clones, not for updates. > Thus, if I rerun the first example, having now the clone in > ~/.cache/guix/checkouts, with a different commit, I get: SWH is not a forge but an archive. :-) Therefore, this update case does not make sense to me. I mean, --8<---------------cut here---------------start------------->8--- $ git -C ~/.cache/guix/checkouts/6k7wvrcpbdsw3pje5b4squybw3jfn3viyrj7gcl7fipa5yjflaza fetch fatal: dépôt 'http://example.org/sdf/' non trouvé --8<---------------cut here---------------end--------------->8--- Well, maybe this cache could be removed if the commit is not found inside this cache and retry to fetch it from SWH. Obviously, the downdate case works. Note that on fresh clone, the error message could be improved: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix build guix --with-git-url=guix=https://example.org --with-commit=guix=ff613c2b68aac539262822490448e637d8f315ba -n updating checkout of 'https://example.org'... guix build: error: Git failure while fetching https://example.org: unexpected http status code: 404 --8<---------------cut here---------------end--------------->8--- where https://example.org is bogus and ff613c2b68aac539262822490448e637d8f315ba is not yet archived on SWH. It could be nice to warn in addition to the 404 that it is not found in SWH. WDYT? > Second, clones from SWH only contain the one branch that the revision > is on. For channels, that means that the ‘keyring’ branch is not fetched, > which is why I commented out ‘introduction’ in /tmp/chan.scm above. To me, it is not an issue. Because you reach a commit from the past knowing the hash. Aside my opinion, I wanted to know which kind of metadata we get back from the Git repo, so I tried: --8<---------------cut here---------------start------------->8--- $ guix build guix --with-git-url=guix=https://example.org --with-commit=guix=c75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada -n updating checkout of 'https://example.org'... SWH: found revision c75b30d58f0becb0a5cd6a8bfe69d1063b0d1ada with directory at 'https://archive.softwareheritage.org/api/1/directory/ca2e8a7222b4850c7bea935dff86b9c2a905efd6/' SWH vault: requested bundle cooking, waiting for completion... SWH vault: Processing... [...] --8<---------------cut here---------------end--------------->8--- then after several hours, I get this: --8<---------------cut here---------------start------------->8--- SWH vault: failure: Internal Server Error. This incident will be reported. SWH vault: retrying... SWH vault: requested bundle cooking, waiting for completion... SWH vault: Processing... --8<---------------cut here---------------end--------------->8--- and after more than 12h, the status is still: «SWH vault: Processing...» and nothing is complete. About this ’keyring’ branch, somehow it could be as a separated repo, so why not effectively do it. :-) I mean, get the branch as it is and mirror this branch in another Git repo saved on SWH; fallback to it if ’keyring’ branch is not there. I do not know… Or simply wait that SWH improves their things. :-) > *Third, and this answers the asterisk above, we must keep in mind that > this is content-addressibility *with SHA1*. Generating a chosen-prefix > collision is becoming affordable³, so users absolutely need an additional > mechanism to authenticate code they fetched. > > For origins, we have the content SHA256, so we’re fine. For channels, > we have Guix’s authentication mechanism¹, except it’s not available yet > via SWH, as I wrote above. For the footswitch example above using > ‘--with-commit’, we don’t have any authentication method, but in fact, > that’s the situation of Git repositories in general: they can rarely be > authenticated. How a chosen-prefix attack could work here? I understand why the second preimage attack is an issue. But I miss how the SHA-1 chosen-prefix attack could be exploited here to compromise the user, because this hash is provided by this very same user. > Ludovic Courtès (3): > swh: Support downloads of bare Git repositories. > git: 'update-cached-checkout' can fall back to SWH when cloning. > git: 'reference-available?' recognizes 'tag-or-commit'. LGTM! Cheers, simon