SWH fallback fails (git-fetch)

DoneSubmitted by zimoun.
Details
2 participants
  • Ludovic Courtès
  • zimoun
Owner
unassigned
Severity
important
Z
Z
zimoun wrote on 9 Jul 2020 02:36
Download
(address . bug-guix@gnu.org)
86imexcymw.fsf@gmail.com
Download
Dear,
I think the fallback with SWH is broken. Here an example to reproduce.
Toggle snippet (8 lines)cd /tmp/tar -xvf $(guix build -S hello)cd hello-2.10git initgit add *git commit -m 'init'
Then I push this fresh repo to my own GitHub account. Because it iseasy to switch between public and private state option, which mimicsupstream loss.
Let create the well-known "hello" recipe, which is simply a copy/pasteof (gnu packages base).
Toggle snippet (28 lines)(define-module (hello) #:use-module (guix packages) #:use-module (guix build-system gnu) #:use-module (guix git-download) #:use-module (guix licenses))
(define-public hi (package (name "hi") (version "2.10") (source (origin (method git-fetch) (uri (git-reference (url "https://github.com/zimoun/hello-example.git") (commit "e1eefd033b8a2c4c81babc6fde08ebb116c6abb8"))) (sha256 (base32 "1im1gglfm4k10bh4mdaqzmx3lm3kivnsmxrvl6vyvmfqqzljq75l")))) (build-system gnu-build-system) (synopsis "Hello, GNU world: An example GNU package") (description "GNU Hello prints the message \"Hello, world!\" and then exits. Itserves as an example of standard GNU coding practices. As such, it supportscommand-line arguments, multiple languages, and so on.") (home-page "https://www.gnu.org/software/hello/") (license gpl3+)))
For the sanity of the test, "guix build -L . hi" builds correctly andreturns the item: /gnu/store/jn8d031zx4znxy7s5zhj4dbr6xjsfq9v-hi-2.10with Guix b7249aa.
The upstream source is archived on SWH with:
guix lint -L . hi -c archival
seehttps://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/zimoun/hello-example.gitand the logs in https://archive.softwareheritage.org/save/#requests
Note that running again "lint -c archival" returns nothing, which meansthat the linter finds it on SWH, I guess. Well, I have not checked what'lookup-content' returns yet. However, I filtered the first ~2000+'packages-with-sources' with 'lookup-content' and all the 'git-fetch'sources are returned #f, and some 'url-fetch' not. Another story,another thread. :-)

Let clean the previous builds and downloads:
guix gc -D $(guix gc --list-dead | grep -E '(hi-2|git-checkout)')
And now let turn off upstream and so the maybe bug:
Toggle snippet (34 lines)guix build -L . hiThe following derivations will be built: /gnu/store/x5vwhv8sn3l49vysq98j7h2rqqlff5zp-hi-2.10.drv /gnu/store/jn6f86hg9zyyhms1vn56hviv4m9yjm8j-git-checkout.drvbuilding /gnu/store/jn6f86hg9zyyhms1vn56hviv4m9yjm8j-git-checkout.drv...guile: warning: failed to install localeenvironment variable `PATH' set to `/gnu/store/378zjf2kgajcfd7mfr98jn5xyc5wa3qv-gzip-1.10/bin:/gnu/store/sf3rbvb6iqcphgm1afbplcs72hsywg25-tar-1.32/bin'Initialized empty Git repository in /gnu/store/884nsva9r8wkp40kbqyvpj1ad57jc5dd-git-checkout/.git/fatal: could not read Username for 'https://github.com': No such device or addressFailed to do a shallow fetch; retrying a full fetch...fatal: could not read Username for 'https://github.com': No such device or addressgit-fetch: '/gnu/store/i2pkspl2vjvas0q85hw46y2li06rcmcg-git-minimal-2.26.2/bin/git fetch origin' failed with exit code 128Trying content-addressed mirror at berlin.guixsd.org...Trying content-addressed mirror at berlin.guixsd.org...Trying to download from Software Heritage...Backtrace: 4 (primitive-load "/gnu/store/s56y8npabah6jc1bqrhsac6wqb1?")In ./guix/swh.scm: 573:13 3 (swh-download "https://github.com/zimoun/hello-example?" ?) 224:22 2 (call "https://archive.softwareheritage.org/api/1/revi?" ?)In web/client.scm: 563:0 1 (http-get "https://archive.softwareheritage.org/api/1/?" ?) 231:6 0 (tls-wrap #<closed: file 7ffff5f1b690> _ # _)
web/client.scm:231:6: In procedure tls-wrap:Error while printing exception.builder for `/gnu/store/jn6f86hg9zyyhms1vn56hviv4m9yjm8j-git-checkout.drv' failed with exit code 1build of /gnu/store/jn6f86hg9zyyhms1vn56hviv4m9yjm8j-git-checkout.drv failedView build log at '/var/log/guix/drvs/jn/6f86hg9zyyhms1vn56hviv4m9yjm8j-git-checkout.drv.bz2'.cannot build derivation `/gnu/store/x5vwhv8sn3l49vysq98j7h2rqqlff5zp-hi-2.10.drv': 1 dependencies couldn't be builtguix build: error: build of `/gnu/store/x5vwhv8sn3l49vysq98j7h2rqqlff5zp-hi-2.10.drv' failed

Let me know if moreinfo are required.
All the best,simon
Z
Z
zimoun wrote on 9 Jul 2020 03:52
Download
swh-download with https->http
(address . 42286@debbugs.gnu.org)
86eeplcv45.fsf@gmail.com
Download
On Thu, 09 Jul 2020 at 02:36, zimoun <zimon.toutoune@gmail.com> wrote:
Toggle quote (3 lines)> --8<---------------cut here---------------start------------->8---> guix build -L . hi
[...]
Toggle quote (13 lines)> Trying to download from Software Heritage...> Backtrace:> 4 (primitive-load "/gnu/store/s56y8npabah6jc1bqrhsac6wqb1?")> In ./guix/swh.scm:> 573:13 3 (swh-download "https://github.com/zimoun/hello-example?" ?)> 224:22 2 (call "https://archive.softwareheritage.org/api/1/revi?" ?)> In web/client.scm:> 563:0 1 (http-get "https://archive.softwareheritage.org/api/1/?" ?)> 231:6 0 (tls-wrap #<closed: file 7ffff5f1b690> _ # _)>> web/client.scm:231:6: In procedure tls-wrap:> Error while printing exception.
[...]
Toggle quote (2 lines)> --8<---------------cut here---------------end--------------->8---
To avoid, this error which should be a missing certificate, I switch to'http:' instead of 'https:' in %swh-base-url. The error is then:
Toggle snippet (13 lines)Trying to download from Software Heritage...Backtrace: 2 (primitive-load "/gnu/store/s56y8npabah6jc1bqrhsac6wqb1?")In ./guix/swh.scm: 573:13 1 (swh-download "https://github.com/zimoun/hello-example?" ?) 243:16 0 (call "http://archive.softwareheritage.org/api/1/revis?" ?)
./guix/swh.scm:243:16: In procedure call:Throw to key `swh-error' with args `("http://archive.softwareheritage.org/api/1/revision/e1eefd033b8a2c4c81babc6fde08ebb116c6abb8/" #<procedure http-get (uri #:key body port version keep-alive? headers decode-body? verify-certificate? streaming?)> #<<response> version: (1 . 1) code: 302 reason-phrase: "Found" headers: ((date . #<date nanosecond: 0 second: 59 minute: 38 hour: 1 day: 9 month: 7 year: 2020 zone-offset: 0>) (server . "Varnish") (x-varnish . "19881791") (location . #<<uri> scheme: https userinfo: #f host: "archive.softwareheritage.org" port: #f path: "/api/1/revision/e1eefd033b8a2c4c81babc6fde08ebb116c6abb8/" query: #f fragment: #f>) (content-length . 0) (connection close)) port: #<closed: file 7ffff5f1b690>>)'.builder for `/gnu/store/9f6648zb0p9z6hxr31wz3d7fm7ndcxzx-git-checkout.drv' failed with exit code 1

Because I hit the rate limit, I need to wait at least one hour...Follow-ups soon. :-)
Cheers,simon
L
L
Ludovic Courtès wrote on 9 Jul 2020 16:31
Download
control message for bug #42286
(address . control@debbugs.gnu.org)
875zawbw04.fsf@gnu.org
Download
severity 42286 importantquit
Z
Z
zimoun wrote on 9 Jul 2020 20:20
Download
Re: bug#42286: SWH fallback fails (git-fetch)
(address . 42286@debbugs.gnu.org)
CAJ3okZ0vpgQ9AHMsLeEbTSuENDtwhCx4CrLOupCWfePSpsqb+w@mail.gmail.com
Download
On Thu, 9 Jul 2020 at 02:39, zimoun <zimon.toutoune@gmail.com> wrote:
Well, "swh-download" perfectly works directly from the REPL.
Toggle quote (3 lines)> --8<---------------cut here---------------start------------->8---> guix build -L . hi
[...]
Toggle quote (13 lines)> Trying to download from Software Heritage...> Backtrace:> 4 (primitive-load "/gnu/store/s56y8npabah6jc1bqrhsac6wqb1?")> In ./guix/swh.scm:> 573:13 3 (swh-download "https://github.com/zimoun/hello-example?" ?)> 224:22 2 (call "https://archive.softwareheritage.org/api/1/revi?" ?)> In web/client.scm:> 563:0 1 (http-get "https://archive.softwareheritage.org/api/1/?" ?)> 231:6 0 (tls-wrap #<closed: file 7ffff5f1b690> _ # _)>> web/client.scm:231:6: In procedure tls-wrap:> --8<---------------cut here---------------end--------------->8---
The error is definitively something related to TLS and the gexp. In(guix git-download) "git-fetch", the "define build" returns a gexp andhere something is missing, even if the extension gnutls (module-ref(resolve-interface '(gnu packages tls)) 'gnutls) is provided. Hum, Iam not sure to understand what.
BTW, if in "git-fetch" from (guix git), I add these lines:
Toggle snippet (9 lines) (setenv "GIT_SSL_NO_VERIFY" "true")
(format #t "git-fetch~%") (http-get "https://archive.softwareheritage.org/api/1/content/sha256:31e066137a962676e89f69d1b65382de95a7ef7d914b8cb956f41ea72e0f516b/") (format #t "ok~%")
(mkdir-p directory)
Then I also hit:
Toggle snippet (32 lines)The following derivations will be built: /gnu/store/wam9fca6vj3rifvqlix9c874vpwn5k82-hi-2.10.drv /gnu/store/cvp65m4wzmzd8pqdfvah4mrl4zkcw3vz-git-checkout.drvbuilding /gnu/store/cvp65m4wzmzd8pqdfvah4mrl4zkcw3vz-git-checkout.drv...guile: warning: failed to install localeenvironment variable `PATH' set to`/gnu/store/378zjf2kgajcfd7mfr98jn5xyc5wa3qv-gzip-1.10/bin:/gnu/store/sf3rbvb6iqcphgm1afbplcs72hsywg25-tar-1.32/bin'git-fetchBacktrace: 4 (primitive-load "/gnu/store/gcr6v6p6c5gwr4l6xzqcy6wln33?")In ice-9/eval.scm: 293:34 3 (_ #<directory (guile-user) 7ffff5bb8f00>)In ./guix/build/git.scm: 44:2 2 (git-fetch "https://github.com/zimoun/hello-example.git" ?)In web/client.scm: 563:0 1 (http-get "https://archive.softwareheritage.org/api/1/?" ?) 231:6 0 (tls-wrap #<closed: file 7ffff4c17150> _ # _)
web/client.scm:231:6: In procedure tls-wrap:Error while printing exception.builder for `/gnu/store/cvp65m4wzmzd8pqdfvah4mrl4zkcw3vz-git-checkout.drv'failed with exit code 1build of /gnu/store/cvp65m4wzmzd8pqdfvah4mrl4zkcw3vz-git-checkout.drv failedView build log at'/var/log/guix/drvs/cv/p65m4wzmzd8pqdfvah4mrl4zkcw3vz-git-checkout.drv.bz2'.cannot build derivation`/gnu/store/wam9fca6vj3rifvqlix9c874vpwn5k82-hi-2.10.drv': 1dependencies couldn't be builtguix build: error: build of`/gnu/store/wam9fca6vj3rifvqlix9c874vpwn5k82-hi-2.10.drv' failed
Then I have tried to turn off the certificate verification with"#:verify-certification #f" for example in "vault-fetch" or "call"used by "define-query" but nothing works. Well, I am a bitcircumspect.
Therefore, I am waiting for a hint or the fix. :-)
Cheers,simon
L
L
Ludovic Courtès wrote on 10 Jul 2020 00:24
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 42286-done@debbugs.gnu.org)
87o8oo9vjp.fsf@gnu.org
Download
Hi!
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (14 lines)> Trying to download from Software Heritage...> Backtrace:> 4 (primitive-load "/gnu/store/s56y8npabah6jc1bqrhsac6wqb1?")> In ./guix/swh.scm:> 573:13 3 (swh-download "https://github.com/zimoun/hello-example?" ?)> 224:22 2 (call "https://archive.softwareheritage.org/api/1/revi?" ?)> In web/client.scm:> 563:0 1 (http-get "https://archive.softwareheritage.org/api/1/?" ?)> 231:6 0 (tls-wrap #<closed: file 7ffff5f1b690> _ # _)>> web/client.scm:231:6: In procedure tls-wrap:> Error while printing exception.> builder for `/gnu/store/jn6f86hg9zyyhms1vn56hviv4m9yjm8j-git-checkout.drv' failed with exit code 1
Should be fixed with commit a7696b9733d4ede9817a0a0accb5ce5b85d9a2d3.Let me know if anything’s amiss.
Thanks!
Ludo’.
Closed
Z
Z
zimoun wrote on 10 Jul 2020 05:18
Download
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 42286-done@debbugs.gnu.org)
86sge0awhq.fsf@gmail.com
Download
Hi,
On Fri, 10 Jul 2020 at 00:24, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (21 lines)> Hi!>> zimoun <zimon.toutoune@gmail.com> skribis:>>> Trying to download from Software Heritage...>> Backtrace:>> 4 (primitive-load "/gnu/store/s56y8npabah6jc1bqrhsac6wqb1?")>> In ./guix/swh.scm:>> 573:13 3 (swh-download "https://github.com/zimoun/hello-example?" ?)>> 224:22 2 (call "https://archive.softwareheritage.org/api/1/revi?" ?)>> In web/client.scm:>> 563:0 1 (http-get "https://archive.softwareheritage.org/api/1/?" ?)>> 231:6 0 (tls-wrap #<closed: file 7ffff5f1b690> _ # _)>>>> web/client.scm:231:6: In procedure tls-wrap:>> Error while printing exception.>> builder for `/gnu/store/jn6f86hg9zyyhms1vn56hviv4m9yjm8j-git-checkout.drv' failed with exit code 1>> Should be fixed with commit a7696b9733d4ede9817a0a0accb5ce5b85d9a2d3.> Let me know if anything’s amiss.
Cool! Works. :-)
I was almost there. :-) The missing trick was because the Guile bug https://bugs.gnu.org/40486I was not aware and so the new "http-get*".
Is it worth to add the test in guix-build.sh?

All the best,simon
Closed
L
L
Ludovic Courtès wrote on 10 Jul 2020 10:07
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 42286-done@debbugs.gnu.org)
87365zaj3u.fsf@gnu.org
Download
Hi!
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (28 lines)> On Fri, 10 Jul 2020 at 00:24, Ludovic Courtès <ludo@gnu.org> wrote:>> Hi!>>>> zimoun <zimon.toutoune@gmail.com> skribis:>>>>> Trying to download from Software Heritage...>>> Backtrace:>>> 4 (primitive-load "/gnu/store/s56y8npabah6jc1bqrhsac6wqb1?")>>> In ./guix/swh.scm:>>> 573:13 3 (swh-download "https://github.com/zimoun/hello-example?" ?)>>> 224:22 2 (call "https://archive.softwareheritage.org/api/1/revi?" ?)>>> In web/client.scm:>>> 563:0 1 (http-get "https://archive.softwareheritage.org/api/1/?" ?)>>> 231:6 0 (tls-wrap #<closed: file 7ffff5f1b690> _ # _)>>>>>> web/client.scm:231:6: In procedure tls-wrap:>>> Error while printing exception.>>> builder for `/gnu/store/jn6f86hg9zyyhms1vn56hviv4m9yjm8j-git-checkout.drv' failed with exit code 1>>>> Should be fixed with commit a7696b9733d4ede9817a0a0accb5ce5b85d9a2d3.>> Let me know if anything’s amiss.>> Cool! Works. :-)>> I was almost there. :-) The missing trick was because the Guile bug> <https://bugs.gnu.org/40486>> I was not aware and so the new "http-get*".
Yeah. :-)
Toggle quote (2 lines)> Is it worth to add the test in guix-build.sh?
We don’t add tests that depend on external services, so we can’t reallydo that. Or we would need to mock the original server, SWH, etc. butthat seems tricky.
Thanks,Ludo’.
Closed
?
Your may also send email to 42286@debbugs.gnu.org to comment.