nar 404 leads to hard ‘guix substitute’ crash

  • Done
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Christopher Baines
  • Simon Tournier
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
important
L
L
Ludovic Courtès wrote on 22 May 2023 00:12
nar 404 leads to hard ‘guix substitute’ crash
(address . bug-guix@gnu.org)
87cz2ttju2.fsf@inria.fr
Like this:

Toggle snippet (63 lines)
gst-plugins-bad-minimal-1.22.2 650KiB 6.8MiB/s 00:00 ???????????????????? 100.0%
guile-bytestructures-1.0.10 222KiB 4.1MiB/s 00:00 ???????????????????? 100.0%
Backtrace:
In ice-9/boot-9.scm:
1752:10 19 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
18 (apply-smob/0 #<thunk 7f6df0f172a0>)
In ice-9/boot-9.scm:
724:2 17 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
In ice-9/eval.scm:
619:8 16 (_ #(#(#<directory (guile-user) 7f6df0f1cc80>)))
In guix/ui.scm:
2300:7 15 (run-guix . _)
2263:10 14 (run-guix-command _ . _)
In ice-9/boot-9.scm:
1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
1752:10 12 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
854:15 11 (_)
646:2 10 (process-substitution _ _ _ #:cache-urls _ #:acl _ # _ # …)
In ice-9/boot-9.scm:
1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
463:7 8 (download-nar #<<narinfo> path: "/gnu/store/dg0rm1mp9q…> …)
In ice-9/boot-9.scm:
1747:15 7 (with-exception-handler #<procedure 7f6df0fa71e0 at ic…> …)
1685:16 6 (raise-exception _ #:continuable? _)
1683:16 5 (raise-exception _ #:continuable? _)
1685:16 4 (raise-exception _ #:continuable? _)
1780:13 3 (_ #<&compound-exception components: (#<&error> #<&irri…>)
1685:16 2 (raise-exception _ #:continuable? _)
1683:16 1 (raise-exception _ #:continuable? _)
1685:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
ERROR:
1. &http-get-error:
uri: #<<uri> scheme: https userinfo: #f host: "bordeaux.guix.gnu.org" port: #f path: "/nar/zstd/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1" query: #f fragment: #f>
code: 404
reason: "Not Found"
headers: ((server . "nginx") (date . #<date nanosecond: 0 second: 39 minute: 59 hour: 21 day: 21 month: 5 year: 2023 zone-offset: 0>) (content-type text/plain (charset . "utf-8")) (content-length . 3) (connection keep-alive))
2. &message: "https://bordeaux.guix.gnu.org/nar/zstd/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1: HTTP download failed: 404 (\"Not Found\")"
substitution of /gnu/store/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1 failed
guix system: error: corrupt input while restoring archive from #<closed: file 7f0e7113b3f0>
$ guix describe
Generation 263 May 21 2023 23:56:04 (current)
guix c5bc698
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: c5bc698e8922d78ed85989985cc2ceb034de2f23
guile 36fd2b4
repository URL: https://git.savannah.gnu.org/git/guile.git
branch: main
commit: 36fd2b4920ae926c79b936c29e739e71a6dff2bc
shepherd 954742c
repository URL: https://git.savannah.gnu.org/git/shepherd.git
branch: master
commit: 954742cd0d066ac4be7de4a031d7e229fd445c5d
$ guix package -p /run/current-system/profile -I guix
guix 1.4.0-6.dc5430c out /gnu/store/cgjddvw9zay626z8hyxl0zmn1354c24k-guix-1.4.0-6.dc5430c
guix-icons 0.1 out /gnu/store/arcpwjvhp3ws1i6g1sdnfxxrxc3zq4m9-guix-icons-0.1

This might be related to 8bd4126917f59f4af9a4323c3d5699201862dca2. The
two ‘download-nar’ calls are guarded against ‘http-get-error?’ though.

Ludo’.
L
L
Ludovic Courtès wrote on 22 May 2023 09:59
control message for bug #63634
(address . control@debbugs.gnu.org)
87o7mczthk.fsf@gnu.org
severity 63634 important
quit
L
L
Ludovic Courtès wrote on 22 May 2023 10:00
Re: bug#63634: nar 404 leads to hard ‘guix subs titute’ crash
(address . 63634@debbugs.gnu.org)
87jzx0ztgi.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (25 lines)
> In guix/scripts/substitute.scm:
> 854:15 11 (_)
> 646:2 10 (process-substitution _ _ _ #:cache-urls _ #:acl _ # _ # …)
> In ice-9/boot-9.scm:
> 1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/scripts/substitute.scm:
> 463:7 8 (download-nar #<<narinfo> path: "/gnu/store/dg0rm1mp9q…> …)
> In ice-9/boot-9.scm:
> 1747:15 7 (with-exception-handler #<procedure 7f6df0fa71e0 at ic…> …)
> 1685:16 6 (raise-exception _ #:continuable? _)
> 1683:16 5 (raise-exception _ #:continuable? _)
> 1685:16 4 (raise-exception _ #:continuable? _)
> 1780:13 3 (_ #<&compound-exception components: (#<&error> #<&irri…>)
> 1685:16 2 (raise-exception _ #:continuable? _)
> 1683:16 1 (raise-exception _ #:continuable? _)
> 1685:16 0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> ERROR:
> 1. &http-get-error:
> uri: #<<uri> scheme: https userinfo: #f host: "bordeaux.guix.gnu.org" port: #f path: "/nar/zstd/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1" query: #f fragment: #f>
> code: 404
> reason: "Not Found"
> headers: ((server . "nginx") (date . #<date nanosecond: 0 second: 39 minute: 59 hour: 21 day: 21 month: 5 year: 2023 zone-offset: 0>) (content-type text/plain (charset . "utf-8")) (content-length . 3) (connection keep-alive))

Possibly duplicate of https://issues.guix.gnu.org/55820.
S
S
Simon Tournier wrote on 22 May 2023 10:19
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 63634@debbugs.gnu.org)
87ttw4ye13.fsf@gmail.com
Hi,


On lun., 22 mai 2023 at 00:12, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (3 lines)
> This might be related to 8bd4126917f59f4af9a4323c3d5699201862dca2. The
> two ‘download-nar’ calls are guarded against ‘http-get-error?’ though.

Well, I am currently hitting something very similar.

Toggle snippet (42 lines)
$ guix build rust-base64@0.13 -S
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
0,0 MB will be downloaded:
/gnu/store/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz
substituting /gnu/store/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz...
Backtrace:
16 (primitive-load "/gnu/store/jpcpva9gg7l337d0icn88pf76l4…")
In guix/ui.scm:
2263:7 15 (run-guix . _)
2226:10 14 (run-guix-command _ . _)
In ice-9/boot-9.scm:
1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
1752:10 12 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
856:15 11 (_)
650:2 10 (process-substitution _ _ _ #:cache-urls _ #:acl _ # _ # …)
In ice-9/boot-9.scm:
1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
467:7 8 (download-nar #<<narinfo> path: "/gnu/store/sx6sr6cs1x…> …)
In ice-9/boot-9.scm:
1747:15 7 (with-exception-handler #<procedure 7f6a0ca624e0 at ic…> …)
1685:16 6 (raise-exception _ #:continuable? _)
1683:16 5 (raise-exception _ #:continuable? _)
1685:16 4 (raise-exception _ #:continuable? _)
1780:13 3 (_ #<&compound-exception components: (#<&error> #<&irri…>)
1685:16 2 (raise-exception _ #:continuable? _)
1683:16 1 (raise-exception _ #:continuable? _)
1685:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
ERROR:
1. &http-get-error:
uri: #<<uri> scheme: https userinfo: #f host: "bordeaux.guix.gnu.org" port: #f path: "/nar/zstd/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz" query: #f fragment: #f>
code: 404
reason: "Not Found"
headers: ((server . "nginx") (date . #<date nanosecond: 0 second: 28 minute: 9 hour: 8 day: 22 month: 5 year: 2023 zone-offset: 0>) (content-type text/plain (charset . "utf-8")) (content-length . 3) (connection keep-alive))
2. &message: "https://bordeaux.guix.gnu.org/nar/zstd/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz: HTTP download failed: 404 (\"Not Found\")"
substitution of /gnu/store/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz failed
guix build: error: some substitutes for the outputs of derivation `/gnu/store/qcpzwblzpa2wprc4vwlm128bwckfj8rj-rust-base64-0.13.1.tar.xz.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source

Then the option “--fallback” works.

Toggle quote (3 lines)
> The
> two ‘download-nar’ calls are guarded against ‘http-get-error?’ though.

Well, the fallback should happen transparently.

It rings this bell

Cheers,
simon
L
L
Ludovic Courtès wrote on 22 May 2023 11:32
(address . 63634@debbugs.gnu.org)(name . Christopher Baines)(address . mail@cbaines.net)
87a5xwzp6w.fsf@gnu.org
I experienced it a second time and have more info:

Toggle snippet (45 lines)
substituting /gnu/store/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0...
Backtrace:
In ice-9/boot-9.scm:
1752:10 19 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
18 (apply-smob/0 #<thunk 7f4f9bf172a0>)
In ice-9/boot-9.scm:
724:2 17 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
In ice-9/eval.scm:
619:8 16 (_ #(#(#<directory (guile-user) 7f4f9bf1cc80>)))
In guix/ui.scm:
2300:7 15 (run-guix . _)
2263:10 14 (run-guix-command _ . _)
In ice-9/boot-9.scm:
1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
1752:10 12 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
854:15 11 (_)
646:2 10 (process-substitution _ _ _ #:cache-urls _ #:acl _ # _ # …)
In ice-9/boot-9.scm:
1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
463:7 8 (download-nar #<<narinfo> path: "/gnu/store/apw1y9nf8r…> …)
In ice-9/boot-9.scm:
1747:15 7 (with-exception-handler #<procedure 7f4f9846af00 at ic…> …)
1685:16 6 (raise-exception _ #:continuable? _)
1683:16 5 (raise-exception _ #:continuable? _)
1685:16 4 (raise-exception _ #:continuable? _)
1780:13 3 (_ #<&compound-exception components: (#<&error> #<&irri…>)
1685:16 2 (raise-exception _ #:continuable? _)
1683:16 1 (raise-exception _ #:continuable? _)
1685:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
ERROR:
1. &http-get-error:
uri: #<<uri> scheme: https userinfo: #f host: "bordeaux.guix.gnu.org" port: #f path: "/nar/zstd/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0" query: #f fragment: #f>
code: 404
reason: "Not Found"
headers: ((server . "nginx") (date . #<date nanosecond: 0 second: 31 minute: 59 hour: 8 day: 22 month: 5 year: 2023 zone-offset: 0>) (content-type text/plain (charset . "utf-8")) (content-length . 3) (connection keep-alive))
2. &message: "https://bordeaux.guix.gnu.org/nar/zstd/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0: HTTP download failed: 404 (\"Not Found\")"
substitution of /gnu/store/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0 failed
guix build: error: corrupt input while restoring archive from #<closed: file 7f1693f9c9a0>

I have ‘--substitute-urls=https://ci.guix.gnu.orghttps://bordeaux.guix.gnu.org’.
Let’s see what’s available on each server:

Toggle snippet (48 lines)
$ wget -O- https://ci.guix.gnu.org/apw1y9nf8rqgxvjnlr1isbhpd502bcs5.narinfo
--2023-05-22 11:00:08-- https://ci.guix.gnu.org/apw1y9nf8rqgxvjnlr1isbhpd502bcs5.narinfo
Resolving ci.guix.gnu.org (ci.guix.gnu.org)... 141.80.181.40
Connecting to ci.guix.gnu.org (ci.guix.gnu.org)|141.80.181.40|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-05-22 11:00:08 ERROR 404: Not Found.
$ wget -O- https://bordeaux.guix.gnu.org/apw1y9nf8rqgxvjnlr1isbhpd502bcs5.narinfo
--2023-05-22 11:03:36-- https://bordeaux.guix.gnu.org/apw1y9nf8rqgxvjnlr1isbhpd502bcs5.narinfo
Resolving bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)... 185.233.100.56, 2a0c:e300::58
Connecting to bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)|185.233.100.56|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1715 (1.7K) [text/plain]
Saving to: ‘STDOUT’

- 0%[ ] 0 --.-KB/s StorePath: /gnu/store/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0
NarHash: sha256:1am3v254jr9dpgd9r4dkjqxfgbp1xm0vl1hxl49cmmh97dg117z8
NarSize: 144790464
References: 8rz7yh6zdzp4b78f4n9wqj3hav2md4d4-isl-0.24 930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0 brzrf6qx5d83j1zvirc1xk14wrhyx4hf-binutils-cross-aarch64-linux-gnu-2.38 cinj0g8krh5s428jawsazawlfarp912k-gcc-cross-aarch64-linux-gnu-11.3.0-lib cs3hw1wnxgijjzsd61whc8ar3qy9wjd6-mpfr-4.2.0 gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35 ib2n2vzqpchc3bhh9i712w5sq9zapn8d-gmp-6.2.1 rbjq50q2ik8c1glkj5f0ksnwfz64g16g-mpc-1.3.1 rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16 rjgpaj54q1q9n3hcvpzyp29fgi2nx6ls-ld-wrapper-aarch64-linux-gnu-0 slzq3zqwj75lbrg4ly51hfhbv2vhryv5-zlib-1.2.13 y4ndp9n5krkaiwgvw58c61n7h8ls9b0f-glibc-cross-aarch64-linux-gnu-2.35
System: x86_64-linux
Deriver: fqb09jygm70cmq03xmjm7al6gzajp1p5-gcc-cross-aarch64-linux-gnu-11.3.0.drv
Signature: 1;bayfront;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjNzA2MTI1QUFGMjcxQzM2MDM3RkY2MTIwMzI0NzI2NTVDMkU3OEY2NTA3OUY0MzgxMzY4NDhDQzA0OTVFRTIxRiMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMDg1RTA2ODE3RTFENTdCQzE1NTAxOUNGNjA0QTQxMTlFRjNBREVGQ0VFOTE4NjQzMDIzQTExQzk4RDM3REVBOCMpCiAgIChzICMwMUIwMkQyNjI0MDhCMDIzMUMxMUVDODI1Q0Q2MTkzNTU5NDRCQUNBMzg4NjEwRDIxOEIzNTVCNjM1NjFCQjk3IykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjN0Q2MDI5MDJEM0EyREJCODNGOEEwRkI5ODYwMkE3NTRDNTQ5M0IwQjc3OEM4RDFERDRFMEY0MURFMTRERTM0RiMpCiAgICkKICApCiApCg==
URL: nar/lzip/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0
Compression: lzip
FileSize: 24767829
- 100%[==================================================================>] 1.67K --.-KB/s in 0s

2023-05-22 11:03:36 (20.6 MB/s) - written to stdout [1715/1715]

$ wget -O/dev/null https://bordeaux.guix.gnu.org/nar/lzip/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0
--2023-05-22 11:04:02-- https://bordeaux.guix.gnu.org/nar/lzip/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0
Resolving bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)... 185.233.100.56, 2a0c:e300::58
Connecting to bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)|185.233.100.56|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24767829 (24M) [text/plain]
Saving to: ‘/dev/null’

/dev/null 100%[==================================================================>] 23.62M 44.8MB/s in 0.5s

2023-05-22 11:04:03 (44.8 MB/s) - ‘/dev/null’ saved [24767829/24767829]

$ wget -O/dev/null https://bordeaux.guix.gnu.org/nar/zstd/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0
--2023-05-22 11:04:28-- https://bordeaux.guix.gnu.org/nar/zstd/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0
Resolving bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)... 185.233.100.56, 2a0c:e300::58
Connecting to bordeaux.guix.gnu.org (bordeaux.guix.gnu.org)|185.233.100.56|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-05-22 11:04:28 ERROR 404: Not Found.

So it seems ‘guix substitute’ picked /nar/zstd, even though
bordeaux.guix is not advertising that.

Or is it?

Toggle snippet (4 lines)
$ sudo cat /var/guix/substitute/cache/kzwjeblndsbkjzmjailrt4bnhguil7tqjmewzcyw22hgajbhfy3q/apw1y9nf8rqgxvjnlr1isbhpd502bcs5
(narinfo (version 2) (cache-uri "https://bordeaux.guix.gnu.org") (date 1683633757) (ttl 15552000) (value "StorePath: /gnu/store/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0\nNarHash: sha256:1am3v254jr9dpgd9r4dkjqxfgbp1xm0vl1hxl49cmmh97dg117z8\nNarSize: 144790464\nReferences: 8rz7yh6zdzp4b78f4n9wqj3hav2md4d4-isl-0.24 930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0 brzrf6qx5d83j1zvirc1xk14wrhyx4hf-binutils-cross-aarch64-linux-gnu-2.38 cinj0g8krh5s428jawsazawlfarp912k-gcc-cross-aarch64-linux-gnu-11.3.0-lib cs3hw1wnxgijjzsd61whc8ar3qy9wjd6-mpfr-4.2.0 gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35 ib2n2vzqpchc3bhh9i712w5sq9zapn8d-gmp-6.2.1 rbjq50q2ik8c1glkj5f0ksnwfz64g16g-mpc-1.3.1 rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16 rjgpaj54q1q9n3hcvpzyp29fgi2nx6ls-ld-wrapper-aarch64-linux-gnu-0 slzq3zqwj75lbrg4ly51hfhbv2vhryv5-zlib-1.2.13 y4ndp9n5krkaiwgvw58c61n7h8ls9b0f-glibc-cross-aarch64-linux-gnu-2.35\nSystem: x86_64-linux\nDeriver: fqb09jygm70cmq03xmjm7al6gzajp1p5-gcc-cross-aarch64-linux-gnu-11.3.0.drv\nSignature: 1;bayfront;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjNzA2MTI1QUFGMjcxQzM2MDM3RkY2MTIwMzI0NzI2NTVDMkU3OEY2NTA3OUY0MzgxMzY4NDhDQzA0OTVFRTIxRiMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMDg1RTA2ODE3RTFENTdCQzE1NTAxOUNGNjA0QTQxMTlFRjNBREVGQ0VFOTE4NjQzMDIzQTExQzk4RDM3REVBOCMpCiAgIChzICMwMUIwMkQyNjI0MDhCMDIzMUMxMUVDODI1Q0Q2MTkzNTU5NDRCQUNBMzg4NjEwRDIxOEIzNTVCNjM1NjFCQjk3IykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjN0Q2MDI5MDJEM0EyREJCODNGOEEwRkI5ODYwMkE3NTRDNTQ5M0IwQjc3OEM4RDFERDRFMEY0MURFMTRERTM0RiMpCiAgICkKICApCiApCg==\nURL: nar/lzip/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0\nCompression: lzip\nFileSize: 24767829\nURL: nar/zstd/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0\nCompression: zstd\nFileSize: 39822104"))ludo@ribbon ~/src/guix [env]$

So it seems that bordeaux.guix did advertise zstd at some point (and
that narinfo is still in cache) but no longer does.

Chris, can you confirm?

I’ll see how to get ‘guix substitute’ to handle that gracefully.

Ludo’.
L
L
Ludovic Courtès wrote on 22 May 2023 11:36
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
875y8kzp17.fsf@gnu.org
Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (2 lines)
Look:

Toggle snippet (13 lines)
$ wget -qO- https://bordeaux.guix.gnu.org/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x.narinfo
StorePath: /gnu/store/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz
NarHash: sha256:0pynnlsiiq03h7ysfkzw53wkf6hfmplb63haz6bjqdc686rnf4q1
NarSize: 49128
References:
System: x86_64-linux
Deriver: qcpzwblzpa2wprc4vwlm128bwckfj8rj-rust-base64-0.13.1.tar.xz.drv
Signature: 1;bayfront;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjRUE2MTcwNUZENjNCOTU1NzcwNjIxN0EyQjczN0U5NzI2RjM4Rjk1NkQwRTRCQjI4MjVBRTA2MTdDQjcxMDQ1MyMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMDNGRjU3ODU1RTNERDMxQkJFMUY2NDEyMkU0MzNBNzcwMEZGNkI1ODExRUQ0NTBCQzVEQTcwMkQyNDk3ODZEMCMpCiAgIChzICMwN0ZCQjREN0YwNTExQTlBNzZENzBFNzUxQTc2NTM1MzE0MzU1Mzk1RUUwNzgzODU3REM2NEYxRDIxMTEzQjMxIykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjN0Q2MDI5MDJEM0EyREJCODNGOEEwRkI5ODYwMkE3NTRDNTQ5M0IwQjc3OEM4RDFERDRFMEY0MURFMTRERTM0RiMpCiAgICkKICApCiApCg==
URL: nar/lzip/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz
Compression: lzip
FileSize: 49781

See? zstd’s gone! This confirms my hypothesis.

BTW, ‘guix publish’ has provisions to not compress already-compressed
files, as in the case above. We should really share code, Chris.

Ludo’.
C
C
Christopher Baines wrote on 22 May 2023 12:59
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 63634@debbugs.gnu.org)
87fs7ofx3v.fsf@cbaines.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (13 lines)
> So it seems ‘guix substitute’ picked /nar/zstd, even though
> bordeaux.guix is not advertising that.
>
> Or is it?
>
> $ sudo cat /var/guix/substitute/cache/kzwjeblndsbkjzmjailrt4bnhguil7tqjmewzcyw22hgajbhfy3q/apw1y9nf8rqgxvjnlr1isbhpd502bcs5
> (narinfo (version 2) (cache-uri "https://bordeaux.guix.gnu.org") (date 1683633757) (ttl 15552000) (value "StorePath: /gnu/store/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0\nNarHash: sha256:1am3v254jr9dpgd9r4dkjqxfgbp1xm0vl1hxl49cmmh97dg117z8\nNarSize: 144790464\nReferences: 8rz7yh6zdzp4b78f4n9wqj3hav2md4d4-isl-0.24 930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0 brzrf6qx5d83j1zvirc1xk14wrhyx4hf-binutils-cross-aarch64-linux-gnu-2.38 cinj0g8krh5s428jawsazawlfarp912k-gcc-cross-aarch64-linux-gnu-11.3.0-lib cs3hw1wnxgijjzsd61whc8ar3qy9wjd6-mpfr-4.2.0 gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35 ib2n2vzqpchc3bhh9i712w5sq9zapn8d-gmp-6.2.1 rbjq50q2ik8c1glkj5f0ksnwfz64g16g-mpc-1.3.1 rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16 rjgpaj54q1q9n3hcvpzyp29fgi2nx6ls-ld-wrapper-aarch64-linux-gnu-0 slzq3zqwj75lbrg4ly51hfhbv2vhryv5-zlib-1.2.13 y4ndp9n5krkaiwgvw58c61n7h8ls9b0f-glibc-cross-aarch64-linux-gnu-2.35\nSystem: x86_64-linux\nDeriver: fqb09jygm70cmq03xmjm7al6gzajp1p5-gcc-cross-aarch64-linux-gnu-11.3.0.drv\nSignature: 1;bayfront;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjNzA2MTI1QUFGMjcxQzM2MDM3RkY2MTIwMzI0NzI2NTVDMkU3OEY2NTA3OUY0MzgxMzY4NDhDQzA0OTVFRTIxRiMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMDg1RTA2ODE3RTFENTdCQzE1NTAxOUNGNjA0QTQxMTlFRjNBREVGQ0VFOTE4NjQzMDIzQTExQzk4RDM3REVBOCMpCiAgIChzICMwMUIwMkQyNjI0MDhCMDIzMUMxMUVDODI1Q0Q2MTkzNTU5NDRCQUNBMzg4NjEwRDIxOEIzNTVCNjM1NjFCQjk3IykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjN0Q2MDI5MDJEM0EyREJCODNGOEEwRkI5ODYwMkE3NTRDNTQ5M0IwQjc3OEM4RDFERDRFMEY0MURFMTRERTM0RiMpCiAgICkKICApCiApCg==\nURL: nar/lzip/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0\nCompression: lzip\nFileSize: 24767829\nURL: nar/zstd/apw1y9nf8rqgxvjnlr1isbhpd502bcs5-gcc-cross-aarch64-linux-gnu-11.3.0\nCompression: zstd\nFileSize: 39822104"))ludo@ribbon ~/src/guix [env]$
>
> So it seems that bordeaux.guix did advertise zstd at some point (and
> that narinfo is still in cache) but no longer does.
>
> Chris, can you confirm?

Yeah, this would have been a consequence of reducing the cache size [1].

-----BEGIN PGP SIGNATURE-----

iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmRrS6RfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9XdLABAAije59Xh6APY5MdwkEXM/z3LnWenZlRRw
ocUKTVT6cIrJdGNsXq1tA6rJTh7+ycp0TK72FbqlCaPUl3ZRBSioS+GUxQN8ptf3
7VIgwWFECYbRgjVQGS5Lb/7AUP1nM2EEeMq395bWe1FdogzC9uu1dqnQm6jvGPBU
fw5XeiniELGhiyNo3vplWAHssnyVSeNhbJJ0ZlMnRsYFMLxH/LgFT9Z04u83TD/Y
nfnv1/PWhkGJiNYes/Gqvfh6lmcWohKIu9NxZMW3duzOtjRaReZ6j49Q7Ie6kF0L
iZa6A9t26CPQurtTPDH5EiUjHlnSzlEW+kvHQu0LZ91mk/ivkaCFY3S320jXIG0A
RBX5XHRMkTvZo874Lp6WbVQ42OYCwVbehlngkfWyw6p+UQ6dlIgH35UfhU239C74
TZxHX51utmwQ5LYEVBfIcPbQVeqwaL08sLsnDLjBSm+36+TdpafB8qTMiDxJysA3
X73c/vtBLHkVCM6gTY4ZrSQWhkCLMqoZe03eWrrExqnvZ+exBMqpharZr7pbNUD/
d05E77fhbk59zFRI7TjcjZ3gfBISu6mgKi4RlgZmJw3HlawXR9Ce5cZMyTHFz0nv
0QyJegXF2UvAfqmf/s0fQEtAzrj+MaRXSmGUoPxlQNKdg8TsfDTN4Stde/4x73A8
HvL2KzFl6wE=
=HFdK
-----END PGP SIGNATURE-----

C
C
Christopher Baines wrote on 22 May 2023 13:02
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 63634@debbugs.gnu.org)
87bkicfw6e.fsf@cbaines.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (25 lines)
> Simon Tournier <zimon.toutoune@gmail.com> skribis:
>
>> 2. &message:
> "https://bordeaux.guix.gnu.org/nar/zstd/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz:
> HTTP download failed: 404 (\"Not Found\")"
>
> Look:
>
> $ wget -qO- https://bordeaux.guix.gnu.org/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x.narinfo
> StorePath: /gnu/store/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz
> NarHash: sha256:0pynnlsiiq03h7ysfkzw53wkf6hfmplb63haz6bjqdc686rnf4q1
> NarSize: 49128
> References:
> System: x86_64-linux
> Deriver: qcpzwblzpa2wprc4vwlm128bwckfj8rj-rust-base64-0.13.1.tar.xz.drv
> Signature: 1;bayfront;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjRUE2MTcwNUZENjNCOTU1NzcwNjIxN0EyQjczN0U5NzI2RjM4Rjk1NkQwRTRCQjI4MjVBRTA2MTdDQjcxMDQ1MyMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMDNGRjU3ODU1RTNERDMxQkJFMUY2NDEyMkU0MzNBNzcwMEZGNkI1ODExRUQ0NTBCQzVEQTcwMkQyNDk3ODZEMCMpCiAgIChzICMwN0ZCQjREN0YwNTExQTlBNzZENzBFNzUxQTc2NTM1MzE0MzU1Mzk1RUUwNzgzODU3REM2NEYxRDIxMTEzQjMxIykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjN0Q2MDI5MDJEM0EyREJCODNGOEEwRkI5ODYwMkE3NTRDNTQ5M0IwQjc3OEM4RDFERDRFMEY0MURFMTRERTM0RiMpCiAgICkKICApCiApCg==
> URL: nar/lzip/sx6sr6cs1x8sf5jhgb65rcr1yxk1q75x-rust-base64-0.13.1.tar.xz
> Compression: lzip
> FileSize: 49781
>
> See? zstd’s gone! This confirms my hypothesis.
>
> BTW, ‘guix publish’ has provisions to not compress already-compressed
> files, as in the case above. We should really share code, Chris.

I'm not sure if this is what you have in mind regarding code sharing,
but I'll use compressed-file? from (guix utils) in the nar-herder to
avoid generating cached compressions for these kind of files in the
future.
-----BEGIN PGP SIGNATURE-----

iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmRrUFlfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh
aW5lcy5uZXQACgkQXiijOwuE9Xc+og//WlWzxg4dlvECpvED5e1mBK1ieJ2xJMD0
yzfItUsFMlvHXfh10WuD8Fb5sVUNkoQHJx8AkuAIlRp77Tg57R42WgiQasKrWfC3
DrBfP4EkO7UMiQHDHEV9nrMCr45sThEwuM5ZM1hdWvWdd/nwXpyCusZhVu50l6Tl
CKUWltM1e9CNwCP4HX+PIFe/U96l8xY4Ai2p6jLYsEUpWax+ja3y1zLW3o5qJJcP
MA8Y2+44aChCmrneWL6oxGLCCAuIPEtMfRq15yIhzJBVw6A+KY46HVy61m5F7uw4
QJ5G9NKQEX3/GmkHsWPARcuO5pl9bo+eOCmR/6Y9Hs1S4kPRteC+La62KLcDMHO6
BV1Q2zh/9ImbVnwubz5/cb3+P7ItAUCUeAvWG+ZnIo/fBPVWWbSv0Gs6eh5t/h84
wbKVkIhits5rp6YGVsuxGDE1pPPmhV3J3GF3yLHLO247Rwma9iModfZNmGd71oeP
szBdjABikAROvtpsdduk/OeAvK0riVgshv3U1fIM3XaeyNW/mvAk8fzdknVz8bLU
6Lc7A+Vq9s6LUjVsEELObqfTZODL+eZLjSyckCPDQN5FjfaoAiRzLDUtmHeA7CQK
JwdukPD3uRL6AmjipLzytJc+Jmu8GbgUgZ8mkPtjC7uZpXBiMsl+zDT1/avkayFb
YTIzGhpdb+o=
=qq8l
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 22 May 2023 13:51
(name . Christopher Baines)(address . mail@cbaines.net)(address . 63634@debbugs.gnu.org)
87y1lgy46x.fsf@gnu.org
Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (2 lines)
> Ludovic Courtès <ludo@gnu.org> writes:

[...]

Toggle quote (8 lines)
>> BTW, ‘guix publish’ has provisions to not compress already-compressed
>> files, as in the case above. We should really share code, Chris.
>
> I'm not sure if this is what you have in mind regarding code sharing,
> but I'll use compressed-file? from (guix utils) in the nar-herder to
> avoid generating cached compressions for these kind of files in the
> future.

Great, thank you.

Ideally we’d have a single ‘guix publish’ implementation, or at least
most of it would be a module shared by both ‘guix publish’ and
‘nar-herder’ because it’s tricky enough that we’d rather not have two of
them. The (guix narinfo) module you introduced is a step in that
direction and as discussed earlier, we need to go further.

Ludo’.
L
L
Ludovic Courtès wrote on 22 May 2023 13:54
(name . Christopher Baines)(address . mail@cbaines.net)(address . 63634@debbugs.gnu.org)
87mt1wy41m.fsf@gnu.org
Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (2 lines)
> Ludovic Courtès <ludo@gnu.org> writes:

[...]

Toggle quote (9 lines)
>> So it seems that bordeaux.guix did advertise zstd at some point (and
>> that narinfo is still in cache) but no longer does.
>>
>> Chris, can you confirm?
>
> Yeah, this would have been a consequence of reducing the cache size [1].
>
> 1: https://git.savannah.gnu.org/cgit/guix/maintenance.git/commit/?id=ce8d3000fda6b80c0cf3e6f8204e0ee293024e6b

As discussed on IRC, I think nar-herder should honor the TTL it
advertises for all the URLs of the narinfo (lzip, zstd, etc.) because
that’s how ‘guix substitute’ currently interprets the TTL.

Eventually it would be nice for ‘guix substitute’ to interpret the TTL
as applying to one of the URLs (not necessarily all of them), but this
kind of change takes months if not years to be widely deployed because
users rarely (if ever) update guix-daemon.

Ludo’.
L
L
Ludovic Courtès wrote on 22 May 2023 17:16
878rdgwg4a.fsf@gnu.org
Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (25 lines)
> 646:2 10 (process-substitution _ _ _ #:cache-urls _ #:acl _ # _ # …)
> In ice-9/boot-9.scm:
> 1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/scripts/substitute.scm:
> 463:7 8 (download-nar #<<narinfo> path: "/gnu/store/dg0rm1mp9q…> …)
> In ice-9/boot-9.scm:
> 1747:15 7 (with-exception-handler #<procedure 7f6df0fa71e0 at ic…> …)
> 1685:16 6 (raise-exception _ #:continuable? _)
> 1683:16 5 (raise-exception _ #:continuable? _)
> 1685:16 4 (raise-exception _ #:continuable? _)
> 1780:13 3 (_ #<&compound-exception components: (#<&error> #<&irri…>)
> 1685:16 2 (raise-exception _ #:continuable? _)
> 1683:16 1 (raise-exception _ #:continuable? _)
> 1685:16 0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> ERROR:
> 1. &http-get-error:
> uri: #<<uri> scheme: https userinfo: #f host: "bordeaux.guix.gnu.org" port: #f path: "/nar/zstd/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1" query: #f fragment: #f>
> code: 404
> reason: "Not Found"
> headers: ((server . "nginx") (date . #<date nanosecond: 0 second: 39 minute: 59 hour: 21 day: 21 month: 5 year: 2023 zone-offset: 0>) (content-type text/plain (charset . "utf-8")) (content-length . 3) (connection keep-alive))
> 2. &message: "https://bordeaux.guix.gnu.org/nar/zstd/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1: HTTP download failed: 404 (\"Not Found\")"
> substitution of /gnu/store/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1 failed

[...]

Toggle quote (3 lines)
> This might be related to 8bd4126917f59f4af9a4323c3d5699201862dca2. The
> two ‘download-nar’ calls are guarded against ‘http-get-error?’ though.

That the exception went through despite the ‘network-error?’ guard is
fixed by 3f59fd6d114548480c719d4b8f8509bdf3e8dcca.

The reason was that the ‘&http-get-error’ condition was getting wrapped
in a ‘kind-and-args’ exception upon rethrow, as illustrated here:

Toggle snippet (12 lines)
scheme@(guile-user)> (catch #t
(lambda () (raise-exception (condition (&message (message "boo")))))
(lambda args args))
$5 = (%exception #<&message message: "boo">)
scheme@(guile-user)> (with-exception-handler (lambda (e) e)
(lambda () (apply throw $5))
#:unwind? #t)
$6 = #<&compound-exception components: (#<&error> #<&irritants irritants: (#<&message message: "boo">)> #<&exception-with-kind-and-args kind: %exception args: (#<&message message: "boo">)>)>
scheme@(guile-user)> (message-condition? $6)
$7 = #f

Ludo’.
L
L
Ludovic Courtès wrote on 22 May 2023 17:35
(address . 63634@debbugs.gnu.org)(name . Christopher Baines)(address . mail@cbaines.net)
87zg5wv0pi.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (3 lines)
> So it seems that bordeaux.guix did advertise zstd at some point (and
> that narinfo is still in cache) but no longer does.

Here’s a fix:


Ludo’.
L
L
Ludovic Courtès wrote on 30 May 2023 00:19
Re: bug#63646: [PATCH] substitute: If a server's nar URL is 404, try the next one(s).
87cz2ieq78.fsf_-_@gnu.org
Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (14 lines)
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> + (define (try-fetch choices)
>> + (match choices
>> + (((uri compression file-size) rest ...)
>> + (guard (c ((and (pair? rest) (network-error? c))
>> + (warning (G_ "download from '~a' failed, trying next URL~%")
>> + (uri->string uri))
>
> I realized we can change ‘network-error?’ to ‘http-get-error?’ above.
> Otherwise, we could find ourselves trying several nar URLs on the same
> server when the error is ETIMEDOUT or ECONNREFUSED, which would be a
> waste of time.

Pushed as 8af9a2aa5fa2fa5b00234c1cbe12e9aff60888a0.

Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 63634@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 63634
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch