Substitute server errors lead to client crashes

  • Done
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Ludovic Courtès
  • Maxime Devos
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
important
L
L
Ludovic Courtès wrote on 6 Jun 2022 14:48
(address . bug-guix@gnu.org)
87mteqc609.fsf@inria.fr
Here’s an example of a substitute server issue that leads to ugly client
crashes:

Toggle snippet (50 lines)
$ guix upgrade

[…]

guix substitute: warning: while fetching https://bordeaux.guix.gnu.org/nar/lzip/fan4wanq1138ds9pj0s63f2iv4qygng0-dstat-0.7.4: server is somewhat slow
guix substitute: warning: try `--no-substitutes' if the problem persists
Backtrace:
In ice-9/boot-9.scm:
1752:10 19 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
18 (apply-smob/0 #<thunk 7f0e562b2f60>)
In ice-9/boot-9.scm:
724:2 17 (call-with-prompt _ _ #<procedure default-prompt-handleâ¦>)
In ice-9/eval.scm:
619:8 16 (_ #(#(#<directory (guile-user) 7f0e562acc80>)))
In guix/ui.scm:
2206:7 15 (run-guix . _)
2169:10 14 (run-guix-command _ . _)
In ice-9/boot-9.scm:
1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
1752:10 12 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
771:15 11 (_)
In ice-9/exceptions.scm:
406:15 10 (process-substitution #<output: file 4> _ "/gnu/store/â¦" â¦)
In ice-9/boot-9.scm:
1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
479:9 8 (_)
In ice-9/boot-9.scm:
1747:15 7 (with-exception-handler #<procedure 7f0e53925f00 at icâ¦> â¦)
1685:16 6 (raise-exception _ #:continuable? _)
1683:16 5 (raise-exception _ #:continuable? _)
1685:16 4 (raise-exception _ #:continuable? _)
1780:13 3 (_ #<&compound-exception components: (#<&error> #<&irriâ¦>)
1685:16 2 (raise-exception _ #:continuable? _)
1683:16 1 (raise-exception _ #:continuable? _)
1685:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
ERROR:
1. &http-get-error:
uri: #<<uri> scheme: https userinfo: #f host: "bordeaux.guix.gnu.org" port: #f path: "/nar/lzip/fan4wanq1138ds9pj0s63f2iv4qygng0-dstat-0.7.4" query: #f fragment: #f>
code: 502
reason: "Bad Gateway"
2. &message: "https://bordeaux.guix.gnu.org/nar/lzip/fan4wanq1138ds9pj0s63f2iv4qygng0-dstat-0.7.4: HTTP download failed: 502 (\"Bad Gateway\")"
substitution of /gnu/store/fan4wanq1138ds9pj0s63f2iv4qygng0-dstat-0.7.4 failed
guix upgrade: error: corrupt input while restoring archive from socket

This should be handled gracefully.

Also, in this case, ‘--fallback’ appears to have no effect.

Ludo’.
M
M
Maxime Devos wrote on 13 Jun 2022 04:02
344c89a58f395b81e09e46ba66f96e972da93387.camel@telenet.be
retitle 55932 Substitute server errors lead to client crashes
merge 55820 55392
thanks

-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYqaashccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hOGAP9rwa34D5Ua1mcXUnlgKGf/FIyU
V3ssUTUUcw3APs9TpAEAs0Bl5d3rm1JCzSWtdIEl7CRZcouTjFKtcJlYCDQ33AA=
=b3Sc
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 22 May 2023 09:57
control message for bug #55820
(address . control@debbugs.gnu.org)
87pm6sztm5.fsf@gnu.org
severity 55820 important
quit
L
L
Ludovic Courtès wrote on 22 May 2023 17:16
Re: bug#63634: nar 404 leads to hard ‘guix subs titute’ crash
878rdgwg4a.fsf@gnu.org
Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (25 lines)
> 646:2 10 (process-substitution _ _ _ #:cache-urls _ #:acl _ # _ # …)
> In ice-9/boot-9.scm:
> 1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/scripts/substitute.scm:
> 463:7 8 (download-nar #<<narinfo> path: "/gnu/store/dg0rm1mp9q…> …)
> In ice-9/boot-9.scm:
> 1747:15 7 (with-exception-handler #<procedure 7f6df0fa71e0 at ic…> …)
> 1685:16 6 (raise-exception _ #:continuable? _)
> 1683:16 5 (raise-exception _ #:continuable? _)
> 1685:16 4 (raise-exception _ #:continuable? _)
> 1780:13 3 (_ #<&compound-exception components: (#<&error> #<&irri…>)
> 1685:16 2 (raise-exception _ #:continuable? _)
> 1683:16 1 (raise-exception _ #:continuable? _)
> 1685:16 0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> ERROR:
> 1. &http-get-error:
> uri: #<<uri> scheme: https userinfo: #f host: "bordeaux.guix.gnu.org" port: #f path: "/nar/zstd/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1" query: #f fragment: #f>
> code: 404
> reason: "Not Found"
> headers: ((server . "nginx") (date . #<date nanosecond: 0 second: 39 minute: 59 hour: 21 day: 21 month: 5 year: 2023 zone-offset: 0>) (content-type text/plain (charset . "utf-8")) (content-length . 3) (connection keep-alive))
> 2. &message: "https://bordeaux.guix.gnu.org/nar/zstd/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1: HTTP download failed: 404 (\"Not Found\")"
> substitution of /gnu/store/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1 failed

[...]

Toggle quote (3 lines)
> This might be related to 8bd4126917f59f4af9a4323c3d5699201862dca2. The
> two ‘download-nar’ calls are guarded against ‘http-get-error?’ though.

That the exception went through despite the ‘network-error?’ guard is
fixed by 3f59fd6d114548480c719d4b8f8509bdf3e8dcca.

The reason was that the ‘&http-get-error’ condition was getting wrapped
in a ‘kind-and-args’ exception upon rethrow, as illustrated here:

Toggle snippet (12 lines)
scheme@(guile-user)> (catch #t
(lambda () (raise-exception (condition (&message (message "boo")))))
(lambda args args))
$5 = (%exception #<&message message: "boo">)
scheme@(guile-user)> (with-exception-handler (lambda (e) e)
(lambda () (apply throw $5))
#:unwind? #t)
$6 = #<&compound-exception components: (#<&error> #<&irritants irritants: (#<&message message: "boo">)> #<&exception-with-kind-and-args kind: %exception args: (#<&message message: "boo">)>)>
scheme@(guile-user)> (message-condition? $6)
$7 = #f

Ludo’.
Closed
?