Substitute server errors lead to client crashes

  • Done
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Ludovic Courtès
  • Maxime Devos
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
important
L
L
Ludovic Courtès wrote on 6 Jun 2022 14:48
(address . bug-guix@gnu.org)
87mteqc609.fsf@inria.fr
Here’s an example of a substitute server issue that leads to ugly client
crashes:

Toggle snippet (50 lines)
$ guix upgrade

[…]

guix substitute: warning: while fetching https://bordeaux.guix.gnu.org/nar/lzip/fan4wanq1138ds9pj0s63f2iv4qygng0-dstat-0.7.4: server is somewhat slow
guix substitute: warning: try `--no-substitutes' if the problem persists
Backtrace:
In ice-9/boot-9.scm:
1752:10 19 (with-exception-handler _ _ #:unwind? _ # _)
In unknown file:
18 (apply-smob/0 #<thunk 7f0e562b2f60>)
In ice-9/boot-9.scm:
724:2 17 (call-with-prompt _ _ #<procedure default-prompt-handleâ¦>)
In ice-9/eval.scm:
619:8 16 (_ #(#(#<directory (guile-user) 7f0e562acc80>)))
In guix/ui.scm:
2206:7 15 (run-guix . _)
2169:10 14 (run-guix-command _ . _)
In ice-9/boot-9.scm:
1752:10 13 (with-exception-handler _ _ #:unwind? _ # _)
1752:10 12 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
771:15 11 (_)
In ice-9/exceptions.scm:
406:15 10 (process-substitution #<output: file 4> _ "/gnu/store/â¦" â¦)
In ice-9/boot-9.scm:
1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/substitute.scm:
479:9 8 (_)
In ice-9/boot-9.scm:
1747:15 7 (with-exception-handler #<procedure 7f0e53925f00 at icâ¦> â¦)
1685:16 6 (raise-exception _ #:continuable? _)
1683:16 5 (raise-exception _ #:continuable? _)
1685:16 4 (raise-exception _ #:continuable? _)
1780:13 3 (_ #<&compound-exception components: (#<&error> #<&irriâ¦>)
1685:16 2 (raise-exception _ #:continuable? _)
1683:16 1 (raise-exception _ #:continuable? _)
1685:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
ERROR:
1. &http-get-error:
uri: #<<uri> scheme: https userinfo: #f host: "bordeaux.guix.gnu.org" port: #f path: "/nar/lzip/fan4wanq1138ds9pj0s63f2iv4qygng0-dstat-0.7.4" query: #f fragment: #f>
code: 502
reason: "Bad Gateway"
2. &message: "https://bordeaux.guix.gnu.org/nar/lzip/fan4wanq1138ds9pj0s63f2iv4qygng0-dstat-0.7.4: HTTP download failed: 502 (\"Bad Gateway\")"
substitution of /gnu/store/fan4wanq1138ds9pj0s63f2iv4qygng0-dstat-0.7.4 failed
guix upgrade: error: corrupt input while restoring archive from socket

This should be handled gracefully.

Also, in this case, ‘--fallback’ appears to have no effect.

Ludo’.
M
M
Maxime Devos wrote on 13 Jun 2022 04:02
344c89a58f395b81e09e46ba66f96e972da93387.camel@telenet.be
retitle 55932 Substitute server errors lead to client crashes
merge 55820 55392
thanks

-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYqaashccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hOGAP9rwa34D5Ua1mcXUnlgKGf/FIyU
V3ssUTUUcw3APs9TpAEAs0Bl5d3rm1JCzSWtdIEl7CRZcouTjFKtcJlYCDQ33AA=
=b3Sc
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 22 May 2023 09:57
control message for bug #55820
(address . control@debbugs.gnu.org)
87pm6sztm5.fsf@gnu.org
severity 55820 important
quit
L
L
Ludovic Courtès wrote on 22 May 2023 17:16
Re: bug#63634: nar 404 leads to hard ‘guix subs titute’ crash
878rdgwg4a.fsf@gnu.org
Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (25 lines)
> 646:2 10 (process-substitution _ _ _ #:cache-urls _ #:acl _ # _ # …)
> In ice-9/boot-9.scm:
> 1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/scripts/substitute.scm:
> 463:7 8 (download-nar #<<narinfo> path: "/gnu/store/dg0rm1mp9q…> …)
> In ice-9/boot-9.scm:
> 1747:15 7 (with-exception-handler #<procedure 7f6df0fa71e0 at ic…> …)
> 1685:16 6 (raise-exception _ #:continuable? _)
> 1683:16 5 (raise-exception _ #:continuable? _)
> 1685:16 4 (raise-exception _ #:continuable? _)
> 1780:13 3 (_ #<&compound-exception components: (#<&error> #<&irri…>)
> 1685:16 2 (raise-exception _ #:continuable? _)
> 1683:16 1 (raise-exception _ #:continuable? _)
> 1685:16 0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> ERROR:
> 1. &http-get-error:
> uri: #<<uri> scheme: https userinfo: #f host: "bordeaux.guix.gnu.org" port: #f path: "/nar/zstd/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1" query: #f fragment: #f>
> code: 404
> reason: "Not Found"
> headers: ((server . "nginx") (date . #<date nanosecond: 0 second: 39 minute: 59 hour: 21 day: 21 month: 5 year: 2023 zone-offset: 0>) (content-type text/plain (charset . "utf-8")) (content-length . 3) (connection keep-alive))
> 2. &message: "https://bordeaux.guix.gnu.org/nar/zstd/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1: HTTP download failed: 404 (\"Not Found\")"
> substitution of /gnu/store/dg0rm1mp9qvjhvlwra5mc8dz5pfky4hj-guile-colorized-0.1 failed

[...]

Toggle quote (3 lines)
> This might be related to 8bd4126917f59f4af9a4323c3d5699201862dca2. The
> two ‘download-nar’ calls are guarded against ‘http-get-error?’ though.

That the exception went through despite the ‘network-error?’ guard is
fixed by 3f59fd6d114548480c719d4b8f8509bdf3e8dcca.

The reason was that the ‘&http-get-error’ condition was getting wrapped
in a ‘kind-and-args’ exception upon rethrow, as illustrated here:

Toggle snippet (12 lines)
scheme@(guile-user)> (catch #t
(lambda () (raise-exception (condition (&message (message "boo")))))
(lambda args args))
$5 = (%exception #<&message message: "boo">)
scheme@(guile-user)> (with-exception-handler (lambda (e) e)
(lambda () (apply throw $5))
#:unwind? #t)
$6 = #<&compound-exception components: (#<&error> #<&irritants irritants: (#<&message message: "boo">)> #<&exception-with-kind-and-args kind: %exception args: (#<&message message: "boo">)>)>
scheme@(guile-user)> (message-condition? $6)
$7 = #f

Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 55820@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 55820
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch