'guix substitute' returns a 504 error without trying to use other servers

  • Open
  • quality assurance status badge
Details
2 participants
  • Clément Lassieur
  • Ludovic Courtès
Owner
unassigned
Submitted by
Clément Lassieur
Severity
normal
C
C
Clément Lassieur wrote on 25 Oct 2018 17:31
(address . bug-guix@gnu.org)
87h8haf3ft.fsf@lassieur.org
Hi,

I get a 504 error while installing Octave, even though Octave is built
on my substitute server (cuirass.lassieur.org), and I cleared the cache.

I think it should automatically fallback on another substitute server if
one of them is down.

Toggle snippet (32 lines)
substitute: updating substitutes from 'https://cuirass.lassieur.org'... 100.0%
substitute: updating substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bayfront.guixsd.org'... 100.0%
substitute: updating substitutes from 'https://cuirass.lassieur.org'... 100.0%
substitute: updating substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bayfront.guixsd.org'... 100.0%
substitute: updating substitutes from 'https://berlin.guixsd.org'... 100.0%
The following derivations will be built:
/gnu/store/nk57a8qz74xnxqd2imxkgsxf2wynhdh9-profile.drv
/gnu/store/zb57j7xxqwp551xcq7v703334hq2kcvw-fonts-dir.drv
/gnu/store/w23wv5s58p1amjyxbdwmvmvzqzl5s7kh-xdg-mime-database.drv
/gnu/store/qw3f7ardwva1r0vcxibv3wpbdhd6wd1z-xdg-desktop-database.drv
/gnu/store/nr8y1gsyw1a2sc033qgn4i2n2537akl9-ca-certificate-bundle.drv
/gnu/store/mf8jfsrg2324wjwh3c10lnbx0ch1amr0-gtk-im-modules.drv
/gnu/store/kkvsygfry98lnfl4j2m6c23lfvnvbf1x-glib-schemas.drv
/gnu/store/89lndcfdvd0k3y1gg2k4zmj4x9m2jvsl-gtk-icon-themes.drv
/gnu/store/7m8196z2sm1cbwi5hlq7m0w25d0cizb7-info-dir.drv
/gnu/store/i4my5nh2k8i4hc854s8lig4qfcccnsvz-graphviz-2.40.1.drv
/gnu/store/3xi9isbhzzwyvw0cc5j2da6cbnic7z89-graphicsmagick-1.3.30.drv
/gnu/store/9z6fsci062frwg5gjylbk916vxbx4y7q-gl2ps-1.4.0.drv
/gnu/store/2x7lfxnz8kjxpki7myffingc4falxg6r-manual-database.drv
/gnu/store/3fyvy9i8i6j7rk1f5rpxvrp3yp4hfiaj-octave-4.4.1.drv
20,8 MB will be downloaded:
/gnu/store/9s72xm4slmqjn62j82y036i730pkf5sa-octave-4.4.1
/gnu/store/7hv88pwzwwcasvfgf80kjajkglpz8mgv-suitesparse-4.5.5
building /gnu/store/9z6fsci062frwg5gjylbk916vxbx4y7q-gl2ps-1.4.0.drv...
guix substitute: error: download from 'https://mirror.hydra.gnu.org/guix/nar/gzip/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0' failed: 504, "Gateway Time-out"
substitution of /gnu/store/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0 failed
building /gnu/store/i4my5nh2k8i4hc854s8lig4qfcccnsvz-graphviz-2.40.1.drv...
guix package: error: build failed: some substitutes for the outputs of derivation `/gnu/store/wj0y6b1cbqjhkfa074kgvgrbnz7ay9sw-octave-4.4.1.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source

Cheers,
Clément
L
L
Ludovic Courtès wrote on 26 Oct 2018 19:50
(name . Clément Lassieur)(address . clement@lassieur.org)(address . 33151@debbugs.gnu.org)
87h8h8d2c6.fsf@gnu.org
Hello!

Clément Lassieur <clement@lassieur.org> skribis:

Toggle quote (3 lines)
> guix substitute: error: download from 'https://mirror.hydra.gnu.org/guix/nar/gzip/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0'failed: 504, "Gateway Time-out"
> substitution of /gnu/store/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0 failed

The problem is that hydra.gnu.org is currently down¹ and
mirror.hydra.gnu.org is just proxying it.

What happens here is that
is available (cached), whereas the corresponding
is not.

I don’t have a good solution to this other than temporarily removing
mirror.hydra.gnu.org from your substitute URLs.

HTH,
Ludo’.

C
C
Clément Lassieur wrote on 27 Oct 2018 03:11
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33151@debbugs.gnu.org)
87ftws6vnc.fsf@lassieur.org
Hello Ludovic, thank you for this explanation!

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (23 lines)
> Hello!
>
> Clément Lassieur <clement@lassieur.org> skribis:
>
>> guix substitute: error: download from 'https://mirror.hydra.gnu.org/guix/nar/gzip/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0' failed: 504, "Gateway Time-out"
>> substitution of /gnu/store/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0 failed
>
> The problem is that hydra.gnu.org is currently down¹ and
> mirror.hydra.gnu.org is just proxying it.
>
> What happens here is that
> <https://mirror.hydra.gnu.org/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5.narinfo>
> is available (cached), whereas the corresponding
> <https://mirror.hydra.gnu.org/guix/nar/gzip/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0>
> is not.
>
> I don’t have a good solution to this other than temporarily removing
> mirror.hydra.gnu.org from your substitute URLs.
>
> HTH,
> Ludo’.
>
> ¹ https://lists.gnu.org/archive/html/guix-devel/2018-10/msg00434.html
L
L
Ludovic Courtès wrote on 3 Nov 2018 14:55
control message for bug #33151
(address . control@debbugs.gnu.org)
875zxeqn80.fsf@gnu.org
tags 33151 fixed
close 33151
C
C
Clément Lassieur wrote on 5 Nov 2018 11:53
Re: bug#33151: 'guix substitute' returns a 504 error without trying to use other servers
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33151@debbugs.gnu.org)
87muqnn6ck.fsf@lassieur.org
Hi Ludovic,

I don't understand why you closed the bug report. Your explanation of
the bug makes sense, but it doesn't mean that it's fixed.

I expect that when a substitute server is down, for whatever reason,
Guix should still work smoothly if there is another server set up.

If you think there is no way to fix it, maybe the tag "wontfix" would be
more appropriate?

Cheers,
Clément

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (23 lines)
> Hello!
>
> Clément Lassieur <clement@lassieur.org> skribis:
>
>> guix substitute: error: download from 'https://mirror.hydra.gnu.org/guix/nar/gzip/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0' failed: 504, "Gateway Time-out"
>> substitution of /gnu/store/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0 failed
>
> The problem is that hydra.gnu.org is currently down¹ and
> mirror.hydra.gnu.org is just proxying it.
>
> What happens here is that
> <https://mirror.hydra.gnu.org/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5.narinfo>
> is available (cached), whereas the corresponding
> <https://mirror.hydra.gnu.org/guix/nar/gzip/9xwrnm3vymsf9w17ayc7dfp5cl1aykq5-metis-5.1.0>
> is not.
>
> I don’t have a good solution to this other than temporarily removing
> mirror.hydra.gnu.org from your substitute URLs.
>
> HTH,
> Ludo’.
>
> ¹ https://lists.gnu.org/archive/html/guix-devel/2018-10/msg00434.html
L
L
Ludovic Courtès wrote on 5 Nov 2018 22:33
(name . Clément Lassieur)(address . clement@lassieur.org)(address . 33151@debbugs.gnu.org)
87zhunw6oz.fsf@gnu.org
Hello Clément,

Clément Lassieur <clement@lassieur.org> skribis:

Toggle quote (9 lines)
> I don't understand why you closed the bug report. Your explanation of
> the bug makes sense, but it doesn't mean that it's fixed.
>
> I expect that when a substitute server is down, for whatever reason,
> Guix should still work smoothly if there is another server set up.
>
> If you think there is no way to fix it, maybe the tag "wontfix" would be
> more appropriate?

No I think you’re right, I shouldn’t have closed it.

I think my first thought was that the problem (narinfo available but
corresponding nar missing) was mostly due to our peculiar nginx setup on
mirror.hydra.gnu.org: this cannot happen under normal circumstances
because ‘guix publish’ really honors the expiry it sets on narinfos.

That said, we could fix ‘process-substitution’ in (guix scripts
substitute) to try other nar URLs (if any) upon failure.

IMO such a fix is not “high-priority” for the reason I gave above, but
OTOH it can be helpful to better cope with the sad reality of situations
like the one we’re in.

Thoughts?

Thanks for the heads-up, and sorry for hitting the “close” button too
quickly!

Ludo’.
C
C
Clément Lassieur wrote on 10 Nov 2018 15:55
control message for bug #33151
(address . control@debbugs.gnu.org)
87muqhvv6o.fsf@lassieur.org
tags 33151 - fixed
C
C
Clément Lassieur wrote on 10 Nov 2018 16:08
Re: bug#33151: 'guix substitute' returns a 504 error without trying to use other servers
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33151@debbugs.gnu.org)
87lg61vul8.fsf@lassieur.org
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (14 lines)
> I think my first thought was that the problem (narinfo available but
> corresponding nar missing) was mostly due to our peculiar nginx setup on
> mirror.hydra.gnu.org: this cannot happen under normal circumstances
> because ‘guix publish’ really honors the expiry it sets on narinfos.
>
> That said, we could fix ‘process-substitution’ in (guix scripts
> substitute) to try other nar URLs (if any) upon failure.
>
> IMO such a fix is not “high-priority” for the reason I gave above, but
> OTOH it can be helpful to better cope with the sad reality of situations
> like the one we’re in.
>
> Thoughts?

I agree that fixing 'process-substitution' would be nice even though
it's not a high priority issue.

Toggle quote (3 lines)
> Thanks for the heads-up, and sorry for hitting the “close” button too
> quickly!

No problem ;) I also removed the "fixed" tag.

Clément
?