Issues with issues.guix.gnu.org (502 Bad Gateway)

  • Open
  • quality assurance status badge
Details
5 participants
  • Andreas Enge
  • Arun Isaac
  • Christina O'Donnell
  • Felix Lechner
  • Maxim Cournoyer
Owner
unassigned
Submitted by
Christina O'Donnell
Severity
normal

Debbugs page

Christina O'Donnell wrote 1 years ago
4dd113ef-8bcf-c48b-548f-29b23b87cfbe@mutix.org
Hi Guix,

From my machine[1] connecting to https://issues.guix.gnu.org/results
in, after 130 seconds[2], a 502 bad gateway. It's been having issues for
over a week, but I only just found a need to test it.

I couldn't see an issue about it on debbugs so I thought it prudent to
raise an issue.

Reproduced with curl, firefox, and ungoogled-chromium on the main page
and on specific issues.

It appears to be under a lot of load.

Kind regards,
 - Christina

[1] based in Cambridge, UK, and via my VPS in Dusseldorf, Germany on
2024-02-03 19:30 UTC+0

[2] Rather consistently 130-140 seconds from both UK and Dusseldorf.
Apparently the server is in Berlin.
Felix Lechner wrote 1 years ago
(name . Christina O'Donnell)(address . cdo@mutix.org)
87eddsbxpm.fsf@lease-up.com
Hi Christina,

On Sat, Feb 03 2024, Christina O'Donnell wrote:

Toggle quote (3 lines)
> connecting to https://issues.guix.gnu.org/results in ... a 502 bad
> gateway.

You are welcome to use my Mumi clone at mumi.juix.org.

Kind regards
Felix
Christina O'Donnell wrote 1 years ago
(name . Felix Lechner)(address . felix.lechner@lease-up.com)
a46ed8f0-d870-39b4-bc10-1283654c4c06@mutix.org
Hi Felix,

Toggle quote (1 lines)
> You are welcome to use my Mumi clone at mumi.juix.org.
Bookmarked, thanks!

Looks like issues.guix.gnu.org is back up again.

Kind regards,
 - Christina

On 04/02/2024 05:33, Felix Lechner wrote:
Toggle quote (10 lines)
> Hi Christina,
>
> On Sat, Feb 03 2024, Christina O'Donnell wrote:
>
>> connecting to https://issues.guix.gnu.org/ results in ... a 502 bad
>> gateway.
>
>
> Kind regards
> Felix
Maxim Cournoyer wrote 1 years ago
Re: bug#68920: Issues with issues.guix.gnu.org (502 Bad Gateway)
(name . Christina O'Donnell)(address . cdo@mutix.org)
87fry54qq4.fsf@gmail.com
Hi Christina,

Christina O'Donnell <cdo@mutix.org> writes:

Toggle quote (14 lines)
> Hi Guix,
>
> From my machine[1] connecting to https://issues.guix.gnu.org/ results
> in, after 130 seconds[2], a 502 bad gateway. It's been having issues
> for over a week, but I only just found a need to test it.
>
> I couldn't see an issue about it on debbugs so I thought it prudent to
> raise an issue.
>
> Reproduced with curl, firefox, and ungoogled-chromium on the main page
> and on specific issues.
>
> It appears to be under a lot of load.

Thanks for the report. It occurred a few times in the past weeks, where
the 'mumi' service had to be restarted on Berlin. Let's keep this open
to see if it'll occur again. Otherwise I'll close it in a week or two.

--
Thanks,
Maxim
Andreas Enge wrote 1 years ago
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
ZcJWf3vFAyj3qUX1@jurong
Am Tue, Feb 06, 2024 at 09:22:43AM -0500 schrieb Maxim Cournoyer:
Toggle quote (4 lines)
> Thanks for the report. It occurred a few times in the past weeks, where
> the 'mumi' service had to be restarted on Berlin. Let's keep this open
> to see if it'll occur again. Otherwise I'll close it in a week or two.

It has happened for me last evening, so it does not seem to be definitely
fixed.

Andreas
Maxim Cournoyer wrote 1 years ago
(name . Andreas Enge)(address . andreas@enge.fr)
87zfwd2h7g.fsf@gmail.com
Hi,

Andreas Enge <andreas@enge.fr> writes:

Toggle quote (10 lines)
> Am Tue, Feb 06, 2024 at 09:22:43AM -0500 schrieb Maxim Cournoyer:
>> Thanks for the report. It occurred a few times in the past weeks, where
>> the 'mumi' service had to be restarted on Berlin. Let's keep this open
>> to see if it'll occur again. Otherwise I'll close it in a week or two.
>
> It has happened for me last evening, so it does not seem to be definitely
> fixed.
>
> Andreas

OK. Next time it happens, let's try to retrieve /var/log/mumi.log,
perhaps a strace of its process, and post it here.

--
Thanks,
Maxim
Felix Lechner wrote 1 years ago
(no subject)
(address . control@debbugs.gnu.org)
87h6ii6d58.fsf@lease-up.com
reassign 65858 mumi
reassign 65809 mumi
reassign 49295 mumi
reassign 64833 mumi
reassign 54204 mumi
reassign 68920 mumi
reassign 55765 mumi
reassign 65923 mumi
reassign 63507 mumi
reassign 60226 mumi
reassign 63936 mumi
reassign 60951 mumi
reassign 66027 mumi
thanks
Arun Isaac wrote 44 hours ago
Re: Issues with issues.guix.gnu.org (502 Bad Gateway)
(address . 68920@debbugs.gnu.org)
875xkgri03.fsf@systemreboot.net
Hi,

I have tried load-testing issues.guix.gnu.org using siege[1] and these 502
errors show up when mumi is under some load. I have now added nginx
caching[2] of mumi endpoints to help mitigate this. It's not a perfect
solution, but this should be a lot less likely to occur now. I'll close
this issue in a month unless it shows up again.

[2]:

If this occurs again in the future, we should consider adding more mumi
processes so that nginx will always have at least one mumi process to
reverse proxy to. We could also consider pre-generating the issue pages
and serving them directly with nginx.

Regards,
Arun
Arun Isaac wrote 44 hours ago
Manipulate bug 68920
(address . control@debbugs.gnu.org)
8734fkrhzg.fsf@systemreboot.net
tags 68920 + fixed
thanks
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 68920@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 68920
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help