https://issues.guix.gnu.org/ cannot be accessed through Tor

  • Done
  • quality assurance status badge
Details
6 participants
  • Altadil
  • Giovanni Biscuolo
  • Ludovic Courtès
  • Maxim Cournoyer
  • Tobias Geerinckx-Rice
  • Ricardo Wurmus
Owner
unassigned
Submitted by
Altadil
Severity
serious
A
A
Altadil wrote on 4 Aug 2023 18:57
(name . bug-guix@gnu.org)(address . bug-guix@gnu.org)
KJ2uCegnng4HzKI2Y5vH1Lt2jO903C2yxw1gSl2d_W449j0amIS58PYlLxTmWsLRdIM1Vds02KhsCYDHtKXxFXqm6mbC4fOTtXXIvlewKJE=@protonmail.com
Hi,
it is no longer possible to get to the bug database at https://issues.guix.gnu.org/when using Tor Browser. The result is an error message saying:?"The connection has timed out".
It looks like a general block of Tor rather than a block of specific IPs, since attempting with different Tor circuits does not change the result.

Best regards,
Altadil
T
T
Tobias Geerinckx-Rice wrote on 4 Aug 2023 21:21
(name . Altadil)(address . Altadil@protonmail.com)(address . 65056@debbugs.gnu.org)
10e9eb490471d421fa709eaa36bdec83@tobias.gr
Hi Altadil,

On 2023-08-04 18:57, Altadil via Bug reports for GNU Guix wrote:
Toggle quote (3 lines)
> it is no longer possible to get to the bug database at
> https://issues.guix.gnu.org/ when using Tor Browser.

I forgot to mention this on IRC, but issues. is ‘simply’ a nicer unified
frontend to the venerable GNU Debbugs instance. You can use its own[1]
interface[2] as a work-around.

Toggle quote (4 lines)
> The result is an error message saying:?"The connection has timed out".
> It looks like a general block of Tor rather than a block of specific
> IPs, since attempting with different Tor circuits does not change the

The Guix project does not block Tor. If the datacentre has decided to
block Tor like it blocked most of Russia, there is little we can do but
ask them to reconsider.

Kind regards,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.

T
T
Tobias Geerinckx-Rice wrote on 4 Aug 2023 21:29
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
0bd1dfa9c3997becec68dc277b6c93e5@tobias.gr
On 2023-08-04 21:21, Tobias Geerinckx-Rice via Bug reports for GNU Guix
wrote:
Toggle quote (4 lines)
> The Guix project does not block Tor. If the datacentre has decided to
> block Tor like it blocked most of Russia, there is little we can do
> but ask them to reconsider.

Didn't mean to sound quite so fatalistic. We could always migrate
issues. to a different machine, like guix.gnu.org was, but it's not very
satisfying.

Kind regards,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.
L
L
Ludovic Courtès wrote on 13 Aug 2023 02:25
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
877cpzbxf4.fsf@gnu.org
Hi,

(Cc: guix-sysadmin.)

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

Toggle quote (2 lines)
> On 2023-08-04 18:57, Altadil via Bug reports for GNU Guix wrote:

[...]

Toggle quote (8 lines)
>> The result is an error message saying:?"The connection has timed out".
>> It looks like a general block of Tor rather than a block of specific
>> IPs, since attempting with different Tor circuits does not change the
>
> The Guix project does not block Tor. If the datacentre has decided to
> block Tor like it blocked most of Russia, there is little we can do
> but ask them to reconsider.

I think it’s worse than this. I noticed that ci.guix.gnu.org (same
machine) would occasionally time out on my side, without Tor, starting
from this week (I was on vacation before, so I don’t know exactly when
it started). From a browser, I get this “DoS attack” HTML page:
The HTML doesn’t contain clues as to where it originates from.

Toggle snippet (12 lines)
$ wget -qO- http://ci.guix.gnu.org | tail
</style>
<title>Attack Detected</title>
</head>
<body><div class="message-container">
<div class="logo"></div>
<h1>Blocked because of DoS Attack</h1>
<p>Your computer has been blocked because a DoS attack originating from your system was detected. For more information, contact the system administrator.</p>
</div></body>
</html>

Some firewall-ish network equipment must be sitting right before our
machine. It’s a problem because fetching narinfos and nars is likely to
count as a “DoS attack”.

Could it be some change at the MDC?

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 13 Aug 2023 02:26
control message for bug #65056
(address . control@debbugs.gnu.org)
875y5jbxer.fsf@gnu.org
severity 65056 serious
quit
T
T
Tobias Geerinckx-Rice wrote on 13 Aug 2023 13:46
Re: bug#65056: https://issues.guix.gn u.org/ cannot be accessed through Tor
1504743B-0B78-4D42-AB22-F25CA2D0194D@tobias.gr
On 13 August 2023 00:25:51 UTC, "Ludovic Courtès" <ludo@gnu.org> wrote:
Toggle quote (5 lines)
>I think it’s worse than this. I noticed that ci.guix.gnu.org (same
>machine) would occasionally time out on my side, without Tor, starting
>from this week (I was on vacation before, so I don’t know exactly when
>it started). From a browser, I get this “DoS attack” HTML page:

Oh, wow. This is new to me.

It's frustrating that $IT keeps adding new significant hurdles with apparently 0 communications, and that our only option is often 'ask rekado, again, to ask things, again'. That's not right.

Ricardo, do you think there's a chance this trend will improve (without you burning out)?

Otherwise, I'd like to suggest wireguarding berlin's impressive hardware resources to bayfront or to a new head node not hosted at the MDC, or something similarly provocative. Just give up on hosting public services there, like we already migrated the home page. This isn't meaningful redundancy.


Kind regards,

T G-R

Sent on the go. Excuse or enjoy my brevity.
R
R
Ricardo Wurmus wrote on 14 Aug 2023 11:07
Re: bug#65056: https://issues.guix.gnu.org/ cannot be accessed through Tor
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
87h6p256ty.fsf@elephly.net
Tobias Geerinckx-Rice <me@tobias.gr> writes:

Toggle quote (14 lines)
> On 13 August 2023 00:25:51 UTC, "Ludovic Courtès" <ludo@gnu.org> wrote:
>>I think it’s worse than this. I noticed that ci.guix.gnu.org (same
>>machine) would occasionally time out on my side, without Tor, starting
>>from this week (I was on vacation before, so I don’t know exactly when
>>it started). From a browser, I get this “DoS attack” HTML page:
>
> Oh, wow. This is new to me.
>
> It's frustrating that $IT keeps adding new significant hurdles with
> apparently 0 communications, and that our only option is often 'ask
> rekado, again, to ask things, again'. That's not right.
>
> Ricardo, do you think there's a chance this trend will improve (without you burning out)?

I don’t know. I’m on holidays now, but I’ve opened yet another ticket
to get a definitive answer to my more elaborate variant of “WTF?”.

Toggle quote (6 lines)
> Otherwise, I'd like to suggest wireguarding berlin's impressive
> hardware resources to bayfront or to a new head node not hosted at the
> MDC, or something similarly provocative. Just give up on hosting
> public services there, like we already migrated the home page. This
> isn't meaningful redundancy.

Good plan.

Sorry about this. It’s frustrating, and I’m stocking up on towels to
throw.

--
Ricardo
L
L
Ludovic Courtès wrote on 8 Sep 2023 22:24
(name . Ricardo Wurmus)(address . rekado@elephly.net)
878r9g2z2o.fsf@gnu.org
Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

Toggle quote (3 lines)
> I don’t know. I’m on holidays now, but I’ve opened yet another ticket
> to get a definitive answer to my more elaborate variant of “WTF?”.

Did you eventually get feedback from them?

If not, we can start looking for a way to move public-facing services
elsewhere. (It may not be trivial because bayfront, which is the other
node we’ve traditionally used for that, is super busy these days.)

Thanks again for your support…

Ludo’.
R
R
Ricardo Wurmus wrote on 12 Sep 2023 10:02
(name . Ludovic Courtès)(address . ludo@gnu.org)
87msxr3jd4.fsf@elephly.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (9 lines)
> Hello!
>
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> I don’t know. I’m on holidays now, but I’ve opened yet another ticket
>> to get a definitive answer to my more elaborate variant of “WTF?”.
>
> Did you eventually get feedback from them?

I got one response to ask for more information, which I supplied.
Nothing since. I requested a response just now.

Toggle quote (4 lines)
> If not, we can start looking for a way to move public-facing services
> elsewhere. (It may not be trivial because bayfront, which is the other
> node we’ve traditionally used for that, is super busy these days.)

Yeah, I’d really like this to be fixed. It worked pretty well for
years, so these seemingly unnecessary changes and the way they are
applied without any recourse (and without anyone being able to confirm
that they have in fact changed somehing) really bother me.

But if our public services keep getting restricted I agree that we
should look for an alternative way to host them.

--
Ricardo
M
M
Maxim Cournoyer wrote on 12 Sep 2023 15:06
(name . Ricardo Wurmus)(address . rekado@elephly.net)
87a5try201.fsf@gmail.com
Hi Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

Toggle quote (23 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> Ricardo Wurmus <rekado@elephly.net> skribis:
>>
>>> I don’t know. I’m on holidays now, but I’ve opened yet another ticket
>>> to get a definitive answer to my more elaborate variant of “WTF?”.
>>
>> Did you eventually get feedback from them?
>
> I got one response to ask for more information, which I supplied.
> Nothing since. I requested a response just now.
>
>> If not, we can start looking for a way to move public-facing services
>> elsewhere. (It may not be trivial because bayfront, which is the other
>> node we’ve traditionally used for that, is super busy these days.)
>
> Yeah, I’d really like this to be fixed. It worked pretty well for
> years, so these seemingly unnecessary changes and the way they are
> applied without any recourse (and without anyone being able to confirm
> that they have in fact changed somehing) really bother me.

Agreed; I think it's premature to jump ship when we've had such a long
and fruitful relationship; let's show some patience and tenacity toward
a resolution.

--
Thanks,
Maxim
L
L
Ludovic Courtès wrote on 14 Sep 2023 23:25
(name . Ricardo Wurmus)(address . rekado@elephly.net)
87led84fbz.fsf@gnu.org
Hi,

Ricardo Wurmus <rekado@elephly.net> skribis:

Toggle quote (14 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> Ricardo Wurmus <rekado@elephly.net> skribis:
>>
>>> I don’t know. I’m on holidays now, but I’ve opened yet another ticket
>>> to get a definitive answer to my more elaborate variant of “WTF?”.
>>
>> Did you eventually get feedback from them?
>
> I got one response to ask for more information, which I supplied.
> Nothing since. I requested a response just now.

I confirm that I still get the problem right now from my home network,
without even really trying:

Toggle snippet (42 lines)
$ wget -qO- --debug http://ci.guix.gnu.org |tail
DEBUG output created by Wget 1.21.3.24-2b723 on linux-gnu.

Reading HSTS entries from /home/ludo/.wget-hsts
URI encoding = ‘UTF-8’
Caching ci.guix.gnu.org => 141.80.181.40
Created socket 3.
Releasing 0x00000000017a8e00 (new refcount 1).

---request begin---
GET / HTTP/1.1
Host: ci.guix.gnu.org
User-Agent: Wget/1.21.3.24-2b723
Accept: */*
Accept-Encoding: identity
Connection: Keep-Alive

---request end---

---response begin---
HTTP/1.1 200 OK
Content-Length: 4401
Connection: Close
Cache-Control: no-cache
Content-Type: text/html; charset=utf-8
X-Frame-Options: SAMEORIGIN

---response end---
URI content encoding = ‘utf-8’
Closed fd 3
Saving HSTS entries to /home/ludo/.wget-hsts
</style>
<title>Attack Detected</title>
</head>
<body><div class="message-container">
<div class="logo"></div>
<h1>Blocked because of DoS Attack</h1>
<p>Your computer has been blocked because a DoS attack originating from your system was detected. For more information, contact the system administrator.</p>
</div></body>
</html>

Ludo’.
R
R
Ricardo Wurmus wrote on 15 Sep 2023 09:41
(name . Ludovic Courtès)(address . ludo@gnu.org)
87v8cbzxum.fsf@elephly.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (3 lines)
> I confirm that I still get the problem right now from my home network,
> without even really trying: […]

Is that through Tor or just your ISP?

--
Ricardo
L
L
Ludovic Courtès wrote on 18 Sep 2023 11:51
(name . Ricardo Wurmus)(address . rekado@elephly.net)
87r0mvizc6.fsf@gnu.org
Ricardo Wurmus <rekado@elephly.net> skribis:

Toggle quote (7 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> I confirm that I still get the problem right now from my home network,
>> without even really trying: […]
>
> Is that through Tor or just your ISP?

It’s just my ISP, no Tor involved. I can share privately my home IP
address if that helps investigate the problem; let me know.

Ludo’.
A
A
Altadil wrote on 18 Sep 2023 22:31
https://issues.guix.gnu.org available through Tor again
(name . 65056@debbugs.gnu.org)(address . 65056@debbugs.gnu.org)
6wSddjg3y1ncXci8SyB5n4ENXh6-22QWx2kOZoqKcdTTEPyj8vl0C4YfugXz5dryIgnTw0FQU89sj3nyaT9JT_FaIDMkfpo48rhecY55Z8s=@protonmail.com
Hi,

it seems https://issues.guix.gnu.orgcan again be accessed when using Tor Browser.

Kind regards,
Altadil
G
G
Giovanni Biscuolo wrote on 20 Sep 2023 23:31
Re: bug#65056: https://issues.guix.gnu.org/ cannot be accessed through Tor
87pm2cbkgp.fsf@xelera.eu
Hi!

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (12 lines)
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> I confirm that I still get the problem right now from my home network,
>>> without even really trying: […]
>>
>> Is that through Tor or just your ISP?
>
> It’s just my ISP, no Tor involved. I can share privately my home IP
> address if that helps investigate the problem; let me know.

Given that Altadil told that the service now can be accessed through Tor
again, can we close this specific bug now?

It would be good to have some feedback from the NOC, just to know /how/
it was resolved (or was just a temporary tech issue), for example that
they do not have a policy to blacklist Tor and whitelist it on demand
(I'm just guessing).

What Ludovic found, anyway, is another issue and we should investigate
if it is worth a new bug report.

I've tested now with the same
wget -qO- --debug http://ci.guix.gnu.org| tail
and all seems fine from my ISP (now)

Anyway the DoS Attack protection of the network hosting ci.guix.gnu.org
/seems/ problematic: how could it be that home IP resposible of a DoS
attack? Was it a false positive or was it some temporary problem from
the originating IP network?

We should carefully track this network issues since they have a great
impact on user experience.


Thanks! Gio'

--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQJABAEBCgAqFiEERcxjuFJYydVfNLI5030Op87MORIFAmULZKYMHGdAeGVsZXJh
LmV1AAoJENN9DqfOzDkSxnEQANxLlUYuzwbhitCb3NQZz+wHDZcKAqigM/RLRuBl
a7l+MS/YdfRLehocG/co15fCyCF0GuA0Q4Ko/V6YBzp4oV21fnRAQ8HU+YREB/uI
8Lg+2yRnnKO9hVrRfhTJOKAPlsBuYmi75XBW/+wHFLP8C5M+6GFAC1AXl1JRQ+vO
fN15ubrImmmaGMGF7tDdS67PDJQ/zpMdYm7EzVKFnCmGIE+ofbJe+m+z2yvi9m1e
NvKZ/nzN255h8gHpmIQZ3LVGNaugMoTmjWugATxgyX7j9zuGYbOVwhJNbsacrkbO
OQFBv42+zM5+A+3vVNO1S5dDde2lKuS5vw8mqnWOvKJbvbYNtz6f3t7ItlQfm0NH
dQSfBphb26OJ0t1Y2OiJAjxvFBqt7PdeoOTB9btc6RQQwOPN/OJY2Y2RIW1wS+wu
7WaNVWgy0OLVVrrLjoLCNIuydluwwFFW2ut75fxSUIx9pN90XVH5VMuQIb3P9t06
vYGoxJC6g0yhcxdd/1UooNob4aRnlvCQd9AtbPxX8rtJJTdOdPH8fmazhjpFNF4Q
4LrlEfQcRUREc3iPaIiOCALkc5+xKAmgHi/iRBR1bdNYbUeSsV/9qhKHjwM/Qu9n
1PbhS5Ln7xpgIR7iXpoogpOUfAbdAo+oweDUaivSxeIH+svP3Mju+0DrOXKJ7vQz
+Kb3
=MDRq
-----END PGP SIGNATURE-----

R
R
Ricardo Wurmus wrote on 21 Sep 2023 00:45
(name . Giovanni Biscuolo)(address . g@xelera.eu)
87o7hw8nsx.fsf@elephly.net
Giovanni Biscuolo <g@xelera.eu> writes:

Toggle quote (5 lines)
> Anyway the DoS Attack protection of the network hosting ci.guix.gnu.org
> /seems/ problematic: how could it be that home IP resposible of a DoS
> attack? Was it a false positive or was it some temporary problem from
> the originating IP network?

IT had installed some DoS attack protection thing for the DMZ with
different thresholds based on past access patterns.

Upon my request they have now disabled this completely for our IPs
corresponding to ci.guix.gnu.org and its sibling node.

--
Ricardo
G
G
Giovanni Biscuolo wrote on 21 Sep 2023 08:27
(name . Ricardo Wurmus)(address . rekado@elephly.net)
87jzskavmz.fsf@xelera.eu
Hello,

Ricardo Wurmus <rekado@elephly.net> writes:

[...]

Toggle quote (3 lines)
> IT had installed some DoS attack protection thing for the DMZ with
> different thresholds based on past access patterns.

Probably some of us is using access patterns the "thing" considers
DoS :-)

Toggle quote (3 lines)
> Upon my request they have now disabled this completely for our IPs
> corresponding to ci.guix.gnu.org and its sibling node.

Thanks a lot! We should probably consider this (disable any firewall
protection) as a requirement when one or more of our public facing hosts
firewalling is not under our direct control.

Now we only have berlin and bayfront as public facing hosts... but for
example milan.guix-1 is connected by our build farm via its public IP
(ehrm, time to set up wireguard for that, too).

Anyway, AFAIU this "thing" in berlin network is no more an issue, we do
not need a new bug report IMO.

Thank you!

[...]

--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQJABAEBCgAqFiEERcxjuFJYydVfNLI5030Op87MORIFAmUL4lQMHGdAeGVsZXJh
LmV1AAoJENN9DqfOzDkS0DkP/AqamkN8OOd/w+nJvXqjiTpdXwiGwpItxlglr3O6
kmC7cNOn6+VNdyDEkqFdJeElJRvFL/l1LBH+Q14UTa0/h3RLWeQCAwhe4oXvrcqg
0fgtNbDp5GxWbybXzZjPUB725jvXvPvv7l1XhaWP+9dVGKZ/ofaGEiYZlu9lE/A1
V7rChpPgdaFEnWSltKehRaMEpXZ8z66m4mF24YkjN4S0iANNOT8hLS2n6sSBZubW
yYGOb1d1T09tgo/w87++dplVe6BAn4MeGh/fR/cRrOmlb6qDnMkAexPjuU1nXZwk
SyK+J8X5vJN1VhxX6rTbd7XXBjiHAYFT2ENR3UcoEW27RipIInjKWNlmlQRXfJwt
ILMWu+ek1t/kCRA57LfJ18jh31R5jMm44+iN6erBaC4BdY9Dn5x2zv/VpguqGNl8
AnOgFLCa3upUfxlo0NS2Ow8VNUxE9E7oCAh2fcyDy+Zp340hfKoBFNy7+IlM8sYv
f/HgbPbqzqAafgB/LG8IRFqa2B9+/DbkJAZlExYiQXuv85HwqZMxcUWOvrt0m5Kb
LeMd15wAvL9V6aJXU9srqO7LYaJu+pHdTuPfhXWct7eleHZ6d8Pl7KiCJv7NyZgC
zB0NlhOdgHNtHe/wyOemrEUFylW11gZ3Et/V/Ni9LazcZmuQgEj6wPQ2ZvYZZut3
RUQ6
=f4w1
-----END PGP SIGNATURE-----

G
G
Giovanni Biscuolo wrote on 21 Sep 2023 08:29
Re: bug#65056: https://issues.guix.gnu.org available through Tor again
87h6noavjz.fsf@xelera.eu
Altadil via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

Toggle quote (5 lines)
> Hi,
>
> it seems https://issues.guix.gnu.org can again be accessed when using
> Tor Browser.

Thank you! I'm closing this bug, feel free to reopen it if needed.

Best regards, Gio'

--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQJABAEBCgAqFiEERcxjuFJYydVfNLI5030Op87MORIFAmUL4sAMHGdAeGVsZXJh
LmV1AAoJENN9DqfOzDkSG/EP/1Es242VB9whLz+1EroXGWrGR5NKSJ8O3HdUDQnm
Ro5IL+A6DmhHpkKPI6dbkLKaxoKrQkQlvT0N9P3rJQ+WffA/dYveuFC7pHbJaNd1
tu0/1xCzpcI8JmMpatiCZaeVx0iKksJCmC5kvfEOltuoEL05+oh9xXtFD5LBIMtm
Ouzben1fr7w06CzI3OYc5bTvxWsn77kV7RzDNalZPEOWG+BOy0i9hmZsz6eTsQji
Y1Qy89LuAadIZQsmhnjcdODt+X4woWNXk+ZwgTJ5JlFtcJQ8YbZ0FwXCVITYmwJ7
jusgMc3IQySnZdtELoPN5oI63ZauvTGi16aTTwGhx5YlKsTMjRJdFETekJpGu9Tc
t/xu5Q5tyMXAuCJHxXqZW9kkVW6d8xdB1xfDnF7Fa/1UzJlbrEf2d40ajVXHsv74
cD9kFOV3I0VU9Lw8xsAG+Tbq2nPMTFoyPdlt7nZ780TEYfxdbugrYoZbpCiyeEgi
ZAEe5abVMrL2XV12DyHFe2xBi3UyfglCweTg6DP8d9t9g52y+sgFwVcZ5tczRlT8
Bk1uSM6gwNGj+yfABb+fsoh3jRzu0tky9HAr1hTQRfhgHERDK+h5kmkM1cce9e1Z
EFvsAX5BDQ9flgDaaxe5DweekqcHyx3x87d8LqUBfbmIHeO+RW7r5337JokodiFz
F2Cq
=PJdr
-----END PGP SIGNATURE-----

Closed
L
L
Ludovic Courtès wrote on 2 Oct 2023 10:48
Re: bug#65056: https://issues.guix.gnu.org/ cannot be accessed through Tor
(name . Ricardo Wurmus)(address . rekado@elephly.net)
87edidl8ai.fsf@gnu.org
Hello,

Ricardo Wurmus <rekado@elephly.net> skribis:

Toggle quote (6 lines)
> IT had installed some DoS attack protection thing for the DMZ with
> different thresholds based on past access patterns.
>
> Upon my request they have now disabled this completely for our IPs
> corresponding to ci.guix.gnu.org and its sibling node.

I’m late to the party but this is excellent news, thank you!

I guess we can close the issue now, right?

Ludo’.
?