qutebrowser/IceCat stuck at Cloudflare "Checking your browser before accessing..."

  • Done
  • quality assurance status badge
Details
5 participants
  • bdju
  • Ben Sturmfels
  • Tobias Geerinckx-Rice
  • Mark H Weaver
  • Michael Rohleder
Owner
unassigned
Submitted by
bdju
Severity
normal
B
B
bdju wrote on 11 Dec 2020 15:41
qutebrowser stuck on browser checks forever
(address . bug-guix@gnu.org)
C7Q19D8JJ2ET.2QEOCH0P4NKLE@masaki
guix (GNU Guix) 91e35e32a4938e0e37499c64fa8ed3e7cf51dce3
some example sites with browser checks:

I already contacted the people in the qutebrowser IRC. The bug persists
when I launch with no config (`qutebrowser --temp-basedir'), and they
could not reproduce the issue on the same version of qutebrowser I'm
using. I suspect it may be an issue with the guix version.
T
T
Tobias Geerinckx-Rice wrote on 11 Dec 2020 20:23
(no subject)
(address . control@debbugs.gnu.org)
874kkskuya.fsf@nckx
retitle 45179 qutebrowser stuck at Cloudflare 'browser checks'
M
M
Michael Rohleder wrote on 22 Dec 2020 17:34
Re: bug#45179: qutebrowser stuck at Cloudflare 'browser checks'
(name . bdju)(address . bdju@tilde.team)(address . 45179@debbugs.gnu.org)
87czz1pzoe.fsf@rohleder.de
Hello bdju,

"bdju" <bdju@tilde.team> writes:
Toggle quote (11 lines)
> guix (GNU Guix) 91e35e32a4938e0e37499c64fa8ed3e7cf51dce3
> some example sites with browser checks:
> https://gitlab.com/users/sign_in
> http://livechart.me/
>
> I already contacted the people in the qutebrowser IRC. The bug persists
> when I launch with no config (`qutebrowser --temp-basedir'), and they
> could not reproduce the issue on the same version of qutebrowser I'm
> using. I suspect it may be an issue with the guix version.
>

A workaround is setting the "User Agent" string to something CF likes, e.g. like this:
qutebrowser --temp-basedir --set content.headers.user_agent 'Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0' https://gitlab.com/users/sign_in

Maybe the reason why the (very helpful) #qutebrowser folks can't
reproduce this, is because they use another qt version (qutebrowser uses
qtwebengine) and perhaps this sets user agent to a string that CF has
whitelisted.

I don't think there is much we could do here (besides updating qt and
mbakke has that in the pipeline, afaik).

--
COFFEE.EXE Missing - Insert Cup and Press Any Key
-----BEGIN PGP SIGNATURE-----

iQFFBAEBCAAvFiEEdV4t5dDVhcUueCgwfHr/vv7yyyUFAl/iICERHG1pa2VAcm9o
bGVkZXIuZGUACgkQfHr/vv7yyyWlTAf/UczseTDl3W8/mltuLRDRt/qsiaJqvhF6
qxSECReHaDucisE+SUC+RUCacYqWgO2BWIhHLZCqylGdRTj86MvaJeFPERUT4wsw
Ykz+w94MszfrBBM56oGR0kVrE3np8KwkQaPKA5VeJDhUastAfxKg/tzzXIA7V1FL
WEvbqdIATpLAZopwq7krsVxf/GSUP1Fg3cGZOcRmxnpXEJhSJ0Nm5D0038aOKcq0
+Pxk/8kSp/Fqw7KAiHUaRPkrK/CTodpeq22ortQDnGfHBxFlbc0Q/eV7Jqsbi4G3
/6W3QkS5nJSzyRBVghDyocVHVVOLLiG3LyHEfx6HBAitNc0VYYMZtw==
=sWXR
-----END PGP SIGNATURE-----

B
B
Ben Sturmfels wrote on 23 Jan 2021 05:58
(name . Michael Rohleder)(address . mike@rohleder.de)
87czxw1c6v.fsf@sturm.com.au
On Tue, 22 Dec 2020, Michael Rohleder wrote:

Toggle quote (21 lines)
> Hello bdju,
>
> "bdju" <bdju@tilde.team> writes:
>> guix (GNU Guix) 91e35e32a4938e0e37499c64fa8ed3e7cf51dce3
>> some example sites with browser checks:
>> https://gitlab.com/users/sign_in
>> http://livechart.me/
>
> A workaround is setting the "User Agent" string to something CF likes, e.g. like this:
> qutebrowser --temp-basedir --set content.headers.user_agent 'Mozilla/5.0
> (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0'
> https://gitlab.com/users/sign_in
>
> Maybe the reason why the (very helpful) #qutebrowser folks can't
> reproduce this, is because they use another qt version (qutebrowser uses
> qtwebengine) and perhaps this sets user agent to a string that CF has
> whitelisted.
>
> I don't think there is much we could do here (besides updating qt and
> mbakke has that in the pipeline, afaik).

I'm also having similar issues accessing gitlab.com with IceCat.
Installing the User Agent Switcher and setting to "Linux / Firefox 82"
fixed this for me.

For context I also tried starting IceCat up in safe mode and switching
tracking protection back to "standard", but no good. The issue appears
to be that Cloudflare are essentially blocking niche user agents.

Regards,
Ben
B
B
Ben Sturmfels wrote on 23 Jan 2021 06:01
(address . control@debbugs.gnu.org)
87a6t01c23.fsf@sturm.com.au
retitle 45179 qutebrowser/IceCat stuck at Cloudflare "Checking your browser before accessing..."
M
M
Mark H Weaver wrote on 24 Jan 2021 01:05
(address . 45179@debbugs.gnu.org)
87eeibfbb6.fsf@netris.org
ben--- via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

Toggle quote (8 lines)
> I'm also having similar issues accessing gitlab.com with IceCat.
> Installing the User Agent Switcher and setting to "Linux / Firefox 82"
> fixed this for me.
>
> For context I also tried starting IceCat up in safe mode and switching
> tracking protection back to "standard", but no good. The issue appears
> to be that Cloudflare are essentially blocking niche user agents.

The IceCat package in Guix sends the same user-agent string as the
'firefox-esr' package in Debian.

Mark
B
B
Ben Sturmfels wrote on 1 Feb 2021 12:58
(name . Mark H Weaver)(address . mhw@netris.org)
874kiwuhi3.fsf@sturm.com.au
On Sun, 24 Jan 2021, Mark H. Weaver wrote:

Toggle quote (13 lines)
> ben--- via Bug reports for GNU Guix <bug-guix@gnu.org> writes:
>
>> I'm also having similar issues accessing gitlab.com with IceCat.
>> Installing the User Agent Switcher and setting to "Linux / Firefox 82"
>> fixed this for me.
>>
>> For context I also tried starting IceCat up in safe mode and switching
>> tracking protection back to "standard", but no good. The issue appears
>> to be that Cloudflare are essentially blocking niche user agents.
>
> The IceCat package in Guix sends the same user-agent string as the
> 'firefox-esr' package in Debian.

I've done a little more testing for interest sake - I still think
CloudFlare are at fault here.

If use a private tab and attempt to access a private GitLab repository I
get stuck at the CloudFlare "checking your browser message". Here's the
IceCat default user-agent string:

"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0"

If I install User Agent Switcher and do the same thing, it works. The
only difference in the user-agent string is the version number:

"Mozilla/5.0 (X11; Linux x86_64; rv:82.0) Gecko/20100101 Firefox/82.0"

If I try the same on Firefox ESR on Debian Buster, it works no problem,
but as Mark says, the Firefox user-agent string matches exactly to
IceCat:

"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0"

That suggests to me that there's more than the user-agent string at play
here.

Regards,
Ben
B
B
bdju wrote on 6 Apr 2021 23:41
(name . Michael Rohleder)(address . mike@rohleder.de)(address . 45179@debbugs.gnu.org)
CAGYUM4RXECK.3RC69VJEKRTBU@masaki
On Tue Dec 22, 2020 at 10:34 AM CST, Michael Rohleder wrote:
Toggle quote (30 lines)
> Hello bdju,
>
> "bdju" <bdju@tilde.team> writes:
> > guix (GNU Guix) 91e35e32a4938e0e37499c64fa8ed3e7cf51dce3
> > some example sites with browser checks:
> > https://gitlab.com/users/sign_in
> > http://livechart.me/
> >
> > I already contacted the people in the qutebrowser IRC. The bug persists
> > when I launch with no config (`qutebrowser --temp-basedir'), and they
> > could not reproduce the issue on the same version of qutebrowser I'm
> > using. I suspect it may be an issue with the guix version.
> >
>
> A workaround is setting the "User Agent" string to something CF likes,
> e.g. like this:
> qutebrowser --temp-basedir --set content.headers.user_agent 'Mozilla/5.0
> (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0'
> https://gitlab.com/users/sign_in
>
> Maybe the reason why the (very helpful) #qutebrowser folks can't
> reproduce this, is because they use another qt version (qutebrowser uses
> qtwebengine) and perhaps this sets user agent to a string that CF has
> whitelisted.
>
> I don't think there is much we could do here (besides updating qt and
> mbakke has that in the pipeline, afaik).
>
> --
> COFFEE.EXE Missing - Insert Cup and Press Any Key
While I was able to bypass the cloudflare checks in IceCat with that
User Agent Switcher addon, I found I so far haven't been able to get
past them in qutebrowser by changing my useragent. I tried both your
example and copying one of the useragents generated by UAS in IceCat.
Still caught in a loop. (and I am using --temp-basedir as you said, so
shouldn't be my config)
B
B
bdju wrote on 6 Apr 2021 23:56
(name . Michael Rohleder)(address . mike@rohleder.de)(address . 45179@debbugs.gnu.org)
CAGZ5M6N9UN3.34SCR6C6KEH3I@masaki
On Tue Apr 6, 2021 at 4:41 PM CDT, bdju wrote:
Toggle quote (6 lines)
> While I was able to bypass the cloudflare checks in IceCat with that
> User Agent Switcher addon, I found I so far haven't been able to get
> past them in qutebrowser by changing my useragent. I tried both your
> example and copying one of the useragents generated by UAS in IceCat.
> Still caught in a loop. (and I am using --temp-basedir as you said, so
> shouldn't be my config)
I was too hasty! I both had to change my useragent and wait a long time!
I didn't measure the time, but I'd guess over 10 minutes for Gitlab to
work. Livechart took even longer. Thanks so much for this workaround.
Oh, and this works without --temp-basedir, by the way. I should be able
to actually use these sites in my normal session now I think!
M
M
Michael Rohleder wrote on 7 Apr 2021 11:55
(name . bdju)(address . bdju@tilde.team)(address . 45179-done@debbugs.gnu.org)
87o8eqxvu0.fsf@rohleder.de
"bdju" <bdju@tilde.team> writes:
Toggle quote (13 lines)
> On Tue Apr 6, 2021 at 4:41 PM CDT, bdju wrote:
>> While I was able to bypass the cloudflare checks in IceCat with that
>> User Agent Switcher addon, I found I so far haven't been able to get
>> past them in qutebrowser by changing my useragent. I tried both your
>> example and copying one of the useragents generated by UAS in IceCat.
>> Still caught in a loop. (and I am using --temp-basedir as you said, so
>> shouldn't be my config)
> I was too hasty! I both had to change my useragent and wait a long time!
> I didn't measure the time, but I'd guess over 10 minutes for Gitlab to
> work. Livechart took even longer. Thanks so much for this workaround.
> Oh, and this works without --temp-basedir, by the way. I should be able
> to actually use these sites in my normal session now I think!

Yay! That's great to hear, so let's close this ;)

--
Practical people would be more practical if they would take a little
more time for dreaming. - J. P. McEvoy
-----BEGIN PGP SIGNATURE-----

iQFFBAEBCAAvFiEEdV4t5dDVhcUueCgwfHr/vv7yyyUFAmBtgZcRHG1pa2VAcm9o
bGVkZXIuZGUACgkQfHr/vv7yyyXrpAf7BaN47NUfT4Rcpe1Vw/NQhZ6S2BPcfhiX
xCQXf0m3Q4Ro1YgPCpUe5s1DoWL6qfugBFcMTLSfFHZhu8J2l8QCHeSp8L9A7Mqa
z3d09ecQBusCOFFdZMrWUoOfWEUoLvaiQxjLTr6NHJUY92nZM3o2R8o6AKrwLMyt
74HFqYmUVITlCHZheQ119XRqk6pRZ4xrv/wezafYGMeNNw+Vl2/rLAco0VIdPilF
kTmSYGWUOxSJFWRHCmoFJU2bnZjC3crHBBLOUbzQLHt3PVIhGN9s58cM6RHGpt98
kfJIGptrQ4wQdDUY0wshA0DCdbKQ1alL1IOtPKt1XEkFQo86dfUdlw==
=3B5r
-----END PGP SIGNATURE-----

Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 45179@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 45179
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch