Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
+--
 Reporter:  arma|  Owner:  metrics-team
 Type:  task| Status:  new
 Priority:  Medium  |  Milestone:
Component:  Metrics/Ideas   |Version:
 Severity:  Normal  | Resolution:
 Keywords:  network-health  |  Actual Points:
Parent ID:  | Points:
 Reviewer:  |Sponsor:
+--
Changes (by karsten):

 * component:  Metrics/Exit Scanner => Metrics/Ideas


Comment:

 This better fits into our Ideas subcomponent.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--
Changes (by gaba):

 * keywords:  network-health => network-health gsoc-ideas


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 If tor project can provide a list of sies which blocking Tor that would be
 useful.


 [http://searxes.nmqn_remove-
 
me_kngye4ct7bgss4bmv5ca3wpa55yugvxen5kz2bbq67lwy6ps54yd.onion/?res&lg=en&alf=t&rq=Cloudflare+MITM
 example link]. Green checkmark: Tor passed, Red: Tor browser smulation
 denied(403). [https://git.openprivacy.ca/you/stop_cloudflare/#data-more-
 information Also this link].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 Replying to [comment:2 cypherpunks]:
 > It's UA discrimination from my personal experience.

 This is true. See
 
"[https://git.openprivacy.ca/you/stop_cloudflare/src/branch/master/README_ethics.md
 #browser-vendor-discrimination Browser vendor discrimination]".
 (it's not secure than torbrowser but there are people who use
 Chromium/Firefox over Tor daemon)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by gk):

 Replying to [ticket:33010 arma]:

 [snip]

 > One catch is that Cloudflare currently gives alt-svc headers in response
 to fetches from Tor addresses. So that means we need a web client that can
 follow alt-srv headers -- maybe we need a full Selenium like client?

 The alt-svc is not kicking in with the first load. So, if we use a really
 simple static page (that is with nothing dynamic and no sub resources
 being requested subsequently) we should not hit that complicating factor.

 That said using Tor Browser for the case where we actually want to find
 out the Tor Browser experience seems like a thing we should investigate,
 and be it alone for the reason mentioned in comment:4. There is
 [https://github.com/webfp/tor-browser-selenium tor-browser-selenium] and
 various forks that should do the trick in combination with
 [https://stem.torproject.org/ stem].

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Old description:

> We should track the rate that cloudflare gives captchas to Tor users over
> time.
>
> My suggested way of doing that tracking is to sign up a very simple
> static webpage to be fronted by cloudflare, and then fetch it via Tor
> over time, and record and graph the rates of getting a captcha vs getting
> the real page.
>
> The reason for the "simple static page" is to make it really easy to
> distinguish whether we're getting hit with a captcha. The "distinguishing
> one dynamic web page from another" challenge makes exitmap tricky in the
> general case, but we can remove that variable here.
>
> One catch is that Cloudflare currently gives alt-svc headers in response
> to fetches from Tor addresses. So that means we need a web client that
> can follow alt-srv headers -- maybe we need a full Selenium like client?
>
> Once we get the infrastructure set up, we would be smart to run a second
> one which is just wget or curl or lynx or something, i.e. which doesn't
> behave like Tor Browser, in order to be able to track the difference
> between how Cloudflare responds to Tor Browser vs other browsers.
>
> I imagine that Cloudflare should be internally tracking how they're
> handling Tor requests, but having a public tracker (a) gives the data to
> everybody, and (b) helps Cloudflare have a second opinion in case their
> internal data diverges from the public version.
>
> The Berkeley ICSI group did research that included this sort of check:
> https://www.freehaven.net/anonbib/#differential-ndss2016
> https://www.freehaven.net/anonbib/#exit-blocking2017
> but what I have in mind here is essentially a simpler subset of this
> research, skipping the complicated part of "how do you tell what kind of
> response you got" and with an emphasis on automation and consistency.
>
> There are two interesting metrics to track over time: one is the fraction
> of exit relays that are getting hit with captchas, and the other is the
> chance that a Tor client, choosing an exit relay in the normal weighted
> faction, will get hit by a captcha.
>
> Then there are other interesting patterns to look for, e.g. "are certain
> IP addresses punished consistently and others never punished, or is
> whether you get a captcha much more probabilistic and transient?" And
> does that pattern change over time?

New description:

 We should track the rate that cloudflare gives captchas to Tor users over
 time.

 My suggested way of doing that tracking is to sign up a very simple static
 webpage to be fronted by cloudflare, and then fetch it via Tor over time,
 and record and graph the rates of getting a captcha vs getting the real
 page.

 The reason for the "simple static page" is to make it really easy to
 distinguish whether we're getting hit with a captcha. The "distinguishing
 one dynamic web page from another" challenge makes exitmap tricky in the
 general case, but we can remove that variable here.

 One catch is that Cloudflare currently gives alt-svc headers in response
 to fetches from Tor addresses. So that means we need a web client that can
 follow alt-srv headers -- maybe we need a full Selenium like client?

 Once we get the infrastructure set up, we would be smart to run a second
 one which is just wget or curl or lynx or something, i.e. which doesn't
 behave like Tor Browser, in order to be able to track the difference
 between how Cloudflare responds to Tor Browser vs other browsers.

 I imagine that Cloudflare should be internally tracking how they're
 handling Tor requests, but having a public tracker (a) gives the data to
 everybody, and (b) helps Cloudflare have a second opinion in case their
 internal data diverges from the public version.

 The Berkeley ICSI group did research that included this sort of check:
 https://www.freehaven.net/anonbib/#differential-ndss2016
 https://www.freehaven.net/anonbib/#exit-blocking2017
 but what I have in mind here is essentially a simpler subset of this
 research, skipping the complicated part of "how do you tell what kind of
 response you got" and with an emphasis on automation and consistency.

 There are two interesting metrics to track over time: one is the fraction
 of exit relays that are getting hit with captchas, and the other is the
 chance that

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by woswos):

 I wanted to conduct a few simple experiments on this issue. I will start
 by explaining my setup and continue with the experiments themselves.

 '''Domain Setup'''
 I registered two domains ([https://captcha.wtf/ captcha.wtf] and
 [https://exit11.online/ exit11.online]) with IPv4 records on Cloudflare.
 After playing with Cloudflare settings, I understood that domain owners
 have an important role in the way Cloudflare blocks Tor users.

 A new free Cloudflare account comes with a default security level (like
 the security levels in the Tor browser and as comment:5 mentioned), and
 the default security level doesn't explicitly block Tor users. I am not
 saying Cloudflare is innocent, but they don't mention a possible Tor user
 blocking at this security level. However, Tor shows up as a country on the
 Cloudflare firewall settings, and it is possible to block Tor users based
 on this firewall rule. I think they have a list of Tor exit node IPs, and
 they use this list to perform the filtering. They "offer" JS and Captcha
 challenges in addition to simple blocking, as shown in the image below:

 [[Image(https://bottomless-pit.barkin.io/tor-firewall-rules.png,
 width=100%)]]

 I think that's why some Tor users face more captcha challenges at higher
 Tor browser security levels. JavaScipt is blocked at higher security
 levels, and they can't pass the Cloudflare JS challenges.
 \\
 Also, if a firewall rule related to Tor is set, Cloudflare applies that
 rule (for example, the never-ending captcha challenge) all the time even
 if the user has somehow managed to pass the challenge 5 seconds ago - I
 think that is the part all of us hate, it just creates an endless loop. A
 sample Cloudflare firewall record below shows that the same IP address is
 continuously challenged over and over again, even after successfully
 passing the captcha challenge.

 [[Image(https://bottomless-pit.barkin.io/tor-firewall-1.png, width=100%)]]
 \\
 exit11.online has the default Cloudflare configuration without any
 additional firewall or protection. I am guessing that this would be the
 case with most of the average Cloudflare users. I also registered the
 [https://bypass.exit11.online/ bypass.exit11.online] subdomain, which
 bypasses the Cloudflare proxy and only utilizes Cloudflare as a DNS
 hosting service and CDN.

 [[Image(https://bottomless-pit.barkin.io/tor-cloudflare-exit11.png,
 width=100%)]]
 \\
 captcha.wtf has the default Cloudflare configuration ''with the additional
 firewall configuration'' for blocking Tor users, as I have mentioned
 previously. I registered this second domain to see the difference between
 using the default Cloudflare settings and adding additional firewall
 rules. I also registered the [https://bypass.captcha.wtf/
 bypass.captcha.wtf] subdomain, which bypasses the Cloudflare proxy and
 only utilizes Cloudflare as a DNS hosting service and CDN.

 [[Image(https://bottomless-pit.barkin.io/tor-cloudflare-wtf.png,
 width=100%)]]

 [[Image(https://bottomless-pit.barkin.io/tor-cloudflare-wtf-firewall.png,
 width=100%)]]
 \\
 Both of these domains have a very simple static "Hello world!" page at
 `/index.html`, and there is a more complicated page at `/complex.html`
 that loads resources from different locations. Additionally, captcha.wtf &
 exit11.online have SSL certificates issued by Cloudflare and
 bypass.captcha.wtf & bypass.exit11.online have SSL certificates issued by
 Let's Encrypt. I thought that these might have an effect on the way
 Cloudflare behaves.

 '''Experimenting'''
 Later, I used the Python script mentioned in comment:7 (it uses httplib)
 and the tor-browser-selenium mentioned in comment:13 to conduct a few
 simple experiments. I wrote another script to fetch different domain
 combinations via tor-browser-selenium and Python's httplib. For example,
 fetching bypass.exit11.online, exit11.online, exit11.online/complex.html,
 and bypass.exit11.online/complex.html via both tor-browser-selenium and
 Python's httplib.

 '''Results'''
 After fetching each combination about 100 times at one minute intervals,
 the domain with the default configuration (exit11.online) was not blocked
 a single time via both Tor and httplib. However, the domain with
 additional f

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by gk):

 Some ideas worth keeping in mind, which irl brought up the other day:

 Is there a ipv4/ipv6 difference?
 Does it matter which day of the week/time of the day sites are getting
 visited?
 Does size of the exit relay play a role (larger might carry "more" abusive
 traffic)?
 If we check Tor Browser we should have a Firefox control group (maybe with
 FPI and RFP on)/Other tool using just tor (curl/Firefox).

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by woswos):

 I did additions to the [https://github.com/woswos/Cloudflare-CAPTCHA-
 Monitoring/ repository] I mentioned in comment:14 and I deployed the code
 to a cloud server, specifically the `automated_fetcher_influxdb` example.

 Now, the server is fetching [https://captcha.wtf/ captcha.wtf] &
 [https://exit11.online/ exit11.online] pages and their combinations with &
 without the Tor browser at 15 minutes intervals. The full list of URLs
 tested is [https://github.com/woswos/Cloudflare-CAPTCHA-
 Monitoring/blob/master/examples/automated_fetcher_influxdb.py here].
 Later, the results are sent to a time series data oriented InfluxDB
 database.

 I created a Grafana dashboard at [http://dashboard.captcha.wtf/
 dashboard.captcha.wtf] to analyze and visualize the collected data. You
 can visit the dashboard to see the data collected so far. I will add more
 panels and analysis to the dashboard as I implement more metrics to track.

 ''Note:'' captcha.wtf & exit11.online websites and the
 `automated_fetcher_influxdb` code are not hosted on the same server. They
 all have different IP addresses if anyone is wondering.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by woswos):

 I wanted to share this lovely(!) patent, just in case anyone missed it:

 ''Blocking via an unsolvable CAPTCHA''
 https://patents.google.com/patent/US9407661

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 Replying to [comment:17 woswos]:

 > ''Blocking via an unsolvable CAPTCHA''
 > https://patents.google.com/patent/US9407661

 yes, they own a so called Troll Captcha patent and recaptha effectively
 presents you this type of unsolvable captcha. or by connecting through
 exit node, just the Message of "generate an unsolvable challenge-response
 test based on identifying the request as being associated with the
 malicious activity."

 While "associated with the malicious activity" is already an high amount
 of requests that any node is processing.

 But did you notice cloudflare seems to have changed captcha provider from
 recaptcha to ?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 Replying to [comment:18 cypherpunks]:
 > Replying to [comment:17 woswos]:

 > But did you notice cloudflare seems to have changed captcha provider
 from recaptcha to ?

 Yes, to hcaptcha.com.

 Here you can see a cloudflared website, that does deliver all of the time
 a captcha to user and of course this have changed from recaptcha to
 hcaptcha too so you can see the difference directly as example site look
 at:

 [https://captcha.website/]


 This means, you should even expect more captchas delivered to users.
 Because now it is a busyness model, (get webmaster to use "free"
 cloudflare service and present users money rewarded captchas) with every
 captcha presented :



 {{{
 runs on the Ethereum blockchain. Websites earn Human Tokens (HMT)
 whenever users use the hCaptcha widget on their site,
 and machine learning companies pay Human Tokens to get their data labeled.

 The Value in Data Labeling
 When you use hCaptcha, companies bid on the work your users do as they
 prove their humanity.
 You get the rewards.
 }}}
 source: hcaptcha.com

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by woswos):

 Wow, I realized the hcaptcha update, but I didn't know Cloudflare was
 making money out of this. Based on the blog post, it seems like Cloudflare
 still utilizes the same decision mechanism to present CAPTCHAs. I wonder
 how that mechanism will change with the hcaptcha update.

 I started collecting data before the hcaptcha update. Let me try to add a
 new panel to the [http://dashboard.captcha.wtf/ dashboard] to see the
 presented CAPTCHA rate changes over time.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--
Changes (by cypherpunks):

 * Attachment "captchawtf.png" added.

 https://captcha.wtf/

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 found a similar repo for this checks on https://github.com/shawa/cfcheck

 {{{
 Attempted Cloudflare CAPTCHA detection on a given site across sample of
 Tor exits
 }}}

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 Can you register 2 other domains with your other identity (e.g your
 friend) and test them too? (And not disclose the domain name in public -
 to defeat the chance that Cloudflare can whitelist it)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 > The domain with default Cloudflare configurations didn't block Tor users

 You must be joking.
 have you tried 'Firefox 7X + HTTP Proxy(like Privoxy for example) + Tor(as
 SOCKS proxy for HTTP Proxy)'?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 And Chrome + Tor. Captcha party.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 {{{
 desc = """Check if a web site returns a CloudFlare CAPTCHA using tor
 browser. By default, this tool is looking for the
 'Attention Required! | Cloudflare' text within the fetched web site.
 """
 }}}


 {{{
 $ getweb --tor https://example.com/hello.php
 URL opened
 Got response:
 Start::
 hello
 end::
 URL closed
 $
 }}}


 Based on your browser's language, Cloudflare return translated string to
 client.
 You better look for '| Cloudflare', or 'Cloudflare'(best).

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--
Changes (by cypherpunks):

 * Attachment "firefoxhttpproxy.jpg" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 [[Image(https://trac.torproject.org/projects/tor/raw-
 attachment/ticket/33010/firefoxhttpproxy.jpg)]]

 Here is a image of captchawtf rejected me.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 {{{
 Expected long-term impact

 Helping Tor users browse the internet without sacrificing privacy and
 getting discriminated
 }}}


 Do you know that CloudFlare is now tracking all users, not just Tor users
 by js_chl_bypass GET paramater?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by cypherpunks):

 CAPTCHA_Monitoring_Project_Diagram.png

 Incorrect:
 "Websites that use Cloudflare as CDN only"

 Correct:
 "Websites that does not route though Cloudflare"


 For bypass.*, user's browser go directly to your WWW server, not
 Cloudflare server.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by ϲypherpunks):

 Hi, i have read your wiki page entry @wosmos

 i have seen under the [wiki:doc/CAPTCHAMonitor#metrics] 1. point, you
 counted my comment into.
 Topic was differences of DualStack. But at the moment, after the change
 from recaptcha to hcaptcha. this might change. Because of time of writing
 this, hcaptcha does not support IPv6 at all, but recaptcha did. While you
 can visit IPv6 Website, the captcha page is including IPv4 hcaptcha.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #33010 [Metrics/Ideas]: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static site

[Tor Bug Tracker & Wiki]

#33010: Monitor cloudflare captcha rate: do a periodic onionperf-like query to a
cloudflare-hosted static site
---+--
 Reporter:  arma   |  Owner:  metrics-team
 Type:  task   | Status:  new
 Priority:  Medium |  Milestone:
Component:  Metrics/Ideas  |Version:
 Severity:  Normal | Resolution:
 Keywords:  network-health gsoc-ideas  |  Actual Points:
Parent ID: | Points:
 Reviewer: |Sponsor:
---+--

Comment (by woswos):

 Replying to [comment:28 cypherpunks]:

 > Incorrect:
 > "Websites that use Cloudflare as CDN only"
 >
 > Correct:
 > "Websites that does not route though Cloudflare"
 > ...

 Fixed it, thank you for the feedback.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs