Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  closed
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:  fixed
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 Replying to [comment:35 dcf]:
 > Replying to [comment:28 isis]:
 > > 2. Either the meek-reflector or the meek-server or the Apache server
 is changing the `X-Forwarded-For` header from `"X-Forwarded-For: 1.2.3.4"`
 to `"X-Forwarded-For: 1.2.3.4, 127.0.0.1"`. This is unfortunate, as it
 means that we're not able to get accurate IP information about the client
 to use for anti-scraping protections (also it bungles the security of the
 CAPTCHAs because it also means clients can "trade" CAPTCHAs and
 solutions). There is a temporary "fix" for this in my `fix/24432-ignore-
 loopback` branch, which simply skips loopback addresses while parsing the
 `X-Forwarded-For` headers.  Eventually we'll need to find which piece of
 infrastructure is setting the IP to 127.0.0.1 and change it to report the
 client's actual IP, then disable this fix (there's a config option). :/
 >
 > I'm pretty sure it's Apache adding the extra `X-Forwarded-For:
 127.0.0.1` (through the `ProxyPass` directive).
 >
 > As for getting the remote client address, you will have to
 [https://gitweb.torproject.org/pluggable-transports/meek.git/tree/meek-
 server/useraddr.go?h=0.28 do what meek-server does]:
 >  1. Use the contents of the `Meek-IP` header, if it exists.
 >  2. Otherwise, use the first entry of `X-Forwarded-For`, if it exists.
 >  3. Otherwise, give up or use the connection's source IP address.
 > The reason you have to check `Meek-IP` is that App Engine
 [https://gitweb.torproject.org/pluggable-
 transports/meek.git/tree/appengine/reflect.go?h=0.28#n77 doesn't set X
 -Forwarded-For], and doesn't allow applications to set it. Other CDNs set
 `X-Forwarded-For` automatically. See #13171.

 Oh thanks! I had no idea about the `Meek-IP` header.

 > I don't think you need special logic for loopback addresses in `x
 -Forwarded-For`: just take the first entry.
 [https://en.wikipedia.org/wiki/X-Forwarded-For#Format According to
 Wikipedia], the client's address is the first entry, not the last:
 >   {{{
 >   X-Forwarded-For: client, proxy1, proxy2
 >   }}}

 BridgeDB's logic (not my code) does exactly the opposite (without any
 comment or explanation as to why).  I've never fully understood why, but I
 assumed it's because it was designed to run only behind the Apache server,
 which is trusted.  So the patch I made to make that code compatible also
 with this new setup—when given a flag to ignore loopback addresses—will
 parse `X-Forwarded-For: 2.2.2.2, 1.2.3.4, 127.0.0.1, 127.0.0.8` into
 `1.2.3.4`. I ''think'' this is probably the most sane thing to do for now.
 I'll go take a took if there's a `Meek-IP` header being sent, and then we
 can just use that.

 > (Hmm, but we should worry about `X-Forwarded-For` spoofing by clients—in
 any case, `Meek-IP` will be trustworthy whenever it exists, because
 reflect.go [https://gitweb.torproject.org/pluggable-
 transports/meek.git/tree/appengine/reflect.go?h=0.28#n35 whitelists] which
 headers it will forward and sets `Meek-IP` itself.)

 Yeah, specifically the part I am worried about is that the `1.2.3.4` only
 ended up in the header because I told curl to put it there… and other than
 that we have no accurate IP for the client.  So right now we're just going
 on whatever they are saying.

 I forgot to mention that I did a temporary config workaround "fix" to try
 to diminish the effect this has, by setting `MOAT_N_IP_CLUSTERS` to `1`
 (instead of `4`).  This has the effect that there are two "clusters" of
 client IPs: those who are using Tor/proxies, and those who aren't. If I
 kept it at `4`, then everyone (who is not using Tor/proxies) could get
 four times as many bridges per period as they are supposed to, simply by
 lying four times (so an average of sixteen tries if they choose IPs
 randomly, but still).

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  closed
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:  fixed
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by dcf):

 Replying to [comment:28 isis]:
 > 2. Either the meek-reflector or the meek-server or the Apache server is
 changing the `X-Forwarded-For` header from `"X-Forwarded-For: 1.2.3.4"` to
 `"X-Forwarded-For: 1.2.3.4, 127.0.0.1"`. This is unfortunate, as it means
 that we're not able to get accurate IP information about the client to use
 for anti-scraping protections (also it bungles the security of the
 CAPTCHAs because it also means clients can "trade" CAPTCHAs and
 solutions). There is a temporary "fix" for this in my `fix/24432-ignore-
 loopback` branch, which simply skips loopback addresses while parsing the
 `X-Forwarded-For` headers.  Eventually we'll need to find which piece of
 infrastructure is setting the IP to 127.0.0.1 and change it to report the
 client's actual IP, then disable this fix (there's a config option). :/

 I'm pretty sure it's Apache adding the extra `X-Forwarded-For: 127.0.0.1`
 (through the `ProxyPass` directive).

 As for getting the remote client address, you will have to
 [https://gitweb.torproject.org/pluggable-transports/meek.git/tree/meek-
 server/useraddr.go?h=0.28 do what meek-server does]:
  1. Use the contents of the `Meek-IP` header, if it exists.
  2. Otherwise, use the first entry of `X-Forwarded-For`, if it exists.
  3. Otherwise, give up or use the connection's source IP address.
 The reason you have to check `Meek-IP` is that App Engine
 [https://gitweb.torproject.org/pluggable-
 transports/meek.git/tree/appengine/reflect.go?h=0.28#n77 doesn't set X
 -Forwarded-For], and doesn't allow applications to set it. Other CDNs set
 `X-Forwarded-For` automatically. See #13171.

 I don't think you need special logic for loopback addresses in `x
 -Forwarded-For`: just take the first entry.
 [https://en.wikipedia.org/wiki/X-Forwarded-For#Format According to
 Wikipedia], the client's address is the first entry, not the last:
   {{{
   X-Forwarded-For: client, proxy1, proxy2
   }}}
 (Hmm, but we should worry about `X-Forwarded-For` spoofing by clients—in
 any case, `Meek-IP` will be trustworthy whenever it exists, because
 reflect.go [https://gitweb.torproject.org/pluggable-
 transports/meek.git/tree/appengine/reflect.go?h=0.28#n35 whitelists] which
 headers it will forward and sets `Meek-IP` itself.)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  closed
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:  fixed
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by isis):

 * status:  reopened => closed
 * resolution:   => fixed


Comment:

 Replying to [comment:33 mcs]:
 > Replying to [comment:30 isis]:
 > > @mcs Do you want to test with your TB against the production server
 again?
 >
 > I did, and it is working. Awesome!

 Yay!

 > We had to change our Tor Launcher Moat client code to send the `id`s as
 strings instead of integers, but that was a trivial change.

 I thought I would need to fix this in the spec too, but the spec already
 said they should be strings not ints, so I was actually doing it wrong
 before.

 > We also switched from asking for `vanilla` bridges to asking for `obfs4`
 ones, because that is what works (I assume the bridges you assigned to the
 Moat distributor were obfs4 ones). For some reason I thought we were
 supposed to request vanilla bridges, but I don't remember where I got that
 idea. Please let Kathy and I know if it is okay to assume that Tor
 Launcher will be able to receive obfs4 bridges via Moat for the
 foreseeable future.

 There are plenty of `obfs4` bridges, so if it's doable on your side
 perhaps we should go with that? They should work better in more countries
 and get blocked less often than vanilla ones, so it'll probably provide a
 better user experience to not have to fill out as many captchas.

 Tentatively closing as "fixed", but of course feel free to reopen if
 there's any more issues.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 Replying to [comment:30 isis]:
 > @mcs Do you want to test with your TB against the production server
 again?

 I did, and it is working. Awesome!

 We had to change our Tor Launcher Moat client code to send the `id`s as
 strings instead of integers, but that was a trivial change.

 We also switched from asking for `vanilla` bridges to asking for `obfs4`
 ones, because that is what works (I assume the bridges you assigned to the
 Moat distributor were obfs4 ones). For some reason I thought we were
 supposed to request vanilla bridges, but I don't remember where I got that
 idea. Please let Kathy and I know if it is okay to assume that Tor
 Launcher will be able to receive obfs4 bridges via Moat for the
 foreseeable future.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 Replying to [comment:31 cypherpunks]:
 > What is moat? Is it a new pluggable transport? Is it documented?

 It is not a new PT; it is a new BridgeDB distribution method. See:
 https://github.com/isislovecruft/bridgedb/#accessing-the-moat-interface

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by cypherpunks):

 What is moat? Is it a new pluggable transport? Is it documented?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 @mcs Do you want to test with your TB against the production server again?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 Okay! Hooray. I've tagged a new version with all these fixes, and uploaded
 to this ticket
 [https://trac.torproject.org/projects/tor/attachment/ticket/24432/moat-
 fetch-and-format-captcha-response_v2.py a fixed version of the test
 script].

 Doing

 {{{
 TEST_PRODUCTION_MOAT=1 ./test-moat fetch > /tmp/moat-fetch
 ./moat-fetch-and-format-captcha-response_v2.py
 }}}

 And (successfully) completing the CAPTCHA, I get the following response:

 {{{
 {"data": [{"qrcode": null, "bridges": ["obfs4 REDACTED:REDACTED REDACTED
 cert=REDACTED iat-mode=0", "obfs4 REDACTED:REDACTED REDACTED cert=REDACTED
 iat-mode=0"], "version": "0.1.0", "type": "moat-bridges", "id": "3"}]}
 }}}

 If I instead type gibberish for the CAPTCHA solution, I get:

 {{{
 {"errors": [{"status": "No You're A Teapot", "code": 419, "detail": "The
 CAPTCHA solution was incorrect.", "version": "0.1.0", "type": "moat-
 bridges", "id": "4"}]}
 }}}

 And if I request a new CAPTCHA and solve it, I get the same two bridges as
 were first returned.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by isis):

 * Attachment "moat-fetch-and-format-captcha-response_v2.py" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 @mcs Thanks, I fixed the typos, the test script seems to be producing the
 same answers as your TB now.

 I've identified a couple more problems:

 1. The "id" field of the JSON that is sent by the client is decoded to a
 string (e.g. `"2"` ''not'' `2`).  (Whooo duck-typing FTL.)  This is
 fixed in my `fix/24432-json-str` branch.
 2. Either the meek-reflector or the meek-server or the Apache server is
 changing the `X-Forwarded-For` header from `"X-Forwarded-For: 1.2.3.4"` to
 `"X-Forwarded-For: 1.2.3.4, 127.0.0.1"`. This is unfortunate, as it means
 that we're not able to get accurate IP information about the client to use
 for anti-scraping protections (also it bungles the security of the
 CAPTCHAs because it also means clients can "trade" CAPTCHAs and
 solutions). There is a temporary "fix" for this in my `fix/24432-ignore-
 loopback` branch, which simply skips loopback addresses while parsing the
 `X-Forwarded-For` headers.  Eventually we'll need to find which piece of
 infrastructure is setting the IP to 127.0.0.1 and change it to report the
 client's actual IP, then disable this fix (there's a config option). :/

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 One more follow up: Kathy and I experimented a little bit with Tor
 Launcher. When communicating with the production BridgeDB server, if we
 include square brackets around the data payload in the `check` request we
 receive a "No bridges available to fulfill request: None." error. If we
 omit the square brackets, we always receive a "The CAPTCHA solution was
 incorrect." error.

 I know we talked about the square brackets / array issue before, at least
 in the context of the JSON responses. Including the brackets in the
 requests makes things work completely when we use a BridgeDB test server
 that we built ourselves (i.e., bridges are returned). But let us know what
 we should be doing, or we can look at your test scripts once they are
 working.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 Replying to [comment:25 isis]:
 >  3. The moat server is erroring with `415 Unsupported Media Type`, but
 that would only happen if it got the HTTP header `Content-Type:
 application/vnd.api+json` but with a media type parameter specified, e.g.
 `Content-Type: application/vnd.api+json;jpeg`. (So it sounds like either
 Apache, the meek reflector, or meek-server is altering the headers?)

 I found a typo in your `moat-fetch-and-format-captcha-response.py` script
 :)
 There is a missing comma at the end of this line:
  '--proxy', 'socks5://127.0.0.1:1/'
 I also had to remove the extra quotes around the header values.

 After I fixed those problems, I always receive a "The CAPTCHA solution was
 incorrect." response (even though I am trying really hard to provide the
 correct answer). With Tor Launcher we still see the "No bridges available
 to fulfill request: None." response, so maybe there is another problem
 with the test script. I did notice that the data "payload" was not
 enclosed in square brackets, but fixing that made no difference.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by isis):

 * Attachment "typescript" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by isis):

 * Attachment "moat-fetch-and-format-captcha-response.py" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 Okay, I think I've found at least one issue, but it appears to be some bad
 interaction between TLS configs between the meek-server, Apache, and the
 moat server:

 If I run:

 {{{
 cd scripts
 TEST_PRODUCTION_MOAT=1 ./test-moat fetch > /tmp/moat-fetch
 ./moat-fetch-and-format-captcha-response.py
 }}}

 where the last script is just something I whipped together for testing
 ([XXX attached]), I get:

 {{{
 
 
 400 Bad Request
 
 Bad Request
 Your browser sent a request that this server could not understand.
 Reason: You're speaking plain HTTP to an SSL-enabled server port.
  Instead use the HTTPS scheme to access this URL, please.
 
 
 Apache Server at bridges.torproject.org Port 443
 
 {"errors": [{"status": "Unsupported Media Type", "code": 415, "detail":
 "", "version": "0.1.0", "type": "", "id": 0}]}
 }}}

 The full log is [XXX attached as a `script` typescript file] (read it with
 `less -r typescript` and beware that it is a raw terminal log including
 escape characters).

 I have no idea why:

  1. Both the Apache server *and* the moat server could answer in the same
 response. (I don't know much about Apache.)
  2. The Apache server is complaining about TLS. (I don't know much about
 meek.)
  3. The moat server is erroring with `415 Unsupported Media Type`, since
 that would only happen if it got the HTTP header `Content-Type:
 application/vnd.api+json` but with a media type parameter specified, e.g.
 `Content-Type: application/vnd.api+json;jpeg`. (It sounds like either
 Apache, the meek reflector, or meek-server is altering the headers?)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 Replying to [comment:23 isis]:
 > For issue !#2 I'll do some more testing and I might need to patch stuff
 a bit to add more verbose logging, because right now I'm not seeing
 anything that indicates why no bridges would be returned.

 Do you want me to open a new ticket to track this issue? Have you had any
 luck tracking own the problem?

 Kathy and I would really like to get the Tor Launcher side of Moat into
 code review (and then to the point where it can be tested by others). But
 I don't think we can do that until this problem is fixed. Thanks!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 Wow, great work, David! Thanks! Let's make #19910 be the ticket for mcs's
 issue !#1 here.

 For issue !#2 I'll do some more testing and I might need to patch stuff a
 bit to add more verbose logging, because right now I'm not seeing anything
 that indicates why no bridges would be returned.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by arma):

 Replying to [comment:21 mcs]:
 > Let's do it! Kathy and I wrestled with the SOCKS optimistic data patch
 once before. It just seems too fragile given the way the networking and
 TLS code is layered inside Firefox. The only thing I don't know is whether
 removing it will impact web page load times in a significant way.
 >
 > Another approach would be add an option to disable the SOCKS optimistic
 data feature on a per-connection basis, which would allow Tor Launcher to
 disable that option when it is using meek directly.

 I figure #19910 is a good place for discussion of whether to rip it out,
 impacts, and workarounds. So I added my thought there.

 (Good stuff here!)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 Replying to [comment:20 gk]:
 > Wow, thanks a lot for this analysis. Let me skip over it to jump to your
 conclusions part...

 Indeed, thanks! David, you are my hero.

 > Given that the analysis shows that at least part of the problem is due
 to the patch itself and how it interacts with the other Firefox networking
 code I think we should back it out and rewrite it if we want to keep it.
 We actually have #19910 for that and I think the OP describes a scenario
 that is compatible with the one you are seeing.

 Let's do it! Kathy and I wrestled with the SOCKS optimistic data patch
 once before. It just seems too fragile given the way the networking and
 TLS code is layered inside Firefox. The only thing I don't know is whether
 removing it will impact web page load times in a significant way.

 Another approach would be add an option to disable the SOCKS optimistic
 data feature on a per-connection basis, which would allow Tor Launcher to
 disable that option when it is using meek directly.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by gk):

 Replying to [comment:19 dcf]:
 > Replying to [comment:10 mcs]:
 > > '''Problem 1:''' The meek tunnel does not work reliably for us.
 Specifically, if we use curl as the SOCKS client it seems to always work
 and if we use Tor Browser it does not. When we test with our own meek-
 server + BridgeDB, things also work fine. I am having trouble debugging
 the meek-client code, probably due to my lack of golang knowledge, but I
 wonder if there is an incompatibility between the meek-client we are
 running and the meek-server that you are running. What version of meek-
 server are you using at tor-bridges-hyphae-channel.appspot.com? Kathy and
 I are using a meek-client that was built from dcf's bug24642 branch (and I
 don't know of any recent changes to meek that would cause this kind of
 communication problem).
 > >
 > > Another data point: if I insert an socat pipe between the meek-client
 SOCKS port and Tor Browser, it started working. Maybe there is a data
 buffering issue at work here. All of our client side testing so far has
 been on macOS.
 >
 > I was debugging this problem today and I traced the cause to Tor
 Browser's [https://gitweb.torproject.org/tor-browser.git/commit/?h=tor-
 browser-52.6.0esr-8.0-2=6d594cc227f153364b91e8310d59a07d7d6ce5f6
 optimistic SOCKS data patch] (#3875), which allows the browser to start
 sending application data right after sending its SOCKS request, before the
 proxy has sent back a reply saying that the connection was successful.

 Wow, thanks a lot for this analysis. Let me skip over it to jump to your
 conclusions part...

 > So a short-term workaround is to add an artificial delay into meek-
 client to prevent it from sending back the SOCKS response immediately. The
 delay has to go [https://gitweb.torproject.org/pluggable-
 transports/meek.git/tree/meek-client/meek-
 client.go?h=bug24642=8c0e2c8601e87687a2f147a875753ac298b52a2e#n278 just
 before conn.Grant] in meek-client.go. I think that's the reason why it
 worked with a socat shim--the extra process tweaked the timing just
 enough.

 Given that the analysis shows that at least part of the problem is due to
 the patch itself and how it interacts with the other Firefox networking
 code I think we should back it out and rewrite it if we want to keep it.
 We actually have #19910 for that and I think the OP describes a scenario
 that is compatible with the one you are seeing.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by dcf):

 * Attachment "demo-socks5.go" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by dcf):

 Replying to [comment:10 mcs]:
 > '''Problem 1:''' The meek tunnel does not work reliably for us.
 Specifically, if we use curl as the SOCKS client it seems to always work
 and if we use Tor Browser it does not. When we test with our own meek-
 server + BridgeDB, things also work fine. I am having trouble debugging
 the meek-client code, probably due to my lack of golang knowledge, but I
 wonder if there is an incompatibility between the meek-client we are
 running and the meek-server that you are running. What version of meek-
 server are you using at tor-bridges-hyphae-channel.appspot.com? Kathy and
 I are using a meek-client that was built from dcf's bug24642 branch (and I
 don't know of any recent changes to meek that would cause this kind of
 communication problem).
 >
 > Another data point: if I insert an socat pipe between the meek-client
 SOCKS port and Tor Browser, it started working. Maybe there is a data
 buffering issue at work here. All of our client side testing so far has
 been on macOS.

 I was debugging this problem today and I traced the cause to Tor Browser's
 [https://gitweb.torproject.org/tor-browser.git/commit/?h=tor-
 browser-52.6.0esr-8.0-2=6d594cc227f153364b91e8310d59a07d7d6ce5f6
 optimistic SOCKS data patch] (#3875), which allows the browser to start
 sending application data right after sending its SOCKS request, before the
 proxy has sent back a reply saying that the connection was successful.
 There are two ways that optimistic data causes problem when using meek-
 client as a SOCKS proxy.

  1. The first is that the SOCKS code in goptlib expects not to get any
 optimistic data: [https://gitweb.torproject.org/pluggable-
 
transports/goptlib.git/tree/socks.go?id=a3ad5df6c9e7dc8117f55958b4ce99bf1e0fe291#n363
 socksReadCommand] finishes by calling [https://gitweb.torproject.org
 /pluggable-
 
transports/goptlib.git/tree/socks.go?id=a3ad5df6c9e7dc8117f55958b4ce99bf1e0fe291#n472
 socksFlushBuffers], which raises an error if there is any data left in the
 read buffer (because otherwise the data would be silently lost when the
 SOCKS code passes the unbuffered socket back to the application). This
 doesn't happen always; it depends on the timing of how fast Firefox sends
 its application data. This issue doesn't seem very serious: we can modify
 the SOCKS code not to use an internal buffer and be tolerant of optimistic
 data. But on its own that is not enough, because
  2. Tor Browser has a race condition when the SOCKS proxy sends back its
 its reply immediately after receiving a request, as meek-client does. The
 SOCKS reply looks like `"\x05\x00\x00\x01\x00\x00\x00\x00\x00\x00"`; when
 it comes back too soon, Tor Browser reads the SOCKS response as if it were
 application data. Trying to parse SOCKS as TLS is what results in the
 connection failure. You can see this by going to
 [https://developer.mozilla.org/en-
 US/docs/Mozilla/Debugging/HTTP_logging#Using_aboutnetworking
 about:networking] and setting the Log Modules
 `timestamp,nsSocketTransport:5,SOCKS:5`. Follow the instructions in
 comment:11, and you will see this in the log:
 {{{
 D/SOCKS socks: DoHandshake(), state = 7
 D/SOCKS socks: ReadFromSocket(), have 2 bytes total
 D/SOCKS socks5: checking auth method reply
 D/SOCKS socks5: server allows connection without authentication
 D/SOCKS socks5: sending connection request (socks5 resolve? yes)
 D/SOCKS socks: DoHandshake(), state = 10
 D/nsSocketTransport   advancing to STATE_TRANSFERRING
 ...
 D/nsSocketTransport nsSocketTransport::OnSocketReady [this=7f2d4c2cbc00
 outFlags=1]
 D/SOCKS socks: DoHandshake(), state = 11
 D/SOCKS socks: ReadFromSocket(), have 5 bytes total
 D/SOCKS socks5: checking connection reply
 E/SOCKS socks5: unexpected version in the reply
 }}}
 "unexpected version in the reply" comes from
 [https://gitweb.torproject.org/tor-
 browser.git/tree/netwerk/socket/nsSOCKSIOLayer.cpp?h=tor-
 browser-52.6.0esr-8.0-2=5e2ac8aed83692c340c8ce219eaa116cf1da2654#n988
 ReadV5ConnectResponseTop]. It's because the application layer and the
 SOCKS layer are fighting over who gets to read data from the socket.
 `state = 10` is [https://gitweb.torproject.org/tor-
 browser.git/tree/netwerk/socket/nsSOCKSIOLayer.cpp?h=tor-
 browser-52.6.0esr-8.0-2=5e2ac8aed83692c340c8ce219eaa116cf1da2654#n55
 SOCKS5_WRITE_CONNECT_REQUEST] and 

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by dcf):

 Replying to [comment:16 mcs]:
 > Replying to [comment:14 isis]:
 > > Google appears to want the passport of someone named "Tor Bridges" and
 the bank statement of the prepaid credit card that I gave, and I have
 neither of these things. Perhaps we can transfer the account ownership to
 TPO? Hopefully? Otherwise we'll need to make a whole new account and do
 #16650 all over again.
 >
 > I hope so. I guess the reflector is still running even though Google has
 suspended the Tor Bridges account? Strange terminology on their part.

 I used prepaid cards for Google and Amazon through 2014 and 2015 (until I
 set up a dedicated bank account for it). I didn't run into this problem,
 but maybe something has changed in their payment processing since then.
 Azure never let me use a prepaid card; it got rejected as such as soon as
 I tried to enter the number.

 If the app works even though the billing is messed up, I would guess that
 the project is running on the 1 GB daily free quota?

 > I also remember that we had to drop meek-google within Tor Browser as a
 PT due to some kind of service violation. If that is likely to happen
 again with the Moat reflector we should switch to Amazon or Azure now I
 suppose.

 The real reason wasn't a ToS violation per se, even though that's what the
 notice said. It was actually because of misuse by a hacking group:
 https://www.bamsoftware.com/papers/thesis/fifield-thesis.pdf#page=63. The
 ToS item that we were unsure about (which is currently
 [https://cloud.google.com/terms/service-terms#20-additional-restrictions
 20.2]), to me, wouldn't apply to Moat: "Customer will not ... use the
 Services to provide a service, Application, or functionality of network
 transport or transmission...".

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 Replying to [comment:16 mcs]:
 > Replying to [comment:14 isis]:
 > > Google appears to want the passport of someone named "Tor Bridges" and
 the bank statement of the prepaid credit card that I gave, and I have
 neither of these things. Perhaps we can transfer the account ownership to
 TPO? Hopefully? Otherwise we'll need to make a whole new account and do
 #16650 all over again.
 >
 > I hope so. I guess the reflector is still running even though Google has
 suspended the Tor Bridges account? Strange terminology on their part.

 Yeah, it's very weird, the account itself is not suspended, I just can't
 pay for anything. Also, I can't access the cloud console to modify or look
 at the instance.

 > I also remember that we had to drop meek-google within Tor Browser as a
 PT due to some kind of service violation. If that is likely to happen
 again with the Moat reflector we should switch to Amazon or Azure now I
 suppose.

 Yeah, probably either Tor's lawyers should try to get a promise from
 Google that we're allowed to do this, or the sysadmin team should set up
 another reflector somewhere else using an account that TPO has financial
 control over?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 Replying to [comment:14 isis]:
 > Google appears to want the passport of someone named "Tor Bridges" and
 the bank statement of the prepaid credit card that I gave, and I have
 neither of these things. Perhaps we can transfer the account ownership to
 TPO? Hopefully? Otherwise we'll need to make a whole new account and do
 #16650 all over again.

 I hope so. I guess the reflector is still running even though Google has
 suspended the Tor Bridges account? Strange terminology on their part.

 I also remember that we had to drop meek-google within Tor Browser as a PT
 due to some kind of service violation. If that is likely to happen again
 with the Moat reflector we should switch to Amazon or Azure now I suppose.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 Replying to [comment:13 isis]:
 > Off the top of my head, I have no idea what this is about, other than
 perhaps the networking code in FF perhaps trying to do something "smart"
 which interacts badly with meek-server?

 That's a good theory, although meek works as a PT within Tor Browser.
 Hmmm.

 > Let me go see what versions are running on the appengine server and on
 polyanthum…
 >
 > The meek-server on polyanthum was from commit
 `017d0f33d7270ff102fe167f1c16def8b3e3be4a` from the end of March. That
 makes sense because that's around the time I did #16650. (I'm not sure if
 anything in the client/server implementations have changed enough to make
 this incompatible?)

 I don't know either. David?

 > So the bridges are assigned to a distributor when they are first seen,
 and they can't be reassigned, in order to keep potential discoverability
 problems with any particular distribution method locally isolated.
 BridgeDB logs these assignments, and, grepping the latest assignments.log,
 it seems like it has 100 bridges for the moat distributor so far. (It's a
 known issue that adding a distributor would take a while to fill up with
 bridges since the way it works is that is has ratios and it assigns
 according to those ratios instead of bringing the levels to the correct
 ratios… I could fix that if we want.) But in any case, with 100 bridges,
 you should be getting at least 1 bridge back? Are you connecting to the
 meek-reflector over tor?

 No, we are not connecting over tor. Is that a requirement?
 I have never receive a bridge, not even one :)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 Replying to [comment:13 isis]:
 > On the AppEngine instance, there should be a meek-reflector from the
 same period. I just logged in however and found this notice? I have no
 idea what this means.
 >
 > [[Image(https://trac.torproject.org/projects/tor/raw-
 attachment/ticket/24432/Screenshot_2018-01-26_00-07-08.png)]]
 >

 Google appears to want the passport of someone named "Tor Bridges" and the
 bank statement of the prepaid credit card that I gave, and I have neither
 of these things. Perhaps we can transfer the account ownership to TPO?
 Hopefully? Otherwise we'll need to make a whole new account and do #16650
 all over again.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by isis):

 * Attachment "Screenshot_2018-01-26_00-07-08.png" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 Replying to [comment:10 mcs]:
 > (I added dcf to the Cc in case he has any insight into problem 1 below).
 >
 > Thanks for your work on this Isis! I feel like we are very close to
 having a working system. Kathy and I have found two problems so far, but I
 am not ready to reopen this ticket yet because I am not sure what
 component is at fault.
 >
 > '''Problem 1:''' The meek tunnel does not work reliably for us.
 Specifically, if we use curl as the SOCKS client it seems to always work
 and if we use Tor Browser it does not. When we test with our own meek-
 server + BridgeDB, things also work fine. I am having trouble debugging
 the meek-client code, probably due to my lack of golang knowledge, but I
 wonder if there is an incompatibility between the meek-client we are
 running and the meek-server that you are running. What version of meek-
 server are you using at tor-bridges-hyphae-channel.appspot.com? Kathy and
 I are using a meek-client that was built from dcf's ​bug24642 branch (and
 I don't know of any recent changes to meek that would cause this kind of
 communication problem).
 >
 > Another data point: if I insert an socat pipe between the meek-client
 SOCKS port and Tor Browser, it started working. Maybe there is a data
 buffering issue at work here. All of our client side testing so far has
 been on macOS.

 Off the top of my head, I have no idea what this is about, other than
 perhaps the networking code in FF perhaps trying to do something "smart"
 which interacts badly with meek-server?

 Let me go see what versions are running on the appengine server and on
 polyanthum…

 The meek-server on polyanthum was from commit
 `017d0f33d7270ff102fe167f1c16def8b3e3be4a` from the end of March. That
 makes sense because that's around the time I did #16650. (I'm not sure if
 anything in the client/server implementations have changed enough to make
 this incompatible?)

 On the AppEngine instance, there should be a meek-reflector from the same
 period. I just logged in however and found this notice? I have no idea
 what this means.

 [[Image()]]

 > '''Problem 2:''' When we do manage to send a good `check` request (one
 that includes the correct response and challenge), we always receive a "No
 bridges available to fulfill request" response. We tried with both
 "vanilla" and "obfs4" transports. Here is a sample response:
 >   {"errors": [{"status": "Not Found", "code": 404, "detail": "No bridges
 available to fulfill request: None.", "version": "0.1.0", "type": "moat-
 bridges", "id": 6}]}
 > Is the Moat responder throttling things so we do not receive too many
 bridges? I don't think we have ever received any bridges from the
 production BridgeDB server via Moat, but even if we did I thought BridgeDB
 would send back the same set of bridges if we ask again. I can get new
 bridges repeatedly if I use a browser to interact with
 https://bridges.torproject.org/bridges?transport=obfs4.

 So the bridges are assigned to a distributor when they are first seen, and
 they can't be reassigned, in order to keep potential discoverability
 problems with any particular distribution method locally isolated.
 BridgeDB logs these assignments, and, grepping the latest assignments.log,
 it seems like it has 100 bridges for the moat distributor so far. (It's a
 known issue that adding a distributor would take a while to fill up with
 bridges since the way it works is that is has ratios and it assigns
 according to those ratios instead of bringing the levels to the correct
 ratios… I could fix that if we want.) But in any case, with 100 bridges,
 you should be getting at least 1 bridge back? Are you connecting to the
 meek-reflector over tor?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 I attached a log from meek-client. It includes some extra logging that
 Kathy and I added to try to make sense of what is going on... but it is
 difficult to reach any solid conclusions without access to the server side
 (it works if we use our own meek-server, although we do not have domain
 fronting in place with our server).

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by mcs):

 * Attachment "meek-client-log1.txt.gz" added.

 meek-client debug log

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  reopened
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by mcs):

 * status:  closed => reopened
 * resolution:  fixed =>


Comment:

 I am reopening now because I have steps to reproduce the Problem 1 that I
 mentioned in comment:10 (and I could still use a response for Problem 2 as
 well). I do not know if Problem 1 is caused by a problem in the meek
 implementation, a problem with the domain fronting setup, or a problem in
 BridgeDB... but we are stuck by it. We have reproduced the problem on
 macOS and Linux. Here are the steps:
 1. Build meek-client from the dcf's bug24642 branch.
 2. Start a standalone copy of the meek-client you built in step 1 and make
 note of which TCP port it is listening on: ./meek-client -url https://tor-
 bridges-hyphae-channel.appspot.com -front www.google.com
 3. Start Tor Browser 8.0a1 and let it connect directly to Tor.
 4. Open about:addons and disable all add-ons.
 5. Restart Tor Browser.
 6. Open about:config again and set network.proxy.socks.port to the port
 that the meek-client is listening on.
 7. Enter https://bridges.torproject.org/moat/fetch in the URL bar.

 Expected result: a 405 Method Now Allowed error page from BridgeDB
 (because GET is not supported for /moat/fetch)

 Actual result: "Unable to connect" error.

 Bonus step: set network.proxy.socks.port to 1 in Tor Browser and
 reload the page after starting a copy of socat:
   socat -v -v TCP-LISTEN:1,fork,reuseaddr TCP-CONNECT:127.0.0:
 (replace  with the port that your meek-client is listening on).
 In my experience, this will work: the 405 error page is correctly returned
 to Tor Browser.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  closed
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:  fixed
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by mcs):

 * cc: dcf (added)


Comment:

 (I added dcf to the Cc in case he has any insight into problem 1 below).

 Thanks for your work on this Isis! I feel like we are very close to having
 a working system. Kathy and I have found two problems so far, but I am not
 ready to reopen this ticket yet because I am not sure what component is at
 fault.

 '''Problem 1:''' The meek tunnel does not work reliably for us.
 Specifically, if we use curl as the SOCKS client it seems to always work
 and if we use Tor Browser it does not. When we test with our own meek-
 server + BridgeDB, things also work fine. I am having trouble debugging
 the meek-client code, probably due to my lack of golang knowledge, but I
 wonder if there is an incompatibility between the meek-client we are
 running and the meek-server that you are running. What version of meek-
 server are you using at tor-bridges-hyphae-channel.appspot.com? Kathy and
 I are using a meek-client that was built from dcf's ​bug24642 branch (and
 I don't know of any recent changes to meek that would cause this kind of
 communication problem).

 Another data point: if I insert an socat pipe between the meek-client
 SOCKS port and Tor Browser, it started working. Maybe there is a data
 buffering issue at work here. All of our client side testing so far has
 been on macOS.

 '''Problem 2:''' When we do manage to send a good `check` request (one
 that includes the correct response and challenge), we always receive a "No
 bridges available to fulfill request" response. We tried with both
 "vanilla" and "obfs4" transports. Here is a sample response:
   {"errors": [{"status": "Not Found", "code": 404, "detail": "No bridges
 available to fulfill request: None.", "version": "0.1.0", "type": "moat-
 bridges", "id": 6}]}
 Is the Moat responder throttling things so we do not receive too many
 bridges? I don't think we have ever received any bridges from the
 production BridgeDB server via Moat, but even if we did I thought BridgeDB
 would send back the same set of bridges if we ask again. I can get new
 bridges repeatedly if I use a browser to interact with
 https://bridges.torproject.org/bridges?transport=obfs4.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  accepted
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 This is issue is fixed in my `fix/24432` branch. The final remaining thing
 causing the "501 missing resources" error was just a bug in the where the
 resources were registered on the Python server versus where the Apache
 server was redirecting to.

 Running the steps above in comment:4 to run the `./scripts/externalize-pt-
 client` script along with `TEST_PRODUCTION_MOAT=1 ./scripts/test-moat`
 script, produces this request:

 {{{
 curl --proxy socks4a://127.0.0.1:1/ -H 'Content-Type:
 application/vnd.api+json' -H 'Accept: application/vnd.api+json' -H 'X
 -Forwarded-For: 1.2.3.4' --data '{"data": [{"supported": ["obfs4"],
 "version": "0.1.0", "type": "client-transports"}]}'
 https://bridges.torproject.org:443/moat/fetch
 }}}

 And the current production server returns the following response:

 {{{
 {"data": [{"challenge":
 "xNAyrt4W7BufeLIWYoRqc4NY1j5Y7XcSPur3nZjjExySWarl0kp3Q-
 
LoFXnCD6net56nvT1FrvyHAGb6ST1-f6KycgMJ4y01nGOKfkCJBqh_PJXajrSh8ruAaXBGwOXOEXnIm3CGLGZXm3pJlaYmynqvo6UVkkRXAi_15AZXQVmll7TMJ_UCpUJmh8QEkVVEjqYRbCJ83V5LRXblQEHR0otDw2FJDjgGHE3
 -0XXl1Gsv5vGq_IJ8LpIrJSQEEGljRWj_dZlHbwdWcQFrFcAD-
 
XMKBh8uHLpPB4ki0eEj9723I1UOFg2TOXxjXiG2kmb6EnsimPDYZMgI2AgSfYTuBUunJOjI4Q8PFEEUHYZ-
 BG2ECCda", "id": 1, "version": "0.1.0", "type": "moat-challenge", "image":
 
"/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQH/2wBDAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQH/wAARCAB9AZADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD+mzV/Hknw4+Id7dfELw9LP4I+EzaPJ4y8T+OoY9L1PxTrWoX2sJ/a/guy1SO7fUPCNlJplre2mi2Gq3+ua1rv2G62PeW11daj2Hgi9+IV74hPi3wn8HfBmofCvxTcvrt94O1jUXv76y0LU4F0+y1G017Vnm8P20l3oPhq
 
G+Xw34auLiy06NtHs5bEWuoSa1bYenw6P4lj1WXQLTxj491nwpr+j2vhDwz411rX9UtNOvi9lrmrfETw/wDEu0tdX16bwhNockWk+HxrB0DSddg07Ura7ab+17meT6D8P6j4H8Xav/wkngu+0241fTPE/h1tdl0O7U6rq32bwjdPa+ILnTbvw4dOs9W1/wAPsmjW2oadb6aNQ08pa2+vSRQ2yqAcPr+sDSfiz8UPD3gd9E8EWfwx+FmgWdlJpj+L7q38HnWGvLvSLC08G20mmeHZje3MGm34m0MXn2y30m90i8QLKsi+E/DnQtA/aB8CWer6r4bji13SviLZaPf3EPiO9sPEfhzwzL8RdP8Aib4G8RaJ4fI1DxPaXHiO/ufCcnjyDWL/AMLanqkNkL+SWWO206eTQ+I3xD8Q+MPFfj74Zw+H9X0z4ieLtc0Ow0y28W2F9rXgS18PX2uNZ+IZfEur6GvhzR7fxDH4Ok0uKTw7ea39o0/Rb7TtQ0W81PVdQuYp+yXxr8M/hr8O/iv8S/DPiDwBqXjT4beN1vfjPN4PtfF3ijw78O9W8N+DNMeXS/GulxX8+o395YaRDo82p6mkOmLbCK18QapZINGuWQA2/F99pfw7m8PeK9QtfDMGqa54g1D4a6JNY+HX8U6z4i8Vatpfh2800+Ira6v9N1OybSrTwbImoWMuqyXphstFM+r3gtZHaTQ9f+ImpeCL7Q9H0i+X4caFH4TuU8eLqNjpN8EuvDeqahe+EvAMlh4huLBrnRr+38P+H7HUotZuNDuDrdzYXQ1G5s9QmuPwe8Vf8HGnwm+OXxYg8AfBXwrc+A/hOfFGkW6/tDfEnQLbSPAVxNqt/ZaZqFn4CmubHUg3xG8eJr2pN4di1Cz1DULrQ9Mm1LTtDvrTU45pP2E/ZF/aJ8DftHfAz4e+M/h1qfhnw98F75fEj6fpvh/U/Fuj2er6PoGg6r4e8TeGNQTxRovhafwj438FytZy3
 
+k3sUds9/p+p3oVLmGSW0APEvgh4+1vxV4o8a+O9MC+IfBfiaS613wH4F17RvDXj/X/AIff8LN025t/7P8ACtx4dNysuoa7r/hqXVrzRbrV/DWm+H9Fku9T16SaaQvae7+G/j/o2jeD/BHhDVPDvxW+H/hnWr+58JR6/o3w91nw3odvZ6xZ6tocGt6lqmn339s6HqHjHxfexahaajHBMt9qD2EWmG/MP9sSewQt4I/ZL+G3gKDQLnSH+Hnhy08X2McegzeHLrUdcvLfwte+JvCWq+Kbux07S5XS00C2unm/sy0vLvzr6yvbm5ktEuryTF+GF78Of2oPhR4C+MXgX4mfFPSND020t/EF1a+L9KsbTQ9dfw3PZazFba5aeLNG1PRrC50/XW00abrilr9IdMebRL94S9ywB5Ro2r6ddeJPCOoxfFTxlqfh/ULLxl4MvvDkiJ4ZbSrHxFsh8H+ITqeiQXGs+I7jV4vF2kTtBqPjm21DUX1mw1+9jgvNLma2+odG+EtxZfDLwpoOn+JtMh0R7i+1zV9Vj8Hadr1p448Iy2lzZ6BoWn6DDNDq+n+JdI0rTvD2oPdaVp6QS3ulNPLb3LyzAcNY+F/A3j7XvilpOrX2keH/ABndeMrL/hK76x07TbHX7Twl4R0G0l0PXNMXVdR1iynabUbCPU9P13Q7bUUYWdvpN/DZ3thLFZdn4NuvFt74317xct1pOufDPS4NT8T+DIL7wRcaT4vtLCbQptKufBmi67cX1le2evXOs2tz4gv3vNMs5rW01JLOyiuLd2ktADw+78VWngrQPD5034u+ArHw1Z+NfHfwuv8ASZb9vB+nxeBpPEGp6Jp/g7wz4d8VvD4WPijwg+iaRp94Fs3NxpD6qlrZrNLaTXVb4GXGs+Dn8CeE9K0D4o6T4t+Lvw5tvE9nZ3mseH9T0LS9C0m1+HvhG6sodMt9T0Xwml9BpljDrGpWMWlR3cmlahql5olzY3ch0ytn4h
 

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  closed
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:  fixed
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by isis):

 * status:  accepted => closed
 * resolution:   => fixed


Comment:

 Tentatively closing as fixed, but please reopen if the issue isn't fixed
 in some way!

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  accepted
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:  #24689| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by mcs):

 * parent:   => #24689


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  accepted
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by mcs):

 Kathy and I are at the point where we would like to give the browser and
 UX team members a new Tor Launcher that supports Moat. To do that, we need
 a test server that is on the public Internet (we have been testing with a
 non-domain fronted server that we built, plus our own copy of meek-
 server). Therefore it would be great if this ticket could be resolved
 soon.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  accepted
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by isis):

 * status:  new => accepted


Old description:

> The apache config has:
>
> ProxyPass /meek/ http://127.0.0.1:2000/
> ProxyPass /moat/ http://127.0.0.1:3881/moat/
> ProxyPass / http://127.0.0.1:3880/ retry=3D10
> ProxyPassReverse / http://127.0.0.1:3880/
>
> (BridgeDB's HTTPS distributor is a Python process listening on port 3880,
> and the moat distributor is listening on 3881.)
>
> The moat-server is run with the following:
>
> ∃!isisⒶwintermute:(master $>)~/code/torproject/bridgedb-admin ∴ cat
> bin/run-meek
> #!/usr/bin/env bash
>
> export TOR_PT_MANAGED_TRANSPORT_VER=1
> export TOR_PT_SERVER_BINDADDR=meek-0.0.0.0:2000
> #export TOR_PT_SERVER_BINDADDR=meek-78.47.38.229:2000
> export TOR_PT_SERVER_TRANSPORTS=meek
> export TOR_PT_ORPORT=127.0.0.1:443
>
> /srv/bridges.torproject.org/bin/meek-server --disable-tls & disown
>
> The moat distributor has two pages, /moat/fetch and /moat/check. In my
> Tor Browser, if I go to https://4-dot-tor-bridges-hyphae-
> channel.appspot.com/meek/moat/fetch I get a "301 Permanent Redirect" from
> the Apache server telling me to go to
> https://bridges.torproject.org/meek/meek/moat/fetch.
>
> Probably I've just configured all the URIs wrong?

New description:

 The apache config has:

 ProxyPass /meek/ http://127.0.0.1:2000/
 ProxyPass /moat/ http://127.0.0.1:3881/moat/
 ProxyPass / http://127.0.0.1:3880/ retry=10
 ProxyPassReverse / http://127.0.0.1:3880/

 (BridgeDB's HTTPS distributor is a Python process listening on port 3880,
 and the moat distributor is listening on 3881.)

 The moat-server is run with the following:

 ∃!isisⒶwintermute:(master $>)~/code/torproject/bridgedb-admin ∴ cat
 bin/run-meek
 #!/usr/bin/env bash

 export TOR_PT_MANAGED_TRANSPORT_VER=1
 export TOR_PT_SERVER_BINDADDR=meek-0.0.0.0:2000
 #export TOR_PT_SERVER_BINDADDR=meek-78.47.38.229:2000
 export TOR_PT_SERVER_TRANSPORTS=meek
 export TOR_PT_ORPORT=127.0.0.1:443

 /srv/bridges.torproject.org/bin/meek-server --disable-tls & disown

 The moat distributor has two pages, /moat/fetch and /moat/check. In my Tor
 Browser, if I go to https://4-dot-tor-bridges-hyphae-
 channel.appspot.com/meek/moat/fetch I get a "301 Permanent Redirect" from
 the Apache server telling me to go to
 https://bridges.torproject.org/meek/meek/moat/fetch.

 Probably I've just configured all the URIs wrong?

--

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  new
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by isis):

 So the meek tunnel is working, but the URL redirections are still not
 quite right…

 David Fifield helped me out by providing this `externalize-pt-client`
 script (now in `bridgedb.git/scripts/externalize-pt-client` in my
 `fix/24432` branch):

 {{{
 #!/bin/bash
 set -e
 CMDLINE="${@:?need a meek-client command line}"
 TRANSPORTS=meek
 # This could be controlled by a command-line arg.
 PORT=1
 TOR_PT_MANAGED_TRANSPORT_VER=1 \
 TOR_PT_CLIENT_TRANSPORTS="$TRANSPORTS" $CMDLINE \
 | sed -n -u -e '/^CMETHOD /{s/^.*127\.0\.0\.1://;p}' | while read
 pt_port; do
 echo "forwarding port $PORT -> $pt_port"
 socat -v -v TCP-LISTEN:$PORT,fork,reuseaddr TCP-
 CONNECT:127.0.0.1:$pt_port &
 done
 }}}

 Also in that branch, I've modified `scripts/test-moat` to also (in
 addition to a localhost server) test the production server through the
 meek tunnel.  This can be accomplished by running David's script like so:

 {{{
 (bdb)∃!isisⒶwintermute:(develop *$>)~/code/torproject/bridgedb ∴ ./scripts
 /externalize-pt-client ~/code/go/src/git.torproject.org/pluggable-
 transports/meek/meek-client/meek-client -url https://tor-bridges-hyphae-
 channel.appspot.com/ -front www.google.com
 }}}

 Then, in another terminal, do:

 {{{
 (bdb)∃!isisⒶwintermute:(fix/24432 $)~/code/torproject/bridgedb ∴
 TEST_PRODUCTION_MOAT=1 ./scripts/test-moat fetch
 }}}

 For me, the bridges.torproject.org server responds with:

 {{{
 {"errors": [{"status": "Not Implemented", "code": 501, "detail": "moat
 version 0.1.0 does not implement POST /moat/fetch", "version": "0.1.0",
 "type": "", "id": 0}]}
 }}}

 Which means:

 1) the meek tunnel through https://tor-bridges-hyphae-channel.appspot.com
 works
 2) the TLS tunneling through the Apache instance on
 bridges.torproject.org:443 works
 3) the meek-server running on polyanthum (the bridges.torproject.org host)
 is correctly stripping the meek layer of TLS
 4) the Apache instance is correctly redirecting to the Twisted Python
 server
 5) the Twisted servers are responding
 6) the moat server specifically is responding, but it isn't giving the
 correct response, since it seems to think there's no such resource

 One thing is that, watching the logs while doing this, the Twisted server
 isn't registering that a request was even made, i.e. there's no logs of
 the request hitting the server (even though its responding with JSON API,
 which is a little weird). So the next step would probably be, in the usual
 Twisted fashion, to just add ridiculous amounts of log statements
 everywhere to figure out why it thinks there's no such resource.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  new
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--

Comment (by dcf):

 It works for me with a few changes.
 {{{
 $ ./externalize-pt-client.sh meek/meek-client/meek-client -url https
 ://tor-bridges-hyphae-channel.appspot.com/ -front www.google.com
 $ curl --proxy socks5://127.0.0.1:1/
 https://bridges.torproject.org/moat/fetch
 {"errors": [{"status": "Not Implemented", "code": 501, "detail": "moat
 version 0.1.0 does not implement GET /moat/fetch", "version": "0.1.0",
 "type": "", "id": 0}]}
 }}}

 4-dot-tor-bridges-hyphae-channel.appspot.com indeed seems to be
 misconfigured (giving the redirect you described), but tor-bridges-hyphae-
 channel.appspot.com seems to work.

 Going to https://tor-bridges-hyphae-channel.appspot.com/moat/fetch in a
 browser is not going to work, because the `/moat/fetch` is at the wrong
 layer. The appspot.com domain is just a tunneling mechanism and doesn't
 use any URL paths. The moat communication, including URL paths, has to
 happen inside the tunnel. In other words,
  * meek-client (outside of tunnel) uses !https://tor-bridges-hyphae-
 channel.appspot.com/
  * curl (inside of tunnel) uses !https://bridges.torproject.org/moat/fetch
 (which indeed it is tunneling to)

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  new
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by dcf):

 * Attachment "externalize-pt-client.sh" added.


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  new
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Changes (by brade):

 * cc: brade, mcs (added)


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+--
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  new
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal| Resolution:
 Keywords:  moat bridgedb-dist|  Actual Points:
Parent ID:| Points:  2
 Reviewer:|Sponsor:  SponsorM
--+--
Description changed by isis:

Old description:

> The apache config has:
>
> ProxyPass /meek/ http://127.0.0.1:2000/
> ProxyPass /moat/ http://127.0.0.1:3881/moat/
> ProxyPass / http://127.0.0.1:3880/ retry=3D10
> ProxyPassReverse / http://127.0.0.1:3880/
>
> (BridgeDB's HTTPS distributor is a Python process listening on port 3880,
> and the moat distributor is listening on 3881.)
>
> The moat-server is run with the following:
>
> ∃!isisⒶwintermute:(master $>)~/code/torproject/bridgedb-admin ∴ cat
> bin/run-meek
> #!/usr/bin/env bash
>
> export TOR_PT_MANAGED_TRANSPORT_VER=3D1
> export TOR_PT_SERVER_BINDADDR=3Dmeek-0.0.0.0:2000
> #export TOR_PT_SERVER_BINDADDR=3Dmeek-78.47.38.229:2000
> export TOR_PT_SERVER_TRANSPORTS=3Dmeek
> export TOR_PT_ORPORT=3D127.0.0.1:443
>
> /srv/bridges.torproject.org/bin/meek-server --disable-tls & disown
>
> The moat distributor has two pages, /moat/fetch and /moat/check. In my
> Tor
> Browser, if I go to
> https://4-dot-tor-bridges-hyphae-channel.appspot.com/meek/moat/fetch I
> get a
> "301 Permanent Redirect" from the Apache server telling me to go to
> https://bridges.torproject.org/meek/meek/moat/fetch.
>
> Probably I've just configured all the URIs wrong?

New description:

 The apache config has:

 ProxyPass /meek/ http://127.0.0.1:2000/
 ProxyPass /moat/ http://127.0.0.1:3881/moat/
 ProxyPass / http://127.0.0.1:3880/ retry=3D10
 ProxyPassReverse / http://127.0.0.1:3880/

 (BridgeDB's HTTPS distributor is a Python process listening on port 3880,
 and the moat distributor is listening on 3881.)

 The moat-server is run with the following:

 ∃!isisⒶwintermute:(master $>)~/code/torproject/bridgedb-admin ∴ cat
 bin/run-meek
 #!/usr/bin/env bash

 export TOR_PT_MANAGED_TRANSPORT_VER=1
 export TOR_PT_SERVER_BINDADDR=meek-0.0.0.0:2000
 #export TOR_PT_SERVER_BINDADDR=meek-78.47.38.229:2000
 export TOR_PT_SERVER_TRANSPORTS=meek
 export TOR_PT_ORPORT=127.0.0.1:443

 /srv/bridges.torproject.org/bin/meek-server --disable-tls & disown

 The moat distributor has two pages, /moat/fetch and /moat/check. In my Tor
 Browser, if I go to https://4-dot-tor-bridges-hyphae-
 channel.appspot.com/meek/moat/fetch I get a "301 Permanent Redirect" from
 the Apache server telling me to go to
 https://bridges.torproject.org/meek/meek/moat/fetch.

 Probably I've just configured all the URIs wrong?

--

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

[tor-bugs] #24432 [Obfuscation/BridgeDB]: The meek<->moat tunneling isn't set up correctly

[Tor Bug Tracker & Wiki]

#24432: The meek<->moat tunneling isn't set up correctly
--+
 Reporter:  isis  |  Owner:  isis
 Type:  defect| Status:  new
 Priority:  High  |  Milestone:
Component:  Obfuscation/BridgeDB  |Version:
 Severity:  Normal|   Keywords:  moat bridgedb-dist
Actual Points:|  Parent ID:
   Points:  2 |   Reviewer:
  Sponsor:  SponsorM  |
--+
 The apache config has:

 ProxyPass /meek/ http://127.0.0.1:2000/
 ProxyPass /moat/ http://127.0.0.1:3881/moat/
 ProxyPass / http://127.0.0.1:3880/ retry=3D10
 ProxyPassReverse / http://127.0.0.1:3880/

 (BridgeDB's HTTPS distributor is a Python process listening on port 3880,
 and the moat distributor is listening on 3881.)

 The moat-server is run with the following:

 ∃!isisⒶwintermute:(master $>)~/code/torproject/bridgedb-admin ∴ cat
 bin/run-meek
 #!/usr/bin/env bash

 export TOR_PT_MANAGED_TRANSPORT_VER=3D1
 export TOR_PT_SERVER_BINDADDR=3Dmeek-0.0.0.0:2000
 #export TOR_PT_SERVER_BINDADDR=3Dmeek-78.47.38.229:2000
 export TOR_PT_SERVER_TRANSPORTS=3Dmeek
 export TOR_PT_ORPORT=3D127.0.0.1:443

 /srv/bridges.torproject.org/bin/meek-server --disable-tls & disown

 The moat distributor has two pages, /moat/fetch and /moat/check. In my Tor
 Browser, if I go to
 https://4-dot-tor-bridges-hyphae-channel.appspot.com/meek/moat/fetch I get
 a
 "301 Permanent Redirect" from the Apache server telling me to go to
 https://bridges.torproject.org/meek/meek/moat/fetch.

 Probably I've just configured all the URIs wrong?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs