[tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
--+
 Reporter:  phw   |  Owner:  phw
 Type:  task  | Status:  assigned
 Priority:  High  |  Milestone:
Component:|Version:
  Circumvention/Obfs4 |
 Severity:  Normal|   Keywords:  sponsor28, anti-censorship-roadmap
Actual Points:|  Parent ID:
   Points:  20|   Reviewer:
  Sponsor:|
  Sponsor28-must  |
--+
 As part of our work for Sponsor 28, we will evaluate and improve the obfs4
 obfuscation protocol.

 Roger started the discussion [https://lists.torproject.org/pipermail/anti-
 censorship-team/2019-May/15.html on our anti-censorship-team mailing
 list]. Relevant reading is the CCS'15 paper
 [https://censorbib.nymity.ch/#Wang2015a Seeing through Network-Protocol
 Obfuscation] and the S&P'16 paper
 [https://censorbib.nymity.ch/#Tschantz2016a SoK: Towards Grounding
 CensorshipCircumvention in Empiricism].

 Let's use this ticket to keep track of this effort.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  october|
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-

Comment (by gaba):

 Do we need a reviewer for this?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  october|
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-

Comment (by phw):

 Replying to [comment:17 gaba]:
 > Do we need a reviewer for this?
 [[br]]
 Eventually, yes, but we're not yet at a point where we would want to merge
 the code.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  october|
Parent ID:   | Points:  20
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor28-must
-+-
Changes (by phw):

 * status:  assigned => needs_review
 * reviewer:   => cohosh


Comment:

 Replying to [comment:18 phw]:
 > Replying to [comment:17 gaba]:
 > > Do we need a reviewer for this?
 > Eventually, yes, but we're not yet at a point where we would want to
 merge the code.
 [[br]]
 I'm currently not working on this, so it's probably a good time to get a
 review and address the resulting feedback.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  october|
Parent ID:   | Points:  20
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor28-must
-+-

Comment (by cohosh):

 This looks great so far. I like the usage of a `net.Conn` to compose
 sharknado with obfs4. I left a few minor comments but no suggested changes
 at the moment. I do have some questions about the implementation:
 - The commit message says
  {{{
 For example, sharknado can -- in
 theory -- turn the following packet sequence:

 client -- 1500 bytes -> server
 client -- 1500 bytes -> server
 client --  500 bytes -> server

 into the following sequences:

 client -- 1500 bytes -> server
 client <- 1500 bytes -- server
 client -- 1500 bytes -> server
 client --  500 bytes -> server
 }}}
  but it looks like it's the client is the one breaking the bursts with
 dummy traffic. Should this instead read
  {{{
 For example, sharknado can -- in
 theory -- turn the following packet sequence:

 client <- 1500 bytes -- server
 client <- 1500 bytes -- server
 client <-  500 bytes -- server

 into the following sequences:

 client <- 1500 bytes -- server
 client -- 1500 bytes -- server
 client <- 1500 bytes -- server
 client <-  500 bytes -- server
 }}}

  Do we want this to happen only at one side of the connection? Or
 eventually for both the client and the server to send dummy traffic? I
 took your comment on deployment speed to mean that we're mostly interested
 in having the client do the shaping.

 - How do servers distinguish between dummy and non-dummy traffic?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  needs_review
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  2020Q1 |
Parent ID:   | Points:  20
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor28-must
-+-
Changes (by gaba):

 * keywords:  sponsor28, anti-censorship-roadmap-october => sponsor28, anti-
 censorship-roadmap-2020Q1


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  needs_revision
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  2020Q1 |
Parent ID:   | Points:  20
 Reviewer:  cohosh   |Sponsor:
 |  Sponsor28-must
-+-
Changes (by phw):

 * status:  needs_review => needs_revision


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
+--
 Reporter:  phw |  Owner:  phw
 Type:  task| Status:  assigned
 Priority:  High|  Milestone:
Component:  Circumvention/Obfs4 |Version:
 Severity:  Normal  | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap  |  Actual Points:
Parent ID:  | Points:  20
 Reviewer:  |Sponsor:
|  Sponsor28-must
+--
Changes (by robgjansen):

 * cc: robgjansen (added)


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
+--
 Reporter:  phw |  Owner:  phw
 Type:  task| Status:  assigned
 Priority:  High|  Milestone:
Component:  Circumvention/Obfs4 |Version:
 Severity:  Normal  | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap  |  Actual Points:
Parent ID:  | Points:  20
 Reviewer:  |Sponsor:
|  Sponsor28-must
+--
Changes (by phw):

 * cc: msherr (added)


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  august |
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-
Changes (by gaba):

 * keywords:  sponsor28, anti-censorship-roadmap => sponsor28, anti-
 censorship-roadmap-august


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  august |
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-
Description changed by phw:

Old description:

> As part of our work for Sponsor 28, we will evaluate and improve the
> obfs4 obfuscation protocol, which may result in obfs5.
>
> Roger started the discussion [https://lists.torproject.org/pipermail
> /anti-censorship-team/2019-May/15.html on our anti-censorship-team
> mailing list]. Relevant reading is the CCS'15 paper
> [https://censorbib.nymity.ch/#Wang2015a Seeing through Network-Protocol
> Obfuscation] and the S&P'16 paper
> [https://censorbib.nymity.ch/#Tschantz2016a SoK: Towards Grounding
> CensorshipCircumvention in Empiricism].
>
> Let's use this ticket to keep track of this effort. Below is a list of
> ideas that we may or may not want to incorporate in obfs5.
>
> == Randomisation
>
> Obfs4 already implements randomisation for packet lengths and inter-
> arrival times but there are other protocol aspects that we can randomise.
> Note that the adoption of these strategies may complicate censorship
> analysis: if obfs5 instance X looks very different from obfs5 instance Y,
> then X may end up getting blocked while Y still works. Instead of saying
> "obfs5 is blocked," one may then have to be more specific and say "the
> obfs5 instances that rely on UDP are blocked."
>
> * **Payload**: All bytes that obfs4 writes to the wire are randomly
> distributed. These high-entropy packets may or may not be common on the
> Internet. We could evade a "high-entropy filter" by having obfs4 servers
> derive a formal language from the shared secret. This language could,
> say, use dummy clear-text headers.
>
> * **Cover traffic**: [https://lists.torproject.org/pipermail/tor-
> dev/2017-June/012310.html dcf] explains that obfs4 only sends data when
> it's given data to send. To improve on this, as dcf suggests, we could
> make obfs5 send data even when the application has nothing to send.
>
> * **Packet directions**: An obfs4 flow begins with the client sending
> data to the server. We could randomise packet directions and have, say,
> the server talk first with a server-specific probability.
>
> * **Transport protocol**: An obfs4 server could talk either TCP or UDP or
> SCTP. This may very well not be worth the effort.
>
> == Lessons learned from [https://censorbib.nymity.ch/#Wang2015a CCS'15
> paper]
>
> * DPI boxes tend to classify flows by only inspecting the first N packets
> of a flow. Keeping state is expensive, after all. We could exploit this
> by relaxing our obfuscation techniques after N packets to increase
> throughput.
>
> * The paper's data set may not be representative of what countries or
> ISPs would see:
>   * It's "only" a university uplink. Universities typically have policies
> that prohibit file sharing such as BitTorrent. BitTorrent's "message
> stream encryption" may look similar to obfs3 and obfs4.
>   * The data sets are from 2014, 2012, and 2010, respectively. That's a
> long time in Internet years.
>   * The detectors' false positive rates are non-trivial and, as the
> authors point out themselves, would be problematic for a censor given
> that non-obfuscated traffic significantly outweighs obfuscated traffic.
>   * Does the data set only contain one obfs4 server instance? This may
> have affected their results.
>
> == Miscellaneous
>
> * [https://trac.torproject.org/projects/tor/ticket/30716#comment:1
> yawning writes] that obfs4 doesn't easily support backward incompatible
> protocol alterations.
>
> * [https://trac.torproject.org/projects/tor/ticket/30716#comment:3
> yawning writes] that the framing could use better cryptography.
>
> * Crazy idea: Use a modified TCP stack that ignores RST and FIN segments,
> so the GFW's on-path devices cannot tear down the connection. Instead,
> the obfs5 protocol could signal the end of the connection in an
> authenticated control frame. We could ignore RST and FIN segments by
> using firewall rules, or to get more crazy, by shipping a user space TCP
> stack (this may be easy to fingerprint, though).

New description:

 As part of our work for Sponsor 28, we will evaluate and improve the obfs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  august |
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-
Changes (by phw):

 * cc: dcf (added)


Comment:

 Website fingerprinting attacks typically operate on traffic traces that
 are frequently encoded as sequences of the form:
 {{{
 ,+/-
 }}}
 `+` refers to packets going from the client to the server
 and `-` refers to packets going from the server to the
 client. For example:
 {{{
 1567548098,+1500
 1567548098,+800
 1567548099,-1500
 1567548099,-1500
 1567548100,-700
 }}}
 Interestingly, packet lengths may not even be necessary. In their
 [https://arxiv.org/pdf/1801.02265.pdf CCS'18 paper], Sirinam et al. write
 in Section 5.1.1:
 > However, we performed preliminary evaluations to compare the WF attack
 performance between using packet lengths and without packet lengths, i.e.,
 only packet direction, as feature representations. Our result showed that
 using packet lengths does not provide a noticeable improvement in the
 accuracy of the attack. Therefore, we follow Wang et al.’s methodology and
 consider only the direction of the packets.
 The traffic trace above can therefore be reduced to:
 {{{
 +1
 +1
 -1
 -1
 -1
 }}}
 Note that obfs4 makes no attempt to defend against website fingerprinting
 attacks. Its goal is to escape protocol classification but these two
 problems (and their respective attacks) overlap to some extent, which is
 why obfs4 would be better off with defences against such attacks.

 [https://lists.torproject.org/pipermail/tor-dev/2017-June/012310.html As
 dcf already pointed out], obfs4 only sends data when the application
 (e.g., Tor) has data to send. Then, depending on what iatMode is used,
 obfs4 may append padding to the application's data and add inter-arrival
 delays. Coming back to the example above, obfs4 can only **extend** a
 packet burst but not **break** a burst. That is, obfs4 can turn the packet
 sequence
 {{{
 +1
 +1
 -1
 -1
 -1
 }}}
 into the sequence
 {{{
 +1
 +1
 +1 (padding packet, which extends a burst)
 -1
 -1
 -1
 }}}
 but not into the sequence
 {{{
 +1
 -1 (padding packet, which breaks a burst)
 +1
 -1
 +1 (padding packet, which breaks a burst)
 -1
 -1
 }}}
 I spent some time looking into ways to fix this issue. It turns out that
 we can add the ability to break packet bursts to obfs4 without losing
 backwards compatibility, allowing a brand-new, burst-breaking obfs4 client
 to talk to an old obfs4 server (however, see below for a caveat). I
 implemented a simple proof-of-concept, for now called
 
[https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports/BabyNameBook
 sharknado], in my
 
[https://dip.torproject.org/phw/obfs4/commit/8da050f29866444b9af685d277c20b7ab142593a
 feature/30716 branch]. The idea is simple: instead of having obfs4 write
 directly to its socket, it now writes to the `SharknadoConn` struct, which
 implements the `net.Conn` interface.  After each call to `Read`, there's a
 1 in 10 chance to send padding, regardless of if the application has data
 waiting or not.

 There are several remaining challenges:
 * Effectively breaking bursts may require the client and the server to
 cooperate. For example, when the client receives the beginning of a burst,
 the adversary (who's somewhere between the client and the server) may
 already have seen the entire packet sequence, so we cannot break it
 anymore. We may be able to address this by having the server send only a
 few packets of its burst and then waiting until it received the client's
 burst-breaking packets.
 * We should find a way to make obfs4's packet sequences server-specific by
 incorporating the server's shared secret into the sequence generation
 process, just like it's done for packet lengths and inter-arrival times.
 * We need to build an evaluation framework to understand what works and
 what doesn't.

 Any thoughts?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bi

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  august |
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-

Comment (by cohosh):

 Nice! Thanks for sharing the update on this!

 I have a few questions and comments about the website fingerprinting
 approach. The observation that these two problems have some overlap is a
 good one, but it's also worth noting that it's not necessarily the case
 that implementing website fingerprinting defences will be strictly better
 for a pluggable transport from an obfuscation point of view.

 The main goal for a website fingerprinting defence is to prevent an
 attacker from learning which site you're visiting, not from learning that
 you are using Tor or even that you are using a website fingerprinting
 defence. And while the website fingerprinting portion is good to have, I
 think the current understanding of website fingerprinting of Tor traffic
 is that it makes more sense to apply to Onion Services at the moment and
 isn't urgent for the rest of Tor traffic yet? This is also being actively
 worked on I believe. Of course PTs don't have to only be used with Tor and
 this would make them better if used for other anti-censorship tools.

 > - We need to build an evaluation framework to understand what works and
 what doesn't.

 So my concern here is that even though the packet breaking is
 probabilistic (and eventually variable, etc.), can we do it without adding
 a fingerprintable feature to the obfuscation traffic? I suppose this is
 related to the "long tail" argument that we have, where hopefully by
 making the sharknado traffic look less and less like any regular thing,
 we're increasing the uncertainty a censor would have in blocking it.

 And it brings us to your bullet point above. It would be an interesting
 research question perhaps to take a look at existing website
 fingerprinting work and see how identifiable the defences are as they are
 implemented in those works given that the adversary knows the client is
 using Tor.

 > - We should find a way to make obfs4's packet sequences server-specific
 by incorporating the server's shared secret into the sequence generation
 process, just like it's done for packet lengths and inter-arrival times.

 Was the purpose of this to maintain some consistency for censors who are
 observing all traffic to a specific bridge IP address (which might make it
 less suspicious)? Or to try to introduce some variability for different
 obfs bridges in order to add to the confusion?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  august |
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-

Comment (by phw):

 Here's a brief overview of where we're at.  We should improve both obfs4's
 payload ''and'' flow obfuscation. My
 [https://trac.torproject.org/projects/tor/ticket/30716#comment:10 above
 comment] gave an overview of how we can improve flow obfuscation. To
 clarify: blindly incorporating WF defences is unlikely to succeed. We
 should instead extend obfs4's flow obfuscation and build an evaluation
 framework (which can be inspired by WF work).

 The main issue with obfs4's payload obfuscation is that it consists
 entirely of uniformly distributed bytes for which Wang et al. built a
 reliable classifier in their CCS'15 paper.  We should find a way to reduce
 obfs4's per-packet entropy – at least for the first ''n'' packets,
 assuming that our adversary classifies a flow in the first ''n'' packets.
 Keeping flow state is costly, after all.

 We identified two approaches for reducing obfs4's per-packet entropy:

 1. Adopting LibFTE. We could derive regular expressions from obfs4's
 shared secret that we then feed into LibFTE. These regular expressions
 should make obfs4 look like "a protocol with structure."

 2. Building an entropy "modifier". A silly implementation of this idea
 could insert a 0-byte after every second obfs4 byte, which would reduce
 the stream's entropy.  Again, the way this entropy modifier works should
 be derived from obfs4's shared secret.  We should parameterise the
 modifier with an inflation factor ''i'', which we can gradually scale back
 to 0, to improve goodput once our adversary classified the flow.

 As cohosh pointed out, the challenge lies in making these improvements
 without having us stand out.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  august |
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-

Comment (by phw):

 We briefly discussed the interaction between Tor's WF defences and obfs4's
 flow obfuscation in #tor-dev. Here's a summary:
 * We want a clear separation of responsibilities: Tor's circuit padding
 defends against WF attacks while obfs4 defends against traffic
 classification.
 * We want defence in depth. If an adversary breaks obfs4, she should not
 be able to fingerprint the encapsulated Tor stream, to learn what website
 the user is visiting.
 * That said, the defence in depth should come with as little overhead as
 possible. Every padding byte (be it from obfs4 or the application) is a
 byte that is no longer goodput. If performance suffers too much, users
 will turn to other circumvention tools.

 Here are some additional research questions:

 * Both obfs4 and Tor add padding to defend against traffic classifiers and
 WF attacks, respectively. Can we combine these two systems in a smart way
 that minimises overhead while retaining our security properties? (Keep in
 mind that obfs4 is only present between client and bridge while Tor's WF
 defences may be present between client and middle relay.)

 * Can obfs4's padding negatively affect Tor's padding and vice versa?

 * How does the application data that goes into obfs4 affect our resistance
 to traffic classifiers? In other words: Is obfs4-transporting-tor more
 resistant to classifiers than obfs4-transporting-vpn?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  october|
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-
Changes (by gaba):

 * keywords:  sponsor28, anti-censorship-roadmap-august => sponsor28, anti-
 censorship-roadmap-october


--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  october|
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-

Comment (by phw):

 I stumbled upon an implementation issue that's also worth fixing:
 obfs4proxy always closes an obfs4 connection after 30 seconds if a client
 was unable to authenticate itself. This facilitates active probing
 attacks. We should instead close connections after a server-specific,
 randomly-determined time interval.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
-+-
 Reporter:  phw  |  Owner:  phw
 Type:  task | Status:
 |  assigned
 Priority:  High |  Milestone:
Component:  Circumvention/Obfs4  |Version:
 Severity:  Normal   | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap-  |  Actual Points:
  october|
Parent ID:   | Points:  20
 Reviewer:   |Sponsor:
 |  Sponsor28-must
-+-

Comment (by phw):

 Here's a prototype of the code I've been working on:
 https://dip.torproject.org/phw/obfs4/tree/feature/30716

 It improves obfs4 by giving it the ability to send dummy traffic at any
 time. In the background, the code determines ''when'' it’s time to send
 dummy traffic and ''how much'' should be sent – both values are sampled
 from a Poisson distribution. The arguments to these two distributions are
 (for now) derived from an obfs4 server's public key, so each obfs4 server
 has its own, unique flow signature.

 The idea is that it should be difficult for an adversary to train a single
 classifier that can detect all obfs4 servers. The adversary should instead
 have to train a classifier for each obfs4 server out there. We expect our
 obfs4 evaluation to 1) show what features traffic classifiers rely on and
 2) how to fine-tune the parameters for our flow shaping algorithm.

 So far, I implemented the flow shaping code as a `net.Conn` wrapper in the
 obfs4 client for the following reasons:
 * **Simplicity**: By having only the client keep track of when it’s time
 to inject padding, we keep the implementation simple and make it easy to
 reason about our approach. Note that our obfs4 evaluation may suggest that
 this approach is insufficient, so we should be open to the possibility of
 making the client and server cooperate on flow shaping.
 * **Deployment speed**: Clients can update more rapidly than servers
 because of Tor Browser's auto-update feature.
 * **Backwards-compatibility**: By having the client do all of the flow
 shaping, there is no need for client-server coordination, which means that
 we don’t need to modify the obfs4 protocol. We can therefore update Tor
 Browser with our obfs4 improvements and benefit from these improvements
 without requiring existing obfs4 bridges to update.
 * **Established design**: Other projects
 [https://lists.torproject.org/pipermail/tor-dev/2015-September/009526.html
 have been stacking net.Conn connections on top of each other], hopefully
 making our design more useful for other projects.

 Note that this is work-in-progress and not ready to be used. Several
 questions remain:
 * How well does our flow obfuscator fend off protocol classifiers?
 * How should we choose the arguments to our flow obfuscator?
 * Should we incorporate additional features like a heartbeat? The
 prototype starts a heartbeat with probability 1/5.
 * Should we turn off flow obfuscation after a certain number of seconds
 have passed, or a certain number of bytes/packets have been transferred?

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
+--
 Reporter:  phw |  Owner:  phw
 Type:  task| Status:  assigned
 Priority:  High|  Milestone:
Component:  Circumvention/Obfs4 |Version:
 Severity:  Normal  | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap  |  Actual Points:
Parent ID:  | Points:  20
 Reviewer:  |Sponsor:
|  Sponsor28-must
+--

Comment (by yawning):

 One of the design deficiencies of the obfs4 protocol is that it doesn't
 easily/efficiently support backward incompatible protocol alterations.

 There are ways around this, but at that point, people are better off
 writing a new different/backwards incompatible protocol entirely, that
 fixes a number of the design flaws in the underlying protocol.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
+--
 Reporter:  phw |  Owner:  phw
 Type:  task| Status:  assigned
 Priority:  High|  Milestone:
Component:  Circumvention/Obfs4 |Version:
 Severity:  Normal  | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap  |  Actual Points:
Parent ID:  | Points:  20
 Reviewer:  |Sponsor:
|  Sponsor28-must
+--

Comment (by dcf):

 The
 
[https://gitlab.com/yawning/obfs4/blob/obfs4proxy-0.0.10/doc/obfs4-spec.txt#L237
 obfs4 framing format] is pretty nice, in that is allow arbitrary shaping:
 both client and server can send any amount of data, at any time. The only
 exception is at one point during the handshake: after the client has sent
 the MAC indicating the end of its padding, the client must remain silent
 until after the server has sent ''its'' part of the handshake. You can see
 the gap in the bottom two graphs at
 https://people.torproject.org/~dcf/obfs4-timing/.

 So one desideratum from me is that the protocol should allow either side
 to send any amount of data at any time, and have it correctly interpreted
 as padding or meaningful data. Ideally it should even be possible for the
 server to send data before the client has sent anything.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
+--
 Reporter:  phw |  Owner:  phw
 Type:  task| Status:  assigned
 Priority:  High|  Milestone:
Component:  Circumvention/Obfs4 |Version:
 Severity:  Normal  | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap  |  Actual Points:
Parent ID:  | Points:  20
 Reviewer:  |Sponsor:
|  Sponsor28-must
+--

Comment (by yawning):

 The framing could use better cryptography and a more sensible design
 overall, but there are larger deficiencies in the protocol.

 > So one desideratum from me is that the protocol should allow either side
 to send any amount of data at any time, and have it correctly interpreted
 as padding or meaningful data. Ideally it should even be possible for the
 server to send data before the client has sent anything.

 At one point I had thoughts of how I would like to implement something
 like this, but it's been years since I gave serious thought about this
 problem.  I personally would have felt uneasy about a responder-speaks-
 first design.

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
+--
 Reporter:  phw |  Owner:  phw
 Type:  task| Status:  assigned
 Priority:  High|  Milestone:
Component:  Circumvention/Obfs4 |Version:
 Severity:  Normal  | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap  |  Actual Points:
Parent ID:  | Points:  20
 Reviewer:  |Sponsor:
|  Sponsor28-must
+--
Description changed by phw:

Old description:

> As part of our work for Sponsor 28, we will evaluate and improve the
> obfs4 obfuscation protocol.
>
> Roger started the discussion [https://lists.torproject.org/pipermail
> /anti-censorship-team/2019-May/15.html on our anti-censorship-team
> mailing list]. Relevant reading is the CCS'15 paper
> [https://censorbib.nymity.ch/#Wang2015a Seeing through Network-Protocol
> Obfuscation] and the S&P'16 paper
> [https://censorbib.nymity.ch/#Tschantz2016a SoK: Towards Grounding
> CensorshipCircumvention in Empiricism].
>
> Let's use this ticket to keep track of this effort.

New description:

 As part of our work for Sponsor 28, we will evaluate and improve the obfs4
 obfuscation protocol.

 Roger started the discussion [https://lists.torproject.org/pipermail/anti-
 censorship-team/2019-May/15.html on our anti-censorship-team mailing
 list]. Relevant reading is the CCS'15 paper
 [https://censorbib.nymity.ch/#Wang2015a Seeing through Network-Protocol
 Obfuscation] and the S&P'16 paper
 [https://censorbib.nymity.ch/#Tschantz2016a SoK: Towards Grounding
 CensorshipCircumvention in Empiricism].

 Let's use this ticket to keep track of this effort.

 Suggestions for improvement:
 * [https://trac.torproject.org/projects/tor/ticket/30716#comment:1 yawning
 writes] that obfs4 doesn't easily support backward incompatible protocol
 alterations.
 * [https://trac.torproject.org/projects/tor/ticket/30716#comment:3 yawning
 writes] that the framing could use better cryptography.
 * [https://trac.torproject.org/projects/tor/ticket/30716#comment:2 dcf
 writes] that during the handshake, the client needs to wait for the server
 before it can send more data. A [https://lists.torproject.org/pipermail
 /tor-dev/2017-June/012310.html tor-dev@] post has more detail.
 * Each obfs4 server has a unique flow signature. Can we make packet
 payload unique to each server too? For example, can we automatically
 derive a formal language, so not all obfs4 instances send high-entropy
 data?

--

--
Ticket URL: 
Tor Bug Tracker & Wiki 
The Tor Project: anonymity online
___
tor-bugs mailing list
tor-bugs@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

Re: [tor-bugs] #30716 [Circumvention/Obfs4]: Improve the obfs4 obfuscation protocol

[Tor Bug Tracker & Wiki]

#30716: Improve the obfs4 obfuscation protocol
+--
 Reporter:  phw |  Owner:  phw
 Type:  task| Status:  assigned
 Priority:  High|  Milestone:
Component:  Circumvention/Obfs4 |Version:
 Severity:  Normal  | Resolution:
 Keywords:  sponsor28, anti-censorship-roadmap  |  Actual Points:
Parent ID:  | Points:  20
 Reviewer:  |Sponsor:
|  Sponsor28-must
+--
Description changed by phw:

Old description:

> As part of our work for Sponsor 28, we will evaluate and improve the
> obfs4 obfuscation protocol.
>
> Roger started the discussion [https://lists.torproject.org/pipermail
> /anti-censorship-team/2019-May/15.html on our anti-censorship-team
> mailing list]. Relevant reading is the CCS'15 paper
> [https://censorbib.nymity.ch/#Wang2015a Seeing through Network-Protocol
> Obfuscation] and the S&P'16 paper
> [https://censorbib.nymity.ch/#Tschantz2016a SoK: Towards Grounding
> CensorshipCircumvention in Empiricism].
>
> Let's use this ticket to keep track of this effort.
>
> Suggestions for improvement:
> * [https://trac.torproject.org/projects/tor/ticket/30716#comment:1
> yawning writes] that obfs4 doesn't easily support backward incompatible
> protocol alterations.
> * [https://trac.torproject.org/projects/tor/ticket/30716#comment:3
> yawning writes] that the framing could use better cryptography.
> * [https://trac.torproject.org/projects/tor/ticket/30716#comment:2 dcf
> writes] that during the handshake, the client needs to wait for the
> server before it can send more data. A
> [https://lists.torproject.org/pipermail/tor-dev/2017-June/012310.html
> tor-dev@] post has more detail.
> * Each obfs4 server has a unique flow signature. Can we make packet
> payload unique to each server too? For example, can we automatically
> derive a formal language, so not all obfs4 instances send high-entropy
> data?

New description:

 As part of our work for Sponsor 28, we will evaluate and improve the obfs4
 obfuscation protocol, which may result in obfs5.

 Roger started the discussion [https://lists.torproject.org/pipermail/anti-
 censorship-team/2019-May/15.html on our anti-censorship-team mailing
 list]. Relevant reading is the CCS'15 paper
 [https://censorbib.nymity.ch/#Wang2015a Seeing through Network-Protocol
 Obfuscation] and the S&P'16 paper
 [https://censorbib.nymity.ch/#Tschantz2016a SoK: Towards Grounding
 CensorshipCircumvention in Empiricism].

 Let's use this ticket to keep track of this effort. Below is a list of
 ideas that we may or may not want to incorporate in obfs5.

 == Randomisation

 Obfs4 already implements randomisation for packet lengths and inter-
 arrival times but there are other protocol aspects that we can randomise.
 Note that the adoption of these strategies may complicate censorship
 analysis: if obfs5 instance X looks very different from obfs5 instance Y,
 then X may end up getting blocked while Y still works. Instead of saying
 "obfs5 is blocked," one may then have to be more specific and say "the
 obfs5 instances that rely on UDP are blocked."

 * **Payload**: All bytes that obfs4 writes to the wire are randomly
 distributed. These high-entropy packets may or may not be common on the
 Internet. We could evade a "high-entropy filter" by having obfs4 servers
 derive a formal language from the shared secret. This language could, say,
 use dummy clear-text headers.

 * **Cover traffic**: [https://lists.torproject.org/pipermail/tor-
 dev/2017-June/012310.html dcf] explains that obfs4 only sends data when
 it's given data to send. To improve on this, as dcf suggests, we could
 make obfs5 send data even when the application has nothing to send.

 * **Packet directions**: An obfs4 flow begins with the client sending data
 to the server. We could randomise packet directions and have, say, the
 server talk first with a server-specific probability.

 * **Transport protocol**: An obfs4 server could talk either TCP or UDP or
 SCTP. This may very well not be worth the effort.

 == Lessons learned from [https://censorbib.nymity.ch/#Wang2015a CCS'15
 paper]

 * DPI boxes tend to classify flows by only inspecting the first N packets
 of a flow. Keeping state is expensive, after all. We could exploit this by
 relaxing our obfuscation techniques after N packets to increase
 throughput.

 * The paper's data set may not be representative of what countries or ISPs
 would see:
   * It's "only" a university uplink. Universities typically have policies
 that prohibit file sharing such as BitTorrent. BitTorrent's "message
 stream encryption" may look similar t