Re: [tor-dev] Have the Tor daemon use more cores
On Tue, 1 Dec 2015 09:19:45 -0500 Jeremy <jer...@sturgix.com> wrote: > I've been working on the volunteer project described here > https://www.torproject.org/getinvolved/volunteer.html.en#useMoreCores > but can't spend much more time on it. > > Right now, I have refactored circuit_receive_relay_cell() in relay.c > (which calls relay_crypt() and eventually the AES crypt routines) to > use the workqueue.c infrastructure similar to cpuworker.c. > > When the refactored code runs in single threaded mode, all seems > good. Once I activate the thread pool and start sending it work with > threadpool_queue_work(), it Bootstraps 100% okay and runs for several > minutes before crashing on cells it doesn't handle properly. > > I'm happy to share my code with anyone interested. Sure I think I'm the person that used the workqueue stuff last, so I'd be up for doing the rest of the work (nb: I am traveling/in meetings for the rest of the week, so it'll be next week at the earliest that I can really spend quality time). Regards, -- Yawning Angel pgp3QgpFuOMph.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] displaying an ed25519 signing key's expiry date
On Thu, 19 Nov 2015 11:42:16 + nusenu <nus...@openmailbox.org> wrote: > >> How can a tor relay op display a given signing key's expiry date? > >> > > > I don't think there is an option for this. > > filed a ticket for it: > https://trac.torproject.org/projects/tor/ticket/17639 > > > Is there a custom openssl command to display the expiry date until > this gets implemented in tor? No. The on disk Ed25519 key format is custom to Tor, and the code doesn't use OpenSSL for any of the Ed25519 operations anyway. Someone that wants to work on this should find the relevant data formats documented in prop 220. Regards, -- Yawning Angel pgpJcH4BDshFu.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal: Link Cryptographic Agility
On Mon, 16 Nov 2015 21:58:41 + isis <i...@torproject.org> wrote: > Hey Yawning, > > I'm generally in favour of merging this, but I'll wait a couple days > to see if anyone has any feedback. Particularly, I'd like to hear if > Nick has any comments. I talked to Nick about this, and now think that adding a line or two to each microdescriptor probably is less painful. So no need to merge this. > What new behaviour do we need from RELAY_EARLY? If I understood > prop#249 correctly, RELAY_EARLY should work as before (in particular > with 8-9 total RELAY_EARLYs allowed per circuit construction), but > that (potentially multipartite) EXTEND2(s) within RELAY_EARLY(s) > should contain variable length authentication data. Are you just > concerned that we'll go over the 8-9 cell limit and open ourselves > back up to infinite circuit attacks? Or am I missing something? Since this is an orthogonal issue... Until the changes that Nick added to #249 last night, RELAY_EARLY behavior was unspecified for fragmented EXTEND cells. The pedantic interpretation could have been "all fragments must be contained in RELAY_EARLY", which wouldn't let you build circuits consisting of more than 1 hop. Since only the first fragment needs to be in RELAY_EARLY now, there's no further issues. I'm gonna poke at some other things (in particular I'm multithreading the rest of our circuit build crypto) for a bit before I come back to this, but I have a rough idea of what I want our PQ options to look like. Regards, -- Yawning Angel pgpqk8e9qQLKD.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Proposal: Link Cryptographic Agility
Hello,
There's been talk of moving to a wide block construct for cell crypto
(#5460), and for adding PQ Forward Secrecy to traffic (#17272). To
facilitate this, there needs to be a method for negotiating which
primitives a given relay supports.
To that end, here's the begining of a proposal (tentatively #260) that
adds a `CIPHERSUITES` cell to the version negotiation process which
allows relays to advertise the link cryptography primitives that they
are willing to accept.
TODOs/TBD:
* Should the new cell be variable length? Our development cycle and
how often we switch handshakes/link crypto tells me "No", the
existing "VERSIONS" cell format tells me "Yes".
* We need to revise prop. 249 to add a cell crypto method specifier to
CREATE2V/EXTEND.
* Proposal 249 does not cover changing RELAY_EARLY behavior, which
will potentially cause problems (Eg: Ring-LWE + ntor will require
between 5 to 9 fragments depending on the exact primitives).
* The actual hybrid construct is deliberately left unspecified.
* With runtime negotiation of primitives on a per-circuit basis, we
introduce the possibility of downgrade attacks. We do calculate
the SHA256 digest of VERSIONS/CERTS/AUTH_CHALLENGE cells (which
will become VERSIONS/CIPHERSUITES/CERTS/AUTH_CHALLENGE), so we could
do something like:
auth_input = verify | ID | B | Y | X | SLOG | PROTOID | "Server"
Where SLOG is the digest without too much difficulty...
The alternative to all of this would be something like including
handshake methods/supported link crypto in the descriptors, but that
seems silly.
Regards,
--
Yawning Angel
Filename: 260-cryptography-agility.txt
Title: Link Cryptography Agility
Created: 2015-11-15
Status: Draft
Extends: 249-large-create-cells.txt
1. Overview
This proposal aims to add per-circuit cryptographic agility, such that
the link handshake and cell layer encryption can be specified on au
per-circuit basis.
Once support for the negotiation mechanism in this proposal has been
deployed, it will be possible to introduce support for new circuit
handshake and/or cell encryption methods without the need for further
wire protocol changes.
2. Negotiation Process
The initator and responder MUST negotiate version 5 or higher to indicate
support for the CIPHERSUITES, CREATE2V and CREATED2V cells along with
fragmented EXTEND2 cells.
3. CIPHERSUITES cells.
Upon negotiating version 5 of the link protocol and sending a VERSIONS
cell, the responder sends a CIPHERSUITES cell befor sending the CERTS
cell to indicate which handshake methods, and which link encryption
primitives it supports.
(XXX: Should this be variable length? At the rate that we will change
handshake/cell crypto, I doubt space will ever be a concern, but
VERSIONS is variable length...)
struct ciphersuites_cell_body {
/* Number of supported handshake types */
u16 hlen;
/* Supported handshake types */
u16 handshake_types[hlen];
/* Number of supported cell crypto types */
u16 clen;
/* Supported cell crypto types */
u16 cell_crypto_types[clen];
/* Padding data to be ignored */
u8 ignored[];
};
The initiator then selects the "best" algorithms supported by the peer
as appropriate.
To prevent downgrade attacks, both parties MUST include the contents of
the CIPHERSUITES cell as part of the link handshake calculations, for
example by including the SLOG (AUTHENTICATE cell) SHA256 hash in the
ntor "auth_input" field.
XXX: It occurs to me that we should include a 2 byte cell crypto specifier
in the CREATE2V/fragmented EXTEND2 cells.
4. Cell Crypto Types
The following cell crypto types are defined and reserved:
0x - CTR-AES128 + SHA1 -- the original Tor cell crypto
0x0001 - reserved (AEZ?)
pgpCMw8okYasS.pgp
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [tor-commits] [tor-messenger-build/master] Bug 17492: Include default bridges configuration
On Fri, 6 Nov 2015 19:29:45 -0800 David Fifield <da...@bamsoftware.com> wrote: > > This could be a great opportunity for Yawning's recent meek variant > > that doesn't use an actual browser. (It won't work as well as the > > real meek, but maybe no actual adversaries care about the > > difference quite yet?) > > You can also just run meek-client without the Firefox helper. It has a > standalone mode. Just change the torrc line > > ClientTransportPlugin meek > exec ./TorBrowser/Tor/PluggableTransports/meek-client-torbrowser > -- ./TorBrowser/Tor/PluggableTransports/meek-client > > to > > ClientTransportPlugin meek > exec ./TorBrowser/Tor/PluggableTransports/meek-client > > There's a man page for meek-client: > https://gitweb.torproject.org/pluggable-transports/meek.git/tree/doc/meek-client.1.txt Yeah, the obfs4proxy change is mostly just to save bundle size/disk space for android and the like. Regards, -- Yawning Angel pgpWolTQ8xEgY.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Pluggable Transports v1 spec rewrite.
On Thu, 29 Oct 2015 10:04:24 -0500 Brandon Wiley <bran...@blanu.net> wrote: > Excellent work on the rewrite. To summarize for those that do not > have time to read the whole document, it's the same spec, it's just > been rewritten to read more clearly. I think it's a great improvement > over the previous version. > > I have two suggestions: > > Section 4, "Tor Configuration", I think should be moved into another > document and a reference should be added instead. While it's useful > for developers reading the PT spec to see how to configure their Tor > nodes so they can test their PTs, this information is going to be > used mostly by bridge operators. Bridge operators shouldn't have to > go to the PT spec to figure out how to configure their bridge. PT > developers though will need to configure a bridge anyway, so I think > it's sensible to at the end of the PT spec send them to the bridge > configuration documentation and the PT configuration section > specifically. Ok, I'll carve it off to a different document. To be honest I think it's somewhat redundant to have the documentation at all (since it's just a bunch of torrc directives), and there's better places for configuration documentation (like the man page). > Section 6 "Future Improvements" I think should be moved into a PT > roadmap document. There is a lot to discuss regarding future > improvements. My list for the most part does not overlap with this > list. We could engage in healthy debate over what should go in the > future improvements section, but this seems to me to be something > better done when drafting a roadmap document rather than in the spec. > It would of course be useful to add a reference to any roadmap > documentation at the end of the spec. The things listed there are the things that need to happen from Tor's perspective, and I'm not currently considering a huge "Change All The Things" type of rework if we were to bump the spec version. But I'll remove the section as unnecessary for now. Regards, -- Yawning Angel pgppIEZaPmNQU.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Pluggable Transports v1 spec rewrite.
Hello all, In an attempt to make Pluggable Transports more accessible to other people, and to have a spec that is more applicable and useful to other projects that seek to use Pluggable Transports for circumvention, I have drafted a re-write of the spec. This is not intended to alter existing behavior, but instead make it clear that the whole "Pluggable Transports" thing isn't just for Tor. Unless people have serious objections, this will replace the existing PT spec, to serve as a stop-gap while the next revision of the PT spec (that does alter behavior) is being drafted/implemented. Regards, -- Yawning Angel Pluggable Transport Specification (Version 1) Abstract Pluggable Transports (PTs) are a generic mechanism for the rapid development and deployment of censorship circumvention, based around the idea of modular sub-processes that transform traffic to defeat censors. This document specifies the sub-process startup, shutdown, and inter-process communication mechanisms required to utilize PTs. Table of Contents 1. Introduction 1.1. Requirements Notation 2. Architecture Overview 3. Specification 3.1. Pluggable Transport Naming 3.2. Pluggable Transport Configuration Environment Variables 3.2.1. Common Environment Variables 3.2.2. Pluggable Transport Client Environment Variables 3.2.3. Pluggable Transport Server Environment Variables 3.3. Pluggable Transport To Parent Process Communication 3.3.1. Common Messages 3.3.2. Pluggable Transport Client Messages 3.3.3. Pluggable Transport Server Messages 3.4. Pluggable Transport Shutdown 3.5. Pluggable Transport Client Per-Connection Arguments 4. Tor Configuration (torrc directives) 4.1. Client Configuration 4.1.1. ClientTransportPlugin 4.1.2. Bridge 4.2. Server Configuration 4.2.1. ServerTransportPlugin 4.2.2. ServerTransportListenAddr 4.2.3. ServerTransportOptions 5. Anonymity Considerations 6. Future Improvements 7. References 8. Acknowledgments Appendix A. Example Client Pluggable Transport Session Appendix B. Example Server Pluggable Transport Session 1. Introduction This specification describes a way to decouple protocol-level obfuscation from an application's client/server code, in a manner that promotes rapid development of obfuscation/circumvention tools and promotes reuse beyond the scope of the Tor Project's efforts in that area. This is accomplished by utilizing helper sub-processes that implement the necessary forward/reverse proxy servers that handle the censorship circumvention, with a well defined and standardized configuration and management interface. Any application code that implements the interfaces as specified in this document will be able to use all spec compliant Pluggable Transports. 1.1. Requirements Notation The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 2. Architecture Overview +++---+ | Client App +-- Local Loopback --+ PT Client (SOCKS Proxy) +--+ +++---+ | | Public Internet (Obfuscated/Transformed traffic) ==> | | +++---+ | | Server App +-- Local Loopback --+ PT Server (Reverse Proxy) +--+ +++---+ On the client's host, the PT Client software exposes a SOCKS proxy [RFC1928] to the client application, and obfuscates or otherwise transforms traffic before forwarding it to the server's host. On the server's host, the PT Server software exposes a reverse proxy that accepts connections from PT Clients, and handles reversing the obfuscation/transformation applied to traffic, before forwarding it to the actual server software. An optional lightweight protocol exists to facilitate communicating connection meta-data that would otherwise be lost such as the source IP address and port [EXTORPORT]. All PT instances are configured by the respective parent process via a set of standardized environment variables (3.2) that are set at launch time, and report status information back to the parent via writing output in a standardized format to stdout (3.3). Each invocation of a PT MUST be either a client OR a server. All PT client forward proxies M
Re: [tor-dev] Pluggable Transports v1 spec rewrite.
On Thu, 29 Oct 2015 10:27:53 -0400 Nathan Freitas <nat...@freitas.net> wrote: > Any small or big changes to highlight that push us to the more > accessible goal? Well, it emphasizes that it's generic, gives a better view of the architecture, and should be usable by anyone competent (unless my rewrite is bad) to add support for Pluggable Transports to any client/server code base. All the Tor specific stuff is neatly contained in it's own section now that people can be free to ignore. Stuff it still should do: * Fold in the ExtORPort spec since that's useful for people that aren't tor. * Document the tor pt external mode torrc configuration format (In my defense, the current spec doesn't do this either). So, yes? Dunno. There's a limit to how far I can de-torify this sort of document, and I think this is a reasonable step in the right direction. Regards, -- Yawning Angel pgp2LOhKYB67e.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] A layered transport
On Mon, 26 Oct 2015 15:44:59 +0800 Da Feng <sunspid...@gmail.com> wrote: > Hi: >I've discovered that the GFW normally doesn't block https > protocols. We can use a https front tier to distribute connections to > actual bridges. The front tier encrypts an internal address identifier > with its private key (no matching public key or public algorithm) and > returns to user the encrypted identifier, part of which also includes > the user's chosen password. Then when submitting requests, the user > encrypt again with his password the items such as his timestamp, > broswer headers. The request line to https server is no different from > an ordinary one and include both the user encrypted item and front > tier encrypted item. After the connection is established, data is > relayed inside https between bridge and user. So... meek (https://trac.torproject.org/projects/tor/wiki/doc/meek), the basis of which will probably also be used for bridge distribution purposes in the future. Regards, -- Yawning Angel pgpYXv2foZmjQ.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Another Virtual Network Environment needs your I/O
On Sun, 18 Oct 2015 19:20:47 +0200 Rob van der Hoeven <robvanderhoe...@ziggo.nl> wrote: > I never adjust the size of the TCP window, that's correct. The code > only sends an ACK for data that is *removed* from the buffer. If data > is added to the buffer, the amount of data the TCP-client is allowed > to send decreases. Eventually becoming zero if no data is removed at > all. Yeah. Since your receive window isn't ever sizable, and the interface is rather reliable, this behavior is probably the simplest. [snip] > > * I couldn't figure out how to get X apps to work at all. > >(Eg: chromium fails with "Gtk: cannot open display: :0") > > Ah, your system probably uses an Abstract Unix Socket for > communication with the X server. Abstract Unix Sockets are created > inside a network namespace, and your X server socket lives inside the > global network namespace. To solve this I have to write proxy code to > create an X server socket inside the network namespace of the TCP > client. Maybe you can temporarily solve the problem by binding your X > server socket to a normal Unix socket? (filesystem Unix Sockets are > not network namespace aware). > > What OS (name+version) are you using? I'm using Debian Wheezy which > does not have this problem. Arch. It's a rolling release system, and it's up to date-ish. It's not a big deal at the moment... > > * There should be documentation that this requires a minimum of > >CAP_SYS_ADMIN (for the various unshare() calls) and > > CAP_NET_ADMIN (to bring the tun interface up). > > AVNE is a suid program at the moment. To do the privileged calls the > program runs (for a while) with root privileges. Oh, well. Using setcap and granting the relevant capabilities also works, though dropping the capabilities requires a bit more work. The elevated capabilities aren't persisted across exec() at least so the child side handling is easy. > I'm going to move to Debian Jessie. This version has a newer kernel > that supports user namespaces. As I understand it (have not played > with user namespaces), these can be used to create programs that can > have full privileges inside the user namespace without having full > privileges themselves. No more suid needed. The downside is that user > namespaces are only available for kernels with versions >= 3.8 Well. Debian requires enabling it by writing to proc (they have a patch for this). Arch flat out doesn't include support for that in the vanilla kernel (https://bugs.archlinux.org/task/36969?project=1), ditto anyone using grsec. It's probably not that much code to support user namespaces, running with just the required capabilities, and running as suid root in the same code base. > > * The config file load/parse routine has a trivially exploitable > >buffer overflow. > > The final config file will be owned by root and stored in /etc. Ah I see (I assume/hope you'll fix this anyway). (Will there also ever be an option for configuring a different tun address?) Neat project. I'll be looking forward to subsequent releases, since this is slick, and I think a better approach than torsocks once it matures a bit. Regards, -- Yawning Angel pgpfz03dZtqv8.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Another Virtual Network Environment needs your I/O
On Sun, 18 Oct 2015 13:08:01 +0200 Rob van der Hoeven <robvanderhoe...@ziggo.nl> wrote: [snip] > When I was researching my idea I came across lwIP and was planning to > use it. Unfortunately I could not find documentation. It's not on the > project homepage and the wikia pages were not helpful too. What I > needed was the info provided by the lwIP paper by Adam Dunkel. Found > this paper yesterday trough the wikipedia article about lwIP. > > I always try to understand a technology that I use, even if I use an > implementation by someone else. So I studied the TCP protocol. From > this I got the idea that a full TCP protocol implementation was not > needed to correctly handle *my* tun traffic. Writing a simplified TCP > protocol would give me some major advantages: perfect integration > with my design and no extra external dependencies. Sounds reasonable, and yeah the lwIP documentation isn't great. > > As your blog post notes "There are still some loose ends", most of > > these should be known issues (From quickly skimming the code, there > > is > > Fair enough, the loose ends are: > > - The proxy uses two connections, an upstream connection and a TCP > connection. If one of the two connections closes, the other connection > is closed immediately. There is however a (very?) small change that > there is still data in a buffer that must be written to the open > connection. The open connection should close after this data is > written. Fairly easy. > - Needs more testing. I have tested the program in a loose way. For > hours and hours I have watched YouTube videos, visited nasty websites > (lots of advertisements, scripts and simultaneous connections), and > downloaded very large files using WGET. Everything seems to work, but > *seems* is not enough it most be *proven* to work correctly. > > - No unit tests. > > > a possibility I've missed things or I'm wrong): > > > > * No IPv6. > > * No congestion control (TCP Tahoe is fairly trivial to implement, > >and would be more than sufficient). > > Congestion control is used to prevent dropped segments. This can not > happen on the User Space <-> kernel connection of a tun interface. The > TCP-window flow control prevents this. Hm. Your code never shrinks the advertised window, and ACKs once when data has been added to the receive buffer. As a perfectionist for such things, this makes me really sad, but since it the stack never sees the light of day outside a TUN interface, the behavior is probably ok. As a side note, the TUN driver will happily drop packets once the backlog exceeds 500 or if a skb allocation fails. Both unlikely, both can happen. [snip] > "Want to" implies that the user is aware of the danger. Most users > have no idea about the dangers of using a particular protocol over > Tor. I want to protect these users. Fair enough. I'd suggest this be configurable behavior. See `WarnPlaintextPorts` in the tor manpage for what should be a sensible list. If you want to protect users from danger, you'd probably also want to disallow port 80 and 443 by default, because they're giving up a lot by using what is probably the Wrong Browser[1]. They may even think that viewing Flash/Java content is safe, when neither are because the plugins are all sorts of terrible. > > Should be easily portable to the U*IXes, Windows will give you pain, > > but I'm not sure you care. > > I care, and I think it can be ported to Windows. Since there are no > kernel namespaces, another way to force all traffic through the tun > interface must be found. It can be done by using the Windows firewall > like this: > > https://community.hide.me/tutorials/bind-your-connection-to-vpn-with-windows-firewall.49/ Guess you'll use the OpenVPN TUN/TAP driver? Some more random things: * PSH is set incorrectly. You set PSH when you're ACKing received data (without piggybacked payload). You're supposed to PSH as part of sending data to the peer (See RFC 1112 4.2.2.2). * DNS resolution does not work if /etc/resolv.conf points at 127.0.0.1. This probably is a documentation thing rather than a code thing. * I couldn't figure out how to get X apps to work at all. (Eg: chromium fails with "Gtk: cannot open display: :0") * There should be documentation that this requires a minimum of CAP_SYS_ADMIN (for the various unshare() calls) and CAP_NET_ADMIN (to bring the tun interface up). * The config file load/parse routine has a trivially exploitable buffer overflow. -- Yawning Angel [1]: Tor Browser would be the Less Wrong Browser. pgpbSQfK93qdj.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Another Virtual Network Environment needs your I/O
On Fri, 16 Oct 2015 15:03:49 +0200 Rob van der Hoeven <robvanderhoe...@ziggo.nl> wrote: [snip] > Because the network interception takes place at the IP-level, no > special Tor configuration is needed. In theory all TCP-clients should > now be able to use the Tor network. Cute. The networking part works in a near-identical manner Orbot's Android VPN mode, under the hood, except they opted to use a 3rd party implementation (that bundles lwIP IIRC). Why did you write your own IP/ICMP/TCP (partial) implementations instead of using something like lwip? As your blog post notes "There are still some loose ends", most of these should be known issues (From quickly skimming the code, there is a possibility I've missed things or I'm wrong): * No IPv6. * No congestion control (TCP Tahoe is fairly trivial to implement, and would be more than sufficient). * Things will break/perform poorly if there ever is loss on the TUN interface (Unlikely, I know). * Out-of-order but in window segments are discarded, instead of stashed for use later. * No slow retransmission timer. * The implementation does not ACK enough. Send a dup. ACK when you receive something that's Out-of-order but in window. Probably doesn't matter much since the stack is never exposed to a public network and is only ever within the confines of a single host. > The second question I have is: are there programs / protocols which > should be blocked by avne? For example: I think its a bad idea to > support unencrypted POP3. I think users should be allowed to shoot themselves in the foot if they want to. > Of course I am also interested in bug-reports and your ideas for > improvement. Code looks fairly clean. I'd feel better if it was in a memory safe language, but I like C/C++11 more than all the new popular stuff so I probably would have made a similar decision. Should be easily portable to the U*IXes, Windows will give you pain, but I'm not sure you care. I assume you use version control for the code and are just not exposing it to the world. Regards, -- Yawning Angel pgpa9OE2cELXj.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hello, I am a Tor Browser user in China. Currently, many obfs4 bridges are blocked by China's firewall.
On Thu, 1 Oct 2015 08:26:50 -0700 David Fifield <da...@bamsoftware.com> wrote: > I don't know about a schedule for deploying SkypeMorph and Dust. They > may not help in your case anyway. The GFW is probably blocking the IP > addresses of your bridges, not detecting the obfs4 protocol itself. SkypeMorph: Never. (Licensing issues among other things) Dust: I assume you mean Dust2, the original Dust is not getting deployed. Not sure yet. [Folding in the 2nd reply] > If you know some details of how I2P resists blocking, please add them > to this wiki page: > https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports It doesn't do anything special. The old TCP based protocol (NTCP) is trivially identifiable. Blocking the more modern UDP protocol (SSU) would require looking for high-entropy UDP or doing statistical attacks IIRC. Active probing is possible if they run a node that's part of the network and obtain enough key material (But, I'd need to look at the floodfill system again to figure out how many nodes for how long is realistically required). Regards, -- Yawning Angel pgptgowGyqoHR.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Desired exit node diversity
On Wed, 23 Sep 2015 06:18:58 + Virgil Griffith <i...@virgil.gr> wrote: > * Would the number of exit nodes constitute exactly 1/3 of all Tor > nodes? Would the total exit node bandwidth constitute 1/3 of all Tor > bandwidth? No. There needs to be more interior bandwidth than externally facing bandwidth since not all Tor traffic traverses through an Exit (Directory queries, anything to do with HSes). The total Exit bandwidth required is always <= the total amount of Guard + Bridge bandwidth, but I do not have HS utilization or Directory query overhead figures to give an accurate representation of how much less. Regards, -- Yawning Angel pgpRhKLezTcGv.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Towards a new version of the PT spec...
On Thu, 17 Sep 2015 14:28:24 -0400 Adam Pritchard <a.pritch...@psiphon.ca> wrote: > At Psiphon we often discuss (and get asked about) using Tor's > pluggable transports directly. The cost/benefit balance hasn't yet > been in favour of doing this, but if there's discussion of a big PT > revamp... maybe Psiphon should indicate how the cost side of the > balance could come down for us. > > We're obviously not a priority for what Tor does with PTs, but there's > surely no harm in us mentioning our wishlist. So... > > What would be best for us is if PTs were written in Go and implemented > the net.Conn[1] interface. We have had good results with the > composability of net.Conn implementations: an obfuscated SSH net.Conn > on top of a meek net.Conn[2] on top of a upstream proxy net.Conn[3] on > top of a TCPConn net.Conn[4]. Layering in a PT net.Conn would be sane > and clean and reasonably easy. > > Conversely: Python is difficult-to-impossible due to runtime > distribution. Separate executables are difficult-to-impossible due to > Android PIE requirements and distribution size bloat. The Guardian project people seem to manage to handle distributing multiple separate executables (tor, meek, obfs4proxy). I do not know what they do, maybe they just take the size hit. > Anyway, if this is of any interest we can discuss it further. I'm interested in hearing what people want but: * I personally am against forcing any particular language on people, since the point of PTs is that they are easy for interested parties to write. * Pluggable Transport implementations sharing address space with the Tor binary is basically never going to happen due to security concerns (iOS may do weird things, but iOS is not an officially supported target). The revamp is more of a "keeping the current model the same (ie: fork/exec a helper process with a known external interface), how can we make said external interface better" than "change the entire Pluggable Transport concept". > (Note: Probably Lantern people are reading this too. And probably they > would benefit from the same things we would, since their architecture > is similar to ours.) Now despite all of this, the obfs2/3/4 and ScrambleSuit implementations I did for obfs4proxy do in fact implement a net.Conn interface[0] and always have[1]. Regards, -- Yawning Angel [0]: https://gitweb.torproject.org/pluggable-transports/obfs4.git/tree/transports/base/base.go [1]: I reserve the right to implement a hypothetical `obfs5` in something that is not Go, depending on the availability of required libraries/primitives, my mood, the phase of the moon, and planetary alignment. pgpwFNWoJiMAq.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal: End-to-end encrypted onion services for non-Tor clients
On Mon, 14 Sep 2015 16:12:23 + Donncha O'Cearbhaill <donn...@donncha.is> wrote: > I have been thinking about ideas to make Tor hidden services more > available and secure for non-Tor users. Inline I've included a draft > proposal which describes an end-to-end encrypted Tor2Web-like system. > > I'm really interested in hearing any suggestions, comments or > criticism about this proposal. In particular I'd like to know if the > trust requirements for the entry proxies and resolvers seem > reasonable? Does this proposal make sense and is it something worth > implementing? I don't understand the use case for this, given the design presented. If the HS operator is willing to register a domain, run a public name server, and pay for a CA cert, why host the actual content on a Tor HS? It should be emphasized that the user has absolutely zero anonymity under this scheme, in that anyone tapping the link between the user and the proxy can see what domain the user is trying to view. Anyone with the capability to inject RSTs can censor on a per-site basis as well. Regards, -- Yawning Angel pgpDz8wgITs4p.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Towards a new version of the PT spec...
On Mon, 14 Sep 2015 16:14:41 -0400 Brandon Wiley <bran...@blanu.net> wrote: > It is my understanding that a sponsored project is coming up to work a > pluggable transport 2.0 specification and implementation. I've also > heard that the first step for this is to have a meeting where we get > together with people that either use pluggable transports or > implement them, for the purpose of soliciting feedback on the > existing specification and gathering design requirements for 2.0. So > perhaps the drafting of a new specification should be deferred until > this is organized. Although of course any feedback gathered in this > email thread can also be presented at the meeting. This would be the first I heard of either. Till someone tells me details about this, I think the sensible thing to do would be to discuss it here... It's not as if lots of people seem interested in giving feedback anyway at present so I'll probably end up drafting something that does what I want it to do (which fixes the major current shortcomings of the interface from my POV), at which point all the other people will appear to complain. Regards, -- Yawning Angel pgpcmJoDQOZYh.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Bridge Guards (prop#188) & Bridge ORPort Reachability Tests
On Thu, 10 Sep 2015 07:01:58 + isis <i...@torproject.org> wrote: > 3. Should we change how the BridgeAuthority tests Bridge ORPort > reachability? If so, how? > > 4. If I'm going to refactor all of this, are there other (future) > things I should take into account? > > For example, if self-testing is disabled for Bridges, and, due to > #7349, [2] the BridgeAuthority isn't testing reachability either, > then how will the BridgeAuthority know if (any of) the Bridge's PTs > are reachable? (One solution might be to, in that case, have the > BridgeAuthority lie and tell BridgeDB that such Bridges were > reachable, letting BridgeDB do the PT reachability testing.) All of this seems to me like we need to sit down somewhere (in meatspace or online), map out how the current system of: * Clients * Bridges * Bridge Guards * The Bridge Authority fits together, and figure out how it can be redesigned in a way that is more inline with how things work now/need to work now (PTs were not a thing when this started being one of the bigger stumbling blocks). Regards, -- Yawning Angel pgpNTCxTZUKfj.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Reproducibility of Pluggable Transports python.msi
On Thu, 10 Sep 2015 10:20:59 -0400 Brandon Wiley <bran...@blanu.net> wrote: > I'm not advocating that the various PT implementations be abandoned, > just that we have a common implementation across products when > possible. If I recall correctly, there was a time when TBB, Tails, > and Orbot were all shipping different implementations. I think the > current state of PT implementation deployment is the following: > > TBB: Go, Python > Tails: Go > Orbot: Go, C++ (on x86) That's correct. It's worth noting that the Python component of TBB is almost entirely FTE that hovers around ~200 users. Out of those, I am unsure how many use FTE because it is the only thing that works in their environment. > The benefit of having the Go implementation ship with all products is > that PT authors can target one implementation and achieve deployment > across all of the products. Sure. Go would be a fine choice for people, but I'd like it to be explicit that I'm open to more options, even if it means reducing the deployment base if that's what it takes for people to write something (I'd rather see more circumvention methods, than fewer). > As far as reproducibility of builds goes, if a reproducible Python > build is a challenge, an alternative is to port FTE to Go and retire > flashproxy. Or port both to Go (flashproxy would be easy)/deprecate both. Regards, -- Yawning Angel pgpeJvU5SJO1N.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Reproducibility of Pluggable Transports python.msi
On Wed, 9 Sep 2015 17:20:11 -0400 Brandon Wiley <bran...@blanu.net> wrote: > Thanks David, great info! Last time I checked, I think the C > implementation was also still shipping with something, I think Orbot > for Android. Perhaps this is also for either flash proxy or FTE > support, since Python is not the best option on Android. Orbot used to use obfsproxy (C), migrated to obfsclient (C++) and the main build uses obfs4proxy (Go) these days. I heard that Android x86 might still need to use the C++ code, but I'm not sure. > From the graphs it looks like FTE is still in use, but that flash > proxy seems to no longer be used. It wasn't really every used due to NAT related issues. WebRTC may be the way forward here but it will be an entirely different codebase. > If I recall correctly, the core FTE code is actually written in C and > is just using the Python PT implementation with Python-C bindings to > the FTE library. So a port of the FTE PT to the Go PT implementation > seems possible. The core FTE code was originally in C++, but IIRC it's in python now. FWIW, I don't particularly think that there must be One True PT language[0], I just recommend Go over the other alternatives due to it being both memory safe and easy to build on mobile. If someone writes a new PT in Python, I don't consider it a deal breaker, though it won't be as useful due to the difficulty of mobile support. All of this sort of ties into the other thread where people are talking about buffing up pyptlib (might make sense), and further deprecating obfsproxy (Python). I don't particularly see a reason to do either things, though the Debian people apparently see obfsproxy as the program and not a library, when it's both. Regards, -- Yawning Angel [0]: MUST be able to be built deterministically. SHOULD be memory safe. Past that, people can do what they want. If they ignore the SHOULD clause, the code needs to undergo more thorough auditing before it will be deployed into production. pgpiHqLAfVSiX.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Towards a new version of the PT spec...
On Tue, 8 Sep 2015 17:39:58 +1000 Tim Wilson-Brown - teor <teor2...@gmail.com> wrote: > > > On 8 Sep 2015, at 08:45, Yawning Angel <yawn...@schwanenlied.me> > > wrote: > > > > So, we currently have a Pluggable Transport (PT) spec, and it > > kind-of sort-of works (The documentation is a mess that I'm working > > on cleaning up, but it's an orthogonal issue for how well it works). > > > > There are a number of problems with the current PT spec that require > > breaking backward compatibility to fix, so eventually I would like > > to do so. > > > > I'm soliciting input on what people would also like to see in a > > (currently hypothetical) PT spec 2.0 beyond what I already have in > > mind: > > > > ... > > > > UNLIKELY: > > * Specify an interface for where fork()/exec() isn't possible > > (iOS). I don't think this is makes sense because it is probably too > >platform/caller specific. > > I imagine that this would require a PT-as-thread(s) interface, which > is out of scope, as iOS is a single platform. It seems to me that > using a PT on iOS could be done in a similar way to using tor. (That > is, if you can’t fork tor from an iOS app, then forking PTs is the > least of your worries.) I was thinking PTs as a shared object of some sort with presumably but not necessarily lots of threads under the hood. But I marked this as UNLIKELY precisely because it's a lot of effort for a single platform (however important) that isn't targeted by an official Tor anything at this point. It also is a platform I don't have access to, and never will, so unless someone is willing to chime in on how to properly support this sort of thing in a way that doesn't clutter up the PT spec and implementations with a lot of extra stuff it won't happen. > I’m hoping someone has developed a generic way of doing this, at > least for network apps. What are the ChatSecure people doing for > their XMMP + Tor chat accounts? Launching pthreads? Don't know, don't care at this moment since it's totally orthogonal to drafting a new Pluggable Transports spec (That doesn't mean that it isn't important, just trying to keep discussion on track.). Regards, -- Yawning Angel pgptXr0MPflc5.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Towards a new version of the PT spec...
So, we currently have a Pluggable Transport (PT) spec, and it kind-of sort-of works (The documentation is a mess that I'm working on cleaning up, but it's an orthogonal issue for how well it works). There are a number of problems with the current PT spec that require breaking backward compatibility to fix, so eventually I would like to do so. I'm soliciting input on what people would also like to see in a (currently hypothetical) PT spec 2.0 beyond what I already have in mind: MUST haves: * Support dual stack Bridges correctly (Multiple server endpoints per transport) * Increase the argument space beyond 510 bytes (Prop. #227). * Mandatory ExtORPort support (currently optional, but metrics are good). * Centralized logging by the calling process (Probably via stderr). * AF_UNIX support where sensible for better sandboxing. MIGHT haves: * Rename the env vars to not start with "TOR_PT". Some people claim that this is a good idea (I think it is stupid and cosmetic). * Ability to force at least clients to stop network activity without tearing the PT down. * Deprecate SOCKS4a, and make SOCKS5 mandatory for clients. UNLIKELY: * Specify an interface for where fork()/exec() isn't possible (iOS). I don't think this is makes sense because it is probably too platform/caller specific. * Allow operating both as a client and a server simultaneously. I don't see a problem with running 2 copies of something for this use case. I probably missed some things. If people have strong opinions about this, do reply, otherwise I *will* design something that I like, which will not include what other people want. Regards, -- Yawning Angel pgpk_4mdbY6Vi.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] fteproxy depends on obfsproxy...
On Mon, 7 Sep 2015 17:03:00 -0700 Kevin P Dyer <kpd...@gmail.com> wrote: > Response inline. > > On Mon, Sep 7, 2015 at 3:29 PM, Yawning Angel > <yawn...@schwanenlied.me> wrote: > > > On Mon, 7 Sep 2015 14:37:07 -0700 > > Kevin P Dyer <kpd...@gmail.com> wrote: > > > > > ...and it shouldn't. > > > > > > Fortunately, the dependency is isolated to a single file. See [1]. > > > > > > My understanding is that pyptlib [2] is no longer maintained. > > > > It isn't? It hasn't gotten any updates because nothing broke and > > the pt spec is relatively static (for now, eventually it'll change > > and pyptlib will get a flurry of updates as needed). > > I was speaking to Brandon Wiley about this last week. I don't want to > speak for him, but I remember him saying that it's no longer being > developed. > > Either way, does pyptlib help solve this issue? It somewhat reduces what you'll need to re-implement but not significantly since you'd still need to re-implement all the stuff that touches the network. -- Yawning Angel pgpWch1oFN68d.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal: Single onion services
On Fri, 4 Sep 2015 15:31:15 -0600 John Brooks <john.bro...@dereferenced.net> wrote: [snip] > > > > Have you considered all the implications? > > Maybe we’ve missed some - what implications are you thinking of, that > aren’t addressed in the proposal? I have two objections to this, one political, one technical: * (The political objection) While this is "cool" and probably(?) "funded", it seems like a poor thing to work on in terms of developmental priority when there are other things Hidden Service related that need a lot of developer attention, primarily in making the existing HSes more resilient against Nation State level adversaries (Eg: Prop. 224). * (The technical objection) It is overly easy for assholes[0] to censor Single Onion Services due to: it’s possible for the previous relay to guess the service you’re connecting to While such a censor would only be able to deny service to clients as a fraction of their relay(s) consensus weight, it's still something that probably should get consideration. Regards, -- Yawning Angel [0]:https://lists.torproject.org/pipermail/tor-dev/2014-July/007167.html pgpGe9WwNRTrz.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Seeing through Network-Protocol Obfuscation
On Fri, 21 Aug 2015 17:51:20 -0700
Kevin P Dyer kpd...@gmail.com wrote:
On Wed, Aug 19, 2015 at 11:58 AM, Yawning Angel
yawn...@schwanenlied.me wrote:
[snip]
The FTE semantic attack they presented isn't the easiest one I know
of (the GET request as defined by the regex is pathologically
malformed).
Very interesting! This is news to me. I'm assuming I did something
silly. (Even though I tested it against bro, wireshark, etc.)
Huh. I brought it up in conversation with a few people and was under
the impression it was passed on. I probably should have e-mailed you
about it or something.
How is it pathologically malformed?
manual-http-request: {
regex: ^GET\\ \\/([a-zA-Z0-9\\.\\/]*) HTTP/1\\.1\\r\\n\\r\\n$
},
No Host header. All complaint requests MUST include one per RFC
2616, and all compliant servers MUST respond with a 400 if it is
missing.
Since requests of that sort should invoke the error path on RFC
compliant servers it's a really good distinguisher since legitimate
clients will not do such a thing. Existing realistic adversaries
already have identify 'suspicious behavior', call back to confirm
style filtering in production, so false positive rate can be reduce to
0 if needed.
Regards,
--
Yawning Angel
pgpt9GOPQh8Cr.pgp
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Seeing through Network-Protocol Obfuscation
On Fri, 21 Aug 2015 17:46:39 -0700 Kevin P Dyer kpd...@gmail.com wrote: The authors suggest active probing to reduce false positives, but don't mention that this doesn't work against obfs4 and meek. I don't want to get too off track here, but do obfs4 and meek really resist against active probing from motivated countries? Don't we still have the unsolved bridge/key distribution problem? meek does because the entry point into the Tor network is a well known high traffic CDN platform. So an adversary can see that there is a meek instance running on a given CDN (since it's not a secret), along with content that people want to see, so distinguishing between normal traffic/meek traffic requires a TLS break or statistical attacks. I personally hold distribution to be orthogonal to circumvention protocol design in the context of obfs4 (scramblesuit, fte, and other bridge based circumvention protocols), because if someone breaks the bridge distribution mechanism, the protocol is irrelevant since the attackers win by virtue of having the IP address/Port of the obfuscated server[0]. Assuming all the adversary sees is the obfs4/scramblesuit stream, then both are active probing resistant, because it requires compromising a separate system to be able to generate a valid handshake for probing. Active probing attacks should be able to defeat a scenario like: I setup a unlisted bridge, firewall off the ORPort and GPG e-mail/OTR/Pond a series of bridge lines to a collaborator in China. The adversary gets to see the IP address/Port of the obfuscated server, and can send traffic as they see fit. Note: There's a few other things an adversary can do under the assumption that whatever is speaking obfs4 is probably a Tor client/bridge pair. But those are attacks against either the Tor network, or limitations of the tor implementation itself[1]. Regards, -- Yawning Angel [0]: Distribution still is an important problem that needs to be solved, and maybe linking it closer to the protocol design is something that is required. Open research questions are open. [1]: Likewise this stuff is important and should be/will be fixed, but they are Tor issues and not obfs4/fte/whatever issues. pgpqGWuRU9dnE.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Seeing through Network-Protocol Obfuscation
On Sat, 22 Aug 2015 14:40:08 -0700 Kevin P Dyer kpd...@gmail.com wrote: Ah, gotcha. It's not RFC compliant. RFC2616 was created in 1999 and there are tons of HTTP-like implementations since then that, ostensibly, don't need to follow it. (e.g., an HTTP-like client/server that only talk to each other.) A network monitor must deal with these cases too, and they'll broadcast HTTP/1.1 in their headers. This [1] paper is a bit dated (2007) but my intuition is that real-world implementations have drifted even further from the RFC over the last 8 years. I swear there's a more recent paper on this topic, but I couldn't find it... I'd be surprised if there were lots of clients that advertise HTTP/1.1 that don't include a Host header, since clients that are broken in that manner will not be able to talk to apache/ngnix/tomcat/etc[0]. Then again, fteproxy is an example of such a thing, so I may be rather sad at the results of an actual survey. Since requests of that sort should invoke the error path on RFC compliant servers it's a really good distinguisher since legitimate clients will not do such a thing. Existing realistic adversaries already have identify 'suspicious behavior', call back to confirm style filtering in production, so false positive rate can be reduce to 0 if needed. Based on our exploration of data, we found there's a wide range of implementations and most of which have non-RFC-compliant behaviors. See Section 4 of our paper for more details. For that reason I'd be very surprised if a host-header-check could result in a 0 FP rate. The point isn't to use non-compliance as the sole discriminator (since people do write broken code), but to cut down the candidate IP/Port list down to something that's reasonable for whatever active probing infrastructure that exists to manage. From there, delta-T later with separate infrastructure attempt a full FTE + Tor handshake, and blacklist/RST inject/etc target candidates that succeed. The second step gets to 0 FP, and precisely this sort of thing is how China currently handles obfs3. The delay (anecdotal) is about 10 mins. Intuitively I think that missing Host header will be extremely rare but I don't have a way to get traces to prove/disprove it. With that being said, I'll add the host-header-check to the list of experiments that we want to do for the full version of our paper. Would be interesting to learn what the data tells us. I would be interested in seeing the results. -- Yawning Angel [0]: Ngnix supports hooking the error handler rather easily, apache less so. pgpufIOwlRYHG.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Number of directory connections
On Fri, 21 Aug 2015 12:45:05 -0400 tordev...@safe-mail.net wrote: I can't think of a good reason for that behavior. Is there a valid security issue using an old consensus to fetch the current one from the entry guard? Your entry guard being rejected as malicious by the DirAuths in the intervening time period. Regards, -- Yawning Angel pgpkGFnUn8J6o.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hash Visualizations to Protect Against Onion Phishing
On Thu, 20 Aug 2015 17:02:24 +0300 George Kadianakis desnac...@riseup.net wrote: Jacek Wielemborek d33...@gmail.com writes: W dniu 20.08.2015 o 15:49, George Kadianakis pisze: Some real UX research needs to be done here, before we decide something terrible. Just curious, has anybody seen any cognitive studies on the SSH randomart visualisation? I always found them impossible to remember. Perhaps adding a bit more color could help... Hm. Indeed. I can remember the general shape and edges of my SSH server's key, but not any details. I doubt I would remember the randomart of like 10 onion websites, especially if I didn't visit them regularly. But maybe I would remember the randomart of my webmail better than my SSH server's. What would be useful here is the number of onion addresses an average user visits. If it's small, something like this would probably be sufficient: 0. Browser generates/stores a long term salt. 1. On onion access, calculate SHAKE(salt | onion address) map that to a poker hand (5 card draw). P(52,5) = 311,875,200 C(52,5) = 2,598,960 2. Goto 1. Benefits: 0. Collisions between addresses are unlikely and don't weaken the scheme (though it will be confusing to the user that does encounter the situation). 1. Brute-forcing is mitigated via the per-client salt. 2. Incomplete recall still useful (User needs to forget all of order, suite, and card value). 3. Easy to memorize. 4. Easy to extend up to a point. 5. Can probably display it with fonts present on the system (or the one that will be bundled). Cons: 0. Having to propagate the salt value if the user uses multiple boxes is somewhat annoying. 1. May get confusing to remember lots and lots of onions-hand mappings. 2. No canonical visualization that can be shared across users. 3. Something Yawning randomly made up before going to bed. Regards and good night, -- Yawning Angel pgpm_xywJhZs5.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hash Visualizations to Protect Against Onion Phishing
On Fri, 21 Aug 2015 04:39:14 +1000 teor teor2...@gmail.com wrote: [snip] Visual schemes are only helpful to users who have the appropriate level of visual ability or processing: * as has already been mentioned, colouring schemes are not as useful to the colourblind; * facial recognition schemes are useless to the face-blind (including many autistic people); and any visual scheme would need to have a text alternative for screen readers or other tools used by the visually impaired. [snip] If we choose a list of English words, is that going to cause recognition issues for people who are non-native English speakers, or whose native script is a non-Latin script? (We could test this out.) As much as all of these are important issues, if we search for a solution that works for absolutely everyone, then we will never implement anything, because IMO no such thing exists. I sort of have mixed feelings about this in general, but since enough people seem to think it's a good idea, what appears to be the obvious way forward is: * Come up with a well defined interface for take an opaque blob, present it to the user. * Implement the underlying firefox glue. * Test deploy a few user select-able representation modules. Without doing so, trying to hash out any sorts of design(s) will likely end badly, and going with write the framework that lets us do UX testing will let us get a better handle on the problem[0]. Regards, -- Yawning Angel [0]: And who knows, an enterprising student may discover that the One Universal Onion Representation For Purposes of Quick Verification is Onions As Interpretive Modern Dance, Music, and Tactile Sensations Conveyed Via A Force Feedback Joystick, and write the appropriate plugin. Personally, I'm done with this subject and I'll stick to using bookmarks. pgpB6q_HC8tpD.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hash Visualizations to Protect Against Onion Phishing
On Thu, 20 Aug 2015 11:00:51 -0400 Ian Goldberg i...@cs.uwaterloo.ca wrote: On Thu, Aug 20, 2015 at 02:41:51PM +, Yawning Angel wrote: What would be useful here is the number of onion addresses an average user visits. If it's small, something like this would probably be sufficient: 0. Browser generates/stores a long term salt. 1. On onion access, calculate SHAKE(salt | onion address) map that to a poker hand (5 card draw). P(52,5) = 311,875,200 C(52,5) = 2,598,960 2. Goto 1. The per-browser salt is a good way to prevent similar-hash attacks, but of course will go astray if the user reinstalls her Tor Browser or has multiple devices. Yeah, that's a downside. I suspect it can be fairly short but I don't have a solid basis for that suspicion, and it also provides the property of being able to semi-arbitrarily increase the brute forcing difficulty at a latter date. I'd caution about the poker hand, though. One year when I taught first-year undergraduate CS, we included an assignment that had to do with decks of cards and card games. A surprising number of people had never seen decks of cards before, and were unfamiliar with the concept. I did not observe whether the (un)familiarity was correlated with what part of the world they came from. It was a hypothetical example. If we're willing to go with the visual equivalent of key poems (which is what my suggestion roughly corresponds to) with a per-client secret to prevent brute forcing, then there's no reason why we couldn't let the user choose a visual representation they're most comfortable with. (My initial example was going to use mahjong tiles. .) Perhaps a notification You've never visited this site before that pushes down from the top like some other notifications might go a long way? People would likely complain about storing did access foo.onion in the past type information to disk. I could argue for/against well, use a per-client keyed bloom filter, false positive rate, but depending on the adversary model, people will probably (rightfully) be uneasy at the thought of persisting even that. The moment people are willing to store I accessed this onion in the past, I'm inclined to think this is functionally equivalent to the user bookmarking said onion. Regards, -- Yawning Angel pgpukHV5j3ArQ.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Seeing through Network-Protocol Obfuscation
NB: quickly responding before I go to bed. On Wed, 19 Aug 2015 14:13:03 -0400 Philipp Winter p...@nymity.ch wrote: https://kpdyer.com/publications/ccs2015-measurement.pdf They claim that they are able to detect obfs3, obfs4, FTE, and meek using entropy analysis and machine learning. Not surprised for obfs3/4 since they're mounting an entropy attack which is explicitly outside of the stated threat model for both protocols. The FTE semantic attack they presented isn't the easiest one I know of (the GET request as defined by the regex is pathologically malformed). Haven't looked at the meek portion of the paper. I wonder if their dataset allows for such a conclusion. They use a (admittedly, large) set of flow traces gathered at a college campus. One of the traces is from 2010. The Internet was a different place back then. I would also expect college traces to be very different from country-level traces. For example, the latter should contain significantly more file sharing, and other traffic that is considered inappropriate in a college setting. Many countries also have popular web sites and applications that might be completely missing in their data sets. Dunno. Others probably have a better idea on what average internet traffic looks like these days. Considering the rate difference between normal and obfuscated traffic, the false positive rate in the analysis is significant. Trained classifiers also seem to do badly when classifying traces they weren't trained for. The authors suggest active probing to reduce false positives, but don't mention that this doesn't work against obfs4 and meek. Coming up with something better than obfs4/meek would be nice. At this point I'm viewing obfs4 as more of a minimum standard than anything else. It's worth noting that Dust2 (mostly done but not yet deployed) can reduce payload entropy to match a target distribution, but will have issues with protocol whitelist based DPI. Regards, -- Yawning Angel pgpFY2al5Ysy5.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is anyone using tor-fw-helper? (Was Re: BOINC-based Tor wrapper)
Just to touch base on this, and to give a rough status of where things are. The tor codebase no longer includes the C tor-fw-helper as of: d2cb92332009567ae778b3570e8fd3420c207446 Closes https://trac.torproject.org/projects/tor/ticket/13338 The new (Go based code) now lives at: https://gitweb.torproject.org/tor-fw-helper.git/ I changed the import paths and what not so that: $ go get git.torproject.org/tor-fw-helper.git/tor-fw-helper $ $GOPATH/bin/tor-fw-helper Does the right thing, at least on my box. In theory as long as the toolchain is properly setup, this will work on Linux, *BSD, OSX, and Windows, though it has been a while since I tested non-Linux (no major functional changes were made so I expect it to still work). If people don't like Go for some reason, they can write a functional replacement in $languageOfChoice, though unless they use library code, it is Not Very Fun. On Tue, 28 Jul 2015 01:18:07 + Jacob Appelbaum ja...@appelbaum.net wrote: * I still intend to move the new code from github.com to git.tp.o, and am willing to provide things like signed release tags, and tarballs of releases if that will make packaging it easier, but I won't be the one making packages (unless I happen to get bored enough to put it in AUR). That sounds fine by me - I think that if that other stuff is done - it is easy to package it. I don't quite have time to do the man page at the moment, but once that is done, I'll tag, and put signed tarballs up somewhere sensible. Since there are no dependencies required beyond a new-ish Go compiler, this should be utterly trivial to package. I'll try to do this sooner rather than later, but no promises since IRL stuff is on fire for the remainder of the week. Regards, -- Yawning Angel pgpwTfSMGIG6Q.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] curve25519_donna vs. crypto_scalarmult_curve25519?
On Tue, 4 Aug 2015 17:55:51 -0400 (EDT) Steve Snyder swsny...@snydernet.net wrote: Given a contemporary release of Tor with a contemporary version of OpenSSL, under what circumstances is the intrinsic curve25519_donna() preferred over the libsodium/NaCl crypto_scalarmult_curve25519(), or vice versa? A quick peek at the libsodium code reveals that it uses curve25519-donna (64 bit targets), and ref10 (32 bit targets?). Standard Tor uses curve25519-donna for both. I *think* donna's 32 bit code should out perform ref10, so libsodium is pointless. djb's NaCl might have assembly versions that are faster, but the difference doesn't look massive. http://bench.cr.yp.to/impl-scalarmult/curve25519.html Does it come down to 32-bit vs. 64-bit? Or CPU instruction sets detected at build time? Or has libsodium/NaCl been deprecated altogether and just not removed from the configure script? I think mostly this, at this point. Deprecating it might make sense, since libsodium is basically never going to outperform donna, and it's not practical to use libsodium/NaCl for Ed25519 due to Tor specific changes that would be required. I'm trying to determine if installation of libsodium in a Tor build environment will yield a better Tor binary. If libsodium, then no. If djb's NaCl then *maybe*, but the difference in practice will probably be fairly negligible (NB: discussing 64 bit platforms. 32 bit platform performance is somewhat non-interesting to me, so I haven't measured things in depth). I'm planning on revisiting this issue at some point, but last I looked into it, using an assembly optimized Curve25519 implementation was potentially a 7-10% gain (but is neither from libsodium nor djb NaCl). https://trac.torproject.org/projects/tor/ticket/8897 -- Yawning Angel ps: 0.2.7.2-alpha and later already does ntor ~25% faster, due to a faster basepoint multiply being used in the key generation step. pgpbYaJ7YxFMb.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is anyone using tor-fw-helper? (Was Re: BOINC-based Tor wrapper)
hilariously broken/insecure uPnP implementations (Documentation issue). * As far as support/bugfixes/maintenance from my end, there's a limit to what I can do for quirky/broken routers, and in a lot of cases this will end up as patches accepted. I hope this clarifies things. Regards, -- Yawning Angel pgpIdmA2SmBnG.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is anyone using tor-fw-helper? (Was Re: BOINC-based Tor wrapper)
On Thu, 23 Jul 2015 23:46:26 + Jacob Appelbaum ja...@appelbaum.net wrote: [snip] Do users know that their router's implementation of NAT-PMP/uPnP is shit? Who knows better than the user? And who better than the user to take an action and to learn it? At this point with all the resources available, I will guess that if the user needs something like tor-fw-helper, they probably have no idea what router firmware is. Eg: https://portforward.com [snip] I don't even understand why this is part of the discussion? Why is this our problem? Or put differently - sure, people don't patch their stuff - are we really making the problem worse? Wouldn't it make them more likely to interact with their router and perhaps apply patches to it? Dunno. Considering there was a bunch of fuss in the media about you should disable UPnP to increase security a while ago, we could be making things worse. Eg: http://www.forbes.com/sites/andygreenberg/2013/01/29/disable-a-protocol-called-upnp-on-your-router-now-to-avoid-a-serious-set-of-security-bugs/ And again, no. If they need tor-fw-helper, I doubt they know what firmware is in the first place. [snip] If they think they can/want to support this sort of thing, then they can say so, and I'll do the integration work on the Tor Browser side, and write the required flashproxy changes to make it work for more than ~2 hours when using NAT-PMP. That seems awesome - I guess I'd ask - does it need to be on by default? I'm not actually advocating for using it by default - I am advocating for shipping some NAT punching tool that can be used at all. tor-fw-helper never shipped as compiled code to end users which I found to be extremely demotivating. For flashproxy to work, yes, it would need to be on since flashproxy currently requires NAT traversal. WebRTC will also fix this, but a version of flashproxy that uses WebRTC does not exist yet. [snip] Any user that can compile a Go program can probably just do the NAT punching on their own anyway... $ go get github.com/yawning/tor-fw-helper $ $GOPATH/bin/tor-fw-helper I can't tell if you're agreeing with me here or if you are suggesting that people who have trouble configuring a program to use to a SOCKS proxy will be able to build and use a commandline tool. I assure you - nearly anyone who can use `go get` will be able to configure their NAT manually. For everyone else, we need some usable automation. A bit of both. In my opinion, people that can't setup port forwarding by hand shouldn't be running a Tor relay in the first place. There are no dependencies beyond what is provided by the Go compiler, so it's the easiest thing to package ever. If someone wants to package binaries for it, I don't care. I'll even add a manpage for it once the upstream git repo is move to git.torproject.org, tag/sign releases, and keep tarballs around if that's what people want. Seems reasonable. I had hoped it would be part of the default Tor build process - so anyone with a Tor can be a NAT punching relay or bridge or pluggable transport. We were very close to this with tor-fw-helper but never flipped the switch. It would be pretty sad if we went even further away from this much needed ability. I guess that is the direction of travel though. :( However, if it breaks, my response will be patches accepted for all but the most trivial bugs since it's not realistic for me to own every single router ever made. And more importantly, compromised routers due to shitty/out of date uPnP implementations are Not My Problem. If we shipped it, I'd say we're still improving on nearly every front over the C tor-fw-helper situation. If that's what people want to do. They should let me know so I sign/tag releases and add the documentation. Unless someone from the support people tells me they're ok with dealing with supporting users when this fails, I won't do the flashproxy work, but someone else is more than welcome to do it. Regards, -- Yawning Angel pgplTso9cEf2L.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is anyone using tor-fw-helper? (Was Re: BOINC-based Tor wrapper)
On Thu, 23 Jul 2015 19:18:34 + Jacob Appelbaum ja...@appelbaum.net wrote: Why are we avoiding allowing users to make this choice because of the above reasons? If a user wants to run a relay or a bridge, we should make it easy. We don't answer the above questions when it is hard - are we really off the hook there? It just seems ridiculous. Do users know that their router's implementation of NAT-PMP/uPnP is shit? It's more a uPnP issue, but I found bugs in at least one NAT-PMP implementation when writing the code (fixed in upstream, don't know how many people are running the newer code). Utterly horrific behavior especially in uPnP implementations is a well known and well documented problem. Eg: * http://www.upnp-hacks.org/annoyances.html * https://tools.ietf.org/html/rfc6886 (Sec. 9.5) * https://community.rapid7.com/servlet/JiveServlet/download/2150-1-16596/SecurityFlawsUPnP.pdf While the situation has probably improved over the years, having users use a feature on their router that should be off until the router firmware is known not to be horrible (See the report on RCE vulnerabilities) doesn't feel great to me. How many average users keep their router firmware up to date? [snippity] But we have a gigantic userbase, and playing consumer router support technician for all of the ones that ship with broken uPnP/NAT-PMP implementations does not fill me with warm fuzzy feelings. I think this is a weird analysis. How many of those people even try to be a relay or a bridge? Do we have numbers on that? Does the support team object or are you objecting on their behalf? It just seems too hand wavy for too many years to punt on dealing with NAT properly. I briefly spoke to Lunar about this at Valencia when I was asked why, given a rewrite exists that it's not being shipped with flashproxy. I was less focused on the relay side of things and more on flashproxy specific issues, which I'm still convinced will be Not Fun to support. If they think they can/want to support this sort of thing, then they can say so, and I'll do the integration work on the Tor Browser side, and write the required flashproxy changes to make it work for more than ~2 hours when using NAT-PMP. I admit, I am pretty frustrated that we implemented it, shipped the code for years and I'm probably the only person who really used it because of what feels like fear, uncertainty and doubt. Some of the concerns makes sense but it mostly just strikes me as a farce at this point. We can always make it harder later but we haven't really tried to make it easier, ever. Part of this sounds like a documentation issue. Any user that can compile a Go program can probably just do the NAT punching on their own anyway... $ go get github.com/yawning/tor-fw-helper $ $GOPATH/bin/tor-fw-helper There are no dependencies beyond what is provided by the Go compiler, so it's the easiest thing to package ever. If someone wants to package binaries for it, I don't care. I'll even add a manpage for it once the upstream git repo is move to git.torproject.org, tag/sign releases, and keep tarballs around if that's what people want. However, if it breaks, my response will be patches accepted for all but the most trivial bugs since it's not realistic for me to own every single router ever made. And more importantly, compromised routers due to shitty/out of date uPnP implementations are Not My Problem. Regards, -- Yawning Angel pgpphKdsowO7U.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is anyone using tor-fw-helper? (Was Re: BOINC-based Tor wrapper)
On Thu, 23 Jul 2015 12:50:29 -0700 David Stainton dstainton...@gmail.com wrote: But we have a gigantic userbase, and playing consumer router support technician for all of the ones that ship with broken uPnP/NAT-PMP implementations does not fill me with warm fuzzy feelings. I think this is a weird analysis. How many of those people even try to be a relay or a bridge? Do we have numbers on that? Does the support team object or are you objecting on their behalf? It just seems too hand wavy for too many years to punt on dealing with NAT properly. If I understand things correctly the uPnP/NAT-PMP is in fact not the proper way to solve this problem because of the reasons Yawning mentioned. IPFS (interplanetary filesystem) currently solves this problem via some complicated protocol with the selection of a rendezvous server... similar to Tor hidden services. Clearly this is the correct way to solve the NAT problem. Am I wrong about this? NAT-PMP (aka PCP) is less awful than uPnP is, may actually be ok (as long as you don't try to remove port mappings due to a bug in older miniupnpd), but is primarily an Apple-ism limiting it's usefulness. OTOH, the far more widely supported/deployed uPnP, on consumer routers at least, should be disabled and treated with extreme suspicion till proven otherwise. Regards, -- Yawning Angel pgpyEKzNPX65d.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is anyone using tor-fw-helper? (Was Re: BOINC-based Tor wrapper)
On Thu, 23 Jul 2015 18:26:33 + Jacob Appelbaum ja...@appelbaum.net wrote: Also - does this mean that after many many years... that this new version of tor-fw-helper be enabled by default at build time? Pretty please? :-) Unlikely, AFAIK the general plan was to have it as a separate package. That is really a major bummer if so - we should be shipping this code and enabling it by default. If a user wants to run a relay, they should only have to express that intent with a single button. The problem with this (and why we're not shipping it in Tor Browser, even if it would make flashproxy actually usable/useful to a large number of users) is because there is no one that is willing/able to deal with every single instance of: * My router crashed * My router crashed and I had to factory reset it * Why do I need to open a UDP port on my computer's firewall for uPnP/NAT-PMP to work, and how do I do that? * Random unrelated port mappings got blown away * My router's NAT TCP session table filled up * My ISP is complaining that I'm on some random asshole's blacklist because they include every single Tor Relay * Sites that used to work no longer work because some random asshole's blacklist includes every single Tor Relay * etc, etc, etc, etc And I certainly can't deal with my router has a strange idea of what the uPnP spec actually says, and it fails to port forward (unless they have/know how to use wireshark). I'm as unhappy at the general situation surrounding the codebase as anyone else, and if I thought deploying it would be a good idea, I'd be strongly pushing for it, since I think the new code I wrote will work for a lot of people. But we have a gigantic userbase, and playing consumer router support technician for all of the ones that ship with broken uPnP/NAT-PMP implementations does not fill me with warm fuzzy feelings. Regards, -- Yawning Angel pgpoHkfbE2rHa.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is anyone using tor-fw-helper? (Was Re: BOINC-based Tor wrapper)
On Thu, 23 Jul 2015 16:54:33 + Jacob Appelbaum ja...@appelbaum.net wrote: On 7/21/15, Nick Mathewson ni...@torproject.org wrote: Yawning's mail below reminds me: I am considering removing the C implementation of tor-fw-helper from the tor distribution, and recommending Yawning's pure-Go implementation instead. But before I do this, I'd like to get some sense of whether folks are shipping tor-fw-helper today, or using it in practice. Does the pure-Go implementation support NAT-PMP or just UPnP? It supports both, though NAT-PMP is limited to Linux, Windows, and *BSD (including Darwin), due to the need to query the routing table to obtain the IP address of the default gateway. It's easy-ish make the new code's NAT-PMP support other platforms (implement one function), but since the existing support covers what's needed I haven't gone and hunted down more obscure things. I still use tor-fw-helper when I hand compile Tor on obscure systems. Generally this means a Novena board when I need a newer version of Tor than is already packaged. Also - does this mean that after many many years... that this new version of tor-fw-helper be enabled by default at build time? Pretty please? :-) Unlikely, AFAIK the general plan was to have it as a separate package. -- Yawning Angel pgplUoRJtq2TV.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is anyone using tor-fw-helper? (Was Re: BOINC-based Tor wrapper)
On Tue, 21 Jul 2015 11:38:00 -0400 Nick Mathewson ni...@torproject.org wrote: Yawning's mail below reminds me: I am considering removing the C implementation of tor-fw-helper from the tor distribution, and recommending Yawning's pure-Go implementation instead. But before I do this, I'd like to get some sense of whether folks are shipping tor-fw-helper today, or using it in practice. On this note, should I move my tor-fw-helper rewrite from github to git.torproject.org? It hasn't had commits for a while, but it's not the kind of code that really rots (and it is complete/portable). Regards, -- Yawning Angel pgptxSAf8ktjE.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] BOINC-based Tor wrapper
On Wed, 22 Jul 2015 01:06:41 +1000 teor teor2...@gmail.com wrote: On 20 Jul 2015, at 11:11 , Serg std.s...@gmail.com wrote: How do you plan to map ports on NAT devices? If it can't be done automatically using UPnP, This must be done manually. No alternative cases. Our experience is that most routers' UPnP / NAT-PMP implementations don't work well with (our) automated tools. So this would have to be done manually, significantly reducing the pool of available volunteers. Just chiming in here. This may well work for a good number of users, but the support overhead for when it fails is utterly gigantic because certain brands of consumer routers have extremely poor UPnP/NAT-PMP implementations. The usual symptom of a poor implementation is the router crashes but certain other behaviors have been documented in the past by people trying to use UPnP in ways that are spec compliant such as the router crashes and requires a NVRAM reset, random port mappings get obliterated, the UPnP/NAT-PMP stack on the router crashes etc. A bigger issue is that a lot of consumer grade routers have a very limited amount of NAT session tracking space (in terms of absolute number of connections), which makes machines behind such devices extremely poor Tor relays (and even worse Guards)[0]. -- Yawning Angel [0]: In an ideal world every relay should be able to handle having a TCP connection open to any other relay simultaneously, plus connections from clients if they are guards, since relays do not get to choose their peers. Relays that fail to meet this criteria are (IMO) harmful to the health of the network. pgp1htVb98QWE.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Performance testing using chutney (was Re: Env variables for chutney)
On Sun, 05 Jul 2015 10:20:50 -0700 Cory Pruce corypr...@gmail.com wrote: Dude, thanks a bunch for you help. I'm really excited to start :D I'm going to read through the initial design and the code to see what functions/structures/constants/etc. need to be changed. Let me know when you release #14175 and I'll be happy to be the test guinea pig =-) Maybe look at these for a start. It's something that's on my TODO list, but I wouldn't complain if someone else happens to do it before I do, and it would help HS scalability considerably[0]. https://trac.torproject.org/projects/tor/ticket/13737 https://trac.torproject.org/projects/tor/ticket/13738 (If you happen to be more interested in making non-HS use cases faster, then look elsewhere. :P) Regards, -- Yawning Angel [0]: I do have a branch that makes circuit build crypto substantially faster that I've been poking at so the situation will be better in 0.2.7.x, but it's not via threading... pgpGJcZNgu2hr.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] How bad is not having 'enable-ec_nistp_64_gcc_128' really? (OpenBSD)
On Mon, 22 Jun 2015 15:55:59 -0400 l.m ter.one.lee...@hush.com wrote: Last I heard NIST groups are rubbish. You're better off without them for security. Am I wrong? DHE is worse (logjam being a recent high profile example), and is far slower. It's important to remember that TLS being broken while far from ideal is insufficient for adversaries since they will need a Curve25519 break as well to actually get plaintext. It is worth noting that as of 0.2.7.x, tor will *require* OpenSSL with ECDH support, and one of P-244 or P-256. There is an IETF draft circulating for standardizing other curves (Ed25519, Ed448) which hopefully will see uptake in the longer run, but ECDHE with the NIST curves is the current least bad choice. Regards, -- Yawning Angel pgptam3OkluOA.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Dumb or-ctl-filter tricks (Was: [tor-talk] SOCKS proxy to sit between user and Tor?)
Hello, I just pushed a fairly large update to or-ctl-filter, that lets you do lots of interesting things, most of them probably unsafe. In particular or-ctl-filter now ships with a SOCKS5 client/server implementation and a stub control port implementation. A picture is worth a thousand words: https://raw.github.com/Yawning/or-ctl-filter/screenshots/or-ctl-filter-tor-i2p.png What it does: * Filters the control port exposed to Tor Browser for things that (IMO) the browser has no business knowing just in terms of attack surface. In particular this intentionally breaks the circuit display feature as part of 4.5.x. * Allows easy integration of Tor Browser with a system tor service (NB: I run a system tor service with the Tor Browser circuit lifespan patch, if you do not, you will get behavior that is different from other users. You have been warned.) * Supports transparently redirecting .i2p requests to an I2P instance. Tor does not need to be running for this. * Enforces isolation to attempt to guard the local I2P web server and management interface from cross protocol trickery, evil Javascript and whatnot. * Supports running without Tor or I2P at all, essentially changing Tor Browser into Firefox with a bunch of patches. Limitations: * NEWNYM does not affect I2P tunnels. * New Tor Circuit For This Site does not affect I2P tunnels either. * Only cookie authentication is supported because I'm lazy, and it is the superior authentication method. * Launching Tor/I2P is not or-ctl-filter's problem and will never be part of the feature set. I have systemd for that. Warning(s): * Very alpha. It is entirely possible that I screwed up enforcing isolation. You can hard disable access to locally hosted i2p services like the management console in the config file. It is still probably 3 million times better than using privoxy/random sketch addons to do something like this because I actually do look at circuit isolation from Tor Browser and propagate it to Tor (or enforce it as best as I can otherwise). * If you enable logging, it will happily splatter destinations, authentication credentials, and everything else to the log, because it is a debugging feature, so don't. * If you enable the option named UnsafeAllowDirect and disable Tor, it will happily connect directly to the internet, destroying your anonymity. * Untested on Windows. Should work, don't care if it doesn't. Patches will sit in my INBOX forever; ignored, and unloved, just like the operating system they target. The same goes for OSX.[0] Code: https://github.com/Yawning/or-ctl-filter/tree/master -- Yawning Angel [0]: Honestly, I'll merge trivial things, but I won't bust out my windows box to test/debug this, and I don't have an OSX box. pgp4kvHS2QXRf.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Quick logjam/Tor analysis.
On Wed, 3 Jun 2015 00:43:50 +1000 teor teor2...@gmail.com wrote: (Mac) OS X Yosemite 10.10 and earlier ship with OpenSSL 0.9.8 and 0.9.7. [snip] While it's possible to build or install OpenSSL 1.0 or 1.1 on OS X, it's not the default. How does this affect Tor and/or Tor Browser on OS X? Tor Browser builds/includes it's own copy of OpenSSL, so there is no impact there. As of a little while ago on master, tor requires OpenSSL 1.0.0 with ECDH support at build time. AFAIK this breaks the build with OSX, FreeBSD 9.x, and certain (Old) versions of Centos/RHEL when compiling against the vendor's OpenSSL. The only resolution is Too bad, so sad, install a modern OpenSSL. See #16034 and #16040 for details. -- Yawning Angel pgp01DoAVjA4U.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Tor Browser IsolateSOCKSAuth behavior questions.
On Mon, 01 Jun 2015 07:43:50 + Georg Koppen g...@torproject.org wrote: My question is, what causes Tor Browser to set the SOCKS username to --unknown-- and what the behavior should be in that case if: Ideally, --unknown-- would only be used for requests originating from privileged browser code and not belonging to a website/resource a user requested. This would encompass things like extensions update requests, browser update requests, blocklist checks, requests issued by installed extensions to name just a few. In reality, however, we are not there yet (see e.g. #13670, #15599, #1, #15569 + plus there is at least one bug I have not filed yet). Ah ok. That explains why I saw --unknown-- when browsing reddit with my shim running. * The destination is a .onion address. #15499 should give you an idea (although I am not sure whether that ticket is still valid) I think spacebook uses subdomains, but in the .onion space, so suffix based validation (socks username, and the requested destination both end in .onion) is sufficient here. * The destination is a .i2p address. I don't know. Maybe we/you should coordinate that with the I2P folks? I'm fairly sure just matching on suffix is also ok here. * The destination is the I2P management console. I'm fairly sure this should be deny. Sounds good. This is the only way I'm messing with/using the Isolation information currently. The management console appears to work. * The destination is any other address (will be dispatched over Tor if running, I don't think I will attempt to support I2P outproxies because they suck). (I think allow because things break otherwise?) I am not sure, honestly. What do you have in mind? [snip] The final form of my shim will support running with any combination of nothing (Tor Browser just for the privacy benefits, probably unsafe, I may reconsider this), I2P, and Tor (Though the most useful configuration is probably I2P + Tor). Sounds useful, indeed. But I think we should make clear to users that this will not be a proper Tor Browser replacement as you need knowledge of the browser state to make correct assumptions on whether to put requests into the --unknown-- bucket or not. And I currently don't see how your shim is able to accomplish that. Well, it's real Tor Browser, just talking to an intermediary SOCKS proxy rather than the one exposed by Tor. I currently propagate the SOCKS username/password if possible (it's not for I2P, and no analog to the isolation exists, so stuff like new identity/new circuit will be a lot of work). Behavior without I2P running (or browsing stuff I dispatch to Tor) is identical to normal Tor Browser, given certain limitations[0]. Tor Browser already does the work of placing requests in the correct bucket for me, I'm just trying to use the work Tor Browser does to prevent cross protocol probing/attacks. Maybe the right thing to do here would be to dispatch based on the isolation info and not the destination, since that doesn't give away the presense of the shim: * If the socks username ends in .i2p send it to I2P regardless of destination. * If the socks username ends in .onion send it to Tor regardless of destination. * Otherwise send it to the default upstream provider (Tor in most cases). (Note: The I2P management interface is special cased, and always will be special cased, since it's extra sensitive.) But yeah. There will be tons of warning labels. Just trying to figure out a good way to get I2P and Tor to co-exist in a single Tor Browser instance while being relatively safe[1]. Regards, -- Yawning Angel [0]: The code assumes it's talking to a system tor instance (it doesn't launch Tor for you), my control port filter is present so circuit display is broken intentionally, etc. [1]: User safety is the #1 goal here, but I know my first stab at this probably won't be perfect. I think it will still be an improvement over install foxyproxy with a bunch of regexes since I do have the option of peeking at the isolation. pgp_2hNyAkk5U.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Tor Browser IsolateSOCKSAuth behavior questions.
Hello, I've been working on a dumb hack that lets me do things like this: https://imgur.com/3mah244 (Yes, that's a single Tor Browser instance, separate windows used for illustrative purposes.) It's still very raw and doesn't do everything I want it to do, so I'm not really releasing the code yet, but I have some questions regarding how Tor Browser behaves when setting the SOCKS username for isolation purposes. Ideally I want my shim to enforce isolation between the various upstreams (Tor, I2P, whatever) correctly to avoid cross-protocol probing (and to shield the I2P administration interface from eeevil websites). This appears to be straight forward if the application is Tor Browser because IsolateSOCKSAuth is always used at first glance (I will assume for now that if users decide to use things like torsocks that do not use isolation this way that they know what they are doing). My question is, what causes Tor Browser to set the SOCKS username to --unknown-- and what the behavior should be in that case if: * The destination is a .onion address. * The destination is a .i2p address. * The destination is the I2P management console. I'm fairly sure this should be deny. * The destination is any other address (will be dispatched over Tor if running, I don't think I will attempt to support I2P outproxies because they suck). (I think allow because things break otherwise?) For destinations that are .onion/.i2p, I plan to be fairly strict about making sure the SOCKS5 target and the username matches (I need to be more relaxed for sites on the regular intertubes since cross-site resources are loaded (I may make this behavior configurable...). Is this dumb? Is it common for foo.onion to load resources off bar.onion? How about in I2P land? The final form of my shim will support running with any combination of nothing (Tor Browser just for the privacy benefits, probably unsafe, I may reconsider this), I2P, and Tor (Though the most useful configuration is probably I2P + Tor). Thanks in advance, -- Yawning Angel pgp4CFhRjXQuC.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [RELEASE] Torsocks 2.1.0
On Thu, 28 May 2015 14:55:07 +0200 anonym ano...@riseup.net wrote: On 05/27/2015 09:19 PM, David Goulet wrote: - IsolatePID is a new option that will make torsocks set the SOCKS5 username and password automatically to provide isolation on Tor side. You can use this with the -i,--isolate command added or TORSOCKS_ISOLATE_PID env. variable. Perhaps I'm overlooking the purpose of this option, but it doesn't look like something you can rely on to get stream isolation between separate torsocks invocations. For instance, on a POSIX-compliant system a PID can be reused as soon as the process using it has terminated. This implies that, for any purpose, a PID should only be used as a unique identifier for the duration of the process using it. Assuming the purpose is to provide stream isolation between torsocks invocations, why not generate a big random numbers for the socks auth user/password instead? 256 bits should be enough even when taking the birthday paradox into account. In practice PID resuage perhaps isn't a problem for any realistic use case, making my remarks into theoretical nitpicking, but YMMV. Ah, this is a matter of the feature is explained in a simple manner in the abbreviated changelog. What the flag actually does is: * Username set to 'torsocks-' PID ':' unixTime * Password set to '0'. (If one day torsocks wants to add something like a signal handler to get tor browser 'new circuit' type behavior, my plan was to increment this, not currently used). While not as great as 256 bit random numbers, PID reuse within the same second is rather unlikely, at least under Linux PIDs are only recycled when the pid wraps (/proc/sys/kernel/pid_max), I assume most other Unixes follow similar behavior. The identifier was chosen such that it would be blatantly obvious in circuit listings as to which torsocks instance things belong to. There is space in the username field, so appending a hexdecimal large random number or something is certainly possible and quite trivial. Regards, -- Yawning Angel pgpAXdf3KDOMa.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [PATCH] Defences against the recent hidden service DoS attacks
Hello, Some minor notes... On Wed, 20 May 2015 20:03:38 +0100 George Kadianakis desnac...@riseup.net wrote: == Instructions == Our patch is not in an official Tor release yet, so you will need to use an unofficial git branch: https://trac.torproject.org/projects/tor/ticket/16052#comment:18 The configuration parameters are now in master (aka 0.2.7.1-alpha-dev). It's likely that a 0.2.6.x backport will happen, but feedback would play an instrumental part in ensuring that happens (either as a reply, or by commenting on the trac ticket). Next, an operator who wants to deploy this experimental fix, should first figure out how many simultaneous TCP connections a normal client would establish. For example, an IRC server would probably not need more than 1 simultaneous connection per user. A web server, depending on the use, might need something between 6 to 12 (?) simultaneous connections. Per discussion with the Tor Browser developers, I have been told that 6 is the correct number for http content, and that if there are any more streams associated with a Tor Browser user accessing a site, it would be a Tor Browser bug. Other browsers/protocols may require a higher or lower limit. A warning is logged periodically (rate limited to avoid log spam/clutter) if circuits exceed the limit, so adjusting the parameter should be relatively straight forward. Regards, -- Yawning Angel pgpEKoiBJTmPv.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] tor's make test fails when run on tails
On Mon, 27 Apr 2015 05:49:37 + warms0x warm...@riseup.net wrote: I am trying to execute tor's tests from git (63a90f2) on tails but the socketpair_ersatz test keeps failing util/socketpair: [forking] OK util/socketpair_ersatz: [forking] FAIL src/test/test_util.c:4212: assert(0 OP_EQ tor_socketpair_fn(family, SOCK_STREAM, 0, fds)): 0 vs -111 [socketpair_ersatz FAILED] util/max_mem: OK The failure seems to be in connect(): https://gitweb.torproject.org/tor.git/tree/src/common/compat.c#n1492 Considering that the error returned is -ECONNREFUSED, this is a reasonable assumption, yes. armadev said on IRC that he/she could make the tests pass, so I wonder if there might be some specific behavior on tails that prevents the test from passing successfully. On a normal system, the tests pass fine, and it would be a bug if they didn't. Did you look at the `tor_ersatz_socketpair()` code at all? It should be obvious why this is failing on Tails. Is there a suggested workaround? Or perhaps is it possible to easily ignore this one test? Suggested workaround(s): * Don't use Tails, the tests are written under the assumption that they are running on a normal system that doesn't have a facist firewall preventing connections to things listening on the loop back interface. * Patch the test code to disable that test. * Maybe use torsocks with AllowOutboundLocalhost, if the version included in Tails supports the option (I'd hope so, I added support for that feature to torsocks specifically for them). Other tests may or may not fail if you chose to go down this path. Regards, -- Yawning Angel pgpf9q_ceZn_j.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Release: obfs4proxy-0.0.5
Hello all, I just tagged obfs4proxy-0.0.5, this time with improvements for both clients and servers. All users are recommended to upgrade. Special thanks to mvdan for various code cleanups. Tarball/Signature: https://people.torproject.org/~yawning/releases/obfs4proxy/obfs4proxy-0.0.5.tar.xz https://people.torproject.org/~yawning/releases/obfs4proxy/obfs4proxy-0.0.5.tar.xz.asc Changes in version 0.0.5 - 2015-04-15: - Go vet/fmt fixes, and misc. code cleanups. Patches by mvdan. - Changed the go.net import path to the new location (golang.org/x/net). - Added limited support for detecting if the parent process crashes. - Support for tor feature #15335 (stdin based termination notification). - Moved the leveled logging wrappers into common/log so they are usable in transport implementations. - Added a DEBUG log level. - Use a bundled SOCKS 5 server instead of goptlib's SocksListener. The big features are: - obfs4proxy now tries really hard to detect if the parent process has crashed, using various system specific hacks (and the new and improved tor assisted mechanism in #15335). This should reduce defunct obfs4proxy processes when tor happens to crash without tearing down the obfs4proxy instance. - Instead of using goptlib's SOCKS4a server, obfs4proxy now includes a SOCKS5 implementation, bringing IPv6 support to clients. Note that running IPv6 bridges has always worked (though dual stack configs are currently somewhat broken due to a tor PT configuration protocol limitation). Notes for packagers: - Like in obfs4proxy-0.0.5, one of the upstream dependency locations has changed. (code.google.com/p/go.net - golang.org/x/net). As far as I am aware, migrating to the new package immediately is not required though, should be done sooner rather than later due to the impending deprecation of code.google.com. To temporarily continue to build against the old go.net dependency, please revert: aed4b723891db1be34eb866a03c62806b58ac148 - It is *strongly* recommended that packages be built against goptlib-0.4 or newer to work around tor bug #15240. Without this workaround, certain bridges will fail to operate correctly when the ExtORPort is enabled (the Tor side fix is in tor-0.2.6.5-rc and newer). Questions, comments, feedback appreciated, -- Yawning Angel pgpyviXcC_LtQ.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Fwd: Orbot v15 RC3... now with x86/ATOM-power!
On Mon, 13 Apr 2015 10:14:43 -0400 Nathan Freitas nat...@freitas.net wrote: One interesting issue is that GoLang 1.4.1, which we are using to cross-compile the Meek and Obfs4 pluggable transports to Android, only supports targeting Android ARM for right now... I assume that will change soon, but if not, we may have to add Obfsclient back into Orbot for supporting x86 devices. Hmm, maybe I should add obfs4 support to obfsclient. I have code for all of the crypto I would need to add. Regards, -- Yawning Angel pgpROO5FvVuvJ.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Are DAC_OVERRIDE CHOWN capabilities required for ControlSocket?
On Sun, 12 Apr 2015 12:46:26 + Nusenu nus...@openmailbox.org wrote: tor will fail to startup with the current systemd service file [1] if your torrc makes use of the ControlSocket feature. To work around the issue one has to additionally allow the following capabilities: CAP_DAC_OVERRIDE CAP_CHOWN since the socket file is create as root and then changed to the tor user (chown). Is it possible to change this to not require CAP_DAC_OVERRIDE and CAP_CHOWN capabilities anymore? I bet using the AF_UNIX SocksPort stuff will break as well, since the code is common. All of the listeners are launched before switching uid/gid and dropping privileges since it's common code. The way to fix this would be to change retry_listener_ports and retry_all_listeners code to additionally allow only launching service ports ( 1024), and staging the listener launch process on config (re)load to something that looks like: 1. Launch listeners that require elevated priviledges (CAP_NET_BIND_SERVICE). 2. Drop priviledges and switch the uid/gid. 3. Launch the rest of the listeners, including all of the AF_UNIX based ones (as the runtime tor user, so neither privilege is required). Patches accepted. -- Yawning Angel pgpCrZZmkj5AW.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] RFC: Ephemeral Hidden Services via the Control Port
Hello all, Just a quick heads up for where this is going: * 'GET_ONIONS' was changed to be 'GETINFO' subcommands for consistency. * Code is in 'needs_review', for the 2nd round. As a proof of concept I wrote: https://github.com/Yawning/onionwrap It's a dumb quick and dirty hack that works like this: $ TOR_CONTROL_PORT=9251 ./onionwrap -port=80,8080 godoc -http=%TADDR INFO: Created onion: 5rn73le64hri2zmp.onion:80 - 127.0.0.1:8080 [onionwrap blocks till killed] The %TADDR notation is a convenience thing for automation, there's also %TPORT, and %VPORT that get rewritten as appropriate. The onionwrap code is still incomplete, so I wouldn't recommend using it quite yet, especially since it: * Only supports creating one shot Onion Services. * Should support talking to the real server over AF_UNIX. * Needs documentation. * Requires a really experimental branch based on tor's master. * Maybe should use stem/txtorcon instead of bulb[0]. But as a proof of concept and a demonstration of the feature, I think it gets the point across. Thanks to special for inspiring me to write this. Regards, -- Yawning Angel [0]: Which is another quick and dirty hack I wrote, so I could write things like this. pgpEG1HqWaqA8.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Wide block cipher experiment.
Hello, Nickm mentioned to me that he was curious as to how LIONESS performs these days (See #5460) with modern cryptographic primitives. I've conveyed the results to several people, but I'm also sending them here for posterity. Code used: https://github.com/yawning/lioness (May be incorrect, don't use for anything other than benchmarking. Numbers taken with a previous version of the code without the initial memcpy, that was added later so that the code in git could be used by the extremely brave for other things.) All measurements taked on an i5-4250U, so the usual caveats about turboboost and hyperthreading apply. Baseline (from tests/bench, AES-NI enabled): = cell_ops = Inbound cells: 231.33 ns per cell. (0.45 ns per byte of payload) Outbound cells: 224.39 ns per cell. (0.44 ns per byte of payload) (Note: Outbound with AES-NI disabled is ~3.0 ns per byte) LIONESS (BLAKE2b/ChaCha, 509 byte block size): * ChaCha20: * Ted Krovetz's AVX2-ed ChaCha20/Ref AVX BLAKE2b: ~6.6 ns/byte (~143 MiB/s) * AVX2ed-ed ChaCha20, Andrew Moon's AVX2-ed Blake2b: ~5.0 ns/byte (~190 MiB/s) * ChaCha12: * Ted Krovetz's AVX2-ed ChaCha12/Ref AVX BLAKE2b: ~6.1 ns/byte (~156 MiB/s) * AVX2ed-ed ChaCha12, Andrew Moon's AVX2-ed Blake2b: ~4.4 ns/byte (~213 MiB/s) * ChaCha8: (Yolo swag 420 blaze it) * Ted Krovetz's AVX2-ed ChaCha8/Ref AVX BLAKE2b: ~5.8 ns/byte (~164 MiB/s) * AVX2ed-ed ChaCha12, Andrew Moon's AVX2-ed Blake2b: ~4.1 ns/byte (~232 MiB/s) NB: Using Andrew Moon's Blake2b isn't in git, because the way I tested it was kind of kludgey. Profiler output: 64.04% lioness_test_av lioness_test_avx2 [.] blake2b_compress 22.43% lioness_test_av lioness_test_avx2 [.] chacha_stream_xor 6.60% lioness_test_av lioness_test_avx2 [.] blake2b_init_key 2.72% lioness_test_av lioness_test_avx2 [.] blake2b 2.41% lioness_test_av libc-2.21.so [.] __memcpy_avx_unaligned 1.07% lioness_test_av lioness_test_avx2 [.] lioness_encrypt_block Ted Krovetz's ChaCha implementation isn't quite the fastest out there, but it doesn't lag massively behind Andrew Moon's. Benchmarks on the same hardware from Andrew Moon's chacha-opt/blake2b-opt are: BLAKE2b: 576 byte(s): avx2, 1468.00 cycles per call, 2.5486 cycles/byte avx, 1674.00 cycles per call, 2.9062 cycles/byte x86, 2020.00 cycles per call, 3.5069 cycles/byte generic/64, 2638.00 cycles per call, 4.5799 cycles/byte ChaCha20: 576 byte(s): avx2, 694.00 cycles per call, 1.2049 cycles/byte avx, 1104.00 cycles per call, 1.9167 cycles/byte ssse3, 1112.00 cycles per call, 1.9306 cycles/byte sse2, 1376.00 cycles per call, 2.3889 cycles/byte x86, 2528.00 cycles per call, 4.3889 cycles/byte generic, 3200.00 cycles per call, 5.5556 cycles/byte I don't think using CTR-AES (with AES-NI) in a LIONESS construct is going to be that big of a win, at least on my hardware, and the sort of performance I'm seeing feels too much of a performance hit to me. Regards, -- Yawning Angel pgpDNb5K2d0Nd.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Pluggable transports meeting tomorrow (16:00 UTC Wednesday 18th of March 2015)
(Trying to get these back onto a regular schedule now that dev-meeting madness is over.) Just wanted to remind you that the regular biweekly pluggable transports meeting is going to occur tomorrow at 16:00 UTC. Place is the #tor-dev IRC channel in the OFTC network. https://trac.torproject.org/projects/tor/wiki/doc/PluggableTransports#PluggableTransportIRCmeetings -- Yawning Angel ps: UTC does not do daylight savings. Those of you that have had clocks change, please double check the time and let me know if we need to change the time. pgpDv6z9btUDC.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] RFC: Ephemeral Hidden Services via the Control Port
On Wed, 11 Mar 2015 02:35:10 + Yawning Angel yawn...@schwanenlied.me wrote: The code: https://github.com/Yawning/tor/compare/feature6411 The spec: https://github.com/Yawning/torspec/compare/feature6411 Minor updates to both over the course of yesterday, thanks to all that gave useful feedback and corrections. The notable changes are: * DiscardPK is now Flags=DiscardPK, to allow for. [0]: Onions added this way will remain tied to the control port connection that created them for now. That particular dead pony has been loaded onto the cart and shipped off to the glue factory, and is no longer available for beatings. * I lied. People that want to shoot themselves in the foot can now specify the Detach flag which explicitly unlinks the newly created HS from any control port instance. Detached Onion Services will persist till tor termination, or explicitly removed by DEL_ONION. * DEL_ONION now will allow the removal of Onion Services belonging to the current connection in addition to any Onion Service created with the Detach flag. * Added GET_ONIONS that returns a list of Onion Services belonging to the current connection in addition to all Onion Services created with the Detach flag. I strongly recommend ignoring the fact that the Detach flag and GET_ONIONS command exists, beyond reviewing my code and making sure they're implemented correctly. That said, I know multiple developers will probably write applications that implements cleanup as DEL_ONION ALL THE THINGS, stomping over unrelated Detach-ed services[0]. Unless something comes up, I plan to rebase/squash my feature branch for review sometime by early next week, so if people have strong opinions on this feature, they should speak up now. Regards, -- Yawning Angel [0]: I have my not a bug/wontfix ready for such situations. pgpZA9563xnv3.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] RFC: Ephemeral Hidden Services via the Control Port
Hello, Not sure about the renaming thing. I'm open to doing so, but a lot of the user facing stuff (the torrc configuration variables etc) use the HS terminology, so I'm not sure if being inconsistent is good. On Wed, 25 Feb 2015 13:51:59 +0100 carlo von lynX l...@time.to.get.psyced.org wrote: On Wed, Feb 25, 2015 at 07:41:01AM +0100, Andreas Krey wrote: The AF_TOR listener would go away with closing the listener socket as well (and thus is bound to the lifetime of the process); so binding a hidden service to the control connection is the obvious analogy. Yes, but as it stands AF_TOR is not the #1 API deployed in network applications. The majority of hidden services are $whatever configured to listen on port localhost:something and zero awareness of any tor router doing the rest of the work. Having to change hundreds of existing apps so that they can work with tor without having to edit torrc is a worse tradeoff than having to edit torrc. So, write a python script that pulls in txtorcon or stem, does the appropriate controller goo, and subprocess.call()s $whatever. Anyway now that master is 0.2.7.x, I'm looking to revisit this in the form of: * Renaming the commands if necessary. * Minor code cleanups (that occurred to me after I let it sit for a while). * Adding an argument to `ADD_EPH_HS` to suppress the private key being returned over the control port, when the caller requests that tor generate the keypair (for cases where the HS is truely oneshot and the user does not wish to preserve it). * Documentation. Regards, -- Yawning Angel pgpAaj3f82G9d.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] RFC: Ephemeral Hidden Services via the Control Port
On Tue, 17 Feb 2015 02:22:54 +0400 meejah mee...@meejah.ca wrote: From my perspective, the entire point of this feature is to allow applications to use the system Tor (or, at least some already-running tor) to put their hidden services on. The design the way it is with more isolation than usual also allows for people that use the tails/whonix/or my control port tinfoil hat filter to potentially whitelist the commands as well, since there isn't any information disclosure (unless I screwed up the implementation). Modifying the filters is left up to the implementors (and I probably won't change or-ctl-filter to allow them since I don't use HSes[0]), but I view it as a step forward for including/using applications that use HSes into such environments. (Or, looking at it another way, if you don't want to share a tor instance with other applications, you can do that easily today -- you start up a tor, and can use __OwningControllerProcess as you see fit). The *only* reason that txtorcon by default launches a new tor process for the onion: endpoints is because nothing else will work reliably. I'd very much like the default to be that it adds a new onion using Yawning's API. This will still fall back to launch a new tor process, but if you know what you're doing you can have a system (or personal) Tor instance running (with cookie authentication) that can have hidden services added and removed from it. Obviously, this will be a lot faster than launching a new Tor (and less load or DirAuths). As a bonus, Yawning's API is nice and simple (versus SETCONF, which makes it super easy to nuke all the *other* hidden services you configured). What I would expect of people using this feature is that the process that's adding the hidden service is the very same one that's providing the service... (If you're doing something with a separate nginx or whatever process running your hidden-service, than you should know enough to be able to add a couple lines to your torrc and make a normal, permanent hidden-service). Or write a trivial script that prompts for a password, decrypts a HS key, and loads it into a tor instance over the control port. I assume if/when my branch gets merged into mainline tor that the controler APIs will support it in a easy to use manner. So, in that context, if your application code isn't perfect and might sometimes crash, definitely the right thing is to nuke the hidden service. If the app didn't even save its private keys before that happened, well, too bad. I guess to put another way: I can't see a use-case to keep the hidden- service around if the application that added it went away. It seems a lose consensus is reached here that this is ok, so I'm going to leave the design as is and write the control-spec.txt patch. -- Yawning Angel [0]: The first HS I ever set up was when I finished my first pass implementation, and got the code to a working state. pgpBl_7Y6_Y46.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] RFC: Ephemeral Hidden Services via the Control Port
On Mon, 16 Feb 2015 10:17:51 -0500 David Goulet dgou...@ev0ke.net wrote: [snip] A hidden service is created using the key and list of port/targets, that will persist till configuration reload or the termination of the tor process. Now, an HS bound to a control connection might be a good idea, I'm not 100% sure but I can see issues with this. Let's say ControlListenAddress is used, this means a TCP socket and it can timeout if no activity, so if that happens, I loose my HS? That's correct, though unless tor or the controller library has code to stomp on long dormant connections (which a casual look says there isn't), this shouldn't happen, because TCP/IP in itself has no idle timeout (See RFC 1122 4.2.3.6 regarding keep alives, which would also not cause HS loss, since the OS will respond to the probe). There may be certain broken middleboxes (loadbalancers etc) that stomp on long idle TCP connections, but anyone that is running a control port connection through such a thing, and sending RSA keying material in the clear, probably has bigger things to worry about. This also put quite a requirement on the user side to add an HS on its tor-ramdisk for instance but has to use a client that keeps the control connection opens for its lifetime?... How does that work with stem, it would have to keep the control connection open and the script using it can't quit else the socket gets closed by the OS? Yup, I don't see you need to leave stem running as being all that bad, since this is mostly targeted at managed applications (chat, filesharing, global leaks, etc). If someone has a suggestion for an alternative interface that can handle applications crashing (possibly before they persist the list of HSes they need to clean up), applications that are just poorly written (and not cleaning up all the ephemeral HSes), and (optionally, though lacking this would be a reduction in features) limiting cross application HS enumeration, I'd be more inclined to change things. Regards, -- Yawning Angel pgpw3rUD3yJJF.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] RFC: Ephemeral Hidden Services via the Control Port
On Mon, 16 Feb 2015 16:11:55 + Leif Ryge l...@synthesize.us wrote: [snippity] However, it seems like in the case of applications which are not HS-specific this will necessitate keeping another process running just to keep the HS alive. I'd rather see two modes: one as you describe, and another in which the ephemeral HS stays running until a new control port connection requests that it be stopped. To avoid allowing enumeration of running services, the stop command could require that the requestor already knows some details of the HS - either a cookie generated at creation time, or perhaps just the private key that was provided when it was started. dgoulet suggested Detach=true as an optional argument, which is what the add side interface would look like if I did this. This of course wouldn't result in crashed applications' HSes being cleaned up automatically, but having a few stale HSes sitting around isn't the end of the world. One approach for cleaning them up could be that tor could remove them automatically after it sees connection refused a few times. I'm not quite sure how I feel about this yet. The code for doing all of this isn't that difficult, but I'd want to hear from a few more people about what the right thing to do here would be. Most importantly since the `ADD_EPH_HS` interface uses key/value pairs for the port/target now, this would be easy to add on at a later date even if it doesn't get included in the first iteration. Something to discuss at the dev-meeting if consensus hasn't been reached by then. Regards, -- Yawning Angel pgpfWLpkffCJ7.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] RFC: Ephemeral Hidden Services via the Control Port
On Mon, 16 Feb 2015 19:35:58 + Michael Rogers mich...@briarproject.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 (CCing the hidden-services list.) (Wonder if my reply will bounce.) On 16/02/15 16:11, Leif Ryge wrote: If someone has a suggestion for an alternative interface that can handle applications crashing (possibly before they persist the list of HSes they need to clean up), applications that are just poorly written (and not cleaning up all the ephemeral HSes), and (optionally, though lacking this would be a reduction in features) limiting cross application HS enumeration, I'd be more inclined to change things. First, thanks for doing this! I think it's a great feature which will make it much easier to create new hidden service applications. Seconded! I like the idea of tying HS lifetime to the control port connection for the reasons you state, namely that cleanup is automatic when applications crash. As an app developer this strikes me as the right approach. But having said that, I wouldn't actually need this feature because Briar already uses __OwningControllerProcess to shut down Tor if the control connection is closed. I imagine the same would apply to any app that manages its own Tor process - so this feature would only be useful for apps that share a Tor process with other apps and communicate directly with it via the control port, rather than indirectly via a controller such as Orbot. Yep. I suspect that well behaved applications won't need this for the most part, but by defining things this way, it avoids unpleasant surprises for apps that aren't written well, or those that do try to share a tor instance. Are there any such apps, and is it a good idea to support such apps (has the rest of the control protocol been examined for cross-controller data leaks, what happens if apps tread on each other's configuration changes, etc)? I haven't looked at other concerns in depth, and AFAIK it's a huge problem area. My concerns in this area are more does my branch make the current situation any worse, rather than solve all the control port problems, and make sure this is totally a safe/fine/recommended thing to do (It's none of those things). If most if not all apps will set _OwningControllerProcess, the current behavior doesn't hurt either since the tor instance will die anyway, and on the off chance that someone writes a not-so-great app (heaven forbid), the right thing happens. However, it seems like in the case of applications which are not HS-specific this will necessitate keeping another process running just to keep the HS alive. Dormant processes are essentially free, so does this matter? I have this mindset too. -- Yawning Angel pgpUVCOT7N8h3.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] RFC: Ephemeral Hidden Services via the Control Port
On Sat, 14 Feb 2015 13:46:04 -0800 Damian Johnson ata...@torproject.org wrote: One gotcha to think about is that ADD_EPH_HS is using a variable number of positional arguments. This will limit your ability to expand this command in the future with new arguments. Also, I'd suggest renaming addrPort to addrTarget (to avoid making this sound restricted to a simple port). Hmm ok. Judging by the feedback, I'm thinking the following: * Auth is still a low-ish priority, I want to get the basic ephemeral stuff done first, and I need to read up more on how it works, and how the code is structured, before I can promise things in this area. * People seem to be ok with the tying ephemeral HSes to the originating control port (and if the only major argument against is it's a bit weird, relative to other things, well, eph. HSes are weird, and this solves a bunch of problems). So this will be implemented as planned. * ADD_EPH_HS syntax changed to something like: ADD_EPH_HS keytype:keyblob 1*(SP Port= virtPort , addrTarget) CRLF So, basically, space separated instances of Port=virtPort,addrTarget entries, of which there must be at least one. This clearly indicates the argument type and should be future proof (and also has the side benefit of being easier for me to validate. Thoughts? -- Yawning Angel pgp7Nxg_N4dT5.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] RFC: Ephemeral Hidden Services via the Control Port
that is intuitive and easy to use? (The initial version I was planning on proposing made key generation entirely the application's problem, but nickm convinced me otherwise). Questions, comments, feedback appreciated, -- Yawning Angel pgpeQtjfKpLIu.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [PATCH] Fix comparison is always true due to limited range of data type
Hello, On Thu, 5 Feb 2015 04:02:15 -0800 (PST) Christian Kujau li...@nerdbynature.de wrote: Hi, I was unable to compile Tor for powerpc32 for a while but I was unable to bisect it because of other compilation errors (and git bisect skip did not help): [snip] The last good version to compile for me was 4ae729683 (Try to fix test_checkdir windows compilation more). Below is my attempt to fix this issue by declaring dir as signed char. It compiles now (4.6.3 and 4.9.1) for powerpc32 and Tor seems to work - but please have a look if this is the Right Thing To Do. Thanks for reporting this. The bug was fixed in master, and should be in the next 0.2.6.x-alpha release. The fix suggested was fine but was tweaked somewhat when applied See: https://trac.torproject.org/projects/tor/ticket/14764 Regards, -- Yawning Angel pgpy4jp7142Fy.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] oppy - an Onion Proxy in Python
On Wed, 21 Jan 2015 10:20:36 -0800 Damian Johnson ata...@torproject.org wrote: Now that I think about it, something that would be great to have in Stem would be path selection capabilities. So something like, say, given a list of RelayDescriptors and some constraints that must hold for a path, return some randomly chosen path that satisfies those constraints. I'd be up for that. This isn't a very commonly requested piece of functionality, but it was part of TorCtl (which Stem deprecated in 2012)... https://gitweb.torproject.org/pytorctl.git/tree/PathSupport.py There's a mostly complete implementation in torps, that uses stem. The code isn't very Python-like as afaik it's a straightforward conversion of the tor path selection code. The last time I tried this, downloading and processing all the documents took forever and consumed a hilarious amount of RAM, but I belive stem has seen performance improvements since I tried using it for this. https://github.com/torps/torps Regards, -- Yawning Angel pgpr2VyLy2bvp.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [PATCH] torify/torsocks and TCP Fast Open
On Wed, 21 Jan 2015 15:26:56 -0500 David Goulet dgou...@ev0ke.net wrote: +/* + * Using TCP Fast Open (TFO) uses sendto() instead of connect() with 'flags' + * set to MSG_FASTOPEN. Without this code, using TFO simply bypasses TOR + * without letting the user know. + * + * This solution simply ignores TFO and falls back to connect(). + * At the time the TOR server supports TFO, socks5.c (client code) could + * implement it in send_data() and connect_socks5(). Could you simply clarify this for me meaning what's needed in socks5 code for TFO to be supported? (no need for an extra comments, just how would I do that :) This doesn't really make sense when a proxy is in the picture. What would you include as the TFO payload? The VER | NMETHODS | METHODS SOCKS5 handshake? In the vast majority of the deployed configurations, the client code talks to tor over a loopback interface, so cutting out 1xRTT isn't worth the added code (There's a case to be made for using TFO for inter-relay traffic, but that's entirely orthogonal to this.). Regards, -- Yawning Angel pgpVhB7cqfu70.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Obfsproxy Address Translation
On Fri, 16 Jan 2015 12:05:41 +0100 Alfredo Palhares masterk...@masterkorp.net wrote: Between the OpenVPN I an obfsproxy server outside the country and a client inside the country. But what I've found out is that Obfsproxy server needs to be running as the OpenVPN server and Obfsproxy client needs to be on the same machine as the OpenVPN client. So what you're saying is, you want to do something like: * There is an obfsproxy client instance running on c.example.com. * There is an obfsproxy server instance running on s.example.com, that feeds into an OpenVPN server instance running on v.example.com. * Multiple clients use c.example.com as the SOCKS proxy for the OpenVPN client, connect to s.example.com to get to the OpenVPN server running on v.example.com. My thoughts on the matter are: 1. This should work. If it can be shown to be broken via a trivial application/test case (Eg: netcat), then it should be fixed (The trival test case requirement is because I don't want to debug OpenVPN again). 2. Oh god, c.example.com is going to be running a public SOCKS proxy. Granted people trying to use it to get to most destinations will have a connection that fails, but bad people can use it as a DDoS amplification host (The SOCKS dialog is much much shorter than any of the client requests that would be sent). 3. I don't know enough about the OpenVPN protocol/implementation to know if there are application specific quirks unique to OpenVPN that would prevent this configuration from working. That would be an OpenVPN problem, unless obfsproxy is altering the data it's relaying (Extremely unlikely). I'll hold off on closing the ticket for now, but unless the code is broken in the 1. sense, I'm inclined to do so. Regards, -- Yawning Angel pgp040JdSEWuS.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] high latency hidden services
On Thu, 8 Jan 2015 03:25:52 -0800 Mike Perry mikepe...@torproject.org wrote: I am unfortunately not optimistic about simple low-overhead padding being successful here. At the very least, this problem will require something more like a congestion-sensitive always pad if there is spare capacity available to grow the CWIND, but no data. We have an approximation of this defense, too, in the form of the Basket pluggable transport: https://lists.torproject.org/pipermail/tor-dev/2014-December/007977.html I believe most of BuFLO's shortcomings documented in Cai, X., Nithyanand, R., Johnson R., New Approaches to Website Fingerprinting Defenses 5.A. apply to the currently proposed defense, though some of the problems have been solved via CS-BuFLO/Basket. CS-BuFLO as implemented in Basket without application assistance (to terminate padding early) has an absolutely gargantuan amount of bandwidth overhead, and the smarter Basket variant that doesn't have stupid amounts of sender side buffering isn't portable (for the same reasons that the new KIST code isn't). If the adversary is *also* capable of Guard node coercion/compromise, then we also want something like Basket to pad all the way to the middle node, but to accomplish that, we actually need fair end-to-end flow control for clients (and by extension, a datagram/UDP transport). Indeed. This is something I'd consider fun, but is really hard to do correctly, since the CS part ideally should be global across all connections (Basket gets that for free since there's only one TCP connection per guard, and only one guard). (Though also note: We should not need such end-to-end flow control to use Adaptive Padding all the way to the middle node, because we can approximate fairness using statistics and consensus load balancing information). None of the schemes I've seen proposed so far fit well into Tor as it is now, due to the fact that multiple circuits are multiplexed over a single channel (that enforces in-order-reliable delivery semantics). HOL blocking is a thing that needs to be considered. Solving this undoubtedly has really interesting anonymity impacts that I haven't given much thought to. Another issue with all of the padding schemes that I currently don't have a solid suggestion for is how to actually detect malicious peers that don't pad/pad incorrectly. I know Marc Juarez is going to be systematically evaluating defenses as part of his research work, so perhaps he can provide more insight into algorithm selection. Regards, -- Yawning Angel pgpdDB_Sg9Y0F.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] gettimeofday() Syscall Issues
On Thu, 01 Jan 2015 23:42:42 -0500 Libertas liber...@mykolab.com wrote: The first two account for the bulk of the calls, as they are in the core data relaying logic. Ultimately, the problem seems to be that the caching is very weak. At most, only half of the calls to tor_gettimeofday_cached_monotonic() use the cache. It appears in the vomiting print statements that loading a single simple HTML page (http://www.openbsd.org/faq/ports/guide.html to be exact) will cause 30 gettimeofday() syscalls. You can imagine how that would accumulate for an exit carrying 800 KB/s if the caching doesn't improve much with additional circuits. So while optimization is cool and all, I'm not seeing why this specifically is the underlying issue. Each cell can contain 498 bytes of user payload. Looking at things simplistically this is 800 KiB/s - 1644 cells/sec, leaving you with approximately 608 microseconds of processing time per cell. On my i5-4250U box, gettimeofday() takes 22 ns on Linux, and 2441 ns on FreeBSD. I'm not sure how accurate the FreeBSD results are as it was in a VirtualBox VM (getpid() on the same VM takes 124 ns). If someone has a OpenBSD box they should benchmark gettimeofday() and see how long the call takes. Taking the FreeBSD case (since we know that tor works fine on Linux), a single gettimeofday() call takes approximately, 0.39% of the per-cell processing budget. For reference (assuming gettimeofday() in *BSD really is this shit performance wise), 7000 calls to gettimeofday() is 17.09 ms worth of calls. The clock code in tor does need love, so I wouldn't object to cleanup, but I'm not sure it's in the state where it's causing the massive performance degradation that you are seeing. Regards, -- Yawning Angel pgpN7QOVVGLMt.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] gettimeofday() Syscall Issues
On Fri, 2 Jan 2015 23:18:16 +1100 teor teor2...@gmail.com wrote: IPredator has complained that tor on Linux spends too much time calling time() when pushing 500Mbit/s, which is an issue for them under 3.x series kernels, but not kernel 2.6. https://ipredator.se/guide/torserver#performance I really don't understand this, unless my benchmark methodology is overly naive. time() in a trivial benchmark takes roughly 3 ns per call. Linux doesn't even do a real syscall for gettimeofday() due to vDSO... I just reviewed my profiling of an exit relay running chutney verify with 200MB of random data. This is on OS X 10.9.5 with tor 0.2.6.2-alpha-dev running the chutney basic-min network. The three leaf functions that take the most time in the call graph are: * channel_timestamp_recv * channel_timestamp_active * time Each of these functions takes around 16% of the execution time, the next nearest function is sha1_block_data_order_avx on 4%. While I understand that OS X, BSD, and Linux syscalls aren't necessarily identical, we now have results for the following platforms suggesting that calling time() too often has a performance impact: * Linux kernel 3.x * OpenBSD * OS X 10.9 My results suggest a maximum performance improvement of 15% on OS X if we reduced the calls to time() to a reasonable number per second. I'm still skeptical, but hey, the code needs love in general. Maybe nickm/dgoulet have more insight into this than I do, and this would also be a good opportunity to switch more things over to monotonic time. Regards, -- Yawning Angel pgpyHZsfFsxzw.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Tor BSD underperformance (was [Tor-BSD] Recognizing Randomness Exhaustion)
On Thu, 1 Jan 2015 14:19:08 +1100 teor teor2...@gmail.com wrote: On 1 Jan 2015, at 07:39 , Greg Troxel g...@lexort.com wrote: Tor 0.2.6.2-alpha (just in the process of being released) has some changes to queuing behaviour using the KIST algorithm. The KIST algorithm keeps the queues inside tor, and makes prioritisation decisions from there, rather than writing as much as possible to the OS TCP queues. I'm not sure how functional it is on *BSDs, but Nick Mathewson should be able to comment on that. (I've cc'd tor-dev and Nick.) I don't think we merged that branch yet, since it's not ready for general use. Additionally, it's not currently functional on the *BSDs. The KIST code last I checked only is used under Linux. While the full portability story is in #12890 it looks roughly like: * Linux - Supported. * Windows - Possible, needs code in tor. * Darwin - Possible, uses interfaces marked as undocumented/internal. * FreeBSD - Requires a trivial kernel patch (interface is there, information exposed is incomplete). * Other BSDs - Requires a kernel patch, which is more involved than the FreeBSD one (implementing the required interface vs exposing more information). The patch is still trivial for anyone that's familiar with the TCP/IP code. I don't think we should be in the business of maintaining kernel patches either, so I'm not sure what the right thing to do would be for non-Darwin *BSD. Regards, -- Yawning Angel pgpus7JlLTwWJ.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] N reasons why the spooks love Tribler (Number N' will surprise you)
StrongRandom actually resembles something that is Strong and Random is due to the Python developers, though in practice it is neither. b) The GMP LCG is seeded with 32/64 bits of entropy, where entropy is Mersenne Twister output. c) GMP's LCG is used for key generation. * How not to do Diffie-Hellman: key = pow(dh_received, dh_secret, DIFFIE_HELLMAN_MODULUS) This is relatively minor since recovering the secret key is trivial via PRNG attacks, so the fact that the modular exponentiation is not constant time matters less. * How not to do RSA: def rsa_encrypt(key, element): assert isinstance(element, (long, int)), type(element) _element = mpz(element) return long(pow(_element, key.e, key.n)) def rsa_decrypt(key, cipher): assert isinstance(cipher, long), type(cipher) _cipher = mpz(cipher) return long(pow(_cipher, key.d, key.n)) RSA being implemented without blinding, using GMP's modular exponentiation, and without OAEP is something I pray that I never see again. Things not covered: * How the intro point/rendezvous system works, including peer discovery and etc (Assuming it is trivial to obtain lots of ECElgamal keys/IP address:Port combinations of endpoints, it appears that it is possible to use the CREATE/CREATED response to mount an UDP amplification attack. But I did not check how easy this would be to mount in practice.) * The bittorrent bits. * The dispersy component beyond ok, at least the ECElgamal key generation uses M2Crypto, assuming the horribly broken keygen code (community.privatesemantic.crypto.ecelgamal.ecelgamal_init()) is dead like I think it is. Recommendations: * For users, don't. Cursory analysis found enough fundamental flaws, and secure protocol design/implementation errors that I would be reluctant to consider this secure, even if the known issues were fixed. It may be worth revisiting in several years when the designers obtain more experience, and a thorough third party audit of the improved code and design has been done. * For the developers: * Use a CSPRNG when doing key generation. Neither GMP's LCG nor Mersenne Twister are CSPRNGs, even if they are seeded from StrongRandom that actually is a StrongRandom (rather than a horrible misnomer). * Add a construct similar to RELAY_EARLY. * Add authenticated encryption to cell payload. This will break wire compatibility. * Use constant time modular exponentiation. As a matter of fact, don't include your own implementations of ECElGamal, RSA, and Diffie-Hellman. Use well tested/known secure library code instead. * Blind RSA operations, use padding as appropriate. * Fix the symmetric encryption somehow, after obtaining a in-depth understanding of what things break security of the mode you wish to use in various ways (Eg: IV/Nonce reuse). See: http://web.cs.ucdavis.edu/~rogaway/papers/modes.pdf * Instead of a DH + ElGamal handshake, consider Ntor. Even with GMPY, doing modular exponentiation is relatively CPU intensive, and there is the threat of CPU exhaustion attacks here. * Clean out all the dead code, after printing out copies to use to scare small children. Eg: Tribler/community/privatesemantic/crypto/elgamal.py Most of the fixes require major revisions to the wire protocol. As it appears that there is no versioning, how that will be done is left as an exercise for the student. Alternatively, rebase the system on I2P. Regards, -- Yawning Angel pgpGSqhEPfHVU.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] basket: More eggs in the Guard basket.
On Wed, 17 Dec 2014 13:51:02 -0500 Nick Mathewson ni...@alum.mit.edu wrote: [snipity] Should the handshake also a signature by Bob of (X|N), and should maybe the shared secret also include a digest of all the other parts of the communication? Hmm, maybe I shouldn't have left bits out, and I really do need to document the handshake component of the protocol. The former is actually done, a BLAKE-256 digest of the entire client request is included in Bob's response and is covered by the signature. The client verifies that Bob received a unmodified request, after checking the signature. There's no reason why the signature can't just include the entire request here if that's better. Including a digest of everything sent as part of the shared secret seems like a good idea, so I shall revise the protocol to do that (digesting 50 KiB of data isn't that big of a deal given how heavyweight the other crypto bits are). Regards, -- Yawning Angel pgpI20qKE8Gnl.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] basket: More eggs in the Guard basket.
Hi all, For several reasons I've been working on a bit of code that I named basket. It's almost at the point where the brave members of the general public should be aware that it exists as a potential option in the privacy toolbox, though using it in any capacity beyond testing on a loopback device IS CURRENTLY ACTIVELY DISCOURAGED unless users are comfortable debugging it (This means, DO NOT USE IT. I will likely break backward compatibility in the future, and you will be sad.). basket is my stab at designing something that significantly increases Tor's resistance to upcoming/future attacks, by providing a link layer cryptographic handshake that uses post-quantum cryptographic primitives and defenses against website fingerprinting (and possibly e2e correlation) attacks. For the ease of development it is in the form of a pluggable transport with the expected tradeoffs (you must absolutely trust your Bridge, since both features only run to the Bridge). It is worth noting that it is anything but subtle, and it is blatantly obvious that a given connection is speaking basket as no attempt was made to obfuscate the handshake. The link layer handshake works roughly like thus: Setup: * Bob generates a long term SPHINCS256 keypair s,S used to sign responses. The handshake: 1. Alice generates a Curve25519 keypair x,X and a NTRUEncrypt EES1171EP1 keypair n,N. 2. Alice sends X | N to Bob. 3. Bob generates a Curve25519 keypair y,Y, and calculates Curve25519(y,X) as the shared secret. 4. Bob sends NTRUEncrypt(N,Y) | S | SPHINCS256(s, ntru_ciphertext | S) to Alice. 5. Alice verifies the SPHINCS256 signature (Alice's copy of S is saved/trusted in a Trust-On-First-Use manner), and decrypts the NTRU ciphertext to obtain Y. 6. Alice calculates Curve25519(x,Y) as the shared secret. NB: Some details omitted for brevity. Passive attackers see Alice's Curve25519 public key, Alice's NTRUEncrypt public key, Bob's SPHINCS256 public key, a NTRUEncrypt ciphertext, and SPHINCS256 signature. Obtaining the link's ephemeral session key requires decrypting the NTRUEncrypt ciphertext (to obtain Bob's Curve25519 public key), and reversing the scalar base multiply on one of X and Y. Active man-in-the-middle attackers need to be able to forge SPHINCS256 signatures to substitute their own NTRUEncrypt-ed payload to mount an attack (Alice's request is also validated/protected but it's one of the details omitted, read the code). As NTRUEncrypt is patent encumbered, there also is a handshake mode that removes the experimental post-quantum cryptography and uses Ed25519/Curve25519. The fingerprinting defenses are in the form of the CS-BuFLO algorithm in CTST (Client total, Server total) mode, without the application level hinting based optimizations to reduce bandwidth consumption (See: http://www3.cs.stonybrook.edu/~rob/papers/csbuflo.pdf). As a minor implementation refinement, basket uses the kernel information as described in the KIST paper to be more responsive to changing network conditions. Most of the CS-BuFLO parameters were adjusted in an attempt to compensate for the lack of said application level hinting, but basket's CS-BuFLO implementation is still a lot more expensive than the one presented in the paper. The code: http://github.com/yawning/basket Notes: * The PQ crypto primitives used are ports to Go by yours truly based on the reference implementations. It is both possible and likely that I messed something up, so users should decide between trusting my implementations and using a handshake based on classical algorithms. * The basket handshake consumes a non-trivial amount of CPU on the server side due to the SPHINCS256 implementation being relatively unoptimized. * CS-BuFLO is EXTREMELY BANDWIDTH INTENSIVE, and if adversaries can observe the Bridge's upstream, it likely can be broken, especially if the Bridge is only servicing a handful of users. Running the defense end-to-end (or even to the middle relay) is *extremely* non-trivial, though that would be required if something more comprehensive is desired. * The KIST code is platform specific, and I only wrote the Linux version. If this ends up being useful to people, support for other platforms is possible. * basket was never written to be a general use transport, though it sits somewhere between obfsproxy-wfpadtools (a pure research framework) and obfs4proxy (something that is intended to be used in production) in terms of completeness. Thanks to Marc Juarez (KU Leuven) and Mike Perry for inspiration to write the CS-BuFLO component of basket. Questions, comments, feedback appreciated as always, -- Yawning Angel ps: Seriously, unless you are a developer or researcher, you REALLY SHOULD NOT use basket. pgpTBcBge4obR.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org
[tor-dev] Pluggable transports meeting tomorrow (16:00UTC Wednesday 17th of December 2014)
Hello! Just wanted to remind you that the regular biweekly pluggable transports meeting is going to occur tomorrow at 16:00 UTC. Place is the #tor-dev IRC channel in the OFTC network. Thanks for your attention! -- Yawning Angel pgpmwTq6Jwj11.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Pluggable transports meeting tomorrow (16:00UTC Wednesday 3rd of December 2014)
Hello! just wanted to remind you that the regular biweekly pluggable transports meeting is going to occur tomorrow at 16:00 UTC. Place is the #tor-dev IRC channel in the OFTC network. Thanks for your attention! -- Yawning Angel pgpgttQ9aqVoM.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Git hosting changes, git:// support discontinued
On Sun, 30 Nov 2014 17:32:05 -0500 Jason Cooper t...@lakedaemon.net wrote: It is unauthenticated and you probably shouldn't use it if at all possible. How does that matter? All of the tags are signed by Nick Mathewson. This allows the server *and* the path to be untrusted. What about intermediary commits between tagged releases? Yes, signing each commit is possible, and probably even a good idea, but it's not currently done. Regards, -- Yawning Angel pgpBtpnfUsqCQ.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Git hosting changes, git:// support discontinued
On Sun, 30 Nov 2014 19:19:58 -0500
Jason Cooper t...@lakedaemon.net wrote:
On Sun, Nov 30, 2014 at 11:55:31PM +, Yawning Angel wrote:
On Sun, 30 Nov 2014 17:32:05 -0500
Jason Cooper t...@lakedaemon.net wrote:
It is unauthenticated and you probably shouldn't use it if at
all possible.
How does that matter? All of the tags are signed by Nick
Mathewson. This allows the server *and* the path to be untrusted.
What about intermediary commits between tagged releases? Yes,
signing each commit is possible, and probably even a good idea, but
it's not currently done.
git uses chained hashes so that verifying the integrity of the tagged
commit also verifies the integrity of the previous commits between the
prior tag and the current one (Actually, across the entire history,
but once I've cloned and validated, I'm primarily concerned with
commits from subsequent pulls).
So, I didn't communicate that well, so I'll try again:
Assuming people use the unauthenticated git protocol, and want to
clone a copy of master, maint-0.2.4 or maint-0.2.5, how do they ensure
that the copy they received is correct?
So intermediary commits as in stuff that happens between releases,
with the next release having not happened yet ('interim' would have
been a better word to use in hindsight). Sure you can validate up to the
last tag, but for all the commits that follow, there's no magic PGP
signed tag that covers those.
I don't see any reason to allow a unauthenticated protocol when
authenticated alternatives exist and are well supported in the first
place, but that's just me.
Regards,
--
Yawning Angel
pgpsUCIBbLyRC.pgp
Description: OpenPGP digital signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] obfs4 questions
On Fri, 28 Nov 2014 13:08:04 + Michael Rogers mich...@briarproject.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, In the obfs4 spec I couldn't find a description of how the secretbox nonces for the frames are constructed. A 16-byte nonce prefix comes from the KDF, but what about the remaining 8 (presumably frame-specific) bytes? It's a simple 64 bit frame counter (Big endian, starts at 1). Guess I never bothered to copy the relevant information from the gigantic comment in framing.go into the spec, oops. The code cuts the connection if the counter would wrap though that's unlikely at any sort of realistic data rate given the use case. If an attacker changes the order of the secretboxes so that the recipient tries to open a secretbox with the wrong nonce, is that guaranteed to fail, as it would if the secretbox had been modified? I can make a hand-wavy argument for why I think it will fail, but I don't know whether the secretbox construct is designed to ensure this. Yes, the poly1305 verification will fail. Any particular reason for using two different MACs (HMAC-SHA256-128 for the handshake, Poly1305 for the frames) and two different hashes (SHA-256 for the handshake, SipHash-2-4 for obfuscation)? No particular reason beyond historical. Ntor is specified to use HMAC-SHA256, ScrambleSuit used HMAC-SHA256-128. I briefly toyed with sending 2 secretboxes one with the length, one with the payload, but that was kind of heavyweight, so I went with something lighter (Thus SipHash). I may go back to the two box design if I do an obfs5, not sure about that yet. Regards, -- Yawning Angel pgpgvpPIf0y5d.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] obfs4 questions
On Fri, 28 Nov 2014 14:47:29 + Michael Rogers mich...@briarproject.org wrote: I believe so too, but is it stated anywhere that this is a guaranteed property of crypto_secretbox? The Poly1305 authenticator is calculated based on the payload and the nonce. In the case of the NaCL secretbox construct, 32 bytes of zeroes encrypted based on a one time key/counter derived from the actual key and the nonce. If the frames are reordered, the derived authenticator would be different. So yes, it is a property of crypto_secretbox because that's how Poly1305 works. It wouldn't be a workable AEAD mode if nonces (which usually are transmitted in the clear) could be modified undetected by attackers either. For more details see the original poly1305 paper[0] (nb: specified in terms of using AES for the nonce-16 byte string mapping, but that is arbitrary). Any particular reason for using two different MACs (HMAC-SHA256-128 for the handshake, Poly1305 for the frames) and two different hashes (SHA-256 for the handshake, SipHash-2-4 for obfuscation)? No particular reason beyond historical. Ntor is specified to use HMAC-SHA256, ScrambleSuit used HMAC-SHA256-128. I briefly toyed with sending 2 secretboxes one with the length, one with the payload, but that was kind of heavyweight, so I went with something lighter (Thus SipHash). I may go back to the two box design if I do an obfs5, not sure about that yet. Interesting, thanks. Would SHA-256 be too slow for obfuscation? It just seems like a lot of dependencies for a simple protocol. Probably not, but at that point, 2 boxes is also likely fine since it wasn't unusably slow. For what it's worth, we're using the two-box approach for the next version of the Briar transport protocol. I'm concerned about the possibility of an attack conceptually similar to [1] against an unauthenticated length field, even though that specific attack wouldn't apply. This was brought up by phw during the obfs4 design phase. The code randomizes the length if it is invalid as per one of the suggested countermeasures in section 6 of the paper (So an identical failure to a modified plaintext/tag is observed). Regards, -- Yawning Angel [0]: http://cr.yp.to/mac/poly1305-20050329.pdf pgpD4SkRCdRkT.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] obfs4 questions
On Fri, 28 Nov 2014 17:57:26 + Michael Rogers mich...@briarproject.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 28/11/14 15:50, Yawning Angel wrote: A one time poly1305 key is calculated for each box, based on 32 bytes of zeroes encrypted with a one time Salsa20 key/counter derived from the nonce and the box key. You can view the use of Salsa20 there as an arbitrary keyed hash function (in the case of the original paper, AES was used). Hope that clarifies things somewhat, Thanks - this is similar to the argument I came up with. I called my argument hand-wavy because it relies on HSalsa20 and Salsa20 being PRFs, and I don't know how big an assumption that is. For what it's worth 7 Nonce and stream both support using a counter here as the nonce, and refers to 'The standard (PRF) security conjecture for Salsa20. IIRC the security proof for the extended nonce variants also hinges on the underlying algorithms being PRFs as well, so it's something I'm comfortable in assuming. http://cr.yp.to/highspeed/naclcrypto-20090310.pdf Regards, -- Yawning Angel pgpVJBifs9PS3.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Pluggable transports meeting tomorrow (16:00UTC Wednesday 19th of November 2014)
Hello! just wanted to remind you that the regular biweekly pluggable transports meeting is going to occur tomorrow at 16:00 UTC. Place is the #tor-dev IRC channel in the OFTC network. Thanks for your attention! -- Yawning Angel pgp17oIBtS3qf.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Hidden Service authorization UI
On Sun, 9 Nov 2014 16:19:24 + Andrea Shepard and...@torproject.org wrote: How would Tor Browser learn about this reason for not being able to connect/ tell Tor the authentication info? This is starting to sound like wanting SOCKS5 extensions to indicate different causes for connection failures in #6031 did. Well prop 229 is on my todo list, though it's not very high up. The last time I spoke to people about this, it seemed like a nice to have but not massively important sort of thing, but I'd be more than happy to rearrange things in that department. Especially as my tenative plans for obfsng (aka obfs6 depending on how long it gets stuck in design and deployment) involves 1 KiB keys... Regards, -- Yawning Angel pgp1Omyydtsp8.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Vidalia Relay Bundle(win) Tor version, obfs4proxy package in deb.tp.o
On Sun, 26 Oct 2014 17:16:46 +0200 s7r s...@sky-ip.org wrote: 3. Last thing, the page https://bridges.torproject.org/ does not contain obfs4 in the drop-down list where the user needs to select the pluggable transport. It only allows requests for obfs2, obfs3, scramblesuit and fteproxy bridges. Don't know about fteproxy, but shouldn't obfs2 be deprecated or is it still working/used? A note on this (I don't handle packaging, sorry). obfs4 propbably should be added to the dropdown once there is a critical mass of bridges in the database, and when obfs4 is in official Tor Browser builds. As far as I am aware, the official integration (as opposed to my obsolete not-really-working-anymore snapshots) is scheduled to happen in the next alpha series. Regarding obfs2, it still works in some environments if the DPI used by the censor is not particularly sophisticated (not targeted specifically at detecting obfs2). I personally think that it should be deprecated sooner rather than later, but others have disagreed with me on this. Hope that helps, -- Yawning Angel pgpRkxhRUfSjM.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Understanding Tor and SOCKS
On Sun, 26 Oct 2014 14:34:59 +0100 Rob van der Hoeven robvanderhoe...@ziggo.nl wrote: So, the SOCKS protocol supports redirection to another SOCKS server. An all-zero address/port simply means: use the server/port that you are currently connected to. That's a really interesting way of interpreting that part of the RFC. The reason why BND.ADDR and BND.PORT are supplied in a SOCKS5 response is to provide the client with the information equivalent to calling getsockname() on a non-proxied socket. In the context of tor, the reason why BND.ADDR and BND.PORT are all NUL bytes is because the RELAY_CONNECTED cell does not propagate BND.PORT backwards to the client from the exit. BND.ADDR could technically be filled in (since the tor client knows where it is exiting from), but I don't see much point (and this information is useless at best in the context of HSes). Regards, -- Yawning Angel pgpGA5tzki8YH.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] obfs4 test bundles
Hello everyone, I just uploaded a new set of obfs4 test bundles to: https://people.torproject.org/~yawning/volatile/tor-browser-obfs4-20140926/ Changes since the last version: * obfs4proxy * Updated to 0.0.2 (Most of the changes are related to packaging and a minor bridge side quality of life feature, no real functional changes since the last version from a client perspective.) * Bundling * Rebased my integration branch on Tor Browser 4.0-alpha-3. * I backed out the goptlib SOCKS5 code since I changed the patch a bunch. IPv6 obfs2/3/4 bridges will probably not work because of this, but that is a know issue, and the patch is sitting in a branch awaiting review and a better opportunity for systematic testing. To be perfectly honest not much has changed to the point where it really warrants another release, but apparently people are still using my old snapshots, so here's a new set for those people that folds in all the changes that went into the Tor Browser side of things. As far as the roadmap for full fledged deployment goes, it's at the get enough bridges point, with the step after that being getting the Tor Browser folks to merge my branch. I have verified that the linux64 and windows (thanks to a friend) bundles appear to be functional. If you wish to follow obfs4 deployment the bug associated with this task is #12130. Questions, comments, feedback welcome. -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] obfs4 test bundles
Hello everyone, I just uploaded a new set of obfs4 test bundles to: https://people.torproject.org/~yawning/volatile/tor-browser-obfs4-20140829/ I forgot to change the file names (oops), but that shouldn't affect the bundles and regenerating them just to rename/reversion would take quite a while, so I'll pass on doing so. Changes since the last version: * obfs4proxy * obfs4proxy -v will print versioning information. * Bridge side logging will sanitize golang networking errors unless the log scrubber is disabled. * Bridge lines now use Base16 instead of Base64 to work around bugs in the pt bridge args processing. * A dumb bug that was causing TYPE_PRNG_SEED frames to be silently ignored was fixed. * Bridge lines now have iat-mode which controls Inter-Arrival Time obfuscation. This is done so the bridge administrator can specify client behavior (Default is to disable IAT obfuscation). * There is now an optional paranoid IAT obfuscation mode, which disables optimizations made around bulk data transfer for additional obfuscation. This has a major impact on bulk data throughput and is not recommended. * Bundling * The #12535 patch is integrated into the goptlib build process, and go based pluggable transports will use SOCKS5. Eventually the patch will be merged into goptlib proper. * The default test bridge was updated to the new bridge line format, and includes the iat-mode argument. * The obfs4proxy and Go licenses are now included in the bundle. I have verified that the linux64 and windows (thanks to a friend) bundles appear to be functional. If you wish to follow obfs4 deployment the bug associated with this task is #12130. Questions, comments, feedback welcome. -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] obfs4 test bundles
Hello all, So after a brief pause in my productivity due to the dev meeting and some IRL things, I'm back in the swing of development and have been working on getting obfs4 ready for development. I've recently mostly finished a fairly large cleanup/refactor of the obfs4 codebase and dusted off the Tor Browser build integration patch. Test bundles signed with my PGP key are available at: https://people.torproject.org/~yawning/volatile/tor-browser-obfs4-20140820/ Some quick notes: * The bundles are alpha. DO NOT USE THEM IF YOU REQUIRE STRONG ANONYMITY OR SECURITY. * While it is possible to pull out the obfs4proxy binary from the bundle and use it to run a bridge, that will be a *terrible* idea because the bundles are missing some logging related fixes. * The Windows and OSX bundles are entirely untested, because I do not have access to either platform. The Linux bundle appears to work. * Using obfs2/obfs3 will also exercise new code (obfs4proxy instead of obfsproxy). * I'm not uploading a 32 bit linux bundle for now. * The one obfs4 test bridge is mine, and is the one I posted to tor-dev@ (port is different, bridge parameters are the same). If you wish to follow obfs4 deployment the bug associated with this task is #12130. Questions, comments, feedback welcome. -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] PKCS#1 ASN.1 Public Key Encoding
On Sun, 17 Aug 2014 16:19:56 +0100
Gareth Owen gareth.o...@port.ac.uk wrote:
I'm trying to generate the fingerprint given just the pubilc key in
Java and after almost a whole day I'm about to give up. Does anyone
have a sample PKCS#1 encoded public key that is used immediately
before SHA-1 to generate the fingerprint? e.g. a hex string is what
I'm after.
Both descriptors and microdescriptors contain this in the appropriate
format (albeit Base64 encoded and with a PEM envelope). Check the data
directory of a running tor instance and look at
cached-microdescs(.new), which will have onion-key entries for all the
relays.
It seems there are subtle ways that an PKCS#1 can vary while encoding
the same information which affects the hash, Java seems to be doing
it one way, OpenSSL another, an example on stack overflow adds an
extra field, etc.
The way that you care about (that matches how tor does it) is specified
in RFC 2313.
7.1 Public-key syntax
An RSA public key shall have ASN.1 type RSAPublicKey:
RSAPublicKey ::= SEQUENCE {
modulus INTEGER, -- n
publicExponent INTEGER -- e }
(This type is specified in X.509 and is retained here for
compatibility.)
How to do this in Java depends on which crypto API you are using, look
at oracle.security.crypto.asn1 or org.bouncycastle.asn1. Additionally
this (http://lapo.it/asn1js/) will probably be useful.
Regards,
--
Yawning Angel
signature.asc
Description: PGP signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] (meek|flashproxy)+(obfs3|fte|scramblesuit|...)
On Sat, 26 Jul 2014 15:08:38 +0100 Kevin P Dyer kpd...@gmail.com wrote: Are there any roadblocks that prevent us from doing the following? 1. Remove the hard-coded bridge_prefs.js in the TBB. Ok. 2. Set meek as the default pluggable transport in the TBB. Sure that's also fairly easy. 3. Use meek to acquire an up-to-date bridge_prefs.js from, say, torproject.org. Whowa, what? I get (from other parts of the thread) that there are compelling reasons for this, but I can think of at least two reasons why I would be massively against this. a) Who is going to pay for this? The amount of data transferred will grow to be non-trivial rather quickly because each user that ends up doing this will do the full bootstrap. Granted, this will be a one time thing per bundle release (and a one time thing over the lifespan of the client in some magical world where TBB has an update mechanism), so the economic side in the future isn't quite as dire, but still. b) Giving anyone a list of a subset of our users (and a particularly vulnerable subset at that, since they need to use PTs), seems dangerous at best. Going from all meek users need to trust $CDN to the default behavior is to give $CDN a list of anyone trying to use PTs at first glance seems like something that will only end badly. 4. Use the information from the acquired bridge_prefs.js to connect to Tor as normal. No clue as to how hard this is. Ostensibly, this doesn't do a better job of hiding bridge addresses. However, it allows us to modify bridge addresses without releasing a new TBB. I don't see that as being a sufficiently compelling reason to give a third party the ability to enumerate (a unknown fraction of) the PT user base (while making them rich at the same time). Regards, -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Email Bridge Distributor Interactive Commands
On Fri, 25 Jul 2014 10:00:01 +0200 Lunar lu...@torproject.org wrote: isis: We can't just make Tor Browser stop accepting obfs2 because some people are using obfs2 bridges right now. But we shouldn't add more people to the set of users of a broken protocol. Obfs3 is also broken, it's just that we haven't yet seen a DPI box do it IRL. That's news to me. Any pointers? Well, the protocol is ok, but it is vulnerable to active probing (eg: See something they don't recognize, flag the destination IP/Port, call back later). Doing so on a mass scale is *quite* expensive since the obfs3 handshake isn't exactly cheap, but probably is in the reach of a nation-state adversary (China springs to mind). There also are a few interesting statistical attacks that are possible vs the obfs3 protocol if you make guesses about the inner payload, but such things are unnecessary for obfs3 (and ScrambleSuit/obfs4 both have some defenses against those, although not all are enabled as a performance tradeoff). Regards, -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Email Bridge Distributor Interactive Commands
On Fri, 25 Jul 2014 13:25:31 +0200 Lunar lu...@torproject.org wrote: isis: We can't just make Tor Browser stop accepting obfs2 because some people are using obfs2 bridges right now. But we shouldn't add more people to the set of users of a broken protocol. Obfs3 is also broken, it's just that we haven't yet seen a DPI box do it IRL. If you want me to only hand out the holy grail, I'm never going to hand anything out. The holy grail will never exist, indeed. I fail too see why this would be a reason to continue giving out solutions that are known to be bad when they have suitable replacement. For what it's worth, the official plan is to kill off obfs2 once we figure out how we want to handle deprecating old transports. https://trac.torproject.org/projects/tor/ticket/10314 Personally I think when we deploy the next round of transports (meek, and either ScrambleSuit or obfs4) would be the right time to revisit this, and I can't think of a good reason to keep obfs2 around beyond there are bridges that only support obfs2 which is a fairly terrible reason keep distributing the protocol to new users. My other objection to the idea a while back was that Orbot only supported obfs2, but that's been fixed for a while now. Regards, -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Email Bridge Distributor Interactive Commands
On Fri, 25 Jul 2014 22:19:40 + isis i...@torproject.org wrote: Personally I think when we deploy the next round of transports (meek, and either ScrambleSuit or obfs4) would be the right time to revisit this, and I can't think of a good reason to keep obfs2 around beyond there are bridges that only support obfs2 which is a fairly terrible reason keep distributing the protocol to new users. Scramblesuit is deployed, if you ask me... We've got roughly 2221 scramblesuit supporting bridges. Kind of. TBB/Orbot and the FirefoxOS code all need to move to 0.2.5.x for those bridges to actually be useful which I belive is Real Soon Now. Just having bridges that only people that build stuff on their own can connect to is a bit silly. My other objection to the idea a while back was that Orbot only supported obfs2, but that's been fixed for a while now. So... I'm going to wait for an update from the Huggable Transport folks, telling me to phase out obfsXYZ, whenever that happens. Until then, obfs3 is still the default transport distributed. Does this sound okay to everyone? Otherwise you're shoving me back into the hell where I get yelled at if I don't make a unilateral decision, and also get yelled at if I do make a decision. It's kind of annoying to get yelled at all the time. :( That's fine by me. I belive obfs3 should be ok for a while, and there are easier ways to identify bridges via active probing than doing on obfs3 handshake anyway (Fixing such things is also on my TODO list). Regards, -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] obfs4 and ntor (question wrt node_id)
On Mon, 02 Jun 2014 16:12:03 +0100
George Kadianakis desnac...@riseup.net wrote:
Yep, that's what I gathered too.
Unfortunately, the server-side obfs4 might not have access to its
address/port (it normally knows that it has to bind to 0.0.0.0:port,
not the actual external IP address).
So we were considering whether generating a random nodeid would be OK
for security.
Or even omitting the nodeid completely, and just using the public key
B in its place (since \hat{B} is just used as an one-to-one map to a
B) Or does this complicate the security proof?
Unless I'm horrifically mistaken, a random nodeid is fine as it is just
as arbitrary as the current node ID. Since there isn't any tight
coupling between pluggable transports and the remote bridges they
connect to, the bridge fingerprint currently in use is also a random
nodeid, at least as far as obfs4 is concerned (The fact that it
coincidentally happens to be the bridge fingerprint has no effect on
the obfs4 protocol itself).
Regards,
--
Yawning Angel
signature.asc
Description: PGP signature
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] wfpadtools: comments about primitives
On Fri, 30 May 2014 17:42:39 +0100 George Kadianakis desnac...@riseup.net wrote: Marc Juarez marc.juarezm...@esat.kuleuven.be writes: Hi all, I am a GSoC student working in a new PT for the development of future Website Fingerprinting countermeasures in Tor. The PT is not targeting any specific defense, but to link padding defenses in general. The idea is to implement a set of primitives that any link padding-based defense would benefit of. In this email I provide a more detailed description of these primitives and give a short update about their state. I forked the obfsproxy repo and made it publicly available in bitbucket: https://bitbucket.org/mjuarezm/obfsproxy-wfpadtools. Excellent. I would appreciate comments from the Tor development community. I'm specially looking for advices that help me generalizing the padding module (padutils) and comments about the primitives I describe below. The envisaged primitives are: 1. A general padding class that provides methods to pad the link This part is almost finished. For that I reused the Scramblesuit's probdist and fifobuffer modules to buffer and flush data according to a given probability distribution. Using this class I have implemented a simple version of BuFLO (which is also included in the padutils module). Sounds reasonable. (BTW, we recently found that scramblesuit's genDistribution() function in probdist.py does not generate a uniform distribution. You can see this, since the way the prob distr is generated, its first element has a mean value of 1/2 instead of 1/n. Yawning fixed this in obfs4 I think, but it remains unfixed in scramblesuit.) https://github.com/Yawning/obfs4/commit/9fe9959c76c96ec3284f43c692cbb099230dcb73 That's an example of how to make it uniform. I'm not sure which behavior is better, real world data on what the packet distribution of non-Tor network protocols look like on the wire would be useful here. [snip] 4. Implement the protocol's handshake. I took a look to the Scramblesuit's handshake.The handshake of this protocol would boil down to the negotiation of the parameters (e.g., probability distributions) for the padding. In the end, I think this handshake will need to be confidential (encrypted) somehow. Otherwise, the adversary could read the probability distributions and unwrap your padding layer more easily. Or is this not in your threat model? Likewise, however (correct me if I'm wrong), this is an orthogonal problem. The vision I have currently is, when when we do obfs6 or whatever, that we would come up with our own unique and clever way to handle authentication and confidentiality and use wfpadtools internally. 5. Implement a flow control protocol This would adjust the padding parameters while running. The fifobuffer we are currently using helps queuing the messages but we will need an extra logic to detect congestion. It's a shame that SIOCOUTQ isn't portable, because checking the send socket buffer's size is one of the better ways to do this. Regards, -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] wfpadtools: comments about primitives
On Fri, 30 May 2014 20:40:07 +0200 Marc Juarez marc.juarezm...@esat.kuleuven.be wrote: 4. Implement the protocol's handshake. I took a look to the Scramblesuit's handshake.The handshake of this protocol would boil down to the negotiation of the parameters (e.g., probability distributions) for the padding. In the end, I think this handshake will need to be confidential (encrypted) somehow. Otherwise, the adversary could read the probability distributions and unwrap your padding layer more easily. Or is this not in your threat model? Yes, you're right. However, I think we shouldn't overlap too much with Scramblesuit. I was thinking that Scramblesuit or obfs3 could be used together with this PT. Actually this is a question I wanted to ask you: can multiple PTs be used together? Except for the overhead of the protocol headers I don't see any technical limitation. Not as well as we would like. Further improving our designs for combining PTs and adding an implementation is one of our GSOC projects this year though. My 0.02 dogecoin here would be to say the PT combiner will fix it, so eventually we could do wfpadtools + obfs3/obfs4/scramblesuit/meek/fte and get the added defenses without (many? any?) code changes. Regards, -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] RFC: obfs4 (Name not final)
Hello, The people that have been following Pluggable Transport development may know that I have been working on something tentatively called obfs4 recently. It's rapidly approaching the point where I would like to open it up for review and feedback, hence the e-mail. A quick and dirty description would be: obfs4 is ScrambleSuit with djb crypto. Instead of obfs3 style UniformDH and CTR-AES256/HMAC-SHA256, obfs4 uses a combination of Curve25519, Elligator2, HMAC-SHA256, XSalsa20/Poly1305 and SipHash-2-4. The feature set offered by obfs4 is comparable to ScrambleSuit with the following differences (implementation specific changes are noted as such): * The key exchange is based on the ntor handshake, and thus authenticates the server to the client. Both obfs3 and ScrambleSuit depend on the encapsulated protocol to handle this on it's own. * obfs4 always does a full handshake. ScrambleSuit style session ticket handshakes are not supported. Even with Elligator2 mapping taken into account, the obfs4 handshake is significantly faster, so there is less of a need for this. * (Impl) obfs4proxy currently does not offer Inter-Arrival Time obfuscation, but this could easily be added if it is something that a lot of people want. It is worth noting that while obfsproxy and obfsclient both support IAT obfuscation, support for it is currently disabled by default as a performance tradeoff. * (Impl) obfs4proxy is currently written in golang. Neither a clear plus or minus, it does allow people to run bridges on reserved ports significantly easier, is probably faster (native code, can use multiple cores for most things), but is more annoying to build and results in rather large binaries (the go runtime is statically linked). * (Impl) obfs4 supports a lot of the protocol improvements made to ScrambleSuit that are pending merge (See #11271, #11203). This difference is temporary. The code and a draft spec is at: https://github.com/Yawning/obfs4 Open questions: * Is the base design and my implementation sane? * Should obfs4proxy also have (disabled) IAT obfuscation? Adding it later will not require wire protocol changes. * The handshake length mimics ScrambleSuit in terms of maximum padding ( 1500 bytes). Should this be increased to be similar to obfs3 (~8k of maximum padding)? The server side cost for this shouldn't be that high. * Is this different enough from ScrambleSuit to be worth deploying? I would be ok with this ending up as just a research project and shelving it if the consensus is otherwise. * Should I have named it ScrambleSuit 2? It's a direct descendant of ScrambleSuit and is an obfs derivative only in name at this point. Future plans: * Implement suggestions/improve the code/fix bugs/write more unit tests. * Improve goptlib. * If this is deployed in meaningful amounts, support it in obfsclient. For the extremely brave, I am running a test bridge with the most recent snapshot of the code: UseBridges 1 ClientTransportPlugin obfs4 exec /path/to/obfs4proxy # Bridge line of doom, all one line. Bridge obfs4 178.209.52.110:52810 67E72FF33D7D41BF11C569646A0A7B4B188340DF node-id=Z+cv8z19Qb8RxWlkagp7SxiDQN8= public-key=vm+w9k57aMIX/o+A9ef5C7o9xKEkhr7kRBj3N4etDn8= As with any PT that requires per-bridge arguments, this also requires tor-0.2.5.x. The node-id in this case happens to be my bridge's fingerprint, the public key is the long term key used to validate my bridge's identity and generate M_[C,S]/MAC in the handshake (The bridge line is a bit of a UI/UX disaster unless people are copy/pasting it). Getting either of them wrong will result in handshake failures and tor not bootstrapping. A few warnings: * The spec assumes familiarity with ntor, ScrambleSuit and NaCl's crypto_secretbox. * The wire protocol is not final and I *will* push builds to the bridge as I break backward compatibility without notifying people (It's been 0 days since a wire protocol change.). The test bridge will randomly be broken/rebooted/etc as I also use it for other things. * Development was done with go1.2.x, older versions of the runtime are not supported. * It would be a terrible idea to use obfs4proxy as anything other than a client at this point. Questions, comments, feedback all appreciated. -- Yawning Angel PS: I also wrote https://github.com/yawning/or-applet recently. It's even less supported/tested than obfs4 is (it is written only for my laptop), but some people may find it cute. signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] RFC: obfs4 (Name not final)
On Wed, 21 May 2014 12:22:46 + David Stainton dstainton...@gmail.com wrote: obfs4 is ScrambleSuit with djb crypto. Instead of obfs3 style UniformDH and CTR-AES256/HMAC-SHA256, obfs4 uses a combination of Curve25519, Elligator2, HMAC-SHA256, XSalsa20/Poly1305 and SipHash-2-4. Elligator... cool! * Development was done with go1.2.x, older versions of the runtime are not supported. Did you get your builds to work with the deterministic build system? Not yet. While it's important I'm sort of working under the assumption that it's mostly a solved problem as the scary build process can handle flashproxy and meek. I've been a bit more focused on getting the protocol design and implementation to a point where I feel generally good about it. -- Yawning Angel signature.asc Description: PGP signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
