[tor-dev] Is it time to drop support for the v1/v2 protos?
Hi all, After reading the Tor spec [1] I did some digging and realized that the old handshakes and link protocols (v1 (certs up-front) and v2 (renegotiation)) are not used anymore as of 0.2.3.6-alpha which introduced link proto v3. Supporting v1 and v2 requires (among other things) supporting SSLv3 which (imho) should be deprecated everywhere. This makes me wonder why Tor still supports these: is it for compatibility with even older versions (consensus health says no) or are there other reasons? If someone were to invest a couple of hours and remove all support for them from the Tor code and the Tor spec, would this hurt the network or would it be a welcome patch? Tom [1] https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt smime.p7s Description: S/MIME-cryptografische ondertekening ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is it time to drop support for the v1/v2 protos?
Nick Mathewson schreef op 29/12/14 om 00:50: On Sat, Dec 27, 2014 at 9:38 AM, Tom van der Woerdt wrote: Hi all, After reading the Tor spec [1] I did some digging and realized that the old handshakes and link protocols (v1 (certs up-front) and v2 (renegotiation)) are not used anymore as of 0.2.3.6-alpha which introduced link proto v3. Supporting v1 and v2 requires (among other things) supporting SSLv3 which (imho) should be deprecated everywhere. This makes me wonder why Tor still supports these: is it for compatibility with even older versions (consensus health says no) or are there other reasons? If someone were to invest a couple of hours and remove all support for them from the Tor code and the Tor spec, would this hurt the network or would it be a welcome patch? There are already a couple of tickets for removing these, and I would like to see them go. The master ticket is https://trac.torproject.org/projects/tor/ticket/9476 (The fact that this ticket is in the 0.2.6 milestone does *not* mean it will automatically get finished in 0.2.6! If somebody writes good patches, then the odds would go up a lot.) One thing that we would need to think about here is the behavior of any zombie 0.2.2 clients and servers that are still lying around. If they just stop connecting to Tor, great. But if they do something obnoxious like fail to connect and then retry repeatedly, we would need to design our code here so as not to inadvertently turn all these non-functional clients and servers into a DDoS botnet against Tor. :) In any case, removing client-side support for these protocol versions is a definite "yes, let's do that". Removing server-side support would need a little safety testing, but I'd take a patch for that too. yrs, Sounds good! I spent some time writing a patch that removes v1 of the link protocol from both the server and client, and so far it seems to work nicely: the code compiles nicely, all test cases pass, and the resulting binary has relayed a few gigabytes of data without any problems. As I didn't really have a place to put the branch, I uploaded it to Github: https://github.com/TvdW/tor/commits/master It's a rather large patch, though not as large as the patch that will remove v2 of the protocol. However, before I write that one, can someone please check whether my patch is sane and I'm not violating any standards or policies? Thanks, Tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is it time to drop support for the v1/v2 protos?
Nick Mathewson schreef op 02/01/15 om 15:27: On Mon, Dec 29, 2014 at 3:33 PM, Tom van der Woerdt wrote: Sounds good! I spent some time writing a patch that removes v1 of the link protocol from both the server and client, and so far it seems to work nicely: the code compiles nicely, all test cases pass, and the resulting binary has relayed a few gigabytes of data without any problems. As I didn't really have a place to put the branch, I uploaded it to Github: https://github.com/TvdW/tor/commits/master It's a rather large patch, though not as large as the patch that will remove v2 of the protocol. However, before I write that one, can someone please check whether my patch is sane and I'm not violating any standards or policies? Hi, Tom! Sorry about the delay; this has been a busy time, between CCC drama and the new year. A few procedural suggestions: * Maybe call branches like this something other than "master"? It's a good idea to have a separate topic branch for each patch series you write, so that the upstream project can merge them independently. * It's a good idea to link to branches like this from the appropriate tickets on the bugtracker, so that we can't lose track of them. Thanks, I'll create an account. * Every Tuesday on the #tor-dev IRC channel on OFTC, at 1800 UTC, it's patch workshop time, where a bunch of people get together to review one another's patches. Maybe you'd like to stop by? :) Sounds fun! And, some fast notes on the patch itself: * Actually, it's not that long. "git log -p -b" only says it's 607 lines, which is comparatively easy to review. * Some of the stuff can't actually get removed yet by the terms of what we've said we're doing. The V2_CIPHER_LIST, for example, is used to detect Tor versions up through 0.2.3.17-beta. We've only mentioned dropping support for 0.2.2 and earlier, right?. Similarly with the tor_tls_session_secret_cb logic. Now, maybe we _should_ drop support for versions before 0.2.3.17-beta as well. If so, we can rip out even more code. (And that might be a good idea.) What do people on the list think? I'd definitely like ripping out more legacy support code :-) According to the consensus, 0.2.3.24-rc is the lowest recommended version right now, and I (mis)interpreted that as the lowest version that must still be supported after I rip out code. On a related note, I've uploaded a branch of the torspec [1] to reword (most of) the handshake part to match what I hope the code will do as soon as I'm done cleaning. Spec changes may need to go through proposals though, should I add it as one? Also, in case anyone is wondering why a complete stranger is suddenly so interested in ripping parts out of Tor: I attempted to implement a Tor client and was constantly faced with legacy stuff in the spec. Might as well get rid of it if you don't need it - also makes my client implementation easier. Tom [1] https://github.com/TvdW/torspec/commits/remove-legacy-protocol-support smime.p7s Description: S/MIME-cryptografische ondertekening ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is it time to drop support for the v1/v2 protos?
> On 12 Jan 2015, at 16:25, Philipp Winter wrote: > >> On Sat, Dec 27, 2014 at 03:38:28PM +0100, Tom van der Woerdt wrote: >> After reading the Tor spec [1] I did some digging and realized that >> the old handshakes and link protocols (v1 (certs up-front) and v2 >> (renegotiation)) are not used anymore as of 0.2.3.6-alpha which >> introduced link proto v3. >> >> Supporting v1 and v2 requires (among other things) supporting SSLv3 >> which (imho) should be deprecated everywhere. > > I was curious about how many relays and clients still want to speak > version 1 and 2. I patched one of my guard relays to keep statistics > about the content of VERSIONS cells. Here's the result after almost > three days. The numbers include relays as well as clients. Awesome! Looks like we can drop it. > > Versions | Amount total | Amount w/o duplicate hosts > -+---+--- > 1 and 2 | 34,648 (9%) | 21,552 (23%) We debugged this last week on IRC, as 1,2 is an invalid combination according to the specification. After correlating the ip addresses, we concluded that this is GFW scanning and not actual client usage. Are you sure you are deduplicating correctly? That's a lot of hosts. > 3 | 73,202 (18%) | 54,307 (59%) > 3 and 4 | 291,807 (73%) | 16,235 (18%) These two are normal. > 4 | 3 (0%) | 2 (0%) That's me! I have a client implementation that will only return the highest understood version matching what the server supports, instead of a list of all supported versions. So yeah, based on that data I conclude that 1 and 2 are no longer used. Tom > > Cheers, > Philipp > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is it time to drop support for the v1/v2 protos?
David Fifield schreef op 12/01/15 om 18:46:
On Mon, Jan 12, 2015 at 06:26:14PM +0100, Tom van der Woerdt wrote:
On 12 Jan 2015, at 16:25, Philipp Winter wrote:
Versions | Amount total | Amount w/o duplicate hosts
-+---+---
1 and 2 | 34,648 (9%) | 21,552 (23%)
We debugged this last week on IRC, as 1,2 is an invalid combination
according to the specification. After correlating the ip addresses, we
concluded that this is GFW scanning and not actual client usage.
I'm sure some of the 1+2 is GFW scanning, but probably not all of it.
Mainstream tor definitely sends 1+2 when using a v2 handshake.
https://gitweb.torproject.org/tor.git/tree/src/or/connection_or.c?id=b0c32106b3559b4ee9fabfb1a49e2e328c850305#n2122
/** Array of recognized link protocol versions. */
static const uint16_t or_protocol_versions[] = { 1, 2, 3, 4 };
/** Number of versions in or_protocol_versions. */
static const int n_or_protocol_versions =
(int)( sizeof(or_protocol_versions)/sizeof(uint16_t) );
/** Send a VERSIONS cell on conn, telling the other host about the
* link protocol versions that this Tor can support.
*
* If v3_plus, this is part of a V3 protocol handshake, so only
* allow protocol version v3 or later. If not v3_plus, this is
* not part of a v3 protocol handshake, so don't allow protocol v3 or
* later.
**/
int
connection_or_send_versions(or_connection_t *conn, int v3_plus)
{
var_cell_t *cell;
int i;
int n_versions = 0;
const int min_version = v3_plus ? 3 : 0;
const int max_version = v3_plus ? UINT16_MAX : 2;
tor_assert(conn->handshake_state &&
!conn->handshake_state->sent_versions_at);
cell = var_cell_new(n_or_protocol_versions * 2);
cell->command = CELL_VERSIONS;
for (i = 0; i < n_or_protocol_versions; ++i) {
uint16_t v = or_protocol_versions[i];
if (v < min_version || v > max_version)
continue;
set_uint16(cell->payload+(2*n_versions), htons(v));
++n_versions;
}
cell->payload_len = n_versions * 2;
connection_or_write_var_cell_to_buf(cell, conn);
conn->handshake_state->sent_versions_at = time(NULL);
var_cell_free(cell);
return 0;
}
Are you sure you are deduplicating correctly? That's a lot of hosts.
Even if it were only GFW probing, GFW rarely uses duplicate IPs, except
for a few. Most IPs you will only see once or twice over the course of
months.
David Fifield
Wow, nice find. Then, based on the fact that 23% of the network (WTF?)
is still running old clients, maybe it's best to wait with dropping the
old link versions for now. (OTOH if it takes that long for people to
update, dropping it will take years anyway, and removing it from a few
relays won't hurt anything)
23% is a lot though - so high that I really doubt it's true. The ratios
between handshakes and deduplicated handshakes is also rather strange.
Is there anything we can do to the dataset to find out why the amount is
so high?
Tom
smime.p7s
Description: S/MIME-cryptografische ondertekening
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Is it time to drop support for the v1/v2 protos?
Philipp Winter schreef op 12/01/15 om 20:14:
On Mon, Jan 12, 2015 at 06:57:01PM +0100, Tom van der Woerdt wrote:
23% is a lot though - so high that I really doubt it's true. The
ratios between handshakes and deduplicated handshakes is also rather
strange. Is there anything we can do to the dataset to find out why
the amount is so high?
When looking at the ratio, consider that the majority of relays runs
newer versions of Tor [0]. Over these three days, my relay has
established hundreds of connections to other relays over and over again.
When deduplicating relays' addresses, all these connections get reduced
to one which explains why the per-host fraction of version 3 and 4 is
much smaller than the per-connection fraction.
Apart from that, I agree that the number of old clients is unexpected.
First, I suspected the Sefnit botnet (which might still account for ~50%
of Tor "users") but apparently the malware uses Tor v0.2.3.25.
I think the same experiment could be repeated by adding the following to
your tor config:
Log [or]info file /path/to/logfile
And then, the negotiated protocol versions can be counted by running,
for example:
grep -c 'Negotiated version 2' /path/to/logfile
[0] <https://metrics.torproject.org/versions.html>
Interestingly, that paints a completely different picture. I added that
line to two machines (guard+exit) and after a few minutes :
# cat /var/lib/tor/node*/infolog | grep Negotiated | awk '{ print $8 }'
| sort | uniq -dc
40 2
76 3
3811 4
# cat /var/lib/tor/node*/infolog | grep Negotiated | awk '{ print $8 }'
| sort | uniq -dc
50 2
122 3
6269 4
I'll let it run a bit longer but these two machines (which are both
exits as well - probably relevant) get almost solely v4 handshakes.
Tom
smime.p7s
Description: S/MIME-cryptografische ondertekening
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] packaging advice needed
Magnus Hedemark schreef op 11/07/15 om 04:35: Hello, I have set up a Jenkins CI server and have it tracking the Tor project's git repository. I'm going to be building packages for the OmniOS platform, which is not yet supported. I tried searching the site for advice for packagers but haven't found any. If I've overlooked it, I'd appreciate a nudge in the right direction. More specifically, I'd like to better understand the release and patch workflow that is followed so that I can build the right commits at the right time. I'll have one job tracking the master branch for those who want to test potentially unstable software, but I have another job intended to promote "stable" packages. By the way, this is all a precursor to a set of articles showing how to create network-isolated hidden services inside of illumos zones. I think OmniOS has a lot of untapped potential to help hidden service operators. TIA, Magnus Hi Magnus, I think you want to be packaging the maint-0.2.6 branch as 'stable', and master as 'unstable'. Those are the branches that releases come from. Tom smime.p7s Description: S/MIME-cryptografische ondertekening ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Linting the ' iff ' ism...
grarpamp schreef op 16/07/15 om 19:21:
There are about 350 places in tor and torspec that are afflicted
by this ism, other locations surely exist...
find . -type f -exec egrep -i ' ' {} \+
Why is it wrong?
https://en.wikipedia.org/wiki/If_and_only_if
smime.p7s
Description: S/MIME-cryptografische ondertekening
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Draft Proposal: Random Number Generation During Tor Voting
I'm not a big fan of automated systems that ban authorities as it may get false positives and it may be gamed and/or attacked. An alternative solution is to make the voting a two-step system: first you publish the sha256 hash of your vote, then a few minutes later you publish the actual vote. If they didn't match, disregard the vote. It may be a bit more work to implement, but should prove valuable in the long run as it mitigates most cases of authorities trying to manipulate the consensus. Tom > On 07 Sep 2015, at 09:05, s7r wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi, > > Sending the comments from #tor-dev here as well. > > This is related to the attack where exactly half of the directory > authorities commit to some values, and the last directory authority > can send different values to both camps, and have the ultimate > decision about the SR value. This isn't the worst thing a compromised > / malicious authority could do, but this would be simple to detect and > take action against, so it doesn't hurt to have it. If we see this > happening in the wild, it will also warn us to take a closer look at > that authority. > > I think we should implement a ban score system, in which we ignore the > shared randomness votes from authorities for which we see different > commitment values within a close time frame: > > A_1 vote: > A_1 -> 7 > A_2 -> 15 > A_3 -> 21 > > A2_vote: > A_1 -> 7 > A_2 -> 15 > A_3 -> 21 > > A_3 vote: > A_1 -> 11 > A_2 -> 15 > A_3 -> 21 > > In this case we give a ban score of 1 to A_1. We put a timestamp on > the ban-score and remember that A_1 stepped wrong for a longer period > of time, maybe 30 days. If the ban score of a directory authority > reaches value 2, we ignore that authority and agree on a SR value > without it. Maybe we don't even need value 2 here, this shouldn't be > possible to happen accidentally (need to properly document what is the > behavior when an authority goes offline, OS/hardware failure, power > cutoff, gets disconnected from the internet, etc. [TODO]). > > For such a system to work without opening the door to other attacks, > each authority needs to include in its vote the commitment values > received from other authorities including their cryptographic > signature (related to the identity of the authority sending the > commitment value), so all other authorities can verify that a certain > authority sent / signed 2 different commitment values in a short time > frame, and A_3 did not lie that it received commitment value 11 from > A_1, while A_2 and A_1 claim the received commitment value from A_1 is > 7. Without this, any authority can increase the ban score of other > authority on purpose, while the targeted authority is in fact "not > guilty". > > We also need a tiebreaker besides the time stamp for when we have an > even number of directory authorities and exactly half of them commit > to something, the other half to something else. In this case we could > implement the same simple system as we currently have for relay > descriptors: shortest digest value wins. > > >> On 8/3/2015 5:03 PM, George Kadianakis wrote: >> Hello there, >> >> we are glad to release a first draft of our proposal on distributed >> random generation using the Tor voting process. It specifies how >> Tor dirauths can generate a fresh random value every day using a >> commit-and-reveal protocol. The protocol piggybacks on top of the >> regular Tor voting procedure which happens every hour. >> >> Our proposal spends lots of effort resolving the various edge cases >> and engineering issues that come up when you are trying to fit such >> a protocol into the Tor voting system. It also introduces a new >> consensus flavor document which is used to host shared randomness >> information, so that the regular consensus does not get affected if >> this feature is flawed. >> >> We are especially interested in feedback from people who are >> familiar with the Tor voting protocol and can tell us whether we >> have messed something up, or whether there are attacks that we did >> not consider. >> >> We hope the document (and the illustration) is helpful for >> understanding the protocol. Let us know if you have any questions, >> or suggestions for improving the proposal. >> >> We believe this proposal is fundamental for the security of hidden >> services and for developing proposal 224, hence we invite everyone >> to provide feedback and improvements. >> >> Thanks! > -BEGIN PGP SIGNATURE- > Version: GnuPG v2.0.22 (MingW32) > > iQEcBAEBCAAGBQJV7Tc8AAoJEIN/pSyBJlsRsPcH/2eVMUWpMwQBsjhbY4Qru2Q/ > IWOFTQY/C3A26Su/viZASzyzV7IoLPTTokbajOOUMVe04zqD7Kl2wvXwPLZ8/LHx > 06wkr4tURTM0DXwvOzhnYJPXURlxDuZwLpUjOXe7YSpHNoRkJBh+rOd+i4CBmo1E > imFa1HDgkMG3zn/GErbzhEZiGUTyh9wsdGdMl4LGcqNjc+6B9Uxccq5wG6lMoSuC > bj9bNFpBROzspPGrRyx/zTfpQW18AHU3G/A+A4zgOW8m53W/krZyl2MxOiLVXTWD > eIvHyV9uo3FKx2Jd4eLVtbVw4/bpKq25/Au+fSlTB0smxDaKb4z77JFyoLYOiL4= > =EsTQ > -EN
Re: [tor-dev] How to introduce new RELAY_COMMAND_* types and new cell fields?
Op 10/09/15 om 04:10 schreef Mike Perry: I'm writing a handful of proposals that need to introduce either new cell sub-payloads or new command types. Specifically, I want to add: 1. Sub-fields to CELL_PADDING to allow clients to tell relays about the amount of link padding they want for Proposal 251. Mobile clients will probably want less or no padding, and need some way to say so. Adding this information to CELL_PADDING itself seemed to be an easy choice. 2. Sub-fields to RELAY_COMMAND_TRUNCATED and CELL_DESTROY to include HMAC data for Proposal 253. 3. Additional RELAY_COMMAND_* types for clients to request out-of-band HMAC request cells for Proposal 253. 4. Additional RELAY_COMMAND_* opcodes for clients to request padding from relays (for an upcoming padding negotiation proposal). I *think* that fields for items #1 and #2 can be simply added to the existing cell definitions, since we specify that cells should already be 0-filled, and I can rely on 0 fields to mean that the additional fields are absent. It should be OK if relays simply ignore/omit these fields until those proposals are implemented. Is this OK? However, for items #3 and #4, if I introduce a new RELAY_COMMAND type and send it to a relay that doesn't support it, then that relay will emit a warning log message from connection_edge_process_relay_cell() in relay.c. How should I detect support? Based on advertised relay version in the consensus? What about non-standard relay implementations that don't use Tor's versioning? I don't think you can use the consensus for this: for HS connections you wont have any relay info. How about introducing very basic version info into created2/extended/rendezvous cells? You'd have to be really careful to not leak too much version info, of course. Tom smime.p7s Description: S/MIME-cryptografische ondertekening ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] How to introduce new RELAY_COMMAND_* types and newcell fields?
On 10 Sep 2015, at 12:37, tordev...@safe-mail.net wrote: >>> 3. Additional RELAY_COMMAND_* types for clients to request out-of-band >>> HMAC request cells for Proposal 253. > > Do you need to request that data? How about always sending it from middle > nodes? (Less leakage about the client.) > >>> 4. Additional RELAY_COMMAND_* opcodes for clients to request padding >>> from relays (for an upcoming padding negotiation proposal). >>> >>> However, for items #3 and #4, if I introduce a new RELAY_COMMAND type >>> and send it to a relay that doesn't support it, then that relay will >>> emit a warning log message from connection_edge_process_relay_cell() in >>> relay.c. How should I detect support? Based on advertised relay version >>> in the consensus? What about non-standard relay implementations that >>> don't use Tor's versioning? >> >> I don't think you can use the consensus for this: for HS connections you >> wont have any relay info. > > Wouldn't matter for 4, right? Indeed, I was replying to the more generic case of 'how to detect support for feature X' Tom > >> How about introducing very basic version info into >> created2/extended/rendezvous cells? You'd have to be really careful to >> not leak too much version info, of course. > > It seems much better to avoid that, if there is a workaround. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] . tor-roster's geo diversity badge and self-ref relays
> On 13 Sep 2015, at 22:09, teor wrote: > > >> On 13 Sep 2015, at 18:18, Sean Saito wrote: >> >> >"No Self-Referencing Relays" >> >> >I'm not sure what exactly you mean by that but I assume it is a MyFamily >> >> >config where a relay includes his own fingerprint. Why does that hurt? >> >> >The unnecessary descriptor space/bw? >> >> >> >> This is something Virgil wanted because he thought self-connections were >> ugly. If the >> >> penalizing of self-connections is found to be uglier than the >> self-connections themselves, we're >> >> both fine with removing it. >> > > Can this be downgraded to an informational message? (or eliminated entirely?) > > Penalties can be quite discouraging, particularly for minor configuration > variants. > > Tim I agree, and this one in particular is important to some operators: by allowing a relay to specify itself in the family, one can just have a single configuration file for all relays in a family. Tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Special-use-TLD support
Hey Jeff, Definitely very interesting and it's nice to see namecoin and friends in the Tor context. Questions : * are those directives handled on the relay or the client? If relay, how will the client know which node to talk to? * please don't add support for .exit here, external parties should never be able to lead users to that (and having cnames point at them would break that) * what happens if two directives compete for the same TLD? Especially if these are handled at the relay... Tom > On 27 Sep 2015, at 19:47, Jeff Burdges wrote: > > > This is the first of two torspec proposals to help Tor work with Sepcial-Use > TLDs, like the GNU Name system or NameCoin. The second part will be an > anycast facility. - Jeff > > > > > > Filename: xxx-special-use-tld-support.txt > Title: Special-Use TLD Support > Author: Jeffrey Burdges > Created: 20 Sept 2015 > Status: Draft > Implemented-In: ? > > Abstract > > Suppose Special-Use TLDs in Tor via external Domain Name System (DNS) > suppliers, such as the GNU Name System and NameCoin. > > Background > > Special-use TLD supplier software integrates with the host operating > system's DNS layer so that other software resolves the special-use TLD > identically to standard DNS TLDs. On Linux for example, a Special-Use > TLD package could create a plugin for the Name Service Switch (NSS) > subsystem of the GNU C Library. > > Tor cannot safely use the local system's own DNS for name resolution, > as doing so risks deanonmizing a user through their DNS queries. > Instead Tor does DNS resolution at a circut's exit relay. It follows > that Tor users cannot currently use special-use TLDs packages in a safe > manor. > > In addition, there are projects to add public key material to DNS, like > TLSA records and DNSSEC, that necessarily go beyond NSS. > > Design > > We denote by N an abstract name service supplier package. > There are two steps required to integrate N safely with Tor : > > Of course, N must be modified so as to (a) employ Tor for it's own > traffic and (b) to use Tor in a safe way. We deem this step outside > the scope of the present document since it concerns modifications to N > that depend upon N's design. We caution however that peer-to-peer > technologies are famous for sharing unwanted information and producing > excessively distinctive traffic profiles, making (b) problematic. > Another proposal seeks to provide rudementary tools to asist with (a). > > We shall instead focus on modifying Tor to route some-but-not-all DNS > queries to N. For this, we propose a NameService configuration option > that tells Tor where to obtain the DNS record lying under some specific > TLD. > > Anytime Tor resolves a DNS name ending in an Special-Use TLD appearing > in an NameService configuration line then Tor makes an RPC request for > the name record using given UNIX domain socket or address and port. > > We should allow CNAME records to refer to .onion domains, and to > regular DNS names, but care must be taken in handling CNAME records > that refer to Special-Use TLDs handled by NameSerice lines. > Tor should reject CNAME records that refer to the .exit domains. > > Configuration > > We propose two Tor configuration options : > >NameSubstitution [.]source_dnspath [.]target_dnspath >NameService [.]dnspath socketspec > [noncannonical] [timeout=num] > [-- service specific options] > > We require that socketspec be either the path to a UNIX domain socket > or an address of the form IP:port. We also require that that each > *dnspath be a string conforming to RFC 952 and RFC 1123 sec. 2.1. > In other words, a dnsspec consists of a series of labels separated by > periods . with each label of up to 63 characters consisting of the > letters a-z in a case insensitive mannor, the digits 0-9, and the > hyphen -, but hyphens may not appear at the beginning or end of labels. > > NameSubstitution rules are applied only to DNS query strings provided > by the user, not CNAME results. If a trailing substring of a query > matches source_dnspath then it is replaced by target_dnspath. > > NameService rules route matching query to to appropriate name service > supplier software. If a trailing substring of a query matches dnspath, > then a query is sent to the socketspec using the RPC protcol descrived > below. Of course, NameService rules are applied only after all the > NameSubstitution rules. > > There is no way to know in advance if N handles cahcing itself, much > less if it handles caching in a way suitable for Tor. > Ideally, we should demands that N return an approporaite expiration > time, which Tor can respect without harming safety or performance. > If this proves problematic, then configuration options could be added > to adjust Tor's caching behavior. > > Seconds is the unit for the timeout option, which defaults to 60 and > appli
Re: [tor-dev] Proposal: Merging Hidden Service Directories and Introduction Points
Op 12/07/15 om 23:48 schreef John Brooks: Hello! George and I, along with the other participants of this hidden services meeting, have written a proposal for the idea to merge hidden service directories and introduction points into the same entity along with proposal 224. Comments are encouraged, especially if there are downsides or side effects that we haven’t written about yet, or that you have a different opinion on. The intent is that we can decide to do this before implementing proposal 224, so they can be implemented together. The proposal is attached, and also available from: https://raw.githubusercontent.com/special/torspec/224-no-hsdir/proposals/ideas/xxx-merge-hsdir-and-intro.txt Thanks! - John Earlier today in Berlin we discussed this proposal a bit and we noticed a small hack we could do here to make #246 backwards compatible: instead of changing the descriptor format, we could choose to publish an only slightly different descriptor to the nodes in the hsdir ring, including only the introduction point that is the node in the hsdir which the descriptor gets published to. That way, the client will already have the circuit to the right node, and could just re-use it. Example: imagine 2 HSdirs, one at 1.2.3.4, and one at 5.6.7.8. The client will talk to one of these HSdirs to fetch the descriptor, and it will find that if it connects to 5.6.7.8 it gets only a single introduction point (that being 5.6.7.8), and if it had asked the directory on 1.2.3.4 it would have been told to introduce itself at 1.2.3.4. The client can then re-use the circuit to do the introduction. The speedup may only be noticeable to nodes that actually re-use the circuit, but all clients will at least keep working. Tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Proposal: Load-balancing hidden services by splitting introduction from rendezvous
Hey all, I'd like your thoughts and comments on this proposal. Tom PS: If you want to deliver them in person, I'm in Berlin. Filename: xxx-intro-rendezvous-controlsocket.txt Title: Load-balancing hidden services by splitting introduction from rendezvous Author: Tom van der Woerdt Created: 2015-09-30 Status: draft 1. Overview and motivation To address scaling concerns with the onion web, we want to be able to spread the load of hidden services across multiple machines. OnionBalance is a great stab at this, and it can currently give us 60x the capacity by publishing 6 separate descriptors, each with 10 introduction points, but more is better. This proposal aims to address hidden service scaling up to a point where we can handle millions of concurrent connections. The basic idea involves splitting the 'introduce' from the 'rendezvous', in the tor implementation, and adding new events and commands to the control specification to allow intercepting introductions and transmitting them to different nodes, which will then take care of the actual rendezvous. External controller code could relay the data to another node or a pool of nodes, all which are run by the hidden service operator, effectively distributing the load of hidden services over multiple processes. By cleverly utilizing the current descriptor methods, we could publish up to sixty unique introduction points, which could translate to many thousands of parallel tor workers. This should allow hidden services to go multi-threaded, with a few small changes. 2. Specification We propose two additions to the control specification, of which one is an event and the other is a new command. We also introduce a new configuration option. 2.1. DisableAutomaticRendezvous configuration option The syntax is: "DisableAutomaticRendezvous" SP [1|0] CRLF This configuration option is defined to be a boolean toggle which, if set, stops the tor implementation from automatically doing a rendezvous when an INTRODUCE2 cell is received. Instead, an event will be sent to the controllers. If no controllers are present, the introduction cell should be dropped, as acting on it instead of dropping it could open a window for a DoS. For security reasons, the configuration should be made available only in the configuration files, and not as an option settable by the controller. 2.2. The "INTRODUCE" event The syntax is: "650" SP "INTRODUCE" SP RendezvousData CRLF RendezvousData = implementation-specific, but must not contain whitespace, must only contain human-readable characters, and should be no longer than 512 bytes The INTRODUCE event should contain sufficient data to allow continuing the rendezvous from another Tor instance. The exact format is left unspecified and left up to the implementation. From this follows that only matching versions can be used safely to coordinate the rendezvous of hidden service connections. 2.3. "PERFORM-RENDEZVOUS" command The syntax is: "PERFORM-RENDEZVOUS" SP RendezvousData CRLF This command allows a controller to perform a rendezvous using data received through an INTRODUCE event. The format of RendezvousData is not specified other than that it must not contain whitespace, and should be no longer than 512 bytes. 3. Compatibility and security The implementation of these methods should, ideally, not change anything in the network, and all control changes are opt-in, so this proposal is fully backwards compatible. Controllers handling this data must be careful to not leak rendezvous data to untrusted parties, as it could be used to intercept and manipulate hidden services traffic. 4. Example Let's take an example where a client (Alice) tries to contact Bob's hidden service. To do this, Bob follows the normal hidden service specification, except he sets up ten servers to do this. One of these publishes the descriptor, the others have this desabled. When the INTRODUCE2 cell arrives at the node which published the descriptor, it does not immediately try to perform the rendezvous, but instead outputs this to the controller. Through an out-of-band process this message is relayed to a controller of another node of Bob's, and this transmits the "PERFORM-RENDEZVOUS" command to that node. This node finally performs the rendezvous, and will continue to serve data to Alice, whose client will now not have to talk to the introduction point anymore. 5. Other considerations We have left the actual format of the rendezvous data in the control protocol unspecified, so that controllers do not need to worry about the various types of hidden service connections, most notably proposal 224. The decision to not implement the actual cell relaying in the tor implementation itself was taken to allow more advanced configurations, and to leave the actual load-b
Re: [tor-dev] Proposal: Load-balancing hidden services by splitting introduction from rendezvous
Hi Tim, Thanks for your great comments, very much appreciated! Comments inline. Op 30/09/15 om 19:40 schreef Tim Wilson-Brown - teor: On 30 Sep 2015, at 17:27, Tom van der Woerdt mailto:i...@tvdw.eu>> wrote: ... Filename: xxx-intro-rendezvous-controlsocket.txt Title: Load-balancing hidden services by splitting introduction from rendezvous Author: Tom van der Woerdt Created: 2015-09-30 Status: draft 1. Overview and motivation To address scaling concerns with the onion web, we want to be able to spread the load of hidden services across multiple machines. OnionBalance is a great stab at this, and it can currently give us 60x the capacity by publishing 6 separate descriptors, each with 10 introduction points, but more is better. This proposal aims to address hidden service scaling up to a point where we can handle millions of concurrent connections. The basic idea involves splitting the 'introduce' from the 'rendezvous', in the tor implementation, and adding new events and commands to the control specification to allow intercepting introductions and transmitting them to different nodes, which will then take care of the actual rendezvous. … 2.1. DisableAutomaticRendezvous configuration option The syntax is: "DisableAutomaticRendezvous" SP [1|0] CRLF This configuration option is defined to be a boolean toggle which, if set, stops the tor implementation from automatically doing a rendezvous when an INTRODUCE2 cell is received. Instead, an event will be sent to the controllers. If no controllers are present, the introduction cell should be dropped, as acting on it instead of dropping it could open a window for a DoS. For security reasons, the configuration should be made available only in the configuration files, and not as an option settable by the controller. I’m not sure it’s necessary to prevent the controller setting this option. We trust the controller, and might need it to be able to set this option for compatibility with ephemeral hidden services. What is the threat model where a controller could set this option, but not do things that are much worse? You're right, this addresses an irrelevant threat model. 2.2. The "INTRODUCE" event The syntax is: "650" SP "INTRODUCE" SP RendezvousData CRLF RendezvousData = implementation-specific, but must not contain whitespace, must only contain human-readable characters, and should be no longer than 512 bytes I don’t think 512 bytes is enough for the current implementation, I recommend at least 2048 bytes. (See below.) Agreed The INTRODUCE event should contain sufficient data to allow continuing the rendezvous from another Tor instance. The exact format is left unspecified and left up to the implementation. From this follows that only matching versions can be used safely to coordinate the rendezvous of hidden service connections. I would appreciate a list of the data needed by the current version of the hidden service protocol to rendezvous, even if we don’t want to specify the exact format, or specify data items for future implementations. This helps ensure that the limits in the proposal are sane, and that the proposal doesn’t have any unexpected implementation issues. From reading rend_service_receive_introduction think the data is at least: * service_id - the hidden service address (16 base32 bytes) * intro_key - the introduction-point specific key (128 binary bytes, 171 base64 bytes) * request - the encrypted portion of the INTRODUCE2 cell (up to 476 binary bytes(?), 635 base64 bytes) Therefore, I think the minimum for the current hidden service implementation is around 830 bytes, at least if we want to offload the maximum processing to the rendezvous instances by sending the entire encrypted INTRODUCE2 cell. Therefore, I’d suggest that a limit of 2048 bytes is much more reasonable for future-proofing this proposal. It also looks like you might need to split rend_service_t into: * introduction point-specific data * rendezvous-specific data * shared data Does any data need to be shared, and, if so, how do you intend to keep the shared data synchronised? (Putting it in the RendezvousData each time might blow out the size considerably.) I’d also appreciate an example of which parts of rend_service_receive_introduction could be performed by each of the cooperating tor instances. I assume that sending the data “as early as possible” would offload the most processing to the rendezvous side. I think that the split could happen right before the decryption of the cell, at the lines: stage_descr = "decryption"; /* Now try to decrypt it */ This would avoid having to share the intro point encrypted replay cache (intro_point->accepted_intro_rsa_parts), but there’s still the hidden service Diffie-Hellman handshake cache (service->accepted_intro_dh_parts). If we don’t share that: * two b
Re: [tor-dev] Proposal: Load-balancing hidden services by splitting introduction from rendezvous
Op 02/10/15 om 14:56 schreef Tim Wilson-Brown - teor: On 2 Oct 2015, at 14:43, Tom van der Woerdt mailto:i...@tvdw.eu>> wrote: Hi Tim, Thanks for your great comments, very much appreciated! Comments inline. Op 30/09/15 om 19:40 schreef Tim Wilson-Brown - teor: On 30 Sep 2015, at 17:27, Tom van der Woerdt mailto:i...@tvdw.eu> <mailto:i...@tvdw.eu>> wrote: ... Filename: xxx-intro-rendezvous-controlsocket.txt Title: Load-balancing hidden services by splitting introduction from rendezvous Author: Tom van der Woerdt Created: 2015-09-30 Status: draft 1. Overview and motivation To address scaling concerns with the onion web, we want to be able to spread the load of hidden services across multiple machines. OnionBalance is a great stab at this, and it can currently give us 60x the capacity by publishing 6 separate descriptors, each with 10 introduction points, but more is better. This proposal aims to address hidden service scaling up to a point where we can handle millions of concurrent connections. The basic idea involves splitting the 'introduce' from the 'rendezvous', in the tor implementation, and adding new events and commands to the control specification to allow intercepting introductions and transmitting them to different nodes, which will then take care of the actual rendezvous. … In general, I’m concerned that we need to think through the implementation of this proposal more carefully, because it will help us decide whether it’s compatible with: * Current Hidden Services * Next-Generation Hidden Services And perhaps make changes to any of these proposals to make them work together. Thoughts welcome! I don't think I'm the right person to address those. I’d also note that it’s definitely not compatible with Single Onion Services as specified in Proposal #252, as there is no rendezvous in that protocol. Indeed. Splitting the introduction and rendezvous is another use case for NAT-punching single-rendezvous-hop onion services. However, splitting hidden services into multiple different implementations provides less cover for users who really need three-hop hidden services. We’ll need to decide what the tradeoff is here. Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F Had some time, implemented the rest of the proposal as well to check feasibility and updated the proposal with some new insights. Looking forward to all comments on this! Patch 1, splitting the intro and rendezvous code (basically a no-op): https://github.com/TvdW/tor/commit/0443e38d776a114458e2f56e435f324c38e7a17a Patch 2, actually implementing the proposal: https://github.com/TvdW/tor/commit/b8d41f66efdb856b7813c4394f8a81c82e1f2e07 Simple controller implementation that proves my proposal works: https://gist.github.com/TvdW/3f720b9c6ffcd71967c1 Tom == Filename: TBD.txt Title: Load-balancing hidden services by splitting introduction from rendezvous Author: Tom van der Woerdt Created: 2015-09-30 Status: draft 1. Overview and motivation To address scaling concerns with the onion web, we want to be able to spread the load of hidden services across multiple machines. OnionBalance is a great stab at this, and it can currently give us 60x the capacity by publishing 6 separate descriptors, each with 10 introduction points, but more is better. This proposal aims to address hidden service scaling up to a point where we can handle millions of concurrent connections. The basic idea involves splitting the 'introduce' from the 'rendezvous', in the tor implementation, and adding new events and commands to the control specification to allow intercepting introductions and transmitting them to different nodes, which will then take care of the actual rendezvous. External controller code could relay the data to another node or a pool of nodes, all which are run by the hidden service operator, effectively distributing the load of hidden services over multiple processes. By cleverly utilizing the current descriptor methods through OnionBalance, we could publish up to sixty unique introduction points, which could translate to many thousands of parallel tor workers after implementing this proposal. This should allow hidden services to go multi-threaded with a few small changes, and continue scaling for a long time. 2. Specification We propose two additions to the control specification, of which one is an event and the other is a new command. We also introduce two new configuration options. 2.1. HiddenServiceAutomaticRendezvous configuration option The syntax is: "HiddenServiceAutomaticRendezvous" SP [1|0] CRLF This configuration option is defined to be a boolean toggle which, if zero, stops the tor implementation from automatically doing a rendezvous when an
Re: [tor-dev] Proposal: Load-balancing hidden services by splitting introduction from rendezvous
Op 04/10/15 om 06:46 schreef Tim Wilson-Brown - teor: On 3 Oct 2015, at 13:34, Tom van der Woerdt mailto:i...@tvdw.eu>> wrote: ... 3. Compatibility and security The implementation of these methods should, ideally, not change anything in the network, and all control changes are opt-in, so this proposal is fully backwards compatible. Controllers handling this data must be careful to not leak rendezvous data to untrusted parties, as it could be used to intercept and manipulate hidden services traffic. After thinking through this, I wonder if the rendezvous data should contain the decrypted cell, rather than the introduction point key and the encrypted cell. That way, if an INTRODUCE event is exposed, only the one rendezvous referred to by the event is vulnerable. (Exposure of the introduction point key means that all introductions from that point are vulnerable until it is rotated, however, there are other layers of encryption protecting the INTRODUCE2 cells [but we shouldn’t rely on these, because we want defence-in-depth].) This is also slightly more efficient, as we are transmitting less data in the INTRODUCE event. The drawback of this change is that decryption places slightly more load on the tor instance that receives the INTRODUCE2 cell. I don't have a particular opinion on which to pick. The proposal leaves this decision up to the implementation for exactly that reason. There are several things to consider that I can think of: * If the crypto is done on the rendezvous side, HSes could potentially scale better and be more resistant to DoSes; * With up to 60 introduction points (= processes) made possible by OnionBalance, the perf difference may not matter any time soon, and 224 should make HSes perform better anyway; * If the crypto is done on the introduction side, DH replays could be detected better, which may be good against DoSes; * The controller is considered trusted, and connections between the controllers needed for this proposal can be encrypted; * The overhead of constantly adding the same key into the events can largely be negated by using a fast compression algorithm such as Snappy -- although I don't think that these introduce events will ever become a bottleneck. Tom PS: So far this thread has been between Tim and myself... Does anyone else have an opinion? ;-) ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Load Balancing in 2.7 series - incompatible with OnionBalance ?
> On 21 Oct 2015, at 00:18, Alec Muffett wrote: > > So I’ve just had a conversation with dgoulet on IRC, which I will reformat > and subedit here as a conversation regarding OnionBalance and issues in 2.6 > and 2.7 when a recently rebooted HS publishes a fresh descriptor: > > […] > > alecm: consider OnionBalance which - being a bunch of daemons on a bunch of > servers - will be a lot more prone to intermittent failures of 1+ daemons > yielding a lot of republishing > > alecm: we tend to move services around, and daemons will be killed in one > place and resurrected elsewhere, and then we'll have to bundle up a new > descriptor and ship it out > > dgoulet: hrm so with that new 027 cache behavior, as long as the IP are > usable, the descriptor will be kept, if they all become unusable, a new > descriptor fetch is triggered and then those IPs will be tried > > alecm: There's a mandatory refresh [of the descriptor] after N minutes? > > dgoulet: we'll retry 3 times and after that all HSDir are in timeout for 15 > minutes (I think, I'll have to validate) before retrying any HSDirs > > alecm: I wonder if descriptors should publish a recommended TTL - [number of > seconds to live before refresh] > > dgoulet: yeah we have an idea for a "revision-counter" in the descriptor > being incremented at each new version for the 24 hours period > > dgoulet: a TTL could be useful for load balancing though! > > alecm: so, here's a scenario: imagine that we run 10 daemons, > > alecm: call these daemons: A B C D E F G H I J - they all have random onion > addresses > > alecm: we steal one IP from each daemon, and bundle the 10 stolen IPs > together to make an onionbalance site descriptor and publish it > > alecm: people pull that descriptor, it's quite popular > > alecm: we then lose power in a datacentre, which takes out half of our onions > - say, A through E > > alecm: we reboot the datacentre and restart A-E merely 10 minutes later > > alecm: everyone who has already loaded our onionbalance site descriptor tests > A B C D E and finds them all dead, because the old IPs for A-E are invalid > > alecm: so they all move to F G H I J - which get overloaded even though (new) > A B C D E are back up > > alecm: and this persists for up to 244, even though the outage was only 10 > minutes > > alecm: net result: large chunks of the world (anyone with an old descriptor + > anyone randomly choosing F-J) have a shitty experience, which is not what > high-availability is all about :-) > > dgoulet: that will be what's going to happen - having a TTL in the desc. > would help here indeed, I see the issue > > dgoulet: TTL would be one thing to add, here we could also add a mechanism > for a client retrying IPs that failed in the situation where some of the IPs > are still working, or making client balance themself randomly could be also > an idea > > dgoulet: definitely there is some content here for tor-dev - I don't have a > good answer but it should definitely be addressed > > alecm: proper random selection of IP would be beneficial for load-balancing; > not perfect, but in the long run, helpful Hi Alec, Most of what you said sounds right, and I agree that caching needs TTLs (not just here, all caches need to have them, always). However, you mention that one DC going down could cause a bad experience for users. In most HA/DR setups I've seen there should be enough capacity if something fails, is that not the case for you? Can a single data center not serve all Tor traffic? If that is a problem, I would suggest adding more data centers to the pool. That way if one fails, you don't lose half of the capacity, but a third (if N=3) or even a tenth (if N=10). Anyway, such a thing is probably off-topic. To get back to the point about TTLs, I just want to note that retrying failed nodes until all fail is scary: what will happen if all ten nodes get a 'rolling restart' throughout the day? Wouldn't you eventually end up with all the traffic on a single node, as it's the only one that hadn't been restarted yet? As far as I can see the only thing that can avoid holes like that is a TTL, either hard coded to something like an hour, or just specified in the descriptor. Then, if you do a rolling restart, make sure you don't do it all within one TTL length, but at least two or three depending on capacity. Tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Proposal: Stop giving Exit flags when only unencrypted traffic can exit
I've had this on my todo list for a while, finally wrote it down. Honestly, it's a minor change, but something that imho needs to be done. Torspec branch: https://github.com/TvdW/torspec/commits/exit-flag-not-all-plaintext Full text below, tldr first: replace [80,443,6667] with [80,443,5222] for Exit flagging. === Filename: 264-exit-flag-not-all-plaintext.txt Title: Stop giving Exit flags when only unencrypted traffic can exit Author: Tom van der Woerdt Created: 2016-01-05 Status: Open 1. Introduction Tor's Exit flags are assigned to relays that have an exit policy that allows exiting to at least two out of three pre-defined ports: 80, 443 and 6667. Since 80 and 6667 (resp. http and irc) are generally used for unencrypted traffic, an attacker could construct an exit policy that relays only unencrypted data. 2. Changes 2.1. Exit flagging By replacing the port 6667 (IRC) entry with a port 5222 (XMPP) entry, Exit flags can no longer be assigned to relays that exit only to unencrypted ports. 2.2. dir-spec.txt A change to dir-spec.txt will be needed to change port 6667 to 5222. 3. Migration This change only needs to be rolled out to directory authorities. Since the flagging system is simple, no special migration is needed for this change, and it will take effect as soon as the deployment of the change has reached a sufficient number of directory authorities. 4. Other considerations While it would have been ideal to drop the port 80 condition as well, in the current state of the internet this is not likely to be a good idea. Too much websites still use unencrypted connections. However, this may be worth reconsidering every few years. XMPP was chosen to replace IRC because nowadays unencrypted XMPP is rare, and because the XMPP protocol is slowly gaining popularity within the communities on the internet. Other popular ports have been considered, such as 22 (SSH), 465 (SMTP), or 995 (POP3), but these are unlikely to be good candidates because of wide spread bruteforce attacks on these ports. === Tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal: Stop giving Exit flags when only unencrypted traffic can exit
Hi Tim,
Thanks for your comments! Appreciated as always :-)
Op 05/01/16 om 02:15 schreef Tim Wilson-Brown - teor:
>
>> On 5 Jan 2016, at 11:29, Tom van der Woerdt > <mailto:i...@tvdw.eu>> wrote:
>> ...
>> 2.1. Exit flagging
>>
>> By replacing the port 6667 (IRC) entry with a port 5222 (XMPP) entry,
>> Exit
>> flags can no longer be assigned to relays that exit only to unencrypted
>> ports.
>
> One consequence of this proposal is that relays that only exit to 443
> and 6667 will lose the Exit flag.
> But these relays do exit to an encrypted port, so this somewhat
> contradicts the goal of the proposal:
> "Exit flags can no longer be assigned to relays that exit only to
> unencrypted ports."
(Sorry for the huge Perl oneliner -- it's a consensus parser...)
$ curl -q http://128.31.0.34:9131/tor/status-vote/current/consensus
2>/dev/null | perl -nle' @l= split /\s/, $_; if ($l[0] eq "r") { if ($r)
{ if (grep { "Exit" eq $_ } @{$r->{s}//[]}) { my @ports= split ",",
$r->{p}[2]; @ports= map { $_ =~ /(\d+)\-(\d+)/ ? eval("$1..$2") : $_ }
@ports; my %p= map { $_ => 1 } @ports; if ($p{443} && !$p{80} &&
$p{6667} && !$p{5222}) { print "$r->{r}[1] $r->{w}[1]"; } } } push @r,
$r={} } $r->{$l[0]}= [@l];'
(tlcr: any relay that currently holds an Exit flag and allows exiting to
443 and 6667, but not 80 or 5222.)
tiggersWeltTor1 Bandwidth=2600
smallegyptrela01 Bandwidth=22
These two relays will be impacted, indeed.
>
> Why not make the rule: "at least one of 80/6667, and at least one of
> 443/5222".
Also sounds good to me. I opted for the smallest possible change
(6667->5222) but what you're suggesting lgtm.
>
> I am also concerned about the choice of XMMP "because the XMPP protocol
> is slowly gaining popularity within the
> communities on the internet".
> Shouldn't we focus on secure protocols that are widely used right now?
>
> Alternately, we could add other widely used SSL ports in addition to
> XMMP, and perhaps increase the rule to "at least two SSL ports".
Imho the challenge is in finding port number(s) that accurately reflect
what Tor is for, while also having a sufficiently large user base for it
to be relevant. XMPP probably has more users than IRC, and is a good
match for what I think Tor would consider important (communication).
Also note that we now have Tor Messenger. Other protocols (SSH, IMAP,
POP3, SMTP) are indeed more popular but I feel that those less reflect
the goals of the project, and they are certainly abused more.
>
> Tim
>
> Tim Wilson-Brown (teor)
>
> teor2345 at gmail dot com
> PGP 968F094B
>
> teor at blah dot im
> OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F
>
>
>
> ___
> tor-dev mailing list
> tor-dev@lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
>
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal: Stop giving Exit flags when only unencrypted traffic can exit
Op 05/01/16 om 10:22 schreef Tim Wilson-Brown - teor: > >> On 5 Jan 2016, at 19:33, Tom van der Woerdt > <mailto:i...@tvdw.eu>> wrote: >> ... >> Op 05/01/16 om 02:15 schreef Tim Wilson-Brown - teor: >>> >>>> On 5 Jan 2016, at 11:29, Tom van der Woerdt >>> <mailto:i...@tvdw.eu> >>>> <mailto:i...@tvdw.eu>> wrote: >>>> ... >>>> 2.1. Exit flagging >>>> >>>> By replacing the port 6667 (IRC) entry with a port 5222 (XMPP) entry, >>>> Exit >>>> flags can no longer be assigned to relays that exit only to unencrypted >>>> ports. >>> >>> One consequence of this proposal is that relays that only exit to 443 >>> and 6667 will lose the Exit flag. >>> But these relays do exit to an encrypted port, so this somewhat >>> contradicts the goal of the proposal: >>> "Exit flags can no longer be assigned to relays that exit only to >>> unencrypted ports." >> >> ... >> >> (tlcr: any relay that currently holds an Exit flag and allows exiting to >> 443 and 6667, but not 80 or 5222.) >> >> tiggersWeltTor1 Bandwidth=2600 >> smallegyptrela01 Bandwidth=22 >> >> These two relays will be impacted, indeed. > > Point taken! > > How many Exits would lose the Exit flag intentionally based on this change? > (That is, how many have 80 & 6667, but not 443?) If we change 6667 to 5222, this changes (where 0->1 means it will become an exit and 1->0 means it will no longer be one) : FruityOatyTorexit Bandwidth=17700 0->1 Alice Bandwidth=25 0->1 tiggersWeltTor1 Bandwidth=3100 1->0 onionnetGOT01 Bandwidth=387 0->1 icubdw2o2xipsdc Bandwidth=137 1->0 miepernl Bandwidth=1420 1->0 ReservoirPi2016 Bandwidth=114 0->1 TORWeazel Bandwidth=98 0->1 HelloWorld Bandwidth=820 1->0 smallegyptrela01 Bandwidth=22 1->0 AnonNodeFin69 Bandwidth=80 0->1 Serveur Bandwidth=703 0->1 Biverse Bandwidth=779 0->1 comaTor1 Bandwidth=148 0->1 Unnamed Bandwidth=138 1->0 Gained bw: 20034 Lost bw: 5637 Tom (source of this data: https://paste.debian.net/360256/) > >>> >>> Why not make the rule: "at least one of 80/6667, and at least one of >>> 443/5222". >> >> Also sounds good to me. I opted for the smallest possible change >> (6667->5222) but what you're suggesting lgtm. >> >>> >>> I am also concerned about the choice of XMMP "because the XMPP protocol >>> is slowly gaining popularity within the >>> communities on the internet". >>> Shouldn't we focus on secure protocols that are widely used right now? >>> >>> Alternately, we could add other widely used SSL ports in addition to >>> XMMP, and perhaps increase the rule to "at least two SSL ports". >> >> Imho the challenge is in finding port number(s) that accurately reflect >> what Tor is for, while also having a sufficiently large user base for it >> to be relevant. XMPP probably has more users than IRC, and is a good >> match for what I think Tor would consider important (communication). >> Also note that we now have Tor Messenger. Other protocols (SSH, IMAP, >> POP3, SMTP) are indeed more popular but I feel that those less reflect >> the goals of the project, and they are certainly abused more. > > 80/443 get us anonymous web browsing, primarily through Tor Browser > 6667/6697 get us anonymous messaging via IRC > (I don't know if 6697 is common enough to be worth changing for.) > 5222 get us anonymous messaging via Tor Messenger > > I can't think of any others right now. > > Tim > > Tim Wilson-Brown (teor) > > teor2345 at gmail dot com > PGP 968F094B > > teor at blah dot im > OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F > > > > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal: Stop giving Exit flags when only unencrypted traffic can exit
Interesting thought. I've followed git history a bit (back then it was svn) and traced it back to 54a6a8f0 (tor.git). It's added to a function "router_is_general_exit" which is described as: > /** Return true iff ri is "useful as an exit node." */ Port 6667 is later chosen by Roger in 0ac3c584, the commit message doesn't describe why. In 19408cf8, code is added to check that at least *two* ports are allowed. Note that all of these happened about 10 years ago. I'd like to hear from Roger/Nick what their opinion is on flagging any exit node with the Exit flag, instead of just the ones that exit on some common ports. It seems that doing this will result in adding a significant amount of exit relays, many of them quite useful: https://paste.debian.net/360365/ (Obviously) no exits will be dropped from the consensus if we choose to do that. Tom PS: Big fan of this relay: Pushkin Bandwidth=3190 21,23,25,80,110,143 Op 05/01/16 om 20:15 schreef Virgil Griffith: >> Other protocols (SSH, IMAP, >> POP3, SMTP) are indeed more popular but I feel that those less reflect >> the goals of the project, and they are certainly abused more. > > I hear you that these are abused more. But I personally think of Tor as > a mere mechanism than a mechanism+policy. For example, should the > command "rm" refuse to remove a file that has the text in it that says > "IMPORTANT! DO NOT DELETE!" Although obviously this is a > well-intentioned feature, presumably rm should not behave this way. The > rm command is a mechanism, the policy for that mechanism judicious use > is a wrapping around the command itself. > > One additional benefit of separating mechanism from policy is that it > makes policies more easily changeable. Well-meaning people have > disagreements on policies, and policies invariably evolve. Separating > policies from the core functionality is helpful to allow easier > experimentation with alternative policies. > > Applying this here, I argue that the ports a relay makes available > should not impact whether they get the exit flag. This is consistent > with treating Tor as a mechanism instead of applying top-down a policy > for how people are "supposed" to use it. > > -V > > > > On Wed, Jan 6, 2016 at 2:25 AM Tom van der Woerdt <mailto:i...@tvdw.eu>> wrote: > > Op 05/01/16 om 10:22 schreef Tim Wilson-Brown - teor: > > > >> On 5 Jan 2016, at 19:33, Tom van der Woerdt <mailto:i...@tvdw.eu> > >> <mailto:i...@tvdw.eu <mailto:i...@tvdw.eu>>> wrote: > >> ... > >> Op 05/01/16 om 02:15 schreef Tim Wilson-Brown - teor: > >>> > >>>> On 5 Jan 2016, at 11:29, Tom van der Woerdt <mailto:i...@tvdw.eu> > >>>> <mailto:i...@tvdw.eu <mailto:i...@tvdw.eu>> > >>>> <mailto:i...@tvdw.eu <mailto:i...@tvdw.eu>>> wrote: > >>>> ... > >>>> 2.1. Exit flagging > >>>> > >>>> By replacing the port 6667 (IRC) entry with a port 5222 (XMPP) > entry, > >>>> Exit > >>>> flags can no longer be assigned to relays that exit only to > unencrypted > >>>> ports. > >>> > >>> One consequence of this proposal is that relays that only exit > to 443 > >>> and 6667 will lose the Exit flag. > >>> But these relays do exit to an encrypted port, so this somewhat > >>> contradicts the goal of the proposal: > >>> "Exit flags can no longer be assigned to relays that exit only to > >>> unencrypted ports." > >> > >> ... > >> > >> (tlcr: any relay that currently holds an Exit flag and allows > exiting to > >> 443 and 6667, but not 80 or 5222.) > >> > >> tiggersWeltTor1 Bandwidth=2600 > >> smallegyptrela01 Bandwidth=22 > >> > >> These two relays will be impacted, indeed. > > > > Point taken! > > > > How many Exits would lose the Exit flag intentionally based on > this change? > > (That is, how many have 80 & 6667, but not 443?) > > If we change 6667 to 5222, this changes (where 0->1 means it will become > an exit and 1->0 means it will no longer be one) : > > FruityOatyTorexit Bandwidth=17700 0->1 > Alice Bandwidth=25 0->1 > tiggersWeltTor1 Bandwidth=3100 1->0 > onionnetGOT01 Bandwidth=387 0->1 > icubdw2o2xipsdc Bandwidth=137
Re: [tor-dev] Tor Control Protocol: multiple commands in a single line possible?
Hi Patrick, No, the protocol does not support this. The only command separator is a newline. Implementing this would likely be hard anyway, as the output of the double command would be hard to parse. Tom Op 10/01/16 om 18:02 schreef Patrick Schleizer: > TLDR: > > Does Tor's control protocol actually support something like ; ? > I.e. something like > signal newnym ; getinfo address > ? > How many of there separators are there? > Can you provide examples please? > > Background: > > At the moment we are implementing support for whitelisting wildcards for > control-port-filter-proxy-python [0] to support Onionshare and Ricochet. > [1] [2] [3] [4] > > Does Tor's control protocol actually support something like ; ? If it > does, then this would complicate the wildcard feature. > > If we whitelisted the wildcard add_onion *, we don't want it to match > some hypothetical feature add_onion * ; I.e. not add_onion * ; > GETINFO address. > > (If it was the case, then we would have to limit the wildcard (*) from > example SETCONF HiddenServicePort * to exactly one [numeric] argument > etc. Or better, correctly parse multi lined commands. Or as stopgap, > reject these separators since no applications using Tor's control > protocol are currently using those.) > > Cheers, > Patrick > > [0] https://github.com/Whonix/control-port-filter-python > https://www.whonix.org/wiki/Dev/Control_Port_Filter_Proxy > [1] https://phabricator.whonix.org/T446 > [2] https://phabricator.whonix.org/T445 > [3] https://www.whonix.org/wiki/Next#onionshare > [4] https://phabricator.whonix.org/T448 > > (Asked Roger by mail, said it's okay to post this on tor-dev.) > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] We need a new RPM maintainer
I can do it. I already package my own RPMs, so I may as well do that properly.
No ldap account though.
Tom
> On 16 Jun 2016, at 15:06, Nick Mathewson wrote:
>
> Hi, all!
>
> Ondrej Mikle, who has built our RPMs for a while, is no longer able to
> do so. (He isn't using RPM distros any more, doesn't have as much free
> time, and etc.)
>
> First: Thanks, Ondrej, for all your hard work!
>
> Here's the instructions he sent on how to do it (quoted with permission):
> {{{
> Basically every part of the process is documented here:
>
> https://gitweb.torproject.org/user/hiviah/rpm-build-scripts.git/tree/README
>
> It's maybe not the best README, because the actual merge-and-build
> part is in the middle (but I always use it, otherwise I'd just forget
> all the necessary steps). Nevertheless, the introduction maybe
> necessary for someone not familiar with the process.
>
> The process in the short is:
>
> 0. have the environment ready (has to be done just once)
> 1. clone my own RPM fork which has RPM changes
> (https://gitweb.torproject.org/user/hiviah/rpm-tor.git)
> 2. Merge and build according to "MERGING REPO USED FOR RPM BUILDING FROM
> ORIGIN"
> 3. Download, review, sign the built RPMs (I used feddei.tpo VM for
> this, but you can have your own VM as I had last time since feddei.tpo
> was down)
> 4. upload to palmeri.tpo
> 5. distribute to everyone else via "static-update-component
> deb.torproject.org" on palmeri.tpo
>
> I could walk somebody through this, as I said, this is not rocket
> surgery, mostly attention to detail :)
>
> If there ever was a critical need for a special build due to some
> unforseen bug, I could still do it, but this is for exceptional case
> only.
> }}}
>
> The easiest way for RPMs to happen would be if somebody who already
> has a Tor LDAP account wants to step forward and volunteer for a
> while. It would be harder if this were somebody that we didn't know
> who volunteered: until our RPM process creates reproducible builds, we
> need to have some amount of trust in the person who creates them.
>
> There may be other issues here too. I am no packaging expert. :)
>
> best wishes,
> --
> Nick Mathewson
> ___
> tor-dev mailing list
> tor-dev@lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
___
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] is the consensus document unpredictable / unique?
Hi Razvan, The consensus has signatures from all directory operators on it, and computing those ahead of time requires a lot of private keys. Because they also all contain the date, they're all unique. So yea, they're both unique and unpredictable. As for your idea: it should be noted that there is not a single valid consensus. At any time there may be several valid ones and clients may have different active ones, as all consensuses are valid for a few hours but generated hourly. Using the hash as a descriptor cookie may thus be troublesome. Tom > On 25 Jun 2016, at 23:52, Razvan Dragomirescu > wrote: > > Hello everyone, > > I couldn't find a detailed description of the Tor consensus, so I'm checking > that my understanding of it is correct. Basically, would it be correct to > assume that the consensus document (or a hash thereof) for a date in the > future is an unpredictable value that will also be unique to all nodes > inquiring about it at that time? > > I'm thinking of using a hash of the consensus document - like > http://171.25.193.9:443/tor/status-vote/current/consensus - as a descriptor > cookie in a hidden service. This way, an attacker cannot generate or publish > a hidden service descriptor for the future (one with a correct cookie). A > client can fetch the consensus at the time it wants to connect, hash it, then > use that as the descriptor cookie to determine the correct descriptor id and > decrypt the introduction point list. > > Does anyone see any issues with this? In my project, the hidden service > private key is on a smartcard, so it can't be copied, you can only ask the > smartcard to sign something with it for you - and I'm trying to prevent an > attacker from generating hidden service descriptors in advance,to be used > without the smartcard. If future descriptors depend on an unpredictable > future value (the hash of the consensus at that time), an attacker can only > generate descriptors for past and current time periods. > > Thank you, > Razvan > > -- > Razvan Dragomirescu > Chief Technology Officer > Cayenne Graphics SRL > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] A meta-package for Pluggable Transports?
How about a conf.d style folder that plugins like bridges can drop files in? $ yum install -y obfs4proxy ... $ cat /etc/tor/torrc.d/obfs4.conf ServerTransportPlugin obfs3,obfs4 exec /usr/bin/obfs4proxy managed ServerTransportListenAddr obfs4 0.0.0.0:9013 ServerTransportListenAddr obfs4 0.0.0.0:9014 $ systemctl restart tor Tom Op 30/06/16 om 21:15 schreef Nima Fatemi: > It’s currently difficult for bridge operators to keep up with the > changes in pluggable transports world. You’ve to be following tor > development and censorship-war very closely to know which transport is > needed currently and how to run them. > > There are many people who are still running vanilla bridges thinking > they’re helping people in censored networks. Unfortunately those bridges > are not anyone any good while burning operator’s resources. > > After some discussion on #tor-project a little while ago, the idea of > having a meta-package that includes all or the most recent transports > came up. Where people would install this meta package and it would > automatically take care of the required steps to get the latest > obfsproxy and set it up. > > From a UX perspective, ideally you’d set up a bridge with small and > consistent steps like this: > > $ sudo apt-get install tor-bridge > $ tor-bridge —-setup OR $ tor-bridge-setup > > and then it will automatically get the most recommended PT (eg obfs4), > tor itself (if not installed), config your torrc, do a reachability > test, publish the bridge to bridgdb automatically and give you the > result in stdout: > > # Congrats! your bridge is up and running on $port > # Your bridge is published in BridgeDB. > # Thanks for fighting censorship! > > Additionally we can have more flags for different transports, ip, port > and so on. For example if you want to run obfs4proxy on an specific port > and not publish it, I imagine running something like this should get you > there: > > $ tor-bridge-setup —-private —-obfs4 —-ip 1.2.3.4 —-port 5000 > > # Congrats! your bridge is up and running on port 5000 > # You have chosen to not to publish your bridge. Users would need to > manually copy and paste the following line in their Tor Browser to use > your bridge. > # > # bridge obfs4 1.2.3.4:5000 C73ADBAC8ADFDBF0FC0F3F4E8091C0107D093716 > cert=gEGKc5WN/bSjFa6UkG9hOcft1tuK+cV8hbZ0H6cqXiMPLqSbCh2Q3PHe5OOr6oMVORhoJA > iat-mode=0 > > The purpose of this email is to see whether this is a good approach (if > not, how can we improve it), and what is needed to move towards it. > > Feedback from everyone, specially packagers and relay operators are > encouraged and welcome :) > > Best, > > > > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] A meta-package for Pluggable Transports?
Op 30/06/16 om 21:31 schreef Nima Fatemi: > Tom van der Woerdt: >> How about a conf.d style folder that plugins like bridges can drop files in? >> >> $ yum install -y obfs4proxy >> ... >> $ cat /etc/tor/torrc.d/obfs4.conf >> ServerTransportPlugin obfs3,obfs4 exec /usr/bin/obfs4proxy managed >> ServerTransportListenAddr obfs4 0.0.0.0:9013 >> ServerTransportListenAddr obfs4 0.0.0.0:9014 >> $ systemctl restart tor > > In this case you'd still have to know what obfs4proxy is or when and > where to get it. It doesn't solve the problem of the need to catch up > with censorship and tor development. And to break a news for you... > obfs5 is coming soon. What are we gonna do when that comes out? What do > we call the package? This also doesn't help running any other transports > like meek. > > Best, Oops, should probably have elaborated a bit more. You can combine conf.d with the meta-package. For example, when you run "yum install tor tor-bridge" the tor-bridge package could bring in obfs4proxy with it and automatically configure that : $ yum install -y tor tor-bridge $ systemctl start tor -- The challenge, imho, is figuring out how to handle upgrades. A virtual package like the tor-bridge in my example would automatically pull in obfs5proxy, leaving obfs4proxy intact, but this may be considered bad behavior to the user, as it may also need firewall changes etc. Questions that would need answering : * Is it acceptable to automatically install newer transports? * If it is, should we automatically deprecate the older transports? And in that case, can we maybe just replace the old transport with the new one while reusing the port? Will that break UX for the user who actually set it up, as it now has a different type of proxy on the same port? * If it is not, how to we notify the user that they should renew their setup? * Most users don't compile Tor from scratch, but use packages that come with distributions such as Ubuntu, Debian and CentOS. How do you properly integrate with those? Tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
