Re: [tor-dev] Proposal: Merging Hidden Service Directories and Introduction Points
Op 12/07/15 om 23:48 schreef John Brooks: Hello! George and I, along with the other participants of this hidden services meeting, have written a proposal for the idea to merge hidden service directories and introduction points into the same entity along with proposal 224. Comments are encouraged, especially if there are downsides or side effects that we haven’t written about yet, or that you have a different opinion on. The intent is that we can decide to do this before implementing proposal 224, so they can be implemented together. The proposal is attached, and also available from: https://raw.githubusercontent.com/special/torspec/224-no-hsdir/proposals/ideas/xxx-merge-hsdir-and-intro.txt Thanks! - John Earlier today in Berlin we discussed this proposal a bit and we noticed a small hack we could do here to make #246 backwards compatible: instead of changing the descriptor format, we could choose to publish an only slightly different descriptor to the nodes in the hsdir ring, including only the introduction point that is the node in the hsdir which the descriptor gets published to. That way, the client will already have the circuit to the right node, and could just re-use it. Example: imagine 2 HSdirs, one at 1.2.3.4, and one at 5.6.7.8. The client will talk to one of these HSdirs to fetch the descriptor, and it will find that if it connects to 5.6.7.8 it gets only a single introduction point (that being 5.6.7.8), and if it had asked the directory on 1.2.3.4 it would have been told to introduce itself at 1.2.3.4. The client can then re-use the circuit to do the introduction. The speedup may only be noticeable to nodes that actually re-use the circuit, but all clients will at least keep working. Tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Draft proposal -- no number yet: How to safely drop support for old clients.
[This isn't done yet, but I've shown it to enough people that I should share it with the list.] 1. Introduction and background 1.1. Motivation Frequently, we find that very old versions of Tor should no longer be supported on the network. To remove relays is easy enough: we simply update the directory authorities to stop listing them. But to disable clients is harder. We can do as we did with v1 directories in Tor 0.2.0.5-alpha, and make them know about no relays at all. Stepping back -- why should we disable very old clients at all? Here are some reasons why: * Security. Anybody who hasn't updated their Tor client in 5 years is probably vulnerable to who-knows-what attacks. They aren't likely to get anonymity either. * Withstand zombie installations. Some Tors out there were once configured to start-on-boot on now-unmaintained systems. They put needless load on the network, and can't be easily disposed of. * Disable backward-compatibility code. Currently, Tor supports some truly ancient protocols in order to avoid breaking ancient versions. If we disable these protocols without turning off the clients that use them, we run the risk that those clients will run wild, retrying their connections to the network over and over. This document describes a designs for use with future versions of Tor to better support client deprecation and removal. It also describes how we can safely disable existing client versions that do not support this proposal. 1.2. History Early versions of Tor checked the recommended-versions field in the directory to see whether they should keep running. If they didn't recognize 1.3. Goals and requirements We need older clients to stop putting (much) load on the network. Having them stop working entirely is not sufficient: they need to stop working in such a way that they don't keep using up resources. In the future, when we disable a client, it would be a good thing to provide some means to warn users in advance that they need to upgrade, and to provide advice about doing so. It might be good to make any disable-switches advisory rather than mandatory. 2. Adding a disable-clients switch to consensus documents I specify a new consensus element: disable-versions. It may appear any number of times in a consensus document. The syntax is: disable-versions SP SOFTWARE SP ROLE SP DATE SP VERSIONS DATE = A date in -MM-DD format ROLE = "relay", or "all". SOFTWARE = "Tor" VERSIONS = VERSION_RANGE VERSIONS = VERSIONS SP VERSION_RANGE VERSION_RANGE = VERSION VERSION_RANGE = VERSION..VERSION [TODO: Specify voting.] If the current version is listed in any disabled-versions line for its current role, then we should warn the user. If the current date is on after the date specified (starting midnight UTC), we should disable the host as follows: * If 'relay' is the disabled role, then we should enter ClientOnly mode. * If 'all' is the disabled role, we should enter DisableNetwork mode. Note that the DisableNetwork behavior means that once a client is disabled, it will not fetch new consensuses. To override this, a Tor instance may include a configuration option, "IgnoreDisabledVersion VERSION". It is an error to set this option when *not* running with a disabled version. [ALTERNATIVE: Make RequireLinkProtocol, RequireCircuitProtocol, etc lines for the consensus. Any Tor not supporting those protocols should behave as above. This allows non-Tor implementations to respect these fields.] 3. Disabling current client versions. 3.1. Disabling versions that use the v2 handshake These versions (before 0.2.3.6-alpha) use a renegotiation-based handshake instead of our current handshake. When we detect a renegotiation attempt, we can close the connection, or stop reading on it. 3.2. Disabling versions prior to 0.2.4.11-alpha. Starting in version Tor 0.2.4.11-alpha, we implemented proposal 214. This proposal added a new link protocol, version 4. Earlier version of Tor never support it; newer versions of Tor always advertise their support for it. We could treat the presence of any link protocol of version 4 or later as indicating the presence of 0.2.4.11-alpha. Upon receiving a connection from such a version, we could throttle the bytes we give it, drop its circuits, or stop reading. [TODO: which?] 3.3. Disabling versions prior to 0.2.5.x In tor 0.2.5.1-alpha, we began ignoring the UseNTorHandshake option, and always preferring the ntor handshake where available. Unfortunately, we can't simply drop all TAP handshakes, since clients and relays can still use them in the hidden service protocol. But we could det
[tor-dev] Proposal: Adding x-namespace to relay descriptor for key:value pairs
Filename: ExtraRelayDescriptorFields.txt Title: Adding x-namespace to relay descriptor for key:value pairs Author: Virgil Griffith Created: 2015-09-30 Status: Open 1. Motivation We wish to allow developers to build new applications atop relays. Towards this end, we wish to add the ability for users to specify arbitrary new key-value entries under the "X-" namespace to the router descriptor [1]. The canonical applications for this are adding a bitcoin donation address and networking of tor2web nodes. 2. Proposal Allow optional key-value lines in the relay's torrc file. These lines will be mirrored in the relay's descriptor which is then published in the directory consensus. The following would be added to section 2.1.2 of the dir-spec (Extra-info document format): "X-" CustomKey SP CustomValue NL CustomKeyChar = "a"-"z" / "0" - "9" / "-" / "_" CustomKey = 1*32 CustomKeyChar CustomValueChar = atext / specials CustomValue = 1*1024 CustomValueChar Custom fields can appear multiple times, for example... X-bitcoin 19mP9FKrXqL46Si58pHdhGKow88SUPy1V8 X-gravatar https://s.gravatar.com/avatar/d27fce46c9ac41a41bb52455ae75701d X-favoritequote Be excellent to each other. Party on dudes! X-foo bar If the same CustomKey appearing more than once is disallowed. Possible values for CustomValueChar as specified per RFC 2822. The sum size accounting for all such custom fields is truncated to 5 kilobytes. To mitigate the chance of a malformed torrc file, I additionally propose that the relay descriptor be scanned and if it does not match the specification, that it exit with error telling her torrc file is a likely culprit. -V ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal: Adding x-namespace to relay descriptor for key:value pairs
Dear Virgil, On 9/30/15, Virgil Griffith wrote: > Filename: ExtraRelayDescriptorFields.txt > Title: Adding x-namespace to relay descriptor for key:value pairs > Author: Virgil Griffith > Created: 2015-09-30 > Status: Open > > > 1. Motivation > We wish to allow developers to build new applications atop relays. Towards > this end, we wish to add the ability for users to specify arbitrary new > key-value entries under the "X-" namespace to the router descriptor [1]. > The canonical applications for this are adding a bitcoin donation address > and networking of tor2web nodes. > > > 2. Proposal > Allow optional key-value lines in the relay's torrc file. These lines will > be mirrored in the relay's descriptor which is then published in the > directory consensus. Do you mean in extra info area? All the best, Jacob ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal: Adding x-namespace to relay descriptor for key:value pairs
Yes I did. Here's the modified proposal. Filename: ExtraRelayDescriptorFields.txt Title: Adding X-namespace to extra-info descriptor for key:value pairs Author: Virgil Griffith Created: 2015-09-30 Status: Open 1. Motivation We wish to allow developers to build new applications atop relays. Towards this end, we wish to add the ability for users to specify arbitrary new key-value entries under the "X-" namespace to the extra-info descriptor. The canonical applications for this are adding a bitcoin donation address, networking of tor2web nodes, and display operator information on a Roster[1] page. 2. Proposal Allow optional key-value lines in the relay's torrc file. The following would be added to section 2.1.2 of the dir-spec [2] (Extra-info document format): "X-" CustomKey SP CustomValue NL CustomKeyChar = "a"-"z" / "0" - "9" / "-" / "_" CustomKey = 1*32 CustomKeyChar CustomValueChar = atext / specials CustomValue = 1*1024 CustomValueChar There can be multiple X-fields, for example... X-bitcoin 19mP9FKrXqL46Si58pHdhGKow88SUPy1V8 X-gravatar https://s.gravatar.com/avatar/d27fce46c9ac41a41bb52455ae75701d X-linkedin https://www.linkedin.com/in/virgilgr X-keybase http://fncuwbiisyh6ak3i.onion/virgil X-favoritequote Be excellent to each other. Party on dudes! X-foo bar The same CustomKey appearing more than once is disallowed. Possible values for CustomValueChar as specified per RFC 2822 sections 3.2.1 and 3.2.4 [3]. The sum size accounting for all such custom fields is truncated to 5 kilobytes. To mitigate the chance of a malformed torrc file, I additionally propose that the relay descriptor be scanned and if it does not match the specification, that it exit with error telling her torrc file is a likely culprit. References [1] http://tor-roster.org [2] https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n700 [3] https://www.ietf.org/rfc/rfc2822.txt On Wed, Sep 30, 2015 at 12:49 PM Virgil Griffith wrote: > Filename: ExtraRelayDescriptorFields.txt > Title: Adding x-namespace to relay descriptor for key:value pairs > Author: Virgil Griffith > Created: 2015-09-30 > Status: Open > > > 1. Motivation > We wish to allow developers to build new applications atop relays. Towards > this end, we wish to add the ability for users to specify arbitrary new > key-value entries under the "X-" namespace to the router descriptor [1]. > The canonical applications for this are adding a bitcoin donation address > and networking of tor2web nodes. > > > 2. Proposal > Allow optional key-value lines in the relay's torrc file. These lines > will be mirrored in the relay's descriptor which is then published in the > directory consensus. > > > The following would be added to section 2.1.2 of the dir-spec > (Extra-info document format): > > > "X-" CustomKey SP CustomValue NL > > CustomKeyChar = "a"-"z" / "0" - "9" / "-" / "_" > CustomKey = 1*32 CustomKeyChar > > CustomValueChar = atext / specials > CustomValue = 1*1024 CustomValueChar > > Custom fields can appear multiple times, for example... > > X-bitcoin 19mP9FKrXqL46Si58pHdhGKow88SUPy1V8 > X-gravatar https://s.gravatar.com/avatar/d27fce46c9ac41a41bb52455ae75701d > X-favoritequote Be excellent to each other. Party on dudes! > X-foo bar > > If the same CustomKey appearing more than once is disallowed. > Possible values for CustomValueChar as specified per RFC 2822. > > The sum size accounting for all such custom fields is truncated to 5 > kilobytes. > > > To mitigate the chance of a malformed torrc file, I additionally propose > that the relay descriptor be scanned and if it does not match the > specification, that it exit with error telling her torrc file is a likely > culprit. > > -V > ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Anycast Exits (related : Special-use-TLD support)
I have attached below the second half of the Special-Use TLD proposal that discusses how a local name service tool contact a peer-to-peer application running on an exist node There is nothing specific here to providing name services, any peer-to -peer application might potentially want an anycast style gateway from Tor to its own network. At the same time, this proposal is *very* hackish since Tor seems almost able to provide the same functionality with a judiciously chosen ExitPolicy lines, and a bunch of work on the application's side. And maybe that's really the right way to do it in the end. There is no discussion here of dealing with bad exit gateways for protocols that Tor does not even know about, presumably that requires some thought as well. I donno if Tor combats exits doing highly targeted DNS manipulation all that well either though. Jeff Filename: xxx-anycast-exit.txt Title: Anycast Exit Author: Jeffrey Burdges Created: 28 September 2015 Status: ? Implemented-In: ? Abstract Provide an anycast operation to help bootstrap Tor aware peer-to-peer applications. Overview Peer-to-peer protocols must define a method by which new peers locate the existing swarm, but available techniques remain rather messy. We propose that Tor provide an "anycast" facility by which peer-to -peer applications built on top of Tor can easily find their peers using the full aka useless descriptors. Server Side We propose an AnycastExit Tor configuration option AnycastExit : Here protocol must be a string consisting of letters, numbers, and underscores. There are two changes Tor's behavior resulting from this option : First, Tor adds the line "ACE :" to the node's full descriptor. Second, Tor allows connections to ip:port as if the torrc contains : ExitPolicy allow: As ExitPolicyRejectPrivate defaults to 1, these policies should be allowed even if the ip lies in a range usually restricted. In particular localhost and 127.0.0.1 are potentially allowed. Client Side Users enable anycast usage by adding the configuration line FetchUselessDescriptors 1 Software queries the Anycast lines in the full descriptor by sending Tor control port the line : GETINFO anycast// This query returns 250-anycast//="..exit:" where is a node identity for a node whose full descriptor contains the line "Anycast :". After receiving such a query for anycast nodes supporting , Tor builds, and later maintains, a list of nodes whose full descriptor contains an "ACE .." line in lexicographic order according to . Tor returns the th node from this list. Also, if exceeds the number of nodes with Anycast lines, then an error is returned. Clients contact the anycast server ..exit on port . As AllowDotExit defaults to off, applications should use the Tor control port to request a circuit to that particular exit using MapAddress: MapAddress ..exit= After this, the peer-to-peer application can connect to over the Tor socks port. MapAddress usually produces a four hop circuit, but many peer-to-peer applications, including name service provides, can accept the small additional latency. Future Work Tor directory authorities could aggregate the lists of anycast supporting nodes, so that clients do not need to download the full descriptors. AnycastExit could support UNIX domain sockets. Hackish Alternative In principle, the ExitPolicy line produced by AnycastExit might suffice if both doing so bypasses ExitPolicyRejectPrivate, and the port could identify the protocol. Additionally, an application could parse the cached-descriptors* files themselves to locate exits with the desired exit policies. Acknowledgments Based on discussions with George Kadianakis, Christian Grothoff. Indirectly based on discussions between Christian Grothoff and Jacob Appelbaum about accessing the GNU Name System over Tor. signature.asc Description: This is a digitally signed message part ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Approaching Deadline: ICGCTI2015 - Malaysia
The Third International Conference on Green Computing, Technology and Innovation (ICGCTI2015) Universiti Putra Malaysia, Selangor, Malaysia 8-10 December 2015 http://sdiwc.net/conferences/icgcti2015/ The proposed conference on the above theme will held over three days, with presentations delivered by researchers from the international community, including presentations from keynote speakers and state-of-the-art lectures. ICGCTI2015 aims to enable researchers build connections between different digital applications. The conference welcomes papers on the following (but not limited to) research topics: - Benefits of, and barriers to, adopting greener IT practices - Carbon metering and user feedback - Climate and ecosystem monitoring - Energy harvesting, storage, and recycling - Energy-aware high performance computing and applications - Energy-aware software - Energy-efficient network services and operations - Green IT metrics, maturity models, standards, and regulations - Green computing models, methodologies and paradigms - Green networking and communication - Life-cycle analysis of IT equipment - Management and profiling tools for energy efficient systems - Modeling-representations, simulation and validation for energy consumption optimization problems - Online dynamic optimization for energy efficient systems - Power-aware algorithms and protocols - Power-efficient delivery and cooling - Renewable energy models and prediction - Smart buildings and urban development - Smart homes, buildings, offices, streets - Stability of smart energy systems - Using IT to reduce carbon emissions - Carbon management policies and ecology- related issues with ICT - Characterization, metrics, and modeling - Creating green awareness using IT - Energy-aware computing - Energy-aware large scale distributed systems, such as Grids, Clouds and service computing - Energy-efficient mass data storage and processing - Governments’ roles in fostering and enforcing green initiatives - Green business process reengineering and management - Green design, manufacture, use, disposal, and recycling of computers and communication systems - Green software engineering - Low-power electronics and systems - Matching energy supply and demand - Network design optimization - Optimization of energy-efficient protocols - Power-aware software and hardware - Reliability, thermal behavior and control - Robustness and performance guarantees - Smart grid and microgrids - Smart transportation and manufacturing - Sustainable computing Researchers are encouraged to submit their work electronically. All papers will be fully refereed by a minimum of two specialized referees. Before final acceptance, all referees comments must be considered. Important Dates == Submission Deadline : November 8, 2015 Final Notification : November 20, 2015 Camera Ready Deadline : November 28, 2015 Registration Deadline : November 28, 2015 Conference Dates: December 8-10, 2015 Drop us an email at icgct...@sdiwc.net ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Anycast Exits (related : Special-use-TLD support)
> On 30 Sep 2015, at 14:53, Jeff Burdges wrote: > ... > Filename: xxx-anycast-exit.txt > Title: Anycast Exit > Author: Jeffrey Burdges > Created: 28 September 2015 > Status: ? > Implemented-In: ? > … > Server Side > > We propose an AnycastExit Tor configuration option > >AnycastExit : > > Here protocol must be a string consisting of letters, numbers, and > underscores. > > There are two changes Tor's behavior resulting from this option : > > First, Tor adds the line "ACE :" to the node's > full descriptor. > > Second, Tor allows connections to ip:port as if the torrc contains : >ExitPolicy allow: > As ExitPolicyRejectPrivate defaults to 1, these policies should be > allowed even if the ip lies in a range usually restricted. > In particular localhost and 127.0.0.1 are potentially allowed. Tor exit policies don’t contain hostnames like “localhost", did you mean 127.0.0.0/8 and ::1? I am concerned about the security considerations of opening up local addresses, as local processes often trust connections from the local machine. Perhaps we could clarify it to say that only the specific port on 127.0.0.0/8 and ::1 is allowed? I also suggest that we specify the following rules based on the current (0.2.7.3) implementation of policies_parse_exit_policy_internal: * Block all IPv6 if IPv6Exit is 0 * If AnycastExit is set, allow 127.0.0.0/8:port and, if IPv6Exit is 1, [::1]:port * If ExitPolicyRejectPrivate is 1: * reject private addresses (0.0.0.0/8, 169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, 172.16.0.0/12, [::]/8, [fc00::]/7, [fe80::]/10, [fec0::]/10, [ff00::]/8, [::]/127) * reject relay’s configured IPv4 and IPv6 address * reject relay’s interfaces’ IPv4 and IPv6 addresses * Then add the default exit policy Regards Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F signature.asc Description: Message signed with OpenPGP using GPGMail ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Draft proposal -- no number yet: How to safely drop support for old clients.
> On 30 Sep 2015, at 12:01, Nick Mathewson wrote: > > [This isn't done yet, but I've shown it to enough people that I should > share it with the list.] > > 1. Introduction and background > > 1.1. Motivation > > Frequently, we find that very old versions of Tor should no longer be > supported on the network. To remove relays is easy enough: we > simply update the directory authorities to stop listing them. > > But to disable clients is harder. We can do as we did with v1 > directories in Tor 0.2.0.5-alpha, and make them know about no relays > at all. > > ... > 1.2. History > > Early versions of Tor checked the recommended-versions field in the > directory to see whether they should keep running. If they didn't > recognize their own version, they would stop (what?). > ... > 3.2. Disabling versions prior to 0.2.4.11-alpha. > > Starting in version Tor 0.2.4.11-alpha, we implemented proposal 214. > This proposal added a new link protocol, version 4. Earlier version versions > of Tor never support it; newer versions of Tor always advertise their > support for it. > > We could treat the presence of any link protocol of version 4 or > later as indicating the presence of 0.2.4.11-alpha. Upon receiving a version 3 or earlier as indicating the presence of a version less than 0.2.4.11-alpha. > connection from such a version, we could throttle the bytes we give > it, drop its circuits, or stop reading. > > [TODO: which?] Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F signature.asc Description: Message signed with OpenPGP using GPGMail ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Draft proposal -- no number yet: How to safely drop support for old clients.
Hi, comments inline. On 09/30/2015 12:01 PM, Nick Mathewson wrote: Early versions of Tor checked the recommended-versions field in the directory to see whether they should keep running. If they didn't recognize you did the thing where you To override this, a Tor instance may include a configuration option, "IgnoreDisabledVersion VERSION". It is an error to set this option when *not* running with a disabled version. This does not work unless the client already has a consensus that they have parsed so they know they're running a disabled version. I appreciate the sentiment here (don't allow people to blindly set the option), but I'm not sure it's worth it. 3.4. Disabling all versions that don't support this proposal We should allow 'n' (short for "node") as a synonym for 'r' in consensus documents. Later, if we want to disable all Tor versions before today, we can change the consensus algorithm so that the consensus (or perhaps only the microdesc consensus) is spelled with 'n' lines instead of 'r' lines. This will create a consensus which older clients and relays parse as having no nodes. Hrm. I'm not a fan of this, it seems like a pretty sad hack. I don't have an alternative ready unfortunately. 3.5. In the event of overzealous retries We need to be sure that client running versions from 0.2.1 through 0.2.6 respond gracefully to the responses above. In particular, we need to make sure that they don't continually retry the connections that fail in these ways: that would put a lot of extra load on the network. Above, I recommend stalling connections rather than just closing them. This may prevent the risk of retries, at the risk of using additional relay resources. Stalling is much less preferable than closing to me. We should actually do the analysis and do it well to know if we actually have to do it, imo. Cheers Sebastian ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Onion Services and NAT Punching
Hi All, Do you know a use case which needs Single Onion Services and NAT punching? We’re wondering if there are mobile or desktop applications / services that would use a single onion service for the performance benefits, but still need NAT punching. (And don’t need the anonymity of a hidden service.) Single Onion Services: * can’t do NAT punching, (they need an ORPort on a publicly accessible IP address), * locations are easier to discover, and * have lower latency. Hidden Services: * can do NAT punching, * locations are hard to discover, and * have higher latency. Are there any use cases that: * need NAT punching, * don’t need service location anonymity, and * would benefit from lower latency? Thanks Tim Tim Wilson-Brown (teor) teor2345 at gmail dot com PGP 968F094B teor at blah dot im OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F signature.asc Description: Message signed with OpenPGP using GPGMail ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Proposal: Load-balancing hidden services by splitting introduction from rendezvous
Hey all, I'd like your thoughts and comments on this proposal. Tom PS: If you want to deliver them in person, I'm in Berlin. Filename: xxx-intro-rendezvous-controlsocket.txt Title: Load-balancing hidden services by splitting introduction from rendezvous Author: Tom van der Woerdt Created: 2015-09-30 Status: draft 1. Overview and motivation To address scaling concerns with the onion web, we want to be able to spread the load of hidden services across multiple machines. OnionBalance is a great stab at this, and it can currently give us 60x the capacity by publishing 6 separate descriptors, each with 10 introduction points, but more is better. This proposal aims to address hidden service scaling up to a point where we can handle millions of concurrent connections. The basic idea involves splitting the 'introduce' from the 'rendezvous', in the tor implementation, and adding new events and commands to the control specification to allow intercepting introductions and transmitting them to different nodes, which will then take care of the actual rendezvous. External controller code could relay the data to another node or a pool of nodes, all which are run by the hidden service operator, effectively distributing the load of hidden services over multiple processes. By cleverly utilizing the current descriptor methods, we could publish up to sixty unique introduction points, which could translate to many thousands of parallel tor workers. This should allow hidden services to go multi-threaded, with a few small changes. 2. Specification We propose two additions to the control specification, of which one is an event and the other is a new command. We also introduce a new configuration option. 2.1. DisableAutomaticRendezvous configuration option The syntax is: "DisableAutomaticRendezvous" SP [1|0] CRLF This configuration option is defined to be a boolean toggle which, if set, stops the tor implementation from automatically doing a rendezvous when an INTRODUCE2 cell is received. Instead, an event will be sent to the controllers. If no controllers are present, the introduction cell should be dropped, as acting on it instead of dropping it could open a window for a DoS. For security reasons, the configuration should be made available only in the configuration files, and not as an option settable by the controller. 2.2. The "INTRODUCE" event The syntax is: "650" SP "INTRODUCE" SP RendezvousData CRLF RendezvousData = implementation-specific, but must not contain whitespace, must only contain human-readable characters, and should be no longer than 512 bytes The INTRODUCE event should contain sufficient data to allow continuing the rendezvous from another Tor instance. The exact format is left unspecified and left up to the implementation. From this follows that only matching versions can be used safely to coordinate the rendezvous of hidden service connections. 2.3. "PERFORM-RENDEZVOUS" command The syntax is: "PERFORM-RENDEZVOUS" SP RendezvousData CRLF This command allows a controller to perform a rendezvous using data received through an INTRODUCE event. The format of RendezvousData is not specified other than that it must not contain whitespace, and should be no longer than 512 bytes. 3. Compatibility and security The implementation of these methods should, ideally, not change anything in the network, and all control changes are opt-in, so this proposal is fully backwards compatible. Controllers handling this data must be careful to not leak rendezvous data to untrusted parties, as it could be used to intercept and manipulate hidden services traffic. 4. Example Let's take an example where a client (Alice) tries to contact Bob's hidden service. To do this, Bob follows the normal hidden service specification, except he sets up ten servers to do this. One of these publishes the descriptor, the others have this desabled. When the INTRODUCE2 cell arrives at the node which published the descriptor, it does not immediately try to perform the rendezvous, but instead outputs this to the controller. Through an out-of-band process this message is relayed to a controller of another node of Bob's, and this transmits the "PERFORM-RENDEZVOUS" command to that node. This node finally performs the rendezvous, and will continue to serve data to Alice, whose client will now not have to talk to the introduction point anymore. 5. Other considerations We have left the actual format of the rendezvous data in the control protocol unspecified, so that controllers do not need to worry about the various types of hidden service connections, most notably proposal 224. The decision to not implement the actual cell relaying in the tor implementation itself was taken to allow more advanced configurations, and to leave the actual load-balancing algorithm to the implementor of the controller. The developer of the tor implementation should not have to choos
[tor-dev] Rate limiting of a hidden service
I developed a scheme to rate limit hidden services using proof-of-work; https://gist.github.com/evandentremont/a3ad12a5cc3a924dae34 The server sends a semi-prime to the client, which then factors it. The client submits the factored primes back with the next request. The 'rate' can be throttled by sending a larger or smaller semiprime. The client has to spend time factoring that number, and the request can simply be dropped if the factored primes aren't correct. It would be effective to hinder brute force attacks on a login screen at the very least. Running as a script on the page isn't ideal as a lot of people disable javascript. There's always the option for a fallback where you calculate the primes yourself and submit them, but I feel like it would be better implemented as part of tor itself. Just throwing this out there for thoughts / feedback / opinions on rate-limiting hidden services. ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal: Load-balancing hidden services by splitting introduction from rendezvous
> On 30 Sep 2015, at 17:27, Tom van der Woerdt wrote: > > ... > > Filename: xxx-intro-rendezvous-controlsocket.txt > Title: Load-balancing hidden services by splitting introduction from > rendezvous > Author: Tom van der Woerdt > Created: 2015-09-30 > Status: draft > > 1. Overview and motivation > > To address scaling concerns with the onion web, we want to be able to > spread the load of hidden services across multiple machines. > OnionBalance is a great stab at this, and it can currently give us 60x > the capacity by publishing 6 separate descriptors, each with 10 > introduction points, but more is better. This proposal aims to address > hidden service scaling up to a point where we can handle millions of > concurrent connections. > > The basic idea involves splitting the 'introduce' from the > 'rendezvous', in the tor implementation, and adding new events and > commands to the control specification to allow intercepting > introductions and transmitting them to different nodes, which will then > take care of the actual rendezvous. > … > 2.1. DisableAutomaticRendezvous configuration option > > The syntax is: >"DisableAutomaticRendezvous" SP [1|0] CRLF > > This configuration option is defined to be a boolean toggle which, if > set, stops the tor implementation from automatically doing a rendezvous > when an INTRODUCE2 cell is received. Instead, an event will be sent to > the controllers. If no controllers are present, the introduction cell > should be dropped, as acting on it instead of dropping it could open a > window for a DoS. > > For security reasons, the configuration should be made available only > in the configuration files, and not as an option settable by the > controller. I’m not sure it’s necessary to prevent the controller setting this option. We trust the controller, and might need it to be able to set this option for compatibility with ephemeral hidden services. What is the threat model where a controller could set this option, but not do things that are much worse? > 2.2. The "INTRODUCE" event > > The syntax is: >"650" SP "INTRODUCE" SP RendezvousData CRLF > >RendezvousData = implementation-specific, but must not contain > whitespace, must only contain human-readable > characters, and should be no longer than 512 bytes I don’t think 512 bytes is enough for the current implementation, I recommend at least 2048 bytes. (See below.) > The INTRODUCE event should contain sufficient data to allow continuing > the rendezvous from another Tor instance. The exact format is left > unspecified and left up to the implementation. From this follows that > only matching versions can be used safely to coordinate the rendezvous > of hidden service connections. I would appreciate a list of the data needed by the current version of the hidden service protocol to rendezvous, even if we don’t want to specify the exact format, or specify data items for future implementations. This helps ensure that the limits in the proposal are sane, and that the proposal doesn’t have any unexpected implementation issues. From reading rend_service_receive_introduction think the data is at least: * service_id - the hidden service address (16 base32 bytes) * intro_key - the introduction-point specific key (128 binary bytes, 171 base64 bytes) * request - the encrypted portion of the INTRODUCE2 cell (up to 476 binary bytes(?), 635 base64 bytes) Therefore, I think the minimum for the current hidden service implementation is around 830 bytes, at least if we want to offload the maximum processing to the rendezvous instances by sending the entire encrypted INTRODUCE2 cell. Therefore, I’d suggest that a limit of 2048 bytes is much more reasonable for future-proofing this proposal. It also looks like you might need to split rend_service_t into: * introduction point-specific data * rendezvous-specific data * shared data Does any data need to be shared, and, if so, how do you intend to keep the shared data synchronised? (Putting it in the RendezvousData each time might blow out the size considerably.) I’d also appreciate an example of which parts of rend_service_receive_introduction could be performed by each of the cooperating tor instances. I assume that sending the data “as early as possible” would offload the most processing to the rendezvous side. I think that the split could happen right before the decryption of the cell, at the lines: stage_descr = "decryption"; /* Now try to decrypt it */ This would avoid having to share the intro point encrypted replay cache (intro_point->accepted_intro_rsa_parts), but there’s still the hidden service Diffie-Hellman handshake cache (service->accepted_intro_dh_parts). If we don’t share that: * two backend instances could accidentally compete for the same rendezvous point if the client times out * a client could more easily DoS the hidden service by using the same Diffie-Hellman handshake We’d
Re: [tor-dev] Anycast Exits (related : Special-use-TLD support)
On Wed, 2015-09-30 at 15:39 +0200, Tim Wilson-Brown - teor wrote: > > First, Tor adds the line "ACE :" to the > > node's > > Second, Tor allows connections to ip:port as if the torrc contains > > : > >ExitPolicy allow: > > As ExitPolicyRejectPrivate defaults to 1, these policies should be > > allowed even if the ip lies in a range usually restricted. > > In particular localhost and 127.0.0.1 are potentially allowed. > Tor exit policies don’t contain hostnames like “localhost", did you > mean 127.0.0.0/8 and ::1? > > I am concerned about the security considerations of opening up local > addresses, as local processes often trust connections from the local > machine. Perhaps we could clarify it to say that only the specific > port on 127.0.0.0/8 and ::1 is allowed? Yes, that's the effect of the ExitPolicy line described. We should not disable ExitPolicyRejectPrivate, merely ensure that the new exit policy be processed before it. I'll add some language to clarify, slightly. I'm futzing around to make sure that just an ExitPolicy line does this already too. Jeff signature.asc Description: This is a digitally signed message part ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Draft proposal -- no number yet: How to safely drop support for old clients.
Hi, Nick Mathewson: Draft proposal -- no number yet: How to safely drop support for old clients. I feel like "safely" sounds too fluffy and not reflective of the proposal. - How to force-drop support for old clients. or - How to depreciate support for old clients. The second of these seems the most suitable. Frequently, we find that very old versions of Tor should no longer be supported on the network. Where can we find research on the impact? 3.4. Disabling all versions that don't support this proposal With all due respect, doesn't Microsoft do stuff like this? Is the impact so large that they require this level of action? if we want to disable all Tor versions before today that do not support this proposal. Is the proposal for 5 years in the past, pre this version, or can/will the cutoff be specified willy-nilly? It might be good to make any disable-switches advisory rather than mandatory. Ultimately, if this is the approach, and it is in the hands of each client operator, this could be good. Though it would be great to hear from those who use previous versions to thwart limitations in "upgrades" to Firefox, such as the way media is streamed, amongst other things :) Wordlife, Spencer ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Onion Services and NAT Punching
Hi All, Hi Tim! > Do you know a use case which needs Single Onion Services and NAT punching? chyaa! NAT has ruined the Internet, violates the end to end principal and make it more difficult to develop decentralized systems. *deep sigh*... Obviously, centralized systems design contributes to human rights violations... so I feel compelled to point out that if the Single Onion service does not provide NAT punching then this will discourage developers from building decentralized systems. pro-human-rights == decentralized NAT punching transport > We’re wondering if there are mobile or desktop applications / services that > would use a single onion service for the performance benefits, but still > need NAT punching. (And don’t need the anonymity of a hidden service.) I am currently working on Tor integration for Tahoe-LAFS and (yes, both of them) IPFS. I think many users would be interested in using a NAT-punching single onion service for this. This would allow slightly better performance for users hosting an IPFS or Tahoe-LAFS storage node at their home behind a NAT device. These tradeoffs you specified for single versus double onion services listed here in my NAT penetration tradeoffs chart here: https://github.com/telekommunisten/nat-penetration-tradeoffs I think developers of decentralized systems will get lots of benefit from using onion services as their NAT penetration method... instead of ICE, NAT-PMP, U-PnP... because it's reliability doesn't depend on the quality of the NAT device and it will work on any network topology. It's true that TURN would work on all network topologies... but this is a central proxy server and therefore not freely available and brittle (it's a single point of failure). Feel free to make changes or corrections to this NAT penetration tradeoffs document, by sending a pull request. Soon I'll add to the reference links on the bottom prop224 and related onion service document links. cheers! David > Single Onion Services: > * can’t do NAT punching, (they need an ORPort on a publicly accessible IP > address), > * locations are easier to discover, and > * have lower latency. > > Hidden Services: > * can do NAT punching, > * locations are hard to discover, and > * have higher latency. > > Are there any use cases that: > * need NAT punching, > * don’t need service location anonymity, and > * would benefit from lower latency? > > Thanks > > Tim > > > Tim Wilson-Brown (teor) > > teor2345 at gmail dot com > PGP 968F094B > > teor at blah dot im > OTR CAD08081 9755866D 89E2A06F E3558B7F B5A9D14F > > > ___ > tor-dev mailing list > tor-dev@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev > ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Second release of OnioNS beta
Heads-up everyone, After some debugging with s7r, it appears that sometimes the programs (onions-server, onions-hs, or onions-client) fail to start because they cannot load the common library. So if you run any of the programs in the terminal, you will see an error that libonions-common.so cannot be found, or they simply won't run if you start them without an active terminal. This is because that library is installed to /usr/lib/onions-common, but the loader is looking for it in /usr/lib/, among other locations. I do set the RPATH in CMake in an attempt to fix this, but in some cases (at least in my build for Debian Wheezy) it may not be set or installed correctly. I'm investigating and looking through https://cmake.org/Wiki/CMake_RPATH_handling If this occurs, there are two workarounds. The first is to run export LD_LIBRARY_PATH="/usr/lib/onions-common" before running any of the software. The second is to add "/usr/lib/onions-common/" (without quotes) to /etc/ld.so.conf Both should tell the loader to look for the library in the correct path. This issue might also occur on Fedora under the for the experimental .rpm, but I'm not sure. The issue doesn't occur on my computer, but if it also applies to Ubuntu or Mint, please let me know. It's always possible to compile from source and run the software from the build directory, but I would really like to identify the scope of this issue and then properly fix it. Thanks, Jesse V. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Onion Services and NAT Punching
On Wed, 30 Sep 2015 17:12:53 +, Tim Wilson-Brown - teor wrote: ... > Are there any use cases that: > * need NAT punching, > * don???t need service location anonymity, and > * would benefit from lower latency? Of course. All the cases where you set up a hidden service exactly because your host is behing a NAT. Like the webcam raspi I'm just booting up. Andreas -- "Totally trivial. Famous last words." From: Linus Torvalds Date: Fri, 22 Jan 2010 07:29:21 -0800 ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Second release of OnioNS beta
Good news! I have identified the issue and fixed it. Turns out that the RPATH wasn't actually embedded in the binary, so the loader couldn't find the common library on most machines. Before: $ objdump -x /usr/bin/onions-client | grep RPATH $ After: $ objdump -x /usr/bin/onions-client | grep RPATH RPATH/usr/lib/onions-common $ I'm pretty sure that the issue affected all flavours of Linux, so I pushed out a quick update to the PPA and to Github. Hopefully it shouldn't be a problem anymore. Jesse V. On 09/30/2015 05:25 PM, Jesse V wrote: > After some debugging with s7r, it appears that sometimes the programs > (onions-server, onions-hs, or onions-client) fail to start because they > cannot load the common library. So if you run any of the programs in the > terminal, you will see an error that libonions-common.so cannot be found, or > they simply won't run if you start them without an active terminal. This is > because that library is installed to /usr/lib/onions-common, but the loader > is looking for it in /usr/lib/, among other locations. I do set the RPATH in > CMake in an attempt to fix this, but in some cases (at least in my build for > Debian Wheezy) it may not be set or installed correctly. I'm investigating > and looking through https://cmake.org/Wiki/CMake_RPATH_handling signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
