[tor-dev] Summer of Privacy applicant, Multi-threading
Hello, I see several folks doing brief intros here so I'll do so as well. Most of my application is in already, just finishing up the patch. I'm a Mathematics undergrad, 3rd year come Fall. I've used multi-threading in coursework and in an internship I did at JPL/NASA. I want to expand the multi-threading in Tor this summer. For my patch I'm working on #3569 (Refactor socks parsing). It won't be a complete implementation of that bug this week, but I aim to complete the first part of it, ie. the refactoring internal to the function to expose the state model clearly in code structure. To that end I have one quick code style question: Is clarity preferred over the (very short) body of a conditional only occurring in one place? I'm encountering at least once instance where the choice is between repeating a (very short) code block, thereby introducing the potential that one section of it may in the future be edited without the other receiving the same edit (even though they're right next to each other), or retaining a gnarly conditional that took a truth table for me to be sure I had the right end of. If code occurring exactly once is a priority, the gnarliness can be repaired with an explanatory comment, but I'd like to know which is preferred. Thanks, -- Kim ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] TOR SoP proposal: extending and improving TOR network anomaly detection
On Tue, Apr 14, 2015 at 01:38:54AM -0400, Kibo Schaffer wrote: I want to improve TOR's ability to detect anomalies such as sybil attacks, and make it easy to include other heuristics for other potential attacks. When a potential attack is detected, users and maintainers are notified (as necessary). There has been research and development with this field with TorDoctor, exitmap, and HoneyConnector. However, as far as I am aware, these projects could use some help being solidified and integrated into TOR. What do you mean by solidified and integrated into TOR? Tor, the network or tor, the C program? exitmap (and I think Doctor and HoneyConnector too) is meant to be a stand-alone tool that only uses the Tor network as a client. And do you already have some concrete ideas about detecting anomalies? It's an interesting topic, but also a theory-heavy one. If we don't have good ideas about concrete things to work on, we can easily spend all three months researching, which is not quite what TSoC is about. While I'm currently working on Sybil attack detection [0], and more broadly anomaly detection, we are still mostly in the process of working out the theory. There might be, however, ways to extend exitmap and add new modules to it, which is mostly programming. The GitHub issue tracker lists two of them [1]. [0] http://notebooks.nymity.ch/detecting_sybils.html [1] https://github.com/NullHypothesis/exitmap/issues Cheers, Philipp ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] TSOP proposal
Hey Remi, It's great to see someone else interested in adversarial stylometry! I have been working on a project quite similar to your proposal since January as my undergraduate thesis. I am considering applying for TSoP in order to fund the continued development of the project, but I think the mentor issue is a strong indication that we should focus our efforts on something more related to Tor. Since I have made considerable progress implementing the tool you described, perhaps we should instead combine our efforts? Rachel Greenstadt, whose work I am sure you are familiar with, also expressed interest in collaboration when I announced this project in December. You will find that I have implemented many of your proposed features in my project on GitHub. Near-term work involves making the project easier to install and implementing Writeprint feature set extractors. If you manage to get it running (for now, you will have to install dependencies manually), I would love to hear some feedback from you! Until then, see my thesis. Contribute at: www.github.com/pagea/unstyle A preliminary version of my thesis can be found below. I will provide a more permanent URL on GitHub in the future. https://www.scribd.com/doc/261957061/Unstyle-A-Tool-for-Evading-Authorship-Attribution Full disclosure: I am applying to TSoP this year, and I am still strongly considering proposing the continued development of Unstyle, in addition to a few other potential applications. Cheers, Alden Page ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Summer of Privacy application, Censorship Analyzer
On Tue, Apr 14, 2015 at 11:56:12AM +0200, Miquel Llobet wrote: As far as coding goes, I played a bit with OONI (did a scan, turns out I'm clean :-) ). and built it from source. What bugs to you recommend to work on as a start? Ideally I can write a patch before the submission is due to attest my skills. Let me know if you have any suggestion or advice on pursuing this project. Hi Miquel, Cool, thanks for your interest! You could find an interesting bug under the OONI component [0]. I'm also copying Arturo, OONI's maintainer, who might have a better tip. Also, do you have some experience with asynchronous programming in general, and Twisted in particular? I'm asking because several students have tried to work on this project in the past and nobody has delivered much code so far. I think that many had issues with learning asynchronous programming and getting familiar with OONI's code base. [0] https://trac.torproject.org/projects/tor/query?status=acceptedstatus=assignedstatus=needs_informationstatus=needs_reviewstatus=needs_revisionstatus=newstatus=reopenedcomponent=Ooniorder=priority Cheers, Philipp ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Tor Summer of Privacy deadline is soon!
Hi all, just a quick friendly heads up that the deadline is coming up soon. If you've been thinking of making an application then now is the time! Application deadline for students is April 17th (end-of-day everywhere [1]). NO LATE APPLICATIONS WILL BE CONSIDERED. We're sorry about needing to be strict on this, but we have a really tight deadline for making selection decisions. Late applications make the process harder, so the only fair thing is to not allow any exceptions. Cheers! -Damian [1] https://en.wikipedia.org/wiki/Anywhere_on_Earth ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] onionoo: historic details.json data
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 By the way, are you aware of Philipp Winter's work on a better Sybil attack detector? If you mean http://notebooks.nymity.ch/detecting_sybils.html then yes, I've seen it. -BEGIN PGP SIGNATURE- iQIcBAEBCgAGBQJVLrcMAAoJEFv7XvVCELh0Q5UQAIV4qIEADzwVNexSCs5JRun5 swV1EKq+OZtQ4BWf2kxGN388zcZvgVGsjSuj4sBYcTFlzgJ5Rx4A6kqMZ+XGA0x5 7FJ7x61rQ/lrvz9HxrFGnKjU28YNhxAkaumrK2690/+ecpk3Wdo0J+sx3nZLRurc 5CpyGmnWv2ArJhXfGbZ3cF5wvU9mYV1hAx4aoXLCbDVOk274zkyb/XA1gTUrJX2d 5BEzbsfDBbiQRIhCzoZNUIX7krZEp/Wi05MqdOoDXmuHRJ6LoGPvp3NPQnoEizaN zUDp/pNaNv1zV8ygTbAueCBLafbYPZc6xfwreI/bAjfLeJST38ABGTwtydsW3VPd UGEcuct8F8G2xq30fGDu6xCgLUhhN9tef41fpiDRRchnCoiDCZwts0rgPywlZyEp 2sWmlkOy/Dxq0GPi2owBIg9wwT8FYb4OwOe+6QiLsXkx1Xb9Adf75pPateNJ5XCB MEgVYl97mxyXonptbjVm3ToEEgljcROcz+7hhX/dD0mEinacB3t092nYHcfLFeq2 RHKEr1bpyjhX8dqjnXK4LrErhQHZlhEiQc+3PglPqq5Z9kLGOcNQowogB5KMVdj0 duBCyYtEDhUlVMCfzP5COLFkJuNfSThc1v+YO0rsnBfMjMX7RmPyCdvfODzbLCtM +yttIXDy9dB4GXHUFYdP =qb7i -END PGP SIGNATURE- ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Relay Database: Existing Schemas?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, I'm planing to store relay data in a database for analysis. I assume others have done so as well, so before going ahead and designing a db schema I'd like to make sure I didn't miss pre-existing db schemas one could build on. Data to be stored: - - (most) descriptor fields - - everything that onionoo provides in a details record (geoip, asn, rdns, tordnsel, cw, ...) - - historic records I didn't find something matching so far, so I'll go ahead, but if you know of other existing relay db schemas I'd like to hear about it. thanks, nusenu GSoC2013: Searchable Tor descriptor archive (Kostas Jakeliunas) https://www.google-melange.com/gsoc/project/details/google/gsoc2013/wfn/ 5866452879933440 https://lists.torproject.org/pipermail/tor-dev/2013-May/004923.html https://lists.torproject.org/pipermail/tor-dev/2013-September/005357.htm l https://github.com/wfn/torsearch (btw, someone knows the license of this?) This is true: the summary/details documents (just like in Onionoo proper) deal with the *last* known info about relays. ernie https://gitweb.torproject.org/metrics-db.git/plain/doc/manual.pdf (didn't find db/tordir.sql mentioned in the pdf) Instructions for setting up relay descriptor database https://lists.torproject.org/pipermail/tor-dev/2010-March/001783.html Set up descriptor database for other researchers https://trac.torproject.org/projects/tor/ticket/1643 -BEGIN PGP SIGNATURE- iQIcBAEBCgAGBQJVLrmVAAoJEFv7XvVCELh0+ycP/0VkRWMGAhx7YwWhixtOPB0q ouitosse3SdLCMd1W7cps7nnQhcGv8E1xeUPYeizNQ+RIK9ldvnwgxhVt9ETOI3e oL6bcx3B/NSbTcGfZhIU6XCCX7+39CXz3/+/gXg4c1a5Cd9rYKuwTn7ZApPUNAK2 9yfckCwHaqh67ZDHB1A7nrHX1BJAjwhri5JRGomHSUXPwNZOyMzpgPkB+I92esqc Z4ajQo2D+Vyk42qF7apqp5ApVBLdt5ycUtW1j4s4KvyJH2BjeP4tCXvvYLKv9IvN TGHMJ111NQXbuk6H1OdaDmMZhhW6utPk8YAbNc7RZHqCdH7No40zFVlikLDrV2Xl uLt8FMlck7hGQv/4udA59Dw3PGHfrSiCbvJXb0S2jarnWOH2O+zSSuMs8jQM/x7k 6tIOlwtRDVGw3VuvNbb4MJnLzdHRxq3qy6ueDYuhmhHslxjD1Cbr3eE1RA7Da9aX kNDgjfXtdah52HWUZbIIxHQYzyCg5G3M5yy51GhUlA2Fe1sbPpNcEBeyEDK/jH3Y +7G+tgR+CdMvSFraCiYKHa1drVPJHeqNvZAqPNHKxpKwOwOoJEYrsW4wb/mJ7ybe o8xNYK691hcaMvHqwI47tw4hZfrI0f0+gQKKzENQQk8/8yUZ/0jgMRlW/Cz0yYIa FZEg+OJ/yEs8vKfYerIN =Rfn4 -END PGP SIGNATURE- ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Summer of Privacy application, Censorship Analyzer
On Wed, Apr 15, 2015 at 6:01 PM, Philipp Winter p...@nymity.ch wrote: You could find an interesting bug under the OONI component [0]. I'm also copying Arturo, OONI's maintainer, who might have a better tip. Great thanks, I'll pick something from the list, hopefully Arturo can offer some guidance. Also, do you have some experience with asynchronous programming in general, and Twisted in particular? I have learned the fundamentals at school and done a bit of work with node.js so I'm familiar with the principles. As far as Twisted goes I did part of this [0] tutorial before and learned a lot, I was looking for some project to put it in practice, that's also why OONI caught my attention. [0] http://krondo.com/?page_id=1327 Miquel Llobet ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Draft of proposal Direct Onion Services: Fast-but-not-hidden services
On 10 April 2015 at 07:58, George Kadianakis desnac...@riseup.net wrote: One negative aspect of the above suggestions, is that if hidden services only listen for connections, then they lose their NAT-punching abilities. But I bet that this is not a problem for some use cases that would appreciate the corresponding performance boost. Theoretically, the above optimizations could also be optional. We should think more. I wobble back and forth on NAT-Punching for DOS (Direct Onion Services ;) ). On one hand it's awesome to not have to worry about NAT. On the other, if you're doing to run a DOS, presumably you are capable enough to either traverse it or not have to worry about it? Ultimately, I think I fall onto the side of wanting to keep the NAT-punching present. As we support more use cases for OSes (Onion Services ;) ) we will probably want to support people behind NATs but willing to compromise anonymity for performance. For example, I could easily envision someone being the DOS in an audio or video chat and being behind a NAT. The DOS end helps boost the performance, the client gets anonymity, everyone gets end-to-end protection. The difference between a DOS connecting to an IP and a DOS _being_ the IP seems small, as it's only used for connection setup. Obviously being the IP would be faster... perhaps the DOS could choose IPs that (it believes) it has a low latency connection to? (If that would be feasible with the information the daemon has available to it?) On 9 April 2015 at 22:33, Jacob H. Haven ja...@jhaven.me wrote: Removing rendezvous nodes is a bigger change to the protocol, but would be very useful. Why not enable the client to just directly establish circuit(s) directly to the onion service? As David pointed out, this would probably be implemented most cleanly with a tag in the HSDir descriptor (this would conveniently identify the service as non-hidden, whether that's a good or bad thing...) |direct onion service| - RP - client middle - client guard - |client| If the RP is removed and the client makes a direct connection to the DOS, the client is using a two-hop circuit, not a three-hop, right? If it's a three-hop circuit, it's no different (performance-wise) than using a RP, right? Another thought. If the DOS makes a direct connection to the HSDir for descriptor publishing the HSDir will be able to (passively) enumerate which HSes are DOSes, right? This seems like it would be something we would want to prevent. (Well, at least require the HSDir to go perform an active fingerprint to learn this information.) The use of three-hop circuits to publish this information is not strenuous either. -tom ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Should popularity-hiding be a security property of hidden services?
Hi George, Thanks for taking up the challenge I raised to you of coming up with use cases where leaking popularity is a threat. Perhaps others have suggested that we don't worry about popularity at all, but for the arguments I had been trying to make these are straw men. I don't suggest that we completely ignore popularity. As one simple example, if you monitored and published the popularity of onion services at the level of seconds or minutes (maybe even courser) adversaries could almost certainly construct practical intersection attacks on users of some onion services whose client-side traffic was being monitored. You noted anonymity is not binary, but you have only addressed popularity at a binary level: protect it or ignore it. We have an unfortunate tendency to sometimes do this in the Tor design community. For example, any design choice that partitions (or more generally statistically separates in any way) clients by the portions of the network about which they've been given information is not even worthy of consideration because partitioning is just bad. On the other hand, some pseudonymous profiling by exits is simply acceptable because of practicality considerations (and indeed, time to keep opening new connections on existing circuits has recently been significantly increased in Tor Browser Bundle for usability reasons---with a bit of discussion, but no significant analysis and no Tor Proposal). These are just single examples on each side for contrast, but others are easy to produce. I don't want to get into addressing the problem of this tendency in general here, I just want to make sure that we avoid specifically doing that for this problem. I think I mentioned to you previously the sorts of popularity statistics I would like to gather. But perhaps I was unclear. I'll set it out here publicly for others to comment on. Details might change, and of course we'd have to worry about particular protocols. That's no different than anything else in Torland. But I want to assume that something like the following is basically feasible. As an argument from authority, I talked to Aaron a bit about how you might do this and we were both convinced it should be feasible to do this securely. So, assume we have an onion service statistics gathering protocol that outputs say weekly the number of connections and bandwidth carried by the highest 5 percent, then 10 percent, then 20 percent, then 40 percent, then bottom 60 percent of onion services. I take it as given that these would be useful for many reasons, some of which you cited. We can revisit usefulness as needed. The question I would like to have answered is what sort of practical threat could be posed by leaks from this. One could imagine an active attacker that hugely fluctuates the volume of a given onion service to determine which bin it had been in assuming very generously that this isn't covered by noise of other onion services or a very long attack on a service whose volume does not otherwise change. These statistics are not a threat in the parkour example. They do not reveal historical volumes of individual onion sites. In the dystopian future scenario, the authorities know which hidden services are run by the rebels but not which ones are popular, and they want to take down the popular ones quickly since the revolution is imminent. If they happen to guess the right few they could inflate the activity (if they can access the onion site) and learn in a week that they were popular (assuming that they are lucky enough to be sure that, e.g., noise doesn't obscure that). This is a pretty iffy and low bang for the buck attack. As a contrasting example, authorities could easily locate the the guard(s) of targeted onion sites (we're assuming they can access targeted onionsites) via congestion attacks and then just monitor the network activity from the guard or its ISP to see the popularity of targeted onionsites in realtime. Not to mention deanonymizing anyone they are watching on the client side. This could be done faster, easier, and more productively than using the statistics. Tor is full of security vs. performance and/or efficiency and/or usability trade-offs. If we're going to rule out any onion service popularity statistics, I'd like some indication of a realistic potential threat. So far I don't feel I've heard that. aloha, Paul ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] TOR SoP proposal: extending and improving TOR network anomaly detection
Hi Philipp, Thanks for your reply. I mean Tor the network. Not integrated into the protocol itself. Sorry for the poor wording. So it would work as exitmap, HonerConnector, and TorDoctor. And do you already have some concrete ideas about detecting anomalies? It's an interesting topic, but also a theory-heavy one. If we don't have good ideas about concrete things to work on, we can easily spend all three months researching, which is not quite what TSoC is about. Agreed. I underestimated how much research it would take, and I haven't had the time this week to look more in-depth into pre-existing projects and research to really gauge this. Since the scale / shape of the project is currently incompatible with TSoP, I won't submit it (I could, but it doesn't make much sense). *However* I still want to contribute to this field, and I think I can look into getting my university to fund me for the summer instead, so I can work towards financial independence. I'll get back in touch soon once things settle down here. Cheers, Kibo On Wed, 15 Apr 2015 17:28:32 +0200 Philipp Winter p...@nymity.ch wrote: On Tue, Apr 14, 2015 at 01:38:54AM -0400, Kibo Schaffer wrote: I want to improve TOR's ability to detect anomalies such as sybil attacks, and make it easy to include other heuristics for other potential attacks. When a potential attack is detected, users and maintainers are notified (as necessary). There has been research and development with this field with TorDoctor, exitmap, and HoneyConnector. However, as far as I am aware, these projects could use some help being solidified and integrated into TOR. What do you mean by solidified and integrated into TOR? Tor, the network or tor, the C program? exitmap (and I think Doctor and HoneyConnector too) is meant to be a stand-alone tool that only uses the Tor network as a client. And do you already have some concrete ideas about detecting anomalies? It's an interesting topic, but also a theory-heavy one. If we don't have good ideas about concrete things to work on, we can easily spend all three months researching, which is not quite what TSoC is about. While I'm currently working on Sybil attack detection [0], and more broadly anomaly detection, we are still mostly in the process of working out the theory. There might be, however, ways to extend exitmap and add new modules to it, which is mostly programming. The GitHub issue tracker lists two of them [1]. [0] http://notebooks.nymity.ch/detecting_sybils.html [1] https://github.com/NullHypothesis/exitmap/issues Cheers, Philipp ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] Release: obfs4proxy-0.0.5
Hello all, I just tagged obfs4proxy-0.0.5, this time with improvements for both clients and servers. All users are recommended to upgrade. Special thanks to mvdan for various code cleanups. Tarball/Signature: https://people.torproject.org/~yawning/releases/obfs4proxy/obfs4proxy-0.0.5.tar.xz https://people.torproject.org/~yawning/releases/obfs4proxy/obfs4proxy-0.0.5.tar.xz.asc Changes in version 0.0.5 - 2015-04-15: - Go vet/fmt fixes, and misc. code cleanups. Patches by mvdan. - Changed the go.net import path to the new location (golang.org/x/net). - Added limited support for detecting if the parent process crashes. - Support for tor feature #15335 (stdin based termination notification). - Moved the leveled logging wrappers into common/log so they are usable in transport implementations. - Added a DEBUG log level. - Use a bundled SOCKS 5 server instead of goptlib's SocksListener. The big features are: - obfs4proxy now tries really hard to detect if the parent process has crashed, using various system specific hacks (and the new and improved tor assisted mechanism in #15335). This should reduce defunct obfs4proxy processes when tor happens to crash without tearing down the obfs4proxy instance. - Instead of using goptlib's SOCKS4a server, obfs4proxy now includes a SOCKS5 implementation, bringing IPv6 support to clients. Note that running IPv6 bridges has always worked (though dual stack configs are currently somewhat broken due to a tor PT configuration protocol limitation). Notes for packagers: - Like in obfs4proxy-0.0.5, one of the upstream dependency locations has changed. (code.google.com/p/go.net - golang.org/x/net). As far as I am aware, migrating to the new package immediately is not required though, should be done sooner rather than later due to the impending deprecation of code.google.com. To temporarily continue to build against the old go.net dependency, please revert: aed4b723891db1be34eb866a03c62806b58ac148 - It is *strongly* recommended that packages be built against goptlib-0.4 or newer to work around tor bug #15240. Without this workaround, certain bridges will fail to operate correctly when the ExtORPort is enabled (the Tor side fix is in tor-0.2.6.5-rc and newer). Questions, comments, feedback appreciated, -- Yawning Angel pgpyviXcC_LtQ.pgp Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
