Re: [tor-dev] Proposal 246: Defending Against Guard Discovery Attacks using Vanguards
I agree with most of what you said regarding the threat of a targeted observer. What I disagree with is that an adversary running his own relays is less of a threat. Running relays is trivial, and running 5% of the guards is fairly cheap (I estimate ~$3000/month (please ask for details)). If you increase the number of third guards to, say, ten, with average expiration times of, say, 6 hours, then after only 24 hours an adversary with 5% of the bandwidth has an 87% chance of compromising at least one third guard. By reducing the number of third guards to 2 and increasing the expiration time to an average of 24 hours, then it takes 20 days until 87% chance of compromise is reached. Observing the network traffic of third parties is, it seems to me, actually much more difficult and excludes most adversaries except legal authorities. And legal authorities often do have to follow some legal procedures to observe network traffic, especially if it is being done between different legal jurisdictions. But ultimately the point of the quickly-rotating third guards is only to *force* the adversary to invest resources into running malicious relays. That is a different type of attack than just traffic observation, and requiring it may stymie some adversaries who don’t have those skills or processes in place. So more and more-quickly rotating third guards would serve that purpose just as well. If the consensus is that an adversary that can observe arbitrary third-party traffic within hours is more of a concern than an adversary that can run 1-5% of the vanguards, then very short-lived third guards would be a reasonable solution, in my opinion. I would not be part of that consensus, but figuring out the speed and size of the adversaries we care about is unfortunately educated guessing at this point. Best, Aaron > On Jul 17, 2015, at 8:11 PM, s7r wrote: > > Signed PGP part > On 7/18/2015 12:49 AM, A. Johnson wrote: > > > > Not having the third guards be selected by every second guard makes > > sense when you consider that the adversary may not be able to > > compromise all relays equally. That was not a consideration I had > > in mind when I argued for “completely connected” guard sets. The > > purpose of multiple guards in a given position is (i) to allow > > onion services that have unusually high load to uses the excess > > capacity of more than one relay, (ii) to lower the variance of > > relay performance, and (iii) to recover more quickly from nodes > > leaving or failing. Note how these are all related to performance. > > The drawbacks of multiple relays are all related to security, and > > they are (i) more relays gives the adversary more chances to have > > his relays selected, (ii) more relays gives the adversary more > > options and opportunities to compromise relays once they are > > observed. > > > > The best design for security purposes would be a single relay in > > each guard "set", with the final relay expiring the fastest. > > Currently, the suggestion was to keep one relay in the first guard > > position (or whatever NumEntryGuards is), because the security > > issue there is very serious (it can observe the onion service > > directly), and also because guards are already selected to have > > somewhat higher bandwidth and stability. Hopefully (currently > > unverified) this means that they have more *excess* capacity as > > well. > > > > If you don’t allow all second guards to connect to all third > > guards, then you have to make a performance / security tradeoff. > > For example, suppose you give each second guard its own set (or > > “bucket”) of third guards. You shouldn’t give each second guard the > > same number of guards that we had planned to have in the third > > guard set, because that only makes things worse. Now Instead of > > choosing one set (of size, say, six), you have for each second > > relay a new chance for the adversary to own one of those relays. > > There is only a benefit if you reduce the number. It makes sense to > > me to reduce that number to one, because there is no reason to > > expect the third guard to be a bottleneck before the second guard > > (unlike the first guards, which it sounds like may be the only ones > > requiring the Guard flag). You still have some of the benefit of > > reducing performance variance and improving failure recovery > > because you have multiple second guards. And you have gained the > > benefit that was Mike’s original motivation of giving the adversary > > that compromises a third guard a view of only one of the second > > guards, with the hope that maybe this one he finds difficult to > > compromise. > > > > There is an unanswered question of what to do with multiple first > > guards. The main reason I can see not to have single second guards > > for each first guard is that the first guard is likely to have more > > excess capacity and thus not be the bottleneck for high amounts of > > onion-service traffic. Assum
[tor-dev] Finding location metadata in large "dark market" datasets
Hello all, I came across a blog post that might interest you all. @techdad did a quick analysis of public images from online black markets (such as Silk Road et al)[2] from 2011-2015, and came to the following conclusion: "After parsing hundreds of thousands of images, I came across about 37 unique images that were not properly sanitized."[1] That's surprisingly low -- 0.00037% if one assumes 100k images analyzed. Given the number of high-profile cases [4] where this location information led to arrests, it's not very surprising that some people likely took the time to remove the EXIF data, but I'm curious whether a given website may have stripped the metadata for uploaded images. The images that tested positive are shown on the blog post, and 8/37 were clearly from the same individual. When mapped out, the location data is primarily in the US (5 locations), along with 1 location in France and Australia. Incidentally, the full 1.6TB dataset from 2011-2015 is available on the Internet Archive [3], just in case the Hacking Team disclosures haven't used up all your hard drive space. ;-) This data on its own is a rather interesting look into the workings of black markets -- many of which no longer exist. Curious to see what you all think and what analyses you'd like to see from this kind of data. best, Griffin [1] http://atechdad.com/Deanonymizing-Darknet-Data/ [2] http://www.gwern.net/Black-market%20archives [3] https://archive.org/details/dnmarchives [4] https://www.eff.org/deeplinks/2012/04/picture-worth-thousand-words-including-your-location ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal 246: Defending Against Guard Discovery Attacks using Vanguards
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 7/18/2015 12:49 AM, A. Johnson wrote: > > Not having the third guards be selected by every second guard makes > sense when you consider that the adversary may not be able to > compromise all relays equally. That was not a consideration I had > in mind when I argued for “completely connected” guard sets. The > purpose of multiple guards in a given position is (i) to allow > onion services that have unusually high load to uses the excess > capacity of more than one relay, (ii) to lower the variance of > relay performance, and (iii) to recover more quickly from nodes > leaving or failing. Note how these are all related to performance. > The drawbacks of multiple relays are all related to security, and > they are (i) more relays gives the adversary more chances to have > his relays selected, (ii) more relays gives the adversary more > options and opportunities to compromise relays once they are > observed. > > The best design for security purposes would be a single relay in > each guard "set", with the final relay expiring the fastest. > Currently, the suggestion was to keep one relay in the first guard > position (or whatever NumEntryGuards is), because the security > issue there is very serious (it can observe the onion service > directly), and also because guards are already selected to have > somewhat higher bandwidth and stability. Hopefully (currently > unverified) this means that they have more *excess* capacity as > well. > > If you don’t allow all second guards to connect to all third > guards, then you have to make a performance / security tradeoff. > For example, suppose you give each second guard its own set (or > “bucket”) of third guards. You shouldn’t give each second guard the > same number of guards that we had planned to have in the third > guard set, because that only makes things worse. Now Instead of > choosing one set (of size, say, six), you have for each second > relay a new chance for the adversary to own one of those relays. > There is only a benefit if you reduce the number. It makes sense to > me to reduce that number to one, because there is no reason to > expect the third guard to be a bottleneck before the second guard > (unlike the first guards, which it sounds like may be the only ones > requiring the Guard flag). You still have some of the benefit of > reducing performance variance and improving failure recovery > because you have multiple second guards. And you have gained the > benefit that was Mike’s original motivation of giving the adversary > that compromises a third guard a view of only one of the second > guards, with the hope that maybe this one he finds difficult to > compromise. > > There is an unanswered question of what to do with multiple first > guards. The main reason I can see not to have single second guards > for each first guard is that the first guard is likely to have more > excess capacity and thus not be the bottleneck for high amounts of > onion-service traffic. Assuming that's true, then I could imagine > having separate second-guard buckets for each first guard as well. > However, there is a bad multiplier effect here. For example, with > three first guards and three second guards per first guard, there > are now *nine* relays in the second-guard position, each of which > presents a chance for a malicious relay to be chosen. > > My current thought is that the following is a better design than > the one currently in the proposal: 1. One first guard 2. Two second > guards per first guard (so two unless NumEntryGuards is changed) 3. > One third guard per second guard (so two unless NumEntryGuards is > changed) This will only perform worse than the design suggested in > the proposal (which is one first guard, two second guards, and six > third guards, with all guards connected to all following guards). I > think it might not much worse, though. > > And I agree with the suggestions to randomize the expiration > times. > > Best, Aaron Agreed. The probability to run into a compromised relay will decrease dramatically if we use fewer second and third guards. However, this will put a penalty over performance, when talking about a popular hidden service. I still see the third hop (speaking from hidden service server start point) is the weak part here. An attacker can connect to a hidden service at his malicious relay selected as rendezvous. Before you know it, all relays in third_guard_set are enumerated by the attacker. This is why I think it's better to have a bigger value for NUM_THIRD_GUARDS and a shorter period for THIRD_GUARD_ROTATION. Here's why I think so: Let's assume a journalist has published some documents offered by a whistleblower regarding the abuses done by "Malicious-ISP" and made them available via a hidden service to remain anonymous, until authorities react to apply the law and punish "Malicious-ISP". Let's assume "Malicious-ISP" is a multinational telecom company with branches
Re: [tor-dev] Proposal: Merging Hidden Service Directories and Introduction Points
> This proposal doubles the default number of IPs and reduces the "cost" > of being an IP since the probability of being selected is no longer > bandwidth-weighted. Is this a fair tradeoff for the performance > improvement? That seems easy to fix. Make the number of Introduction Points the same as it was before and make them be selected in a bandwidth-weight way. There is no cost to this. You need IPs to be online, and so whatever number was used in the past will yield the same availability now. And bandwidth-weighting should actually improve both performance and security. Aaron ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] Proposal 246: Defending Against Guard Discovery Attacks using Vanguards
>> Here's another crazy idea that would potentially bring this Vanguards >> idea closer to "Virtual Circuits": What if you divided your third-level >> Vanguards into NUM_SECOND_GUARDS isolated buckets, and mapped exactly >> one these buckets to each of your second-level guards? ... >> That way, if one third-level guard was compromised via a successful >> Sybil attack, the adversary would learn at most 1 of your second-level >> guards, instead of learning all of them and then getting to take their >> pick which one(s) they want to try to compromise. > > Neat idea! Concept is interesting and makes sense, since we need to > protect second_guard_set for way longer period of time, and > first_guard_set for even longer. Not having the third guards be selected by every second guard makes sense when you consider that the adversary may not be able to compromise all relays equally. That was not a consideration I had in mind when I argued for “completely connected” guard sets. The purpose of multiple guards in a given position is (i) to allow onion services that have unusually high load to uses the excess capacity of more than one relay, (ii) to lower the variance of relay performance, and (iii) to recover more quickly from nodes leaving or failing. Note how these are all related to performance. The drawbacks of multiple relays are all related to security, and they are (i) more relays gives the adversary more chances to have his relays selected, (ii) more relays gives the adversary more options and opportunities to compromise relays once they are observed. The best design for security purposes would be a single relay in each guard "set", with the final relay expiring the fastest. Currently, the suggestion was to keep one relay in the first guard position (or whatever NumEntryGuards is), because the security issue there is very serious (it can observe the onion service directly), and also because guards are already selected to have somewhat higher bandwidth and stability. Hopefully (currently unverified) this means that they have more *excess* capacity as well. If you don’t allow all second guards to connect to all third guards, then you have to make a performance / security tradeoff. For example, suppose you give each second guard its own set (or “bucket”) of third guards. You shouldn’t give each second guard the same number of guards that we had planned to have in the third guard set, because that only makes things worse. Now Instead of choosing one set (of size, say, six), you have for each second relay a new chance for the adversary to own one of those relays. There is only a benefit if you reduce the number. It makes sense to me to reduce that number to one, because there is no reason to expect the third guard to be a bottleneck before the second guard (unlike the first guards, which it sounds like may be the only ones requiring the Guard flag). You still have some of the benefit of reducing performance variance and improving failure recovery because you have multiple second guards. And you have gained the benefit that was Mike’s original motivation of giving the adversary that compromises a third guard a view of only one of the second guards, with the hope that maybe this one he finds difficult to compromise. There is an unanswered question of what to do with multiple first guards. The main reason I can see not to have single second guards for each first guard is that the first guard is likely to have more excess capacity and thus not be the bottleneck for high amounts of onion-service traffic. Assuming that's true, then I could imagine having separate second-guard buckets for each first guard as well. However, there is a bad multiplier effect here. For example, with three first guards and three second guards per first guard, there are now *nine* relays in the second-guard position, each of which presents a chance for a malicious relay to be chosen. My current thought is that the following is a better design than the one currently in the proposal: 1. One first guard 2. Two second guards per first guard (so two unless NumEntryGuards is changed) 3. One third guard per second guard (so two unless NumEntryGuards is changed) This will only perform worse than the design suggested in the proposal (which is one first guard, two second guards, and six third guards, with all guards connected to all following guards). I think it might not much worse, though. And I agree with the suggestions to randomize the expiration times. Best, Aaron ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
[tor-dev] 4th status report for OnioNS
Hello again everyone, This report covers the period of time that I spent in Washington, DC at the hidden service meetings. I made excellent progress on this project. On the 10th, for example, I pushed nearly 30 commits. I fixed many significant bugs and improved many areas that would make the software easier to understand and to configure. * Created a separate and fully-fledged manpage for the hidden service, server, and client aspects of the software. * Client-side and HS-side are now complete and working reliably. The hidden service can generate a Record (a claim on a domain name) and transmit it over a Tor circuit to a remote server. The client now has a binary that launches the Tor executable, the onions-client binary, and a Stem script when the Tor Browser opens, and can now shut them all down properly now. * Fixed a show-stopping bug that prevented the Stem script from launching automatically with the Tor Browser. (George, this bug was why I had to launch Stem manually when I showed the software to you.) The error thrown for this bug was large and unhelpful, but the fix was amusingly simple: https://github.com/Jesse-V/OnioNS-client/commit/9e80691b02e2a843b26ac21b6b0b88daf6840667 * Many bug fixes and enhancement updates, including some points of confusion. * Improved flexibility for hosting a server, including the capability to bind to a custom TCP port. * Finalized event logging and a flag to specify where the log goes. Logs on the client-side now go in an OnioNS folder inside the Tor Browser directory. * Migrated flag-parsing from the tclap library I found long ago on SourceForge to Unix popt. * Added a system installation of Tor as a dependency for the hidden service code. * Minor code cleanup. Global sources of randomness were also discussed at the hidden service meetings. I need a source of timestamped archivable randomness that the whole network agrees on in order to securely build OnioNS into a distributed system. The global randomness also has implications for the next generation of hidden service protocols, so there was an overall need to get something together. We made good progress towards this goal. I was able to show Roger and George a demonstration of OnioNS. I registered example.tor and arma.example.tor, pointed example.tor at a HS that I was hosting on my laptop and arma.example.tor at Roger's hidden service (duskgytldkxiuqc6.onion) and uploaded the Record over a Tor circuit to a remote machine, Server A. I then manually transfered the JSON-encoded data structures from Server A to Server B, another machine. Then I launched the Tor Browser (with my binary substitution in place) so that all of the client-side programs were running. I entered "example.tor" into the Tor Browser and as expected arrived at my hidden service, and then typed arma.example.tor into the browser and as expected loaded duskgytldkxiuqc6.onion. This was a great test. The server-to-server communication needs a few bug fixes, but most of that code is in place. As soon as that is complete, I should be about ready for a beta test. Jesse V. signature.asc Description: OpenPGP digital signature ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
Re: [tor-dev] [Patch] or/config.c for MSVC
Nick Mathewson wrote: > I made the changes conditional on not having GCC, since the GCC syntax will work with older versions of GCC. (Somebody should check whether we care about those versions.) I saw that; from the Changelog: o Minor features (portability): - Use C99 variadic macros when the compiler is not GCC. This avoids failing compilations on MSVC, and fixes a log-file-based race condition in our old workarounds. Original patch from Gisle Vanem. I don't know what the "race conditions" is about, but this (?) change broke MSVC again by a '__PRETTY_FUNCTION__' which MSVC doesn't have. I suggest you add a: #define __PRETTY_FUNCTION__ __FUNCTION__ to win32/orconfig.h. Trying to find the change under https://trac.torproject.org/projects/tor/browse proved impossible. That site is so slow so I add some more MSVC-porting issues here: 1) Base-types like 'uint8_t' have been part of MSVC's since ver. 1600. So this patch was needed here: --- a/src/ext/trunnel/trunnel-impl.h2015-06-04 19:32:02 + +++ b/ext/trunnel/trunnel-impl.h 2015-06-04 19:49:06 + @@ -18,7 +18,7 @@ #include "trunnel-local.h" #endif -#ifdef _MSC_VER +#if defined(_MSC_VER) && (_MSC_VER < 1600) #define uint8_t unsigned char #define uint16_t unsigned short #define uint32_t unsigned int 2) MSVC have never had : --- a/src/test/test_checkdir.c 2015-03-03 20:36:19 + +++ b/src/test/test_checkdir.c2015-03-04 15:29:02 + @@ -3,7 +3,11 @@ #include "orconfig.h" #include "or.h" + +#ifndef _MSC_VER #include +#endif + #include "config.h" #include "test.h" #include "util.h" -- --gv ___ tor-dev mailing list tor-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
