VisiTor - or: a script to tell you how many of your users are probably Tor users
Hi everyone! We have heard from web server operators who are wondering how many of their visitors are using Tor to do so. We can now answer this question. We have an exit list archive back to February 2010 and a tool to compare an Apache HTTP server log to these lists. We can further guess how many of the requests come from Torbutton users by looking at the user-agent string. There's a lot of sensitive data involved here, which is why we give out the archived exit lists and our tool so that web server operators can run the comparison themselves. You'll find the exit list archive and the VisiTor tool here: https://metrics.torproject.org/data.html#exitlist https://metrics.torproject.org/tools.html#visitor If you are running a web server and want to help us make the tool better, please download the sources, review the 300+ lines of finest Java*, run the tool on your machine, and let us know how that works! Thanks, --Karsten *There will be a Python version once we're happy with the Java version. Want to help writing (and maintaining) a Python version? *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: blutmagie TNS / v0.2.2.15 nodes
Hi Olaf,
On 8/25/10 12:10 PM, Olaf Selke wrote:
blutmagie Tor network status site apparently displays incorrect
bandwidth values for all nodes running version 0.2.2.15. Unlike other
tns sites blutmagie calculated bw as an average from the extra-info data
instead of using the bw peak value. So far I don't have a clue what's
going wrong. The extra-info format might have changed or my Perl script
populating the mysql db might be buggy.
Blutmagie4 which is is running v0.2.2.14 for testing purpose still shows
up with the correct bw
http://torstatus.blutmagie.de/index.php?SR=BandwidthSO=Desc. All
0.2.2.15 nodes like trusted, teunTest, or the other three blutmagie
nodes are displayed with a bw being obviously much too low.
This might be related to:
Changes in version 0.2.2.15-alpha - 2010-08-18
- Relays report the number of bytes spent on answering directory
requests in extra-info descriptors similar to {read,write}-history.
Implements enhancement 1790.
There are now two new lines dirreq-read-history ... and
dirreq-write-history ... containing the bytes spent on the dir
protocol. Maybe TNS greps for read-history and not ^read-history
when parsing descriptors?
I'll have more time to investigate this tomorrow. Please let me know if
you find something interesting in the meantime.
Thanks,
--Karsten
***
To unsubscribe, send an e-mail to majord...@torproject.org with
unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: GeoIP database comparison
On 5/13/10 4:19 AM, grarpamp wrote: Wasn't there a user driven opensource geoip database project somewhere? Sortof like DynDNS, users go to the website, it pops up their ip address, they enter their location in the DB. Thought it had some advanced stuff too, network admins could enter CIDR blocks, contact info and such. In theory, this sounds like a great idea. But I'm not sure if this works for smaller countries with only a few IP address ranges. Is the database you had in mind contained in the table that I attached to my first message in this thread? http://archives.seul.org/or/dev/Apr-2010/msg00021.html If not, can you look up the ranges for Tunisia and post them here? Thanks, --Karsten *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Drop Tor users via bridges by over 2/3 in the beginning of March
On 3/10/10 5:20 PM, Andrew Lewman wrote: On Wed, 10 Mar 2010 08:31:06 -0500, Flamsmark flamsm...@gmail.com wrote: :At the beginning of March, the great firewall of China blocked all (then) :known tor exits and relays, and a substantial number of bridges - presumably :enumerated over a prior, somewhat extended period. This is our working theory as well. Pending research involves which set of bridges were blocked; website, email, twitter/qq account, or all of them. No obvious problems with the measurements or presentation, so these numbers are probably real. --Karsten *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Tor in China
On 2/19/10 5:00 PM, Andrew Lewman wrote: On 02/19/2010 05:20 AM, onion.s...@nym.hush.com wrote: http://metrics.torproject.org/bridge-users-graphs.html#china if there is no clear explanation account for the doubling of the usage figure in the whole December, i would speculate that this is an error in the estimation. could anyone confirm this? The best person to answer this is Karsten, and he's currently traveling. We await his answer. That's a fine question. It's already on my list. I'll let you know as soon as I have a better answer than probably something wrong with the measurements. --Karsten *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: bridge relay: GeoIPFile config option
On 2/16/10 6:17 PM, Olaf Selke wrote: am I right the bridge relay config option GeoIPFile means the path to GeoIP.dat provided by MaxMind? No. Tor can only handle the text-based ip-to-country database, but none of Maxmind's databases. You can the database that Tor understands in src/config/geoip in the sources: http://gitweb.torproject.org//tor.git?a=blob;f=src/config/geoip;h=31c721a9fe1d554e6a404b046eeaa4d83162f49b;hb=HEAD Or do you want to use Maxmind's database for some reason? If so, you can probably convert the text-based one (not .dat) easily. --Karsten *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Exit archives
On 12/03/2009 07:11 PM, downie - wrote: You might want to look at this Python script that parses the descriptor archives to tell you exactly what you are looking for. It also comes with a HOWTO explaining which files you need: https://svn.torproject.org/svn/projects/archives/trunk/exonerator/ I see there is generally a 6 day lag. Is there any chance of getting the November data out early? I'm afraid there isn't, sorry. --Karsten Hi Karsten, I have Python 2.3 installed and I don't compile - can I still use the script? GD If it doesn't compile, you might want to try the Java version of the script (requires Java 6; maybe 5 works, too) that has identical results. Or you need to upgrade to Python 2.6.2 or higher. --Karsten *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Exit archives
On Dec 3, 2009, at 6:54 AM, downie - wrote: Date: Wed, 2 Dec 2009 23:45:01 -0500 From: and...@torproject.org To: or-talk@freehaven.net Subject: Re: Exit archives On Wed, Dec 02, 2009 at 10:57:00PM -0500, downgeo...@hotmail.com wrote 1.4K bytes in 48 lines about: : could you remind me please where to query the historical data of exit nodes' IPs? http://archive.torproject.org/ Thanks Andrew, which of those groups of files is best just to check if an IP was an exit on a given date? You might want to look at this Python script that parses the descriptor archives to tell you exactly what you are looking for. It also comes with a HOWTO explaining which files you need: https://svn.torproject.org/svn/projects/archives/trunk/exonerator/ I see there is generally a 6 day lag. Is there any chance of getting the November data out early? I'm afraid there isn't, sorry. --Karsten *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: any rough stats on bridges ?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/19/2009 04:10 PM, John Case wrote: It would be interesting if someone in the know could let us know how many bridges are running ... I'd further be interested in the total number that have been submitted over time, vs. the number that are actually running now ... maybe some rough ideas as to their average bandwidth, etc. My understanding of the protocol leads me to believe that this is benign information. The latest information that I can give you is from June 22: https://www.torproject.org/projects/metrics in particular https://git.torproject.org/checkout/metrics/master/report/bridges/bridges-2009-06-22.pdf Let me know if there's something else you are interested in that could be extracted from the bridge descriptors, and I can include it in the next report. Thanks, - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkrctvsACgkQ0M+WPffBEmVsKgCeIBSb9vE93GDAgeTA8s0x3LZn tjgAoKN1qtfYH3Qi8vR+w5HpcXooHgUc =Jzyh -END PGP SIGNATURE- *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Directory History
Sorry, there's no web interface, yet. I talked to Martin Mulazzani about integrating that functionality in TorStatus, but that might not happen very soon. Also, it would be great to have some feedback on the Java/Python script first that can be included in the web version. (Sent from Android, so please excuse typos and mailing list rudeness of all kinds.) Best, Karsten Flamsmark flamsm...@gmail.com wrote: Is there a web interface to the archives, or would users of the archives have to check manually? On Tue, Oct 6, 2009 at 07:12, Karsten Loesing karsten.loes...@gmx.netwrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/06/2009 12:19 PM, morphium wrote: I think this question has already been asked before, but I can't find it...so: Is there an archive anywhere where I can see what Tor nodes have been active at a specific date? You can find the descriptor archive here: http://archive.torproject.org/tor-directory-authority-archive/ I'm in court (Jena, Germany) in 3 weeks because someone ordered something (worth about 50 Euro) and they're accusing me now. I think it would be good not only to explain them what Tor is, but to have an excerpt from a directory listing around the date, so I can prove my Tor node was active that time. You may find this script useful that parses these archives and tells you whether an IP address was a relay at a given time: https://tor-svn.freehaven.net/svn/projects/archives/trunk/exonerator/ The script is available in Java and in Python. Good luck with your court case! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkrLJgEACgkQ0M+WPffBEmWwpQCgq5MQK8Tx45sasE/RP/QAUUeB CZIAn203QG0IIlfZ3wJyAtg65OQLhNnD =SPx2 -END PGP SIGNATURE- *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Directory History
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/06/2009 12:19 PM, morphium wrote: I think this question has already been asked before, but I can't find it...so: Is there an archive anywhere where I can see what Tor nodes have been active at a specific date? You can find the descriptor archive here: http://archive.torproject.org/tor-directory-authority-archive/ I'm in court (Jena, Germany) in 3 weeks because someone ordered something (worth about 50 Euro) and they're accusing me now. I think it would be good not only to explain them what Tor is, but to have an excerpt from a directory listing around the date, so I can prove my Tor node was active that time. You may find this script useful that parses these archives and tells you whether an IP address was a relay at a given time: https://tor-svn.freehaven.net/svn/projects/archives/trunk/exonerator/ The script is available in Java and in Python. Good luck with your court case! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkrLJgEACgkQ0M+WPffBEmWwpQCgq5MQK8Tx45sasE/RP/QAUUeB CZIAn203QG0IIlfZ3wJyAtg65OQLhNnD =SPx2 -END PGP SIGNATURE- *** To unsubscribe, send an e-mail to majord...@torproject.org with unsubscribe or-talkin the body. http://archives.seul.org/or/talk/
Re: Hidden service usage
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/21/2009 06:54 AM, leandro noferini wrote: Ciao a tutti, I would like to control the usage (the amount of connections or something like) to a hidden service I have. In the man page (0.2.2.1-alpha-1 version) I found these directives: No, these config options won't help you in finding out what usage _your_ hidden service sees. The option AuthoritativeDirectory 0|1 is only used by around ten relays that act as directory authorities; and HSAuthoritativeDir 0|1 HSAuthorityRecordStats 0|1 DataDirectory/hsusage are only useful on three of these directory authorities that store (the old version of) hidden service descriptors. As I can understand I need to enable all to have something, right? The first option is not only for directory servers? And also, what kind of information I will have? So, these options won't help you. You shouldn't enable them, or your Tor will behave funny. Can you instead learn the number of connections to your hidden service from your webserver (or whatever kind of server that is)? Your local Tor opens a new connection for every incoming request to your hidden service. Maybe you can count those connections? Best, - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkq3U1AACgkQ0M+WPffBEmVzRgCfRM0hb6EiCYK9s3Jk/SNd+M4y Yn4An2BOvLuOSzaHJjIhdkR5KV+PAtjb =ygBQ -END PGP SIGNATURE-
Re: warning message question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/26/2009 09:52 AM, Roger Dingledine wrote: On Sun, Jul 26, 2009 at 02:32:45AM -0500, Scott Bennett wrote: Saturday morning, I got the following message. Jul 25 09:33:57.004 [warn] Received http status code 502 (Proxy Error) from server '80.190.246.100:80' while fetching consensus directory. Can anyone explain the situation(s) that can result in such a message? 80.190.246.100:80 is the DirPort on gabelmoo, one of the v3 authorities. So your relay was trying to get an updated version of the v3 consensus from gabelmoo. It looks like the way gabelmoo is listening on port 80 is by proxypassing it through apache. You can read more about that approach in this poorly organized faq entry: https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#ServerForFirewalledClients and at that particular moment, for some reason apache decided that sending back a 502 proxy error was better than passing your request on. FYI, gabelmoo is passing directory requests through Apache for two reasons: First, I have been using Apache as a first attempt to measure how long clients take to download network statuses in order to derive how fast clients are; this functionality is now in 0.2.2.0-alpha-dev, so that Apache is not required anymore for this: http://archives.seul.org/or/cvs/Jul-2009/msg00244.html Second, I'm using port 80 both for serving the Tor directory and for serving files for performance measurements: http://freehaven.net/~karsten/volatile/torperf-2009-07-01.pdf For example, I bet this could happen if the gabelmoo relay was down at the time. It's back online now. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpsMc8ACgkQ0M+WPffBEmWTTQCdFth19N02fKpJc7vAJE+pSMOW +3sAoLGWwxPlkCXmSLD/BHldeeqGdpTU =FvlG -END PGP SIGNATURE-
Re: Bad Exit Node
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/10/2009 07:36 PM, Robert Marquardt wrote: I've setup a tor exit node in russia yesterday and today it's flagged as Bad Exit. Router Name: Romulus Fingerprint: FF7D 3F88 EEB8 C7E1 0D04 005B 45D7 FD24 E572 93E9 Contact: Robert Marquardt email AT robert minus marquardt dot com IP Address: 92.241.164.157 Hostname: tor-proxy-readme.robert-marquardt.com Why do you think it's flagged as Bad Exit? This is what the current network status says about your node: r Romulus /30/iO64x+ENBABbRdf9JOVyk+k wcVamAnXtevgQeBzsOZ5TuX0YAc 2009-07-10 15:48:21 92.241.164.157 9001 9030 s Exit Fast Running V2Dir Valid v Tor 0.2.0.35 w Bandwidth=50 p reject 25,119,135-139,445,465,587,1214,4661-4666,6346-6429,6699,6881-6999 Where did you see that your node has the BadExit flag? - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpXf2IACgkQ0M+WPffBEmURtwCgycYvJrBq7YXZPmNcdTpPiJ0A 6tAAoJb5qw8+T947exnvosbxMmTOGerV =yPUU -END PGP SIGNATURE-
Re: Bad Exit Node
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/10/2009 07:56 PM, Robert Marquardt wrote: In the directory entries: http://torstatus.blutmagie.de/router_detail.php?FP=ff7d3f88eeb8c7e10d04005b45d7fd24e57293e9 That page says that your node does not have the BadExit flag. All flags are listed, but your node only has those with a green check mark. Your node didn't have the BadExit flag the whole day. HTH, - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpXhHcACgkQ0M+WPffBEmV36wCfYhOG5c6/QOx9xCWu5iEG9fd9 SDkAnAz2gtKHUA3uAiLjhvAym3NSxAUK =I2D9 -END PGP SIGNATURE-
Re: many new relays
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On 07/04/2009 11:59 AM, grarpamp wrote:
I usually play with this form of output as it is the most verbose.
getinfo desc/all-recent | perl_splitter - separate files
by fingerprint or other tag.
Even one of these (or a cached-descriptors) from before the
jump could be enough. An ls -al of your dirs could probably
find a good pre jump candidate by size. The 19th seems to
be the first of the permanent influx.
If someone needs ... server descriptors ... please let me know
So sure, just one from the 17th might be good.
It's not that simple. Every descriptor is stored in a separate file.
Archiving the cached-* files would add a lot of redundancy.
As to future distribution and if size is a problem ...
My current view of getinfo desc/all-recent is 4.356MiB, 1.801MiB
compressed. Removing this uncompressible info irrelevent to
the subject:
if (/^opt (extra\-info\-digest|(read|write)\-history) /) {
if (/^(onion\-key|signing\-key|router\-signature)$/) {
gives 1.641MiB, .292MiB compressed.
Good idea. Removing the crypto parts did the trick. The compressed June
descriptors are now 20 MB rather than about 100 MB before. I think we
can afford the bandwidth (even if 50 or-talkers download the thing):
http://freehaven.net/~karsten/volatile/server-descriptors-2009-06-short.tar.bz2
You'll probably want to use the published timestamps or write your own
little parsing application to match descriptors with network status
lines in the consensuses:
http://freehaven.net/~karsten/volatile/consensuses-2009-06.tar.gz
(I'll remove both links in 1 month from now.)
Let us know what you find!
Best,
- --Karsten
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkpPPdkACgkQ0M+WPffBEmUznQCgtsg63PNBedW52JOKYfRAtVHk
DX4AnjOs+pH/nS6dZecV1t6For/Sbrwr
=fsQ1
-END PGP SIGNATURE-
Re: many new relays
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/28/2009 09:05 AM, grarpamp wrote: I'd give it a 15 minute mile high eyeball if I had the 'before the jump' cache files or a 'getinfo desc/all-recent' from back then. I just don't have that dataset. I have uploaded a tarball of the 00:00 UTC consensuses from June 1 to 30, 2009 here (3.3 M): http://freehaven.net/~karsten/volatile/consensuses-2009-06.tar.gz If someone needs the consensuses in between (709 M including votes) or the server descriptors (760 M uncompressed), please let me know via private email. (We're still in the process of finding a better way to make these files public, but then there are always tasks with higher priority..) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpKEtcACgkQ0M+WPffBEmWg1gCffDinyt8/6wwH+C4PjaD9f4U/ B+MAoNksVGRxVXkfsl2XvpU+L9gbUIcm =R9aq -END PGP SIGNATURE-
Re: Obfuscated URLs?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/30/2009 08:47 PM, Martin Fick wrote: Would it be possible to create a URL or some longer string that describes a hidden path through the tor network to a specific hidden URL and to implement a routing mechanism to access documents (files) using this Obfuscated URL? Two thoughts: - - What you describe as obfuscated URLs sounds a lot like precursor designs of hidden services. For example, encoding a path into the locator works only as long all nodes in that path are functional. Hidden services (and other designs) have directory services to overcome that problem. Why make a step backwards? - - Tor is made for interactive communication, not for exchanging single files. Even if you don't intend to exchange bulk files, others will do so. Unfortunately, the Tor network does not have the necessary capacity. Best, - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkpKZfAACgkQ0M+WPffBEmWljwCdEOJBNfRbMXJcOyWwZF9GcSBN 7LgAniCxgTT/eNlvMmBWHIVPuIUGvTo+ =iwWY -END PGP SIGNATURE-
Re: Hackers exploiting tor clients on .onion sites?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 So, did I get this right? You are concerned about certain log messages, you even searched them on the Net, but you deleted them afterwards (including the searches in your browser history) and are telling us now that something strange is going on when visiting .onion sites? I'm not saying there cannot be bugs in this part of the Tor code. But what you describe is rather unlikely. I'm not aware how someone could write anything to your log file---nor why he/she should want to do that. Should you encounter these messages again, please retain them and file a bug report: https://bugs.torproject.org/ For the fun of it you might also want to verify that you are running an official Tor version: https://www.torproject.org/verifying-signatures - --Karsten On 06/14/2009 11:29 AM, pigpo...@safe-mail.net wrote: I explored a few of the common .onion sites listed at Wikipedia's tor page listed within the external links footer. These sites loaded well, but I noticed several errors in my tor client logs. I googled for info on the errors, some of the error messages turned up in cvs related pages and bug talks, but lacked definite information. I didn't retain the error messages, but I won't be visiting .onion sites in the future as it feels to me by the open nature of some of the .onion sites where users post messages or edit wikis anonymously, malicious users may have the ability to target the small audience of users at these sites: tor users and their clients. I only discovered these errors when visiting a few .onion sites. My tor client did not exhibit strange behavior during or after these error messages were displayed, but as some of the error messages appeared entirely in CAPS, I thought to post here. As I didn't retain the error messages, this post is vague - sorry! I am concerned. This is not to discourage others from visiting .onion sites, but to raise awareness about the possibility of rogue behavior on some .onion sites, most probably by anonymous users, not the .onion admins. Should this concern me? What steps could be taken to safeguard the tor client from possible attacks, if any, from .onion websites? Am I considering errors too seriously? Hello, my final thread for this day, thank you. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAko02kkACgkQ0M+WPffBEmXRDACbBCkY0hUv51gIlwP/oO7Fnvc2 tOAAoNJCLJ5WMEX2A+PSs2QSVXdRo4gx =OtHg -END PGP SIGNATURE-
Hidden services on Tor versions 0.1.2.x should be upgraded soon!
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi folks, if you run a hidden service on Tor version 0.1.2.x or lower, you should upgrade to 0.2.0.x or 0.2.1.x soon. Otherwise, people running Tor versions 0.2.2.x or higher won't be able to reach your hidden service. Why is this the case? We added a new format for hidden service descriptors in 0.2.0.x and made hidden services and clients speak both the old and the new format. 0.2.1.x didn't change that. But in 0.2.2.x we have just dropped support for the old format. Speaking both formats at the same time means an unnecessary message overhead that we have to stop at some point. That means that a hidden service running 0.1.2.x and a client running 0.2.2.x won't be able to connect; the same applies to hidden services on 0.2.2.x and clients on 0.1.2.x. This is also a reminder that 0.1.2.x is obsolete. End-of-life for 0.1.2.x was announced in February 2009 [0]. There are known security holes in 0.1.2.x that are fixed in later versions. Please upgrade! Best, - --Karsten [0] http://archives.seul.org/or/announce/Feb-2009/msg0.html -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoUSP8ACgkQ0M+WPffBEmXrGwCgz0K1wlUgNZmzbyeQ4CbukbOh TZIAoJ/iNHYpCqESCSuo41gl5/DfEcA9 =bdsr -END PGP SIGNATURE-
Re: My tor exit node is gone from the node list?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/10/2009 10:01 AM, Alexandru Cezar wrote: How odd. It is still publishing descriptors, and the directory authorities are still testing its reachability. In particular, here are the six votes from the six directory authorities: [...] So that's why it's missing. But, why is it not considered reachable from three of them? I'm not sure. I am still trying to solve this. Since my last mail, I also let TOR regenerate the keys, so kyirong's fingerprint now is 849D 45A3 2335 5EB3 4F73 2EF5 DB43 0B90 6A21 DAAE (89.248.169.108, DirPort 80, ORPort 8010; uptime 24/7). It is still not listed. The node is reachable from multiple locations (judging from my limited way of testing). If someone can give me hints towards unreachable routes, I can ask my ISP about that. We discussed this problem on IRC today and found out that your node is not reachable from multiple locations. To be more precise, your node does not present an SSL certificate when accessed from these locations. You can test this from different machines using the following command: % openssl s_client -connect 89.248.169.108:8010 You should see a certificate then. If the output consists only of the following line, something's wrong: CONNECTED(0003) This problem seems to be related to your port 8010. From some locations your node presents an SSL certificate on port 443 but not on 8010. You might want to ask your ISP why that is the case. (A workaround might be to switch your OR port from 8010 to 443, but let's try to figure out the reason for the original problem first.) While looking at your problem we found that many relays have similar reachability problems. This is a list of relays that are missing the Running flag by at least one authority: http://pastebin.com/mf7e2d7c If someone else on this list finds her/his node in this list and can help us figure out what's going on, that would be grand. Single events of missing Running flags are nothing to worry about, but if there is a pattern we should have a look at it. Thanks! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoHSiUACgkQ0M+WPffBEmWPTQCePmwGjrm14YPVKdsK2AdBzm/i /fYAn0rlrNH9whjMrkn7NuiHGaWgn8nm =Kj/E -END PGP SIGNATURE-
Re: ExitNodes for encrypted connects only are not possible. Why?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/09/2009 11:19 AM, Gitano wrote: In 'git.torproject.org/checkout/tor/master/doc/spec/dir-spec.txt' ExitNodes are defined as: Exit -- A router is called an 'Exit' iff it allows exits to at least two of the ports 80, 443, and 6667 and allows exits to at least one /8 address space. I would like to setup my ExitNode for ports 443, 465, 563, 993, 995 (https, ssmtp, nntps, imaps, pop3s) only, but this is not possible. What's the reason behind this? Is there any chance to loose this restriction in one of the next releases? Feel free to configure your node to exit to those 5 ports only. That makes your node an exit node for connections to those ports. Your node won't get the Exit flag, though, but that's not required for being an exit node. The Exit flag is used by clients for path selection. Relays with the Exit flag are selected less often for non-exit positions, so that their bandwidth is saved for exiting connections. That means that your node will be selected more often as middle node and less often as exit node compared to relays that have the Exit flag. It's unlikely that the criteria you pasted above will be changed. There need to be some criteria, and if almost every node matches them, the flag would be useless. Hope that helps! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoFTeEACgkQ0M+WPffBEmX4jgCgncZIgKLe1t4nK3Fau0NWirws eCgAnRC4XUqHvaBHpv9WZ9y1hP+JZb6T =yEhk -END PGP SIGNATURE-
Re: ExitNodes for encrypted connects only are not possible. Why?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/09/2009 01:38 PM, Gitano wrote: It's unlikely that the criteria you pasted above will be changed. There need to be some criteria, and if almost every node matches them, the flag would be useless. Ok, but adding one more 'secure' port beside 443 would be enough in this case. I'm not sure what you are trying to achieve with that. The idea is not to flag as many nodes that permit exiting as Exit nodes. The idea is to relieve the exit nodes carrying most of the exit traffic from acting as middle nodes, so that they can push more exit traffic. The same is done for guard nodes, by the way. It's unlikely that your node would carry as much exit traffic with the five ports you mentioned as compared to other nodes that already meet the requirements for the Exit flag. Of course the requirements could be lowered to assign the Exit flag to more relays. But it defeats the purpose if too many nodes have that flag. In the end, all nodes would see the same load as before, without the Exit flag. I'm not saying that the current definition for the Exit flag is perfect. But right now we lack good data to come up with a better definition. Best, - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkoFvy8ACgkQ0M+WPffBEmXMawCgkzkbYdk1J4F6y7VSxdfxUKTm LeoAoMNHbXYG6BqSIFu2dpq3VQ+He56t =O2DW -END PGP SIGNATURE-
Re: Google Summer of Code 2009
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/21/2009 12:42 AM, Stephen Tyree wrote: I want to start by saying thank you. It is an honor to be selected for Google Summer of Code on the Tor Project. Glad to have you with us! :) And thank you for starting the introductions on this list. It would be neat if the other GSoC students did the same. A paragraph or two about you and your projects would be nice. Just go ahead, it's the community bonding period [0]. Community meet $student, $student meet community! Thanks! - --Karsten [0] http://googlesummerofcode.blogspot.com/2007/04/so-what-is-this-community-bonding-all.html -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAknts0wACgkQ0M+WPffBEmXGlACfeyq2+h1+c7iW6QChbBYeG5WK fAkAoNsKbyyhfLvOz89O2U5W12dyNFkv =35Jj -END PGP SIGNATURE-
Re: Questions about gathering information and statistics about the tor-network
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Martin, who maintains TorStatus these days? Is it you, Kasimir, both, even more people...? :) Is there a mailing list, IRC channel, or some other way to contact you rather than or-talk? none wrote: I tried something similar recently. My approach was to monitor the infrastructure only, as this is public knowledge, and collect no informations on the clients whatsoever. I incorporated this monitoring already into TorStatus. Check out the SVN of TorStatus for the implementation and see the trunk version of TorStatus @ http://trunk.torstatus.kgprog.com/ or http://trunk.torstatus.kgprog.com/network_history.php It is not yet perfect. The timeslot for updating is too short, so the graphs look frayed. Short summary: * collect the total number of running, running exits, running guards and running fast servers and save them in an RRD over time. * On the top 11 countries these values are collected as well. Looks like a great start to me! Right now, I'm investigating options to display more statistics about the Tor network: https://www.torproject.org/projects/metrics . TorStatus seems to be a promising tool for that. What would you say, how hard would it be to add router descriptors from the past years to the database and make nice graphs from them? I have written a Java application to feed descriptor archives into a PostgreSQL database that could be adapted to MySQL and the TorStatus schema. Also, how hard would it be to add more graphs displaying other information than the numbers of servers with certain flags? For an example what output I'm interested in, see the evaluation of the 2008 data: http://freehaven.net/~karsten/metrics/dirarch-2009-02-11.pdf And finally, how extensible is TorStatus regarding other data than descriptor archives? Given that there are other interesting data about the Tor network than what relays advertise, would it be feasible to extend the TorStatus database and add more graphs? Thanks! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJqpo40M+WPffBEmURArFjAJ9leJhzSfSgJ/HyoxxCg659PyFOLQCg2xnF DxWLJNwh5LVCDa5+xfm8W/g= =Z5Mf -END PGP SIGNATURE-
Re: Questions about gathering information and statistics about the tor-network
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Sebastian, Sebastian Schmidt wrote: Right, making data available might turn out to be difficult. I haven't looked at specific frameworks, yet. One option would be to integrate more graphics into TorStatus (http://trunk.torstatus.kgprog.com/). Kasimir Gabert did a great job displaying bandwidth histories for the past day, week, month, and so on. Maybe these graphics can be extended, given that we have good data to present. Yes but there are so many links you can make between different informations, you can't show them all on one page. I think a dynamicaly solution like giving one the possibilty to say: show me one single graph with the development of all exitnodes at all and all exitnodes in britain between this dates, would be pretty cool. If stator's ready does everything on the console and with gnuplot which I want I'm going to look further into this. Yes, the existing TorStatus pages already contain enough information, so this would mean adding more pages. I rather meant using and extending the infrastructure of TorStatus to collect, store, and present data about the Tor network. This extension could allow users to select what data they are interested in and generate graphs for them. But to be honest, I haven't looked into any technical realizations so far. It's just an idea. As soon as it can be used by others I will make a public release of it. At the moment both apis need way more coding work to be used by others than me and aren't documented at all. That looks great so far! You shouldn't wait for the perfect time to make your code available -- there is no such time. Nobody expects your code to be perfect from the beginning. You might consider putting it into the Tor SVN repository or make it available at some other place. And of course: Commit early, commit often! Also, when your exams are done, you might want to hang out at #tor (if you aren't there already). It's probably easier to discuss designs there than filling the mailboxes of the people here. :) Best, - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJezU20M+WPffBEmURApoQAJ43opvCt3bS3sOMr56d0ZluCm4/DQCg1c1H tfteXqHwwi1AmZ3vdxYwHcA= =GjDm -END PGP SIGNATURE-
Re: Questions about gathering information and statistics about the tor-network
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Sebastian, Sebastian Schmidt wrote: That's about how geolocations of directory users can be collected right now. This sounds interesting. Can those informations be questioned somehow from the dir-servers or are they non-public? It's not that these data are non-public, but there is currently no way to request them from the directory servers. And unfortunately, I don't have a good sample at hand, as I don't collect these data right now. But I think you should be able to generate them on a directory mirror, too. Have a look at the DirRecordUsage* config options (grep for them in the code). You may also need to configure --enable-geoip-stats. Those stats you gathered about the bridges here are really interesting! Since I read it I'm thinking how to interprete them. It looks like we have already many bridges for the short time they are supported in stable tor but just a small number of overall traffic (based on the bandwith consumption) on them. This could be intresting for people who want to support the network because they don't need to setup a 4TB-root for running a bridge. Correct. Often, bridge admins are disappointed that they don't help Tor 'enough', because they see so little bandwidth. But people who don't have much bandwidth to share do help the network by setting up a bridge. As Roger has stated several times, bridges might become even more useful in the future. So it's good to already have a solid number of bridges when they are needed. Also most users seem to be germans/americans and not people of countrys one would think who would be the number one. I'm thinking why? Afaik no provider in germany restricts the access to tor. Do people use bridges because they think this extra hop increases their anonymity instead of letting the bandwith for the people who really needs them? One explanation that comes to mind is that some corporate filtering systems make it necessary to use bridges. I would have to look this up in the spec, but I think there is no anonymity gain by using bridges in terms of an 'extra hop'. Bridges replace the first node in a circuit, so that circuit are 3 hops long, too. Well there are many interesing information which could be gathered without touching users anonymity at all. In contrast there are information which needs to be collected to protect their anonymity. Stats like the sudden increase of nodes in a fascist country like e.g. burma,china and so on shouldn't happen without people noticing it. For the beginning I want to get all the interesting information out of the service-descriptors and make them visible. Starting with the descriptors is a good idea. You can collect them with Peter Palfrader's directory archive scripts in the contrib/ directory. Have you already thought about a good way to present the data? I think best would be a dynamic solution so one gatheres all the information and users can throw exactly the information they want in one pot which they want to see joined in one graph for a timeline they can choose. But I don't know any good framework which offers this. At the moment I just found cewolf(http://cewolf.sourceforge.net/new/index.html) and I don't know if it fits the needs of which I'm thinking off. So I think gathering the already available information is more easily than finding a good way to make them easily public for the average user. Right, making data available might turn out to be difficult. I haven't looked at specific frameworks, yet. One option would be to integrate more graphics into TorStatus (http://trunk.torstatus.kgprog.com/). Kasimir Gabert did a great job displaying bandwidth histories for the past day, week, month, and so on. Maybe these graphics can be extended, given that we have good data to present. I'll be really busy the next month but as soon as I have something to show I'll let you know! Ah, the exam phase? :) If so, good luck with that! Thanks! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJcx3N0M+WPffBEmURAvZlAKDOXtUQbbZ4weziWOD3q67HjjfFqgCeK8uN DXLHN9/NlBZp/dLd1wW2Wq4= =i68w -END PGP SIGNATURE-
Re: Questions about gathering information and statistics about the tor-network
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Sebastian, Sebastian Schmidt wrote: Hi, I'm writing a tool right now to gather some longtime statistics about the tor-network. That sounds like a fun project! :) I'm a bit in a hurry and cannot answer your posting in detail, sorry for that. But let me give you some pointers now. Well, first of all, I should say that your concerns about possibly endangering anonymity of Tor users are very important. The data you collect should not be usable to deanonymize Tor users. For example, you mention collection of data on entry nodes (and that you don't want to collect them, okay). What you should _not_ do is collect precise data about who connected to your entry node at what time. Someone else could collect similar data on their exit nodes what targets are requested at what time. Both data sets don't pose a risk on their own, but put together... *ouch* A better way to collect such data would be to aggregate them over, say, 24 or 48 hours, aggregate them by country instead of memorizing single IP addresses, and round them up to multiples of 8 or 16. That's about how geolocations of directory users can be collected right now. If you wanted to experience a few dozen enraged privacy researchers, you should have been at last PETS when a study on the Tor network, 'Shining Light in Dark Places: Understanding the Tor Network', was presented. Apart from the authors' consideration to make their data available to the research community in an 'anonymized way' (I don't recall their full plan for anonymizing them), that paper is a good read! ;) So, the right way to collect data about an anonymity network is for sure a hot topic. Prepare for a lively discussion here. ;) Anyway, I wanted to give you some pointers. Did you know that gathering good statistics of the Tor network is on the 3-year roadmap (Section 5.7)? https://svn.torproject.org/svn/tor/trunk/doc/roadmaps/2008-12-19-roadmap-full.pdf This should really not stop you from doing your own statistics! We have just started with that and there's definitely enough fun work left to do. :) But maybe some thoughts in that document are interesting for you. Also, you might be interested in an analysis of bridge usage in Tor. The bridge authority Tonga collects data about all bridges in the network in order to give them out to bridge clients. These data are also archived for later statistical analysis. The approach of evaluating these data might be interesting for you. The data model is more or less the same as for non-bridge data. Ah, and please keep in mind that this is only an early draft of the analysis *cough*. If you want, you can find the evaluation scripts in the parent directory of the same SVN repository: https://svn.torproject.org/svn/projects/dir-stats/trunk/bridge-stats/report/bridge-stats-2008-12-25.pdf There will be some more statistics on the Tor network within the next weeks. My plan is to evaluate archived network statuses, router descriptors, and extra-info documents of the past 12 months to get a better idea on the network growth and related facts. Further, I'd like to evaluate geolocations of client requests to the directory authorities and directory mirrors. And I want to finish that bridge data analysis. So, to answer one of your questions: Yes, people are interested in such statistics. :) If you have ideas on what data should be collected (and how that can be done in an anonymity-preserving way) or what statistics should be performed with existing data, your input is most welcome! And sorry again for ignoring most of your posting. I'll try to get to it the next days. Thanks! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJbmLC0M+WPffBEmURAtRzAJwIbTfQk2gJ9f8KGv3fxjiF4dsqqgCgzq5o V33eHw/Q8GdBIzvWFsZL5kw= =KVOW -END PGP SIGNATURE-
Re: How many hidden service circuits built?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Bernhard, Bernhard Fischer wrote: Sorry, I didn't see this before. I'll read your paper and I appreciate all improvements regarding hidden services. You might also want to read the documents that are linked from the NLnet project page, for example: http://freehaven.net/~karsten/hidserv/perfanalysis-2008-06-15.pdf While TOR is building circuits there's always some kind of randomness involved. As far as I know TOR chooses nodes based on directory flags (like fast, stable, ...) and the randomizes those matching some criteria. Obviously the flag fast is somehow misleading because bandwidth is a local property and does not necessarily mean that it's also fast across the network to any other node. Okay, we didn't change anything about path selection so far. One reason is that this might have serious consequences on anonymity. While it would be great to make Tor and hidden services faster by using only the best nodes available, this largely destroys anonymity. All changes here should be made with extra caution! I'm interested in performance improvements of hidden services, but I'm talking about RTT once the connections are established and not so much on the connection setup time (which of course is also important but this time is only spent once) I did some RTT measurements and my observations are really ugly. It usually is never below 5s. What you can observe is that when the circuit is rotated the RTT also changes signifficant. See the measurements in the analysis linked above. This document contains some data about message transfer times after connections are established. Basically, we excluded message transfer times from the project, because they didn't seem to be a problem of hidden services, but rather of Tor in general. My idea now was to open several circuits to the same hidden service. If they're connected through different nodes (because of the random selection) also the RTT will be different. Then I continuously do RTT measurements on all those circuits and always use that one with the lowest time for user data. Even though this would constitute a local optimization, the effects on overall network load would be seriously bad. There must be ways to improve RTTs which waste less resources than this approach. One solution might be to change path selection for rendezvous circuits, both on client- and server-side. If we knew what relays to pick for these circuits which are likely to deliver good RTTs, we could improve RTTs for the 6-hop circuit from client to server. Again, changing path selection requires caution as stated above. Another solution is to start performing QoS for hidden services. In combination with client authorization (see proposal 121), hidden servers could decide whether to pick an extra-fast circuit to connect to the client's rendezvous point, or not. Having said that, did you look at proposal 121 for OnionCat. I could imagine that OnionCat would make good use of the additional security that client authorization offers for hidden services. See also a Technical Report on that topic: http://petsymposium.org/2008/hotpets/vrtprsvc.pdf - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJQ8hE0M+WPffBEmURAgzYAJ95qU0k+V9Ic9hRVvMsNJWAbf8tSQCfYfnK goWrW1jA183eTsvj5BJfcXo= =5Mur -END PGP SIGNATURE-
Re: How many hidden service circuits built?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bernhard Fischer wrote: On Thursday 11 December 2008, Roger Dingledine wrote: On Thu, Dec 11, 2008 at 11:25:40PM +0100, Bernhard Fischer wrote: If I connect through the SOCKS interface several times at the same time to the same hidden service, does TOR open a bunch of circuits in parallel to the designated hidden service or does it just open a single one and then reuse it for every of the incoming SOCKS request? It should try to reuse the same circuit. (You will see a bunch of circuits going to make the rendezvous happen, but only one of them will be the one that all your streams get connected to.) --Roger Is it possible to change this behavior (maybe by a slight modification of the source)? I'm not sure what you are up to, so I'm guessing. Are you asking for a) parallelizing connection establishment in order to reduce delay, b) having a separate circuit to the hidden server for every application-level stream, or something else? As for a), we are already working on improvements to reduce the delay in connection establishment. Did you have a look at this page?: https://www.torproject.org/projects/hidserv.html Part of the solution is to parallelize some of the substeps. One example are circuits to introduction points which are built in parallel after a delay of 15 seconds. Future ideas are to request hidden service descriptors from the directories in parallel. But making two (or even more) full connection establishments with all steps being performed twice (or more times) is a bit too brute-force, isn't it? The goal is to make hidden services faster, but in a way that doesn't put too much new load on the network. As for b), I don't know if this makes sense, either. Why separate the circuits when you can multiplex an arbitrary number of streams over them? Fault tolerance? Unlinkability of streams? But instead of guessing what you had in mind, I'll just ask: Why do you want to do this? - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJQkiP0M+WPffBEmURAsmuAJ4lf5aPZBg7IEXw0hzW4aCb0Ve2CgCfW37x ki2Nf2vTOF9Z+jRX8GevDfU= =1DHP -END PGP SIGNATURE-
Re: Error Making tunnel to dirserver failed
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 leandro noferini wrote: Nov 13 08:38:23 nemo Tor[2370]: Requested exit point '$847B1F850344D7876491A54892F845934E4EB85D' is not known. Closing. Nov 13 08:38:23 nemo Tor[2370]: Making tunnel to dirserver failed. What does it mean? Looks like bug 767: https://bugs.torproject.org/flyspray/index.php?do=detailsid=767 It's fixed in 0.2.1.6-alpha and will be fixed in 0.2.0.32, too. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJG/IY0M+WPffBEmURAmNwAKDRra8AsRTd2Xl/T0SE4Xhffv/C3QCdE160 x43edCpDL/kJvs8XIoc+On4= =FeS7 -END PGP SIGNATURE-
Re: Hidden service route
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Erilenz wrote: If I connect to a Tor hidden service am I right in thinking it goes like: Web browser - Tor client - Entry Node - Middle Node - Hidden Service No, that's not how it works. There are 6 nodes between you and the hidden service, three chosen by the hidden service, three chosen by you. See https://www.torproject.org/hidden-services for a description of the hidden service protocol. If I then change routelen to '2' in circuitbuild.c as per http://www.mail-archive.com/or-talk@freehaven.net/msg08747.html does that give me: Web browser - Tor client - Entry Node - Hidden Service Changing the route length should have minimal impact on performance. The step that takes time is to extend an existing circuit by another hop. I guess it has only minimal impact on performance whether you extend a 3-hop circuit to a fourth node, or a 2-hop circuit to a third node. You might want to try the latest alpha (0.2.1.7-alpha). It contains some improvements to speed up hidden services. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJGbe/0M+WPffBEmURAoAiAKCX8/i7JiFGdZz1a7NwU6H8eW1hSQCfZ8yK fY50qwXYpSMStMMQnAQjhKw= =Cw2y -END PGP SIGNATURE-
Re: Version deprecated?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roger Dingledine wrote: Looks like gabelmoo isn't recommending quite the set of versions it should be recommending. That is, it's missing 0.2.0.29-rc, 0.2.0.30, 0.2.0.31. Whoops. They were missing in the config after moving gabelmoo to new hardware and recreating its config from an older backup. Fixed. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJGVKN0M+WPffBEmURArOvAKC2J/kahX5XdxMLZyuNqfms5QobtQCgyXhe 4vuxB8hOdSioGzdKAKFD+j0= =bTkg -END PGP SIGNATURE-
Re: any middlemen seeing DoS currently?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 CyberRax wrote: Could the No current certificate known for authority ides; launching request. message that my client's been displaying every minute or so for the last 4 hours be related to that, or is my problem just a coincident? This might be the problem, yes. The reason is that ides' authority certificate has expired. But clients do not differentiate between a (temporary) download problem and the situation that a certificate has expired and a new one needs to be created (which is rather unlikely to happen within the next few minutes). So, clients send another request for a certificate every minute. All clients running 0.2.0.x or higher do this, which is why there is so much additional traffic in the network. The problem of clients downloading certificates that often will be solved with the next alpha. But the main solution is to upgrade the authority certificate which should happen some time today. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJFGJI0M+WPffBEmURAs5cAJ97vQ6F+WZKLjux4ubjWirV3KyIfACggWiN emK+fqenVFWlN5aOcxkPo2M= =A9gY -END PGP SIGNATURE-
Re: any middlemen seeing DoS currently?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Freemor wrote: I think that it might be an idea to send out an official announcement here on or-announce and perhaps on the website to tell people to stop their inactive tor clients (i.e. sudo /etc/init.d/tor stop ) to take the pressure off the exits and middles. When I read the above thread I checked and my computer has been pinging away all day looking for an updated certificate.. and I've only used Tor a little.. It's now turned off till I need it. or the problem is fixed. A new certificate is now in place. This should clear things up really soon. The authorities should exchange the new certificate during the next voting process (in roughly 30 minutes). Then clients will be satisfied with the new certificate and stop requesting a new one repeatedly. That means there is not enough time for an official announcement. Or rather, the effect would not be as significant as compared to the resulting confusion. Sorry for the trouble! This will be fixed in future Tor versions. And we will pay more attention to expiring certificates. The next certificate expires on January 17, 2009. Mark the date. ;) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJFJXg0M+WPffBEmURAgj4AJ9viyb2hSad/dG4Ho2dgbB036MaBwCfXtjZ cOoynU24phoc1M7i7+FT7dQ= =Z+94 -END PGP SIGNATURE-
Future Development on Hidden Services
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi list, as some of you may know, there have been several improvements to hidden services lately. First, hidden services publish their descriptors to a distributed directory [1] consisting of currently 71 nodes. Second, hidden services may require client authorization already during connection establishment to block unauthorized requests as early as possible [2]. Third, hidden service performance has been improved with respect to advertising and accessing hidden services [3,4]. Certainly, there are still things that can (and should) be improved. This is an attempt to compile a good list of future development tasks on hidden services. Comments are most welcome. If there are other things with hidden services that need improvement, please let us know. - --Karsten 1. Further improve performance and reliability - -- In the current NLnet project on speeding up hidden services [3] we found and fixed a lot of performance-related bugs, identified the likely bottlenecks of the protocol, and added the most important performance improvements to the code [4]. But there are still some possible improvements left that require evaluation and possibly coding if they turn out to be useful: a. Descriptor fetches on client side are an issue. Most failed connection establishment occur when trying to fetch the descriptor of a hidden service. We could parallelize this step as well (maybe also in combination with a certain delay to avoid extensive network load), just as we do with client-side requests to introduction points [4]. This might speed things up and lead to better reliability (at the price of increasing the number of requests to the distributed directory). b. The client-side request timeout of 60 seconds could be improved. It doesn't make sense to have a 60-seconds timeout for 5 substeps and stick to it even when realizing that the first substep has already taken 55 seconds. The other 4 steps (that are invisible to the client) simply cannot succeed in that time. This would also improve reliability, because we are otherwise giving up on requests that succeed soon after. c. The INTRODUCE1 cells could be combined with the first BEGIN cell to initialize an application stream. After establishing a 6-hop circuit from client to service, the client still needs to initialize an application stream over it which takes another 6 seconds in the mean. Maybe there is a chance to send the stream initialization message already as part of the introduction request. Or the hidden service could initialize the first stream back to the client. There might be security implications that prevent this, so more thoughts are needed here. d. Adapt the protocol to low-bandwidth environments: The effects of low-bandwidth connections on either accessing or running hidden services has not been investigated so far. Maybe such environments require us to change parameters like timeouts when we realize that our connection is bad. Jörg Lenhard is currently working on measurements using telephone and cell phone connections that hopefully give us some new insights. e. One of the big performance issues is general Tor circuit-building performance. This comes in two pieces: First, extending a circuit in Tor has a nontrivial chance of timing out or failing. Second, extending a circuit in Tor takes too long, both in terms of mean and in terms of variance. These problems are magnified for hidden service use, a) because the path is twice as long, and b) because some components of the path really do need to be made on demand, whereas for normal Tor circuits you make the whole circuit beforehand. So, if this improves for general circuits in Tor, hidden services should inherit these performance gains 'for free'. 2. Improve hidden services with client authorization - Beginning with version 0.2.1.6-alpha, hidden services support client authorization. That means that hidden services can restrict access to a small set of clients and prevent other clients (or introduction points/directory nodes) from establishing a connection. When client authorization is applicable, it reduces the risk of certain attacks on locating hidden servers [6] or denial of service. Once this feature is more tested, people should be told that it exists and how to use it. a. Make client authorization for hidden services easier to use: Domenik Bork has made a start on making Vidalia support configuration of hidden services with client authorization in his GSoC project [7]. His changes will hopefully be merged into Vidalia trunk quite soon. The question is whether people understand the interface, or if this needs improvement. b. Write good howtos for both service operators and clients: First, explain to the service operator what steps are required to add or remove authorization for a given client. Second, help end users understand how to
PET Convention 2008.2 on Sep 30, 2008 in Darmstadt, Germany
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi everybody, even though this is not perfectly on-topic (sorry for that!), I figured that some people on this list might be interested in the next Privacy-Enhancing Technologies Convention that will take place on September 30, 2008 in Darmstadt, Germany. From the homepage: PET-CON is a convention to help junior researchers, master and diploma students, to come together and exchange ideas. For this purpose, we're holding this event every six months at an easily reachable location somewhere in Germany or nearby. The convention is organized according to the grass roots approach: from young researchers for young researchers. Therefore, there is no formal dress code, no filtering of contributions, and no participation fee. If possible, we plan the convention in a way which allows people to travel there and back home on the same day -- so that busy people can participate as well. Please find more information on the convention on Sep 30 here: http://www.pet-con.org/index.php/PET_Convention_2008.2 - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI0OXA0M+WPffBEmURAkRlAKDQFjSPupfVRhoBHtTtz+RLWBrypACeJOve V2297DYkjuZ3lh1ZUIdnmUs= =Oy0a -END PGP SIGNATURE-
Re: invitation to directory server operators
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, the quoting approach doesn't work here any more, so that I try to address the main questions directly; if I should have overlooked something important, please let me know: One question was why we didn't announce the feature of configuring a node as v2 hidden service directories (HSDir in the folling) earlier: This feature was introduced in one of the alphas of the 0.2.0.x series. Back then I asked some people I knew to configure their node as HSDir to have a number of 3--6 HSDirs as a basis to get it running. Unfortunately, there was a major bug in one of the alphas (I don't recall if it was in the HSDir code or not, but anyway, it's fixed long ago, so no worries). The result was that the one of the more high-bandwidth nodes crashed and the node administrator downgraded to 0.1.2.x. At that time I refrained from asking more people to be beta testers before being more sure that it works more stable. Now that the HSDir code runs for quite some time without making trouble, I would say it is stable; which doesn't rule out the possibility of bugs completely, though. It was also on my TODO list to make an announcement, but not on top position, so that Scott got ahead of me with his announcement. It wasn't urgent, though, because the v0 directory is still running in parallel. Scott asked whether enough people turned on this option now: Not if we want the distributed directory be as stable and reliable as it was planned in its design. It is really awesome that so many people followed the announcement here, but we need as many HSDirs as possible. The concept depends on distributing descriptors among hundreds of nodes in the long term. This is required for higher reliability in face of single failing and corrupt nodes. Plus, it even gains more importance for hidden services with client authorization (see proposal 121) where you have separate hidden service descriptors for different clients that should not be linked together. With only a few HSDirs we need to rely on delaying descriptor publication for different descriptors from the same hidden service going to the same HSDir. With hundreds of HSDirs we can make this significantly faster. But this whole thing is not even completely implemented in trunk, so give us some time before announcing it here. (See proposal 121 for more details if you are interested in that.) Andrew found out that it is not required to open the DirPort in addition to setting the HSDir configuration. While this could on the one hand be considered a bug, it shows on the other hand that this requirement is really redundant and can be dropped. Originally, this requirement stems from a time when it was not clear that we can tunnel directory requests over the OR port. This works by extending a circuit to the OR port of a relay and sending a so-called BEGIN_DIR cell that contains a directory request and can be answered directly instead of a command to open a connection to another server or something like that. Then there was a question why nodes need to have an uptime of 24 hours or more: As was discussed earlier on this list, this is a means to ensure high availability of HSDirs. If one looks at the number of nodes over time and removes nodes with lower uptime than 24 hours, one gets a very smooth graph with low variations. Unfortunately this excludes people on daily disconnected DSL lines. Sorry for that, but if we want a reliable distributed hidden service directory, we really need reliable nodes that don't change their IP address. Hidden service clients shall be able to find a hidden service descriptor even when it was published a few hours ago. Finally, there were some questions about legal issues when configuring a relay as hidden service directory. I can't answer those, sorry. Please consult your lawyer, or turn off this option. We will add a remark in the sample torrc (and maybe other places) that this option can be turned off when 0.2.1.x goes stable (at the latest). - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIy6W70M+WPffBEmURAn6nAKDLAeBjtuGEFeE4erWE1Ce8CLYPPQCgl/km Adgs1qh0en59PyJ/caR1d8E= =Oz3x -END PGP SIGNATURE-
Re: SIGHUP without effect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Hans, Scott, Hans Schnehl wrote: | What I tried to achieve is getting closer to the servers upper limits | in bandwidth/performance, therefore publishing descriptors with different | bandwidths, sometimes within relatively short times. What I did not know | is the authorities' not accepting so called 'cosmetic' changes within a | certain timeframe. This is perfectly logic and makes sense, explains | most my observations, though not all. Correct. dir-spec.txt says that relays generate and upload a new router descriptor when: - Bandwidth has changed by a factor of 2 from the last time a descriptor was generated, and at least a given interval of time (20 mins by default) has passed since then. (They further do in other situations, but this is the only one that is bandwidth-related.) The directory authorities perform the same check when deciding whether or not to store a router descriptor. But you are right, this doesn't explain why Tor should stop publishing descriptors afterwards. Scott Bennett wrote: | I first noticed that traffic had dropped to only a few, occasional | packets per second (not even every second). I then checked the most recent | consensus and status documents and found that MYCROFTsOtherChild was missing | from them. What log statements did you have in mind? Perhaps something like | Oops! I forgot to upload an updated descriptor, so I'm going into hiding? | It forgot to do it, so it had nothing to log. Ah, that's reasonable. Well, log statements are my primary source to see that something is going wrong. So in this case I wondered which log statement might have lead you to your conclusions. I didn't find one that was directly related, so I asked if there was another one I might have overlooked. After all, it could be something like There is something wrong with the descriptor I just wanted to upload, so I'll try again later (or something similar). So, it makes sense to turn on logging in this case (probably even on debug level) to track down this bug. | That was the next thing I | checked. If left unmolested, tor normally uploads a new descriptor to the | authorities about every eighteen hours, but if it forgets to do that, then | that's pretty much the death knell of its usefulness until it has been | restarted. Giving it a SIGHUP at this point does not then cause a new | descriptor to be uploaded, apparently regardless of ExitPolicy changes. Just to get this right: You changed your configuration once or multiple times to force Tor to upload new descriptors, and at some time later (about 18 hours) you realize that it has stopped publishing new descriptors, correct? At that point even changes to your exit policy don't make Tor publish a new descriptor any more, but you have to restart it. (All of this should be fine to do, I'm just trying to understand what is happening.) Okay, I'll try to reproduce the bug myself and have a second look at the code. However, if either of you finds out information that might help in finding this bug, please share. Thanks! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFImcLR0M+WPffBEmURAo9mAKCXg+7aQqQW4xdkP2cPIGVrY7NG9gCfQauy a3T02ufl9vCEXqt14jt0p/I= =2PjA -END PGP SIGNATURE-
Re: questions about MinUptimeHidServDirectoryV2 in 0.2.1.2-alpha
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Scott, Scott Bennett wrote: | The default of 24 hours ensures that hidden service directories are | available for the next few hours with a certain probability. The idea is | that there are hundreds of hidden service directories at some point | which are not authoritative any more, but provide a more scalable and | robust storage than the three authoritative ones can. Hidden services | and clients need to have a view as consistent as possible of which | hidden service directories are out there, so that clients can find | previously stored hidden service descriptors. The 24 hours have turned | | How is that different from the situation of normal directory mirrors? The big difference is that every hidden service directory is responsible for a certain set of hidden service descriptors and not all of them. If you run a hidden service you determine six responsible hidden service directories depending on a) the onion address of your service and b) the node identities of the hidden service directories. Then you store your descriptor on exactly those directories. Your clients need the same or at least very similar information about hidden service directories as you have. If their list of directories contains further directories or misses some of those that you know, your clients will end up requesting your descriptor from the wrong directories. A further difficulty comes from the fact that your clients and you might use consensuses with up to two hours time difference. A certain number of differences is tolerable by performing replication, but these shouldn't get too big. | In other words, it is already covered by the Guard and Stable | flags from the authorities, right? These flags have different purposes, and their definitions might change in the future (as might the definition of HSDir). Apart from that there needs to be an HSDir flag anyway to denote which relays want to participate in the hidden service directory. Admittedly, a further evaluation could compare the approach taken here with your approach to derive usefulness of a hidden service directory from Guard and/or Stable flags instead. Honestly, I don't know what the result would be. Seems that there's always work left to do. ;) | http://freehaven.net/~karsten/dirnodesminuptime.pdf | | It's very pretty, but, given the legend, which I assume denotes | uptimes in hours, the axis labels are not helpful. What exactly do | Directory Size and Time Index refer to? Directory Size refers to the size of the distributed hidden service directory, assuming that *all* relays that have their directory port open work as hidden service directory. Of course, this number differs significantly from the current number of hidden service directories. That's why one proposed change of proposal 143 (which is planned to be implemented in 0.2.1.x) is to make all relays with open dir port act as hidden service directories by default, with the possibility to opt-out. Time Index is admittedly a bad axis description, which however comes from the fact that I didn't know how to write month names at the appropriate places of the x axis in R. :) The x values denote the hours of evaluated consensuses between mid-January and end of March. The peaks that you see for minimum uptimes, e.g., of 4 to 16 hours are the days in that interval. Now compare the peaks of using no minimum uptime at all with those of requiring an uptime of 24 hours. If there would be no requirement for minimum uptime, replication would have to be increased significantly. | The option MinUptimeHidServDirectoryV2 is mainly there to perform tests | with the distributed hidden service directory without having to wait for | 24 hours. It is not required to set it in the public Tor network. (It | only has an effect on directory authorities anyway.) | | I understand that, though it is also useful for the operators of the | current authorities should the policy change. What I still don't see is | the need for a 24-hour delay before a server stops being only potentially | useful and becomes actually useful. Earlier today the HSDir count was | down to only *4*. How is it thus helpful to keep other servers from being | available for use? I think I have already answered these questions above. If I should have left out something, please ask. Hope this helps a bit more now. :) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFImcwL0M+WPffBEmURAiUkAJ9MIRiesdLEenA9BxSJJ4T6mC2fZACfYWNT 2Net1ADWsE/ywa7V1l4Iqtw= =IlPj -END PGP SIGNATURE-
Re: questions about MinUptimeHidServDirectoryV2 in 0.2.1.2-alpha
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 | The tor man page says, | | MinUptimeHidServDirectoryV2 N seconds|minutes|hours|days|weeks | Minimum uptime of a v2 hidden service directory to be accepted | as such by authoritative directories. (Default: 24 hours) | | My questions are, what is the justification for the default of 24 hours? And | why have this particular option at all? Why not instead have a no longer | fresh/up to date indicator somewhere, much like the fresh-until line for | consensus/status documents, so that a server that beomes disconnected or goes | down for only a brief time will remain available to provide hidden service | directory service as much of the time as possible? Or, better yet, why not | simply handle this issue the same way that it is handled for normal directory | (mirror) service? The default of 24 hours ensures that hidden service directories are available for the next few hours with a certain probability. The idea is that there are hundreds of hidden service directories at some point which are not authoritative any more, but provide a more scalable and robust storage than the three authoritative ones can. Hidden services and clients need to have a view as consistent as possible of which hidden service directories are out there, so that clients can find previously stored hidden service descriptors. The 24 hours have turned out to be a characteristic that allows distinguishing highly available relays from others. The rationale behind it is that a certain number of relay operators turn their relays off over night. The following diagram shows the variation of relays with different minimum uptimes over an interval of 2+ months. You can see the difference between minimum uptimes of 16 hours and lower and those of 20 hours and higher. That is the reason for the default of 24 hours. http://freehaven.net/~karsten/dirnodesminuptime.pdf The option MinUptimeHidServDirectoryV2 is mainly there to perform tests with the distributed hidden service directory without having to wait for 24 hours. It is not required to set it in the public Tor network. (It only has an effect on directory authorities anyway.) I should probably make the design paper of the distributed hidden service directory available rather soon. It answers questions like yours. Hope that helps! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFImE8S0M+WPffBEmURAseDAJ9zbmc9Fr0u1NDSdfBZCMf3IHxAnwCghAYp ioWjbih5vuaFVbydCthSGu0= =BusG -END PGP SIGNATURE-
Re: SIGHUP without effect
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Scott, Hans, this might indeed be a bug, or at least a behavior that should be changed. However, some more information about it would be helpful in finding out what exactly is wrong. The first thing is a description of what needs to be done in order to reproduce the problem. Is it just setting up a relay and, well, waiting? If so, how long does one have to wait? Can you provide your torrc files? Could you reproduce the behavior yourself after you experienced it the first time? What was the first Tor version that had this problem, and can you confirm that previous versions didn't have it? At some time, preferably when we have more information about what's going on, someone should open a flyspray task for this bug. or-talk is not the best place to discuss bugs. :) | On Thu, Jul 24, 2008 at 06:53:36AM -0500, Scott Bennett wrote: | I'm running 0.2.1.2-alpha and have noticed a recurring problem. It | appears that tor is sometimes not uploading a new descriptor on schedule. How did you figure that out? Could you paste the log statements? | Once this happens, it appears that it will *never* upload a descriptor | update on its own, though it can be tricked into doing so by making some | significant change in torrc, then giving it a SIGHUP. I've been trying | to keep an eye on it and forcing it to update when the authorities stop | listing it in the consensus documents by commenting/uncommenting an | ExitPolicy line and giving it SIGHUP. The explanation might be that the authorities don't store router descriptors when changes are considered cosmetic. However, after 12 hours time difference between storing two descriptors, changes should never be considered cosmetic, regardless of how subtle they are. Hans Schnehl wrote: | Very simular, on 0.2.0.28-rc (r15188) sending a SIGHUP did not do what | it is supposed to. Okay, so what is SIGHUP supposed to do here? From the manpage: SIGHUP The signal instructs Tor to reload its configuration (including closing and reopening logs), fetch a new directory, and kill and restart its helper processes if applicable. I might be wrong, but as far as I understand this, sending a HUP signal does not necessarily include uploading a new descriptor that is then accepted by the authorities. Why should this happen if the currently stored descriptor is fine? | Trying to publish new descriptors (bandwidth) lead to quite unexpected | results. The authorities (cached-consensus) simply stopped listing the | node and cached-descriptors(.new) were not updated any more. | This though did not have an immediate effect as there were enough machines | using the node because of the previously downloaded descriptors and | consensus. It simply died away slowly when more and more machines 'forgot' | the node to exist. | Some 12 hours later Tor had to be restartet when it was finally running | on some 30% of its previous capacity, but then uploading the new | descriptors then accepted (or recognized) correctly by the authorities. Well, this behavior is not intended. It would be quite interesting to figure out when this situation occurs. Please provide more information (as stated at the top of this mail). Thanks! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIlr800M+WPffBEmURAtdfAKCGnTNxge5FiOLqlzkVlilzssd4nACgwWB5 dOfs2+JRGiTbRHYb+8l0mT4= =DBNp -END PGP SIGNATURE-
Re: ContactInfo?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hey Nathaniel, | In the torrc file is the ContactInfo option. Here's an example. | | #ContactInfo 1234D/ Random Person nobody AT example dot com | | My question is, what format should I put my GPG key? That doesn't matter so much. The intention of the contact line is not to parse it automatically (previous attempts were not very successful), but are read by humans. In fact, it might be better to obfuscate that line a bit in order to prevent the bots from collecting your address -- or make their lives a bit harder. Further, in most cases your GPG key won't be used to encrypt notice message to you or verify your mails to us anyway. By the way, thank you for running a relay! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIMKJu0M+WPffBEmURAiRYAJ4kt9GAxLzj8ZFb1MvU8k4ZSCljBgCcCwwe 5fjeF2xi8RUc2fH7QLiKuj0= =v3ae -END PGP SIGNATURE-
Re: Tor On Private Network
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ringo Kamens wrote: | Is there a way to make tor not check this file? Any ideas? ServerDNSAllowBrokenResolvConf sounds like a useful option here. Have a look at the last section of proposal 135 that contains a bunch of useful config options for private Tor networks: https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/proposals/135-private-tor-networks.txt - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIId6L0M+WPffBEmURAgwRAKDB4oSnUO7l6fx92CDJkF5snJ3H1gCeKA0p ybDyFPiLHoogcOXUfxtu4A8= =ZHHB -END PGP SIGNATURE-
Re: .onion sites fail to load with: (waiting for rendezvous desc)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, | [notice] Tried for 120 seconds to get a connection to [scrubbed]:80. | Giving up. (waiting for rendezvous desc), followed by a Privoxy 502 | error. Some days ago there was a guy on #tor with a similar problem. You? :) After trying out some things which did not appear to have any direct effect, it suddenly worked again for him. I just tried to access two hidden services. Worked fine. But obviously, there is a bug that we should track down. Maybe you can help us with that? | Others have verified the .onion sites I try to connect to as up and | running, but I can almost never connect to them anymore. I'm running | the latest Tor alpha. Every other use of Tor works fine, only this | problem when trying .onion sites. What does almost never mean? How often does it work? What happens when you restart Tor and try again? What if you wait for some minutes after starting Tor before trying? With latest Tor alpha you mean 0.2.0.19-alpha? Did you try to compile and run current trunk? Unfortunately one bug fix for 0.2.0.19-alpha turned out to be erroneous and was reverted in current trunk. But I doubt that it's the one causing the problem you describe. What versions are the people running who say that it's working for them? Can you tell the .onion address in public or in private mail to me? Or another service that you can tell? If not, do you know by any chance who is running such a service and whether they are running version 0.2.0.10-alpha or higher? Can you reach the example hidden service http://duskgytldkxiuqc6.onion/ or my v2 test service http://uulvbbixcufpmmg4.onion/ ? Thanks for your help! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHt0u+0M+WPffBEmURAlZiAKCyKH9YJcyy9PQx2j5n54Z1lvWhVACfcGaw K1GuMu9rxNeUpIACfMVYzLM= =XXuE -END PGP SIGNATURE-
Re: puppetor setup
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi skaoth, | When I execute the program I get this error: | | Feb 12, 2008 10:57:21 PM de.uniba.wiai.lspi.puppetor.impl.ProxyNodeImpl startNode | WARNING: Could not start Tor process! Tor exited with exit value 255! Please go check the config options in /home/skaoth/workspace/TorTest/test-env/1202885841439/proxy/torrc manually! Did you try to run Tor with the mentioned config file like tor -f that-torrc-file? There is probably a config option in that file that is unrecognized by Tor... ... which brings us to the question which Tor version you use. Unfortunately PuppeTor only works with versions around 0.2.0.7-alpha which was the last version that I used it for. Later versions require v3 directory settings and earlier versions don't provide some of the config options (for example ClientDNSRejectInternalAddresses is such a troublemaker). So, you could either try to run PuppeTor with other Tor versions, or remove those unknown config options before writing node configurations (see comments below for how to do that). Sorry for the inconvenience! I planned to make PuppeTor compatible with current 0.2.0.x versions for a long time, but always deferred it for other things. Well, at least you helped prioritizing this task in my todo list. :) Ah, damnit, this is open source: Feel free to send a patch. :D | I've also go questions about PuppeTor as far as simulating exit-node policies | and was wondering where/who/list to contact You can change the configurations of all nodes created by PuppeTor, either by changing their template (for directory nodes, routers or proxies) or single configurations, e.g. with the methods ProxyNode.addConfiguration() or Network.addTemplateConfiguration(). Please also have a look at the Javadocs for that. If you have questions about PuppeTor, you can either mail me directly or bother the or-talk list. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHtVvg0M+WPffBEmURAigxAJ9Euv/0GXuSCbzF5k/59Ox8uXH6swCeOL8W lGJ0y+Ax74FrVp2/OzNSM6A= =lxID -END PGP SIGNATURE-
Re: HidServDirectoryV2 option
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 | Is there a design document on this DHT-like thing? Yes, there are multiple documents on different technical levels. The first is my GSoC 2007 application which contains the general idea, some pre-studies, and a brief security discussion; however, the design as described there has slightly changed while writing the specification and implementing it, so it is only about 90 % accurate: http://www.uni-bamberg.de/fileadmin/uni/fakultaeten/wiai_lehrstuehle/praktische_informatik/Dateien/Forschung/Tor/loesing-distributed-storage.pdf Then, proposal 114 contains a more accurate description of the design as it is implemented now, but with fewer explanations: https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/proposals/114-distributed-storage.txt The relevant parts of the proposal are also included in rend-spec.txt: https://tor-svn.freehaven.net/svn/tor/trunk/doc/spec/rend-spec.txt Just in case you need something more citable: I'm currently writing a paper about it (and some other stuff). If you like, I could send you the submitted version (as soon as it is submitted) via private e-mail. If you have comments on any of these documents, please feel free! Hope this helps! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHnksW0M+WPffBEmURAraxAKCT6X4z+tFOGSRcD3xN9QHfuqmqxwCgh4KF 3D97PHXQr8YFqv9eG1jzhBE= =mb7t -END PGP SIGNATURE-
Re: tornode lefkada
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 | Every minute there is a log message: | | [Notice] We're missing a certificate from authority lefkada with signing key : launching request. | | I put lefkada in my excludenode list, no change. Nobody can remove this node from the list of nodes? There have been several reports of this problem and several answers. Here is one: http://archives.seul.org/or/talk/Jan-2008/msg00186.html - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHig/e0M+WPffBEmURAoOrAJ9ShvWIVKWb6ZmXPefVOcsjsNO0FQCgohF0 MvrGHvDvF9yxldm7B8cvQtg= =M6WB -END PGP SIGNATURE-
Re: Suspicious Circuits
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
A) This explains why it is trying the old introduction points, and it
explains why it's building a long circuit trying each one in turn. But
why is it trying the same introduction point more than once?
Uhhhm, right. The problem is that introduction points are not removed on
failure, which they should be. It's quite unusual that introduction
points fail (and most people won't notice that), but okay, we should
better fix it. My first guess is that this problem was introduced in
0.2.0.7-alpha (ChangeLog entry: - Hidden services were choosing
introduction points uniquely by hexdigest, but when constructing the
hidden service descriptor they merely wrote the (potentially ambiguous)
nickname.) and that hex-encoded identities are compared with nicknames
when removing introduction points. But it's quite hard to tell without
running it. On the other hand, this change was also backported to
0.1.2.18 which did not have this problem. Hmm.
Let's find it out. :) As discussed on #tor yesterday, I attached a patch
to this message containing some more logging statements. Could you,
Kyle, apply it and run the same configuration as you did last time? You
should also enforce the same timing problem, because without it's
unlikely that intro points fail and respond with a NAK message.
Thanks for that! :)
B) Do you think it's possible to reduce the 30 second delay to make
this sort of behavior happen less often? It would be nice to have hidden
services launch more 'immediately'. But on more thought, I think 30
seconds may already be a bare minimum, if we consider users on crappy
connections setting up hidden services. Hm.
Good question. 30 seconds are not really much compared to the overall
performance of hidden services. How important is it that hidden services
are more immediately available after setting them up? This is only done
once in a while. Does this affect user experience so much? I think that
the behavior in Kyle's case is a really rare event, compared to the
normal use cases for hidden services. So, I would not change it in favor
of fewer and more accurate descriptor publications.
- --Karsten
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFHak1D0M+WPffBEmURAhe+AJwMWLUlZCiroiU7BBxIwL5vkd813ACfc4Q8
AKAxnBr9Apq13uhz5gubZIM=
=jY7A
-END PGP SIGNATURE-
Index: /home/karsten/diss/tor/tor-0_2_0_12_alpha/src/or/rendclient.c
===
--- /home/karsten/diss/tor/tor-0_2_0_12_alpha/src/or/rendclient.c
(revision 12873)
+++ /home/karsten/diss/tor/tor-0_2_0_12_alpha/src/or/rendclient.c
(working copy)
@@ -337,11 +337,21 @@
return 0;
}
+ log_info(LD_REND, Removing failed intro point for service %s.
+We still have a valid descriptor with %d intro points.,
+escaped_safe_str(query), ent-parsed-n_intro_points);
if (ent-parsed-intro_point_extend_info) {
+log_info(LD_REND, The descriptor has its intro points stored as extend
+ infos rather than nicknames. Failed intro point is %s,
+ hex_str(failed_intro-identity_digest, DIGEST_LEN));
for (i=0; i ent-parsed-n_intro_points; ++i) {
+ log_info(LD_REND, Comparing with intro point %s...,
+ hex_str(ent-parsed-intro_point_extend_info[i]-identity_digest,
+ DIGEST_LEN));
if (!memcmp(failed_intro-identity_digest,
ent-parsed-intro_point_extend_info[i]-identity_digest,
DIGEST_LEN)) {
+log_info(LD_REND, Found a match! Removing intro point.);
tor_assert(!strcmp(ent-parsed-intro_points[i],
ent-parsed-intro_point_extend_info[i]-nickname));
tor_free(ent-parsed-intro_points[i]);
@@ -352,11 +362,19 @@
ent-parsed-intro_point_extend_info[i] =
ent-parsed-intro_point_extend_info[ent-parsed-n_intro_points];
break;
+ } else {
+log_info(LD_REND, No match.);
}
}
} else {
+log_info(LD_REND, The descriptor has its intro points stored as
+ nicknames rather than extend infos. Failed intro
+ point is %s, failed_intro-nickname);
for (i=0; i ent-parsed-n_intro_points; ++i) {
+log_info(LD_REND, Comparing with intro point %s...,
+ ent-parsed-intro_points[i]);
if (!strcasecmp(ent-parsed-intro_points[i], failed_intro-nickname)) {
+log_info(LD_REND, Found a match! Removing intro point.);
tor_free(ent-parsed-intro_points[i]);
ent-parsed-intro_points[i] =
ent-parsed-intro_points[--ent-parsed-n_intro_points];
@@ -361,9 +379,13 @@
ent-parsed-intro_points[i] =
ent-parsed-intro_points[--ent-parsed-n_intro_points];
break;
+ } else {
+log_info(LD_REND, No
Re: Suspicious Circuits
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Roger Dingledine wrote: On Sun, Dec 09, 2007 at 09:19:53PM -0800, Kyle Williams wrote: I've been having problems getting to hidden services the last couple of days. I noticed something odd in Vidalia the other day, but it was gone before I could take a screenshot. However this evening, I was having a lot of problems with .onion addresses, and Vidalia was showing several (more than 6) nodes in a circuit almost every time I tried to reach any hidden service, including my own. Exciting. Looks like a bug of some sort. No, I don't think it's a bug. - From the log file that you, Kyle, gave me yesterday I can see that you started Tor at 16:04:51 which established introduction points at Slowpoke, dpujtk, and server and published a descriptor for your service at 16:05:27. The delay of 36 seconds comes from the fact that Tor waits at least 30 seconds for a descriptor to be stable before publishing it. Then you made a connection attempt at 16:06:26 which succeeded at 16:06:53 and another attempt at 16:07:00 succeeding at 16:07:02. Everything fine so far. Subsequently, at 16:07:12 you restarted Tor and made it establish new introduction points at otherator2, crelm, bytebutlerfive and publish a new descriptor containing these introduction points at 16:07:53. Again, the delay of 41 seconds is intentional. But---and this is the problem---when accessing your service at 16:07:25, Tor downloaded the first descriptor without being able to know that it's obsolete. So, Tor tried to connect to Slowpoke and the other introduction points which were not acting as introduction points for your service any more. That's why you get those NAKs which lead to re-extending the failed introduction circuits which is also normal behavior. Hence, there is not a problem in the Tor code. In general, when performing tests, you should give Tor a little bit more time to stabilize, especially for hidden services. You should also consider not to run both server and client on the same Tor instance. If the problem persists even when waiting for some more time, please report! Timing is everything! :) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHaSwW0M+WPffBEmURAhRlAJ90qOxY2wYA6Vq/sw0VCMzn75zRZgCeJoZH MUzuCbvRrIRN/4705ieI+s4= =Uad9 -END PGP SIGNATURE-
Re: Setting up a private tor network
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Csaba, I have seen similar error and warning messages to what you have mentioned, both with 0.1.2.17 and with 0.2.0.8-alpha. Quoting from your private mail (with your permission): I've seen in your doc that you are killing and restarting Tor instances in order to have the thing running. I have also seen a note on that, but till now I was trying to avoid that, I was hoping that there is a combination of configurations with which things start up smoothly. Before I go into the restart game, I would like to be sure if my torrc files are good or not. In PuppeTor I did not get 0.2.0.8-alpha to work in a private network setting, but only versions up to 0.2.0.7-alpha. Further, the current trunk (or what will become 0.2.0.9-alpha these days) introduces the new v3 directories that make things a little bit more complicated: The solution for building a private network with all versions up to 0.2.0.7-alpha is to periodically send HUP signals to the nodes until they start building circuits. In principal you don't have to, but it accelerates things a lot; as an example, I tried to create a private network with 2 directories and 4 routers _without_ sending HUP commands: 3 out of 10 attempts built circuits after 15 minutes and a few seconds, and the other 7 attempts took 60 minutes and a few seconds for it. The multiples of 15 minutes should come from the interval in which directory mirrors fetch networkstatuses from the directory authorities. When sending HUP signals, the whole process takes about half a minute. The reason is that directory mirrors refetch the networkstatus immediately when reloading their configuration. As a side note: proxies behave differently for this. If you want to read more, have a look at the Javadocs of PuppeTor's ProxyNode class: https://tor-svn.freehaven.net/svn/puppetor/trunk/src/de/uniba/wiai/lspi/puppetor/ProxyNode.java In 0.2.0.8-alpha-dev (and newer versions) you need to configure v3 directory authorities to get things working. There is a description how to do this here: https://tor-svn.freehaven.net/svn/tor/trunk/doc/v3-authority-howto.txt . In order to speed up the process you can configure Tor to build consensuses in shorter intervals. The following configuration worked for me: V3AuthVotingInterval 10 minutes, V3AuthVoteDelay 1 minute, V3AuthDistDelay 1 minute. Unfortunately, the process still takes about half an hour, so this is only a first solution to get it working. If you find a better solution, please let us know! After seeing PuppeTor I've realized that mine is quite similar to it in its goals, [...] First of all, it's good to have multiple approaches to this problem. We could both learn from the other approach and improve our tools. My decision to not use a virtual machine for each node was that I did not see why it should be necessary. In PuppeTor every Tor node has it's own working directory and set of ports and should not interfere with the other local Tor processes. The only output that I care about is what Tor writes to its log files. My primary motivation for writing PuppeTor was to test my developments on Tor hidden services which are rather high-level in Tor. However, when it comes to lower levels, like sniffing or altering packets, my approach might be too limited. I'm not sure about that, because I rarely used that. Thus, there is room for other approaches! :) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHH2+h0M+WPffBEmURAkpEAKC3NsvLDFvc4uu52OwYEPSSBy84kQCgnnOk g+jjUhZrPXutUSQ0hIIcSPs= =520x -END PGP SIGNATURE-
Re: Setting up a private tor network
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Shreyas, But nowadays when i start the network is says do not have enough directory information to build circuits. Which Tor version do you use? I had a potentially related problem with the current trunk version that had to do with private IP addresses and the directories. You could try to set the new config option ClientDNSRejectInternalAddresses to 0. That option is not described in the wiki, yet. But I'm not sure if that will solve your problem, too. Apart from that you might consider using PuppeTor for creating private Tor network configurations and running whatever you want to test in it. We developed it for testing and measuring hidden-service related things, but it could also be useful for you. It also contains all our wisdom measured in necessary configuration options and sending HUP signals to create private Tor networks. You can find it here: https://tor-svn.freehaven.net/svn/puppetor/ - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHCSCK0M+WPffBEmURAo7oAKDO5KMXelzav5I7b+Bqb1YAxfqE+QCfajSc IHSdYr0Ksp6NVezk10tOq/c= =3evC -END PGP SIGNATURE-
Re: Setting up a private tor network
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I am using 0.1.2.17. I am planning to run an application over tor so i was not sure puppetor will work. I think i will try using that. Then you might encounter problems with 0.1.2.17, because PuppeTor is configured to be used with the development versions. This is kind of a dilemma: Newer Tor version require certain configuration options to be used in a private setting which are not understood by older Tor versions. So, you will need to remove some configuration strings before being able to use PuppeTor with 0.1.2.17. Or use the trunk version. Or I could include a version check and select configurations appropriately -- sometime. You could also use PuppeTor only to establish and initialize private network configurations, without performing actual test. Afterwards, you can re-use the working directories with their configuration files and state files and start the Tor processes on your own. Up to you. My problem is that the logs say that there is enough directory information but still it does not try to make a circuit. I changed the code so that it builds circuits all the time. But, it is like tor is not running at all. It is supposed to make a circuit once it gets directory information but is not doing so. Are there any reasons why it is not able to do so? Hard to say without your log files. From PuppeTor I know that newly configurated private Tor networks require multiple reloads before being stable. And this process also fails quite often. In general you should not have to change the Tor code to create a private Tor network. Maybe your changes are what prevents Tor from working properly?! Could you try whether PuppeTor is able to create a private network configuration for you -- with your changed and the unchanged Tor? If you have specific questions on PuppeTor, e.g. how to configure it for 0.1.2.17, you could also mail me off the list. And if this all fails, you could post a link to your info-level log files here. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHCV/O0M+WPffBEmURAnmuAKCXzm/layHGwWeEWmhRFx25PPlKLgCgrQUJ 84LpzLGGnTD5GesN35Eh/mM= =YIv6 -END PGP SIGNATURE-
Re: Rejecting truncated ESTABLISH_INTRO cell warns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi everyone, A number of server operators have been reporting seeing these lines in their logs lately: Sep 25 04:59:13.165 [warn] Rejecting truncated ESTABLISH_INTRO cell. ... Hmm, it looks like it was us (our university working group), trying to include authentication to Tor hidden services. For the moment, consider this as a rather agressive marketing strategy to promote our new proposal on hidden service authentication, which we by the way just sent to or-dev: http://archives.seul.org/or/dev/Sep-2007/msg00026.html And the plan worked, didn't it? ;) However, if this explanation did not convince you: It simply was a modified Tor client going wild. The test was intended to be performed in a private Tor network, but for some reason our Tor nodes connected to public Tor nodes. Sorry for the confusion! In the future, we will try to keep our packets at home. If we should fail at this, please mail me, as it is unlikely that it was someone else. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFG+mw90M+WPffBEmURArt/AKCEcqTHBrp8JxHRAKwbOlV96mXTMgCglObT 6QjbfU/9QTmq+KyMewWLoks= =a/rI -END PGP SIGNATURE-
Re: Proposal of a new hidden wiki
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I like the distributed private key idea. Yes, that's really a nice idea. And it might even work. My question is: what would determine which server got chosen? I think that if two or more hidden services used the same private key, thus the same .onion hostname, the master servers would always point to the latest updated. Correct. A hidden service uploads a current descriptor (containing contact information) if a) there is some significant change in contact information or b) an hour passes. Then, they could, just to equal host usage, schedule tor restarts each 2 hours. So, in even hours host A would respond, and in pair hours host B would respond. And this, automatically. That's a bad idea, because it does not really improve availability if a hidden service is restarted every two hours. The two services should rather be run in parallel all the time. Then, after some maths, one would (probably -- am no mathematician) find that both services have their own descriptors published half the time, and thus receive half of the client accesses. (Note that the one-hour intervals break as soon as the list of introduction points changes -- that means that starting the nodes with a certain timing does not significantly improve this solution.) However, I am quite sure that the developers did not have this variant of content replication in mind when they designed the hidden services. That means that it might break. But why not try it? :) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGuhxn0M+WPffBEmURAubmAJ9Or3XmcxgmnGxXJgDHGSXHPvaK5gCbB90/ qeNETEE1FYc9bNxUeJi8niU= =8nZG -END PGP SIGNATURE-
Length of new onion addresses
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 (posting to or-talk and or-dev, because it concerns both, usability and development) Hi, at the moment I am designing the new ASCII-based format for hidden service descriptors, including new security features like encryption of introduction points and the ability to be distributed among onion routers unpredictably for non-clients. This incorporates a secret cookie that needs to be passed between the hidden service provider and his clients in addition to the service id which is the current onion address. You can read about all the details in proposal #114 in the svn repository. As the new descriptor might replace the current descriptor some day and the format of onion addresses would affect all hidden service users, I would like to discuss this decision of the onion address format in public, rather than make a decision on my own and being confronted with incomprehension when it might be introduced. The current onion addresses consist of 80 bits and (as you all know) look like this (address of the hidden wiki): http://6sxoyfb3h2nvok2d.onion/ The new onion addresses would consist of two parts: (1) the service id and (2) the secret cookie. (1) In contrast to the current format, the service id is not used to identify the service (bad name then, I know), but to generate an unpredictable descriptor id where to find the service descriptor. If an adversary can create an own key pair with a fingerprint equal to the service id, she can prevent the actual hidden service from announcing his service. Though, the effect of this is limited, because descriptor ids are automatically changed every day. My idea was to use 32 bits for the service id. (2) The secret cookie is the key for encrypting and decrypting the introduction points and to calculate the current descriptor id. Whoever finds out the secret cookie could observe hidden service activity and attack introduction points which both would otherwise not be possible. My plan was to use a 128 bit key as secret cookie. In total, new onion addresses would be 160 bits long. The question is now, if an onion address of that size is still manageable for human beings? (Is the current size manageable after all?) For illustration purposes, the new addresses would look like this: http://6sxoyfb3h2nvok2d6sxoyfb3h2nvok2d.onion/ Or are my assumptions concerning the length of the service id still too incautious, and would 200 bits (72 bits for service id and 128 bits for the secret cookie), resulting in the following onion address, be better? http://6sxoyfb3h2nvok2d6sxoyfb3h2nvok2d6sxoyfb3.onion/ For downward compatibility reasons, those 200 bits could also be distributed by using 80 bits for the service id and 120 bits for the secret key. Then, people could start using the new descriptor by simply adding a dot and a secret cookie to their current (unchanged) onion address. This would look like this: http://6sxoyfb3h2nvok2d.6sxoyfb3h2nvok2d6sxoyfb3.onion/ To the (probably upcoming) question, why one needs a secret cookie at all, or if it could also be used optionally in the long run: The plan is to distribute the storage of descriptors, primarily for scalability reasons. But this raises new security issues, because anyone running a stable onion router could become responsible for storing a descriptor, so that we simply need new security mechanisms. Otherwise, security would be worse by the distribution, but with the secret cookie, security even gets better than before. But perhaps we should rather aim for usability than for security and use only 120 bit long onion addresses, e.g. by using 32 bits for the service id and 88 bits for the cookie, resulting in the following onion address? http://6sxoyfb3h2nvok2d6sxoyfb3.onion/ Maybe we shouldn't even extend the onion addresses at all, but allocate the 80 bits in another way, e.g. 24 bits for the service id and 56 bits for the secret cookie? Then we should use another virtual top level domain to distinguish current and new descriptors, resulting in something like the following: http://6sxoyfb3h2nvok2d.hidden/ What do you guys prefer? How do you exchange onion addresses? Publishing them on non-hidden web pages, pasting them to IRC chats, writing them on business cards, memorizing and telling them, ...? I think it's important to find a balance between security and usability here. The question is: Does size matter? :) Any comments are welcome! Thanks! - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGX/ks0M+WPffBEmURAg42AJ9l6aDu++f1Ozaesouxxm4d82rdwgCgsC8l l0858q0gkfWYlcOG3odyT+s= =BGOH -END PGP SIGNATURE-
Re: question about A/B communication with dir servers for hidden services
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Are the streams from Bob and Alice to put get the descriptor of a hidden service always established over Tor circuits Yes, they are. or sometimes direct streams from the OP's to the Tor directory server? No, never. In other words: Is it assured, that the directory server doesn't know, that Bob has established a hidden service and Alice has asked about it? Correct, the directory server never learns about the IP addresses of the service provider and its clients. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGX9em0M+WPffBEmURAqJaAJ41JL/Vba+WIC2l5Y1oIiNbjGUHrACgvfrn TQPzLmLsOE0ihY2oPwFPjYY= =aGRk -END PGP SIGNATURE-
Re: Length of new onion addresses
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Michael, Length is not nearly as important as bookmarkability. You mentioned that you are going to be changing stuff every day. That worries me. My bad. No worry, this is just a misunderstanding. What I should have written is that a service's onion address (what clients bookmark or type into their browsers) stays the same all the time. What changes are the descriptor identifiers which are created from the service id and the secret cookie. This allows for storing descriptors on changing nodes all the time, which is a novel security feature that becomes possible from incorporating the secret cookie. It prevents persons from tracking a service's activity or usage pattern. I only mentioned it to stress that the attack of generating a key pair with the same id as an honest service would be limited to one day. Such an attack would become more likely the fewer bits the service id has. But the changing descriptor ids have no impact on the usage by hidden service providers or clients. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGYFh20M+WPffBEmURAhyaAKDU+qHjsTVn1LNsDIsyBP05kXGkrwCeM3yT v8ziwd3VBWtIyv7AEyW1W9A= =Li4l -END PGP SIGNATURE-
Re: All authorities have failed. Not trying any.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Though I have no idea about the __AllDir... stuff, I can give you a first hint to solve it: [Info] update_networkstatus_client_downloads(): Our most recent network-status document (from nobody) is 1180650256 seconds old; and where is that age coming from? that amounts to 37 years 1970-01-01 00:00 + 37+ years = today Now the rest is up to you. ;) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGX1CE0M+WPffBEmURAq9bAKCTiyHDKkJttAuFKbLNtqRsa28aHACeNU4F JU9fxyZBJq1zWtjArWY3WQ0= =G1vB -END PGP SIGNATURE-
Re: Setup Directory Server
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Eric However, I would like to set up my own directory server on a real network (PuppeTor is only a simulation). I could not find the source code or the binary where I can compile and deploy my own directory server, where my own Tor ORs will contact. Any ideas if that is possible? There are no special sources or binaries for directory servers. You can use the same sources/binaries as for an onion proxy or onion router. The only thing you need to change is the configuration in torrc. Additionaly, you need to change the configurations of all clients to contact your directory instead of the public directories using DirServer. I think to remember that you need at least two directories to run a Tor network. If you are unsure which configuration options that are, look into the man page or let PuppeTor create an example configuration that you can change afterwards. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGShso0M+WPffBEmURAgXuAJ0b8d2ypT/dbDJHgX20R537ebL2nwCfY2Cg s9Dtza7WhK2lXMxxhScxOBg= =KqNf -END PGP SIGNATURE-
Re: Setup Directory Server
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 He wants to create an authoritative directory service, like the ones that already exist and run it on the big bad internet. I think this is good and will help decentralize trust. Just edit the torcc comrade. Comrade Ringo Kamens Either I missed the irony in your post, or you have got something seriously wrong. Of course, you cannot simply start a sixth authoritative directory service by yourself that will be trusted by all clients. And if you could, I wouldn't call that decentralization of trust, but a serious security leak. But it's a joke, right? (In this case I would like to promote the use of smileys, so that the silly non-native speakers can understand the real meaning, too.) :) - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGSiFf0M+WPffBEmURApm6AKDRlwe65aacBB85ydkI0H+z47SKpwCeK+qV Xitv/c/MY4JFZvOs6E2Nock= =XTT3 -END PGP SIGNATURE-
Re: Setup Directory Server
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 From what I understand, you can start your own dir server if you email the other dir servers and ask them to trust you. Unless you show drastically different results there's no reason for them not to. Nope, no way! You can run a directory *mirror* by storing and forwarding the directory information of authoritative directories to clients. But you cannot simply run your own authoritative directory server and write an email to the Tor ops so that they trust you and add you to the list of authoritative directories. The anonymity of Tor relies to a certain extent on the trustworthiness of the directories. If you (and your friends) could control a majority of the directories and convince clients to use your own, potentially modified directory information, you could pass different router lists to different clients and defeat their anonymity by observing which routers they pick. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGSihP0M+WPffBEmURAkqIAJ9mK3aOgt7EM2ryG8RN9XBnkXRVqQCfarxf jViK/eCFTQ4KbSix3f2iGmk= =HCy0 -END PGP SIGNATURE-
Re: [Fwd: High-traffic Colluding Tor Routers in Washington, D.C. Confirmed]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, this question is not directly related to the described case. I would like to contribute some more Tor servers running at different providers across Germany (probably not in the same /16 network). My current server is a virtual server at 1blu that has a bandwidth of 931 KB/s which makes it the 71st fastest Tor server in the network. Maybe other providers are even faster than 1blu. Just as a comparison: the fastest Tor server at the moment has 4533 KB/s. Do you think it's a privacy problem to run 3 to 5 servers? All servers would be non-exit servers because of the current habit of the German police to collect all exit servers. Of course, I will set the family entry. Just want to ask in advance. - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGJeir0M+WPffBEmURAt8wAKCvxrHh2adEKZwkTkcMuKEzstGTZgCg0Sai 3Q5QfDp6+Nv8JDhffwBUUGs= =ahDa -END PGP SIGNATURE-
Re: [Fwd: High-traffic Colluding Tor Routers in Washington, D.C. Confirmed]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 What kind of traffic plan to you have with 1blu, and how much do you pay for it? They offer 1blu-vServer Unlimited with unlimited traffic volume for 17 euros per month. I don't know if it's the best offering, so I decided to give them a try. Are there other good offerings for (virtual) Linux servers with unlimited traffic? FWIW, I don't see any problems in running two middleman servers (shrek, shrek2), with proper family setting, of course. OK. Does someone else have scruples about it? -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGJfBs0M+WPffBEmURAkV0AJ4md2knpz29e0XkXbXd3nWcyL8G6QCfTVNl s5eGtelrtzBi2Z2UpiNc9m0= =lzMy -END PGP SIGNATURE-
Re: [Fwd: High-traffic Colluding Tor Routers in Washington, D.C. Confirmed]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Karsten, (strange to write that *g*) do you run a TOR server on a virtual server without connection faults? A year ago, I tested a tor server on virtual hardware (Virtuozzo) and I got many TCP connection faults in /proc/user_beancounters. Is a TOR server now ready to run with less then 1024 TCP connections? Or do you have a virtual server, which does not have low limits for TCP connections? In this case the offer of 1blu is very nice for TOR. At the moment I count 630 TCP connections using netstat. And I don't know about /proc/user_beancounters, but that file is empty. I don't have any long-term experience with 1blu so far. Maybe they shut down my node as soon as they find out why it produces so much traffic. And maybe they change their contracts as soon as everybody is running Tor servers at them from now on. Let's wait and see. - - Begin Off-Topic --- I know, it is a Tor list. But please let me write this: What do you think about a remailer (Mixmaster or Mixminion), something like TOR for emails. Emails are more private than surfing in my opinion. If you did have the power to admin a few tor server, you may run a remailer too. It may share a server together with TOR. The traffic is not very high: 5.000 mails per day. It uses at max. 16 TCP connections. And it can act as a middle-man like TOR. For Mixmaster a working MTA (exim4 or something else) is required, for a Mixminion middle-man nothing. The size of the remailer networks decreases in the last 6 month down to 35 nodes for Mixminion and less than 30 nodes for Mixmaster. Hope, we can stop this trend. Large networks for high anonymity are needed. I am ready for help, if somebody needed any docs. (in German too) Personally, I don't know so much about e-mail anonymizers, yet. So, if you have information that I cannot find in a two-minutes Google session, yes, please send it to me. - -- End Off-Topic -- - --Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGJnGz0M+WPffBEmURAg0+AKDUnONqZSlnhxxb/29QWIevsg1tbgCgza10 9NGVDrMDsAxIVj5oDGswbbE= =9zMm -END PGP SIGNATURE-
GSoC project: Distributed Tor Directory
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi everyone, Google has announced which applications have been accepted for Google Summer of Code. I am happy to say that my application has been accepted! :) It's about distributing the Tor directory for hidden service descriptors (not router descriptors) to all onion routers or at least a subset of it. If you like to read more on my project and its progress, stop by at http://distributed-tor-directory.blogspot.com/ What about the other three students who have been accepted for a Tor project in GSoC? Who are you? Show yourself! :) Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGH1hA0M+WPffBEmURAlpIAKCXiVhkfhSm+aJm9fnHdNrrUEIT7QCeMEsP UaC77AG9qPxHO0QGAq4zPKU= =mgJK -END PGP SIGNATURE-
Re: Is this for real?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yeah, that's the problem, given it says NSA, who knows for sure. As arrogant as govt. has gotten in the last few years it would not surprise me one bit to see them run a server under that name. But wouldn't they have even more fun, if they called their server WeLovePrivacy? At least, I would have, if I were them... ;) Don't worry, the probability that it's an NSA server is the same as for every other server in the Tor network. Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGDmj50M+WPffBEmURApuKAJ41aZS/OgkrljDaX/KkK+XzGLU1zwCfZl4S Wrgmie15uH96Wtf6hYgjVhE= =J1BJ -END PGP SIGNATURE-
Re: Example hidden service issue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Mike, In the documentation it tells you to set up an example hidden service pointing at google.com, eg: HiddenServicePort 80 www.google.com:80 I've just started looking at hidden services so I'm not exactly sure how they work yet, but if I'm correct, by setting that up and testing it surely you'll be connecting to www.google.com on port 80 from the server with your hidden service and doing a: GET / HTTP/1.1 Host: youronionaddress Wont that give google a map of Real IP - Hidden service name? In fact, that is not the information you want to hide. The server that is to be hidden may know which Tor node is actually hiding it. Hidden services are meant to hide the locations of the servers (here: Google) from others. Perhaps it's better if you think of another server than Google which you would like to hide. I mean, for me, Google means the opposite of anonymity---apart from Google summer of code supporting Tor which is a step into the right direction. ;) If you set up a hidden service, you provide access to a service in the non-Tor network to a client connecting to you over the Tor network (simplified picture): client -- Tor proxy -- some Tor routers -- Tor proxy (YOU) -- Google You advertise the server to the Tor network using an onion address. As soon as you receive a request to the hidden service from a client, you connect to Google with your own IP, perform the request, and respond to the client over Tor. I hope that this makes it a little clearer to you. Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGDnfk0M+WPffBEmURAjrFAKC/IovXsmvrTeVhlhu4MLkkvKWSTACdFi+F zlY9cyJMpdZFdUij/z95ebc= =s9c6 -END PGP SIGNATURE-
Re: Example hidden service issue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, That's exactly the way I should have described the issue in my original post. I didn't think I'd need to spell it out in so much detail. :) Was that me confusing everyone?! :( Sorry for that, my fault! The descriptions above seem right to me. If you assume that everyone that has set up a hidden service has done the google test as described in the documentation and hasn't then changed the onion address afterwards. Also assume that google logs the Host header, eg using apache common+host format and that they archive the logs. This gives google the ability to grep for an onion address and get the real ip of the hidden service if they're ever asked for it. Further to this, there is still a problem even if you *do* change the onion address after doing the test. The fact that google can see that someone was testing setting up a hidden tor service from a particular IP on a particular date is often going to be enough info to expose the *probable* real location of a hidden service. These could indeed be new threats to hidden services; the first being more threatening than the second. I could imagine that nobody has ever thought about an untrustworthy (to be hidden) server, but only about all the other untrustworthy nodes in the network. I assume I also need more thinking on that... and more coffee... Maybe it could help to switch steps one and two in the howto? First set up the web server and try if it's available over http://localhost:5222, and then make it available over Tor. Or is there a special reason for this order that I overlooked? Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGDpqy0M+WPffBEmURAqIdAJ91mYQp37R9vfW4IbJXPtTUF9twfwCfWlUK ziM7iOR7SiSP3j2eaEQvR34= =djF6 -END PGP SIGNATURE-
Re: Hidden services
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi JT, I read the docs and slides on hidden services. But I still don't quite get it. Maybe I can help you with this. On slide 19 it looks as if there was only one hop between the client and the server. Is this the case or has the diagram been simplified? All connections to introduction and rendezvous points are sender-anonymous. This is depicted by the big onions on the slides. These connections consist of more than one hop just as with circuits to public servers. The standard hop count for each sender-anonymous connection is 3. If only client and server are for real and the all tor servers along the path are compromised then can the operator find out what the hidden service is offering and who is communicating. Well, if _all_ Tor servers in the path from a client to a hidden server were compromised, they could find out that the two are communicating. Communication between the two is still end-to-end encrypted from the client's to the server's Tor node. But the adversaries could make an own attempt to connect to the hidden server and find out what it is offering. Anyway, we are talking about at least 6 routers of which 3 are picked by the client and 3 by the hidden service. So, it's not so likely that they are all compromised. In fact, this is what Tor relies on. I think, you should not be too nervous about that kind of attack. Inside the Tor network(not using exits) everything is encrypted, right?! So does the last node in the path, connected to the hidden service know, that it is talking to a hidden service? As far as I understand hidden services can be run by servers and clients. The last node in the circuit, which is closest to the hidden server, does not know that it is talking to the hidden service. The hidden server opened a circuit to that router as done with every other circuit. So, this router cannot conclude what the hidden server is doing. It could also be - which is more likely - a usual client. If you are more interested in attacks on this, you might want to read the paper by Øverlier and Syverson on locating hidden servers. Hope this helps. Karsten -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGA5qm0M+WPffBEmURAnRiAKCi0SCx4kD7nqh/7Y1zAFtFOZO7BgCffIMP UpT0Vm7Bs7OUu9wn1UDsMCc= =9Lkd -END PGP SIGNATURE-
