Re: Suggestions for Advocacy
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 pho...@rootme.org wrote: > On Fri, Dec 19, 2008 at 08:25:49PM -0600, onionrou...@gmail.com wrote 1.6K > bytes in 28 lines about: > : I'm a member of one of those local organizations that regularly presents > : information related to security, technology, etc to each other. I'm > : wondering if there are suggestions for local advocacy and training > : guidelines so not to mis-represent Tor (although it would be obvious that I > : have no directly affiliations). I read on the three year road map that > : training and promotion is a priority of course but discussion about > : anonymity systems can become a heated topic especially related to the > : Privacy Debate. Maybe something from the CCC meetings or one of Roger's > : presentations. > > A number of people have created their own presentations. Roger's > presentations are listed at > https://www.torproject.org/documentation.html.en#UpToSpeed numbers 10 > and 11. > > Depending where you are, someone from Tor may be available to show up > and also help out. > > But you're right, it would be good to have a general Tor presentation > template/demo for others to use for their own advocacy purposes > available on the website. I'm happy to work with others on such a thing > that we can give away with some creative commons license. > I've been giving workshops on security for activists for about a year now. One of the sections includes a tor walkthrough, I'm in the Oly Hackbloc (olyhackbloc.org) and we're doing another workshop in mid-jan. All of our speaker notes and slides will be available online at that point. Ringo -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFJTIBH6pWcWSc5BE4RAsSpAJ4x/vcpnXhRnlPpdyEUEtNkxzk2zgCfZBEl QauwzEvjsZJzsekyxa222+8= =jfv5 -END PGP SIGNATURE-
Re: Performance optimizations for high-bandwidth Tor exit
On Sat, Dec 20, 2008 at 03:13:00AM -, 6cnf6c...@sneakemail.com wrote 1.1K bytes in 26 lines about: : Most of the time, the Tor process maxes out the CPU (85-100%), : while memory consumption stays at ~10%; until today, this didn't : pose much of a problem as log files show no errors and the machine : has been used exclusively for Tor. Any chance you could run oprofile and tell us where the cpu is burning cycles? My guess is it's in openssl over actual tor code. : I now want to play around with hidden services, and noticed that : Apache takes a very long time to reply, even to local requests. Is this because the cpus are consumed with handling tor? : configuration much. Also, niceness is set to +15 for the tor : process. I want to avoid setting a fixed bandwidth limit, Tor : should use "as much as it gets". Will upgrading the CPU help? So, my personal experience is that the switch from Xeon to AMD64 cpus dramatically changed the cpu usage with as many other variables staying constant. However, it's probably not a practical change for you. -- Andrew
Re: Suggestions for Advocacy
On Fri, Dec 19, 2008 at 08:25:49PM -0600, onionrou...@gmail.com wrote 1.6K bytes in 28 lines about: : I'm a member of one of those local organizations that regularly presents : information related to security, technology, etc to each other. I'm : wondering if there are suggestions for local advocacy and training : guidelines so not to mis-represent Tor (although it would be obvious that I : have no directly affiliations). I read on the three year road map that : training and promotion is a priority of course but discussion about : anonymity systems can become a heated topic especially related to the : Privacy Debate. Maybe something from the CCC meetings or one of Roger's : presentations. A number of people have created their own presentations. Roger's presentations are listed at https://www.torproject.org/documentation.html.en#UpToSpeed numbers 10 and 11. Depending where you are, someone from Tor may be available to show up and also help out. But you're right, it would be good to have a general Tor presentation template/demo for others to use for their own advocacy purposes available on the website. I'm happy to work with others on such a thing that we can give away with some creative commons license. -- Andrew
Re: UDP and data retention
On Fri, Dec 19, 2008 at 02:40:23PM +0100, eu...@leitl.org wrote 0.8K bytes in 18 lines about: : On Fri, Dec 19, 2008 at 08:23:40AM -0500, pho...@rootme.org wrote: : > On Fri, Dec 19, 2008 at 11:24:01AM +0100, eu...@leitl.org wrote 0.1K bytes in 3 lines about: : > : : > : This is off-topic, but isn't UDP making data retention more difficult : > : than TCP/IP. : > : > How would UDP make data retention more difficult? : : That was posed as a question, but I accidently dropped the question mark. I really doubt the politicians know the difference between TCP and UDP. I'd expect the law to cover IP communications, regardless of what runs over IP. -- Andrew
Performance optimizations for high-bandwidth Tor exit
Hallo, I've been running a Tor exit node on one of my machines (Intel Dual E2160 (1.8GHz), 2GB RAM, Xen domU for Tor, encrypted HDD) for some months. It is on a shared 100MBit/s line (500GB in/out daily). I have not configured any bandwidth limits within Tor. Most of the time, the Tor process maxes out the CPU (85-100%), while memory consumption stays at ~10%; until today, this didn't pose much of a problem as log files show no errors and the machine has been used exclusively for Tor. I now want to play around with hidden services, and noticed that Apache takes a very long time to reply, even to local requests. As I am already using Xen, I thought it might be possible to share CPU and memory intelligently between two domUs (with the Tor domU having lower priority), but I didn't find any useful information how to do that. Are there any performance tweaks to limit Tor's CPU consumption? NumCPUs is set to 2, other than that I didn't modify the default configuration much. Also, niceness is set to +15 for the tor process. I want to avoid setting a fixed bandwidth limit, Tor should use "as much as it gets". Will upgrading the CPU help? -Daniel
Suggestions for Advocacy
looking for advice - I'm a member of one of those local organizations that regularly presents information related to security, technology, etc to each other. I'm wondering if there are suggestions for local advocacy and training guidelines so not to mis-represent Tor (although it would be obvious that I have no directly affiliations). I read on the three year road map that training and promotion is a priority of course but discussion about anonymity systems can become a heated topic especially related to the Privacy Debate. Maybe something from the CCC meetings or one of Roger's presentations. Roc Tor Admin
Re: Failed to hand off onionskin
Hi! On Sat, Dec 20, 2008 at 12:45 AM, Scott Bennett wrote: > That is odd. In my experience, tor has 4 + NumCPUs threads, except > right after a SIGHUP or during initialization. I normally only see two > threads do any work, and most of it is done by one of those two. Although > I run it on a P4 Prescott, it is HT-enabled, so I set NumCPUs 2. Most > likely, the onionskin-decoding threads are used so infrequently that I only > see one in use at a time anyway. I am using FreeBSD. And if I set NumCPUs to 1, then I have 2 threads reported by top, if I set NumCPUs to 3, then I have 3 threads reported by top. > Are you seeing any significant paging activity on that system? If not, > then I doubt that is the problem. No. And I was not thinking that paging/swaping is a problem. But it is 5 MB/s of data to be processed what could be a problem if it is has to be copied a lot around in the memory (RAM). > Even small amounts of paging activity are not likely a problem. Is that > machine dedicated to tor? Or is it also running other jobs with large > (relative to the real memory of the machine) memory requirements? It is not dedicated to Tor but at the moment I am testing it only with Tor. So currently it does not have much other load, not CPU nor memory. But later on (when I will tweak it correctly) it will have also some other things running. Mitar
Re: Failed to hand off onionskin
Hi! On Sat, Dec 20, 2008 at 12:30 AM, Scott Bennett wrote: > Roger missed mentioning (step two) that you can adjust the queue limit > yourself, e.g., by adding to torrc > > MaxONionsPending 200 > > to double the default limit of 100. Try 200, and if that takes care of it, > fine. If not, set it to something higher. It does increase tor's memory > requirement, but only by a small amount. But is not this value more something of a "warn when there is so many onions queued"? OK, it is true that higher values somehow smooth out peeks, but otherwise it also means that I would introduce larger delays for onions going through my relay. From a perspective of "real time" processing it means that I would increase the limit of "real time". So the question is what is better: that I decrease advertised bandwidth and have onions routed almost immediately or that I advertise more bandwidth (which I do have) but that from time to time it could happen that it takes a lot of time for some of them to get through. On the other hand it could be true that the default value is meant for relays with lower bandwidth and so there is a lot of onions going through the system and they are also processed very fast: only that there are so many of them at a given moment that the queue gets full. It is not so much a problem of delay than of a supporting high throughput. Mitar
Re: Failed to hand off onionskin
On Wed, 17 Dec 2008 02:00:46 +0100 Mitar wrote: >On Wed, Dec 17, 2008 at 12:44 AM, Roger Dingledine wrote: >> Well, step one is to set >> NumCPUs 2 >> but I think you already did that. > >Yes. That was something I tried first. But it did not change those >warnings much (I have not really measured or counted them but I >believe that there is the same amount of them in a day with one or two >processors enabled). > >The only time I really see increase in CPU % (in top output on >FreeBSD) is when I send HUP signal - then it gets much over 100 % for >a few seconds. Several things happen when tor catches a SIGHUP that can eat CPU time for a bit, including checking to see whether anything has changed that would require publishing a new descriptor (e.g., checking to see if the ExitPolicy specifications have changed) and building a new set of circuits. > >> Assuming you're referring to the relay "Arlequin", this is a very fast >> relay, so you will be running up against various constraints that not >> many people hit. > >Yes. But the system really does not seem busy. Most of tor's work seems to be done by a single thread, although one can see occasional CPU usage by others. Onionskin decrypting is handled by NumCPUs threads. > >> How much ram is Tor using? If it's a lot, you might consider the -alpha >> versions, as on some platforms they use much less ram. > >top reports 369M of total memory footprint. It has 3 threads. There >are around 14000 TCP connections established (I can count this as the >system has a dedicated IP address for Tor traffic). That is odd. In my experience, tor has 4 + NumCPUs threads, except right after a SIGHUP or during initialization. I normally only see two threads do any work, and most of it is done by one of those two. Although I run it on a P4 Prescott, it is HT-enabled, so I set NumCPUs 2. Most likely, the onionskin-decoding threads are used so infrequently that I only see one in use at a time anyway. > >Could it be a memory use bound? So that memory access cannot keep with >the use? And this is why the CPU is not maxed? > Are you seeing any significant paging activity on that system? If not, then I doubt that is the problem. Even small amounts of paging activity are not likely a problem. Is that machine dedicated to tor? Or is it also running other jobs with large (relative to the real memory of the machine) memory requirements? Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at cs.niu.edu * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * **
Re: Failed to hand off onionskin
On Tue, 16 Dec 2008 18:44:58 -0500 Roger Dingledine wrote: >On Wed, Dec 17, 2008 at 12:28:37AM +0100, Mitar wrote: >> From time to time I am getting this warning: >> >> Failed to hand off onionskin. Closing. >> Your computer is too slow to handle this many circuit creation >> requests! Please consider using the MaxAdvertisedBandwidth config >> option or choosing a more restricted exit policy. > >This warning happens when you have more than 100 create cells queued >for processing, so presumably your CPU isn't able to keep up with them. > >> I have been monitoring the system and while it is true that sometimes >> it tops one processor, it occupies most of the time just 50 % of one >> processor. I have also configured Tor daemon to use two threads so >> even if it tops one it could still switch to another. But it rarely >> passes 100 % (that is, it rarely really uses two processors). The >> system as a whole has also not topped its CPU power. And while load >> does not seem to be so high I get at the same time this errors. Is >> there some other system bound which would be causing this and not CPU? >> Are there some other performance tweaks I could try? > >Well, step one is to set >NumCPUs 2 >but I think you already did that. > Roger missed mentioning (step two) that you can adjust the queue limit yourself, e.g., by adding to torrc MaxONionsPending 200 to double the default limit of 100. Try 200, and if that takes care of it, fine. If not, set it to something higher. It does increase tor's memory requirement, but only by a small amount. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at cs.niu.edu * ** * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * *-- Gov. John Hancock, New York Journal, 28 January 1790 * **
Re: Windows buffer problems
On Fri, Dec 19, 2008 at 7:54 AM, Lee wrote: > ... > "Manipulating > HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\GlobalMaxTcpWindowSize > and TcpWindowSize to 0xfaf00 (1027840) seemed to increase the time to > failure when running Tor and BitTorrent." > > seems backwards. Instead of buffering up to 16KB of data for each > open connection you're telling the system to buffer up to 1MB of data > for each open connection. How can increasing system buffer usage help > if the problem is insufficient buffer space? this is a little odd. there is a different option useful for virtual servers (and seems to help with the WSAENOBUFS too) called ConstrainedSockets. this will use setsockopt() to limit the size of send and receive buffers associated with each socket. the ConstrainedSockSize parameter can lower or increase from the default constrained size. i wonder if the person reporting that behavior allowed TCP windows to scale above 1M so constraining them to 1M limit helped (but still seems high). > So I'm wondering if the problem could be that the system runs out of > available ports. i don't think this is quite as common, and is reported as an error distinct from the too many concurrent pending and buffer exhaustion errors. > XP defaults to using something like 4K ports and 240 seconds for > keeping a closed socket in the timed wait state. Has anyone tried > bumping the allowable port numbers up to 64K and dropped the timed > wait state time to 16 seconds? dropping the time wait state is a good idea. i have to do this for other connect heavy applications in a different setting to keep within global kernel limits for max open files. by default this is usually around 4 minutes. that's a long time to tie up scarce kernel buffer resources and/or file descriptors. i don't know how useful this would be for Tor servers; a good experiment for someone to try perhaps! :) best regards,
Re: UDP and data retention
This is off-topic, but isn't UDP making data retention more difficult than TCP/IP. I don't see how .. "tcpdump -s 1514 -w evidence.pcap ip proto \\udp" is any harder than .. "tcpdump -s 1514 -w evidence.pcap ip proto \\tcp" Now I guess you could rig a communications "network" that dealt entirely in header-source forged UDP packets, but as best practices dictate (not the everybody follows them) .. one should filter egress of packets with a source address not within your netblock. Cheers, Michael Holstein Cleveland State University
Re: UDP and data retention
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Eugen Leitl wrote: > On Fri, Dec 19, 2008 at 08:23:40AM -0500, pho...@rootme.org wrote: >> On Fri, Dec 19, 2008 at 11:24:01AM +0100, eu...@leitl.org wrote 0.1K bytes >> in 3 lines about: >> : >> : This is off-topic, but isn't UDP making data retention more difficult >> : than TCP/IP. >> >> How would UDP make data retention more difficult? > > That was posed as a question, but I accidently dropped the question mark. > > The idea is that UDP is a connectionless protocol, while the bulk of > off-shelf lawful interception software and intent behind the data > retention legislation as well as ISP-side financial investment into > interception infrastructure will be initially mostly focused on HTTP, SMTP, > POP3 and its ilk. This might open up a loophole which could take > several years to close. > > That's the hypothesis. What do you think? > I think it is missleading to talk about "connectionless" here, it is "stateless". There is a relationship between sender and recipient as is for TCP, however the state and handshake are missing. UDP can be observed just as well as TCP unless you go for an extra mile by using random source/destination ports which however still leaves the sender/recipient relationship. Which however you could break by falsifying the sender address.. getting some bad thoughts here. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (Darwin) iD8DBQFJS+HyOMmnRrmEoQkRAvl0AJ0ckadcyoD+xXsLkeEt8HcWQYaYQACbBMWy 0rdUVvcIALN8yfYf0Jf/Byc= =TVvZ -END PGP SIGNATURE-
Re: Windows buffer problems
On 12/19/08, coderman wrote: > > there are actually two issues (or more?) for non-server Windows > running Tor. the usual problem Tor encounters is not related to the > number of concurrent attempts but to kernel non-paged memory resources > consumed to exhaustion when lots of active non-overlapped-I/O sockets > are in use. details here: > https://wiki.torproject.org/noreply/TheOnionRouter/WindowsBufferProblems This bit from the web page: "Manipulating HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\GlobalMaxTcpWindowSize and TcpWindowSize to 0xfaf00 (1027840) seemed to increase the time to failure when running Tor and BitTorrent." seems backwards. Instead of buffering up to 16KB of data for each open connection you're telling the system to buffer up to 1MB of data for each open connection. How can increasing system buffer usage help if the problem is insufficient buffer space? So I'm wondering if the problem could be that the system runs out of available ports. XP defaults to using something like 4K ports and 240 seconds for keeping a closed socket in the timed wait state. Has anyone tried bumping the allowable port numbers up to 64K and dropped the timed wait state time to 16 seconds? Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters] "MaxUserPort"=dword:fffe "TcpTimedWaitDelay"=dword:0010 "StrictTimeWaitSeqCheck"=dword:0001 Lee
Re: UDP and data retention
Am 19.12.2008 um 14:32 schrieb Sven Anderson:
Am 19.12.2008 um 11:24 schrieb Eugen Leitl:
This is off-topic, but isn't UDP making data retention more difficult
than TCP/IP.
Since you seem to talk about Germany: Again, data retention does and
will not happen on a per-packet basis and especially not on the
transport layer (TCP/UDP) with the current law. There will "only" be
records which dynamic IP-address was assigned to which customer at
which time. That's it. See Paragraph 4 in [1] (German).
[1]
http://de.wikipedia.org/wiki/Vorratsdatenspeicherung#Verkehrsdatenspeicherung
I should add that anonymizing services, as far as the law can be
applied to them, only have to record the mapping of data replacements,
but _only_ for data that has to be recorded by another party anyways.
This is only true for IP adresses in case of Tor (not so for email
anonymizers). So, port numbers and the like are never allowed to be
recorded by anonymizing services regarding data retention law, since
port numbers are also not allowed to be recorded by the internet
access providers or any other party.
Beside that, the data retention law does only apply to services in
return for payments ("in der Regel gegen Entgelt erbrachte Dienste").
Since Tor is a completely free service (no payments, no ads), it is
very likely that Tor operators are not allowed to store _any_ data.
In any case, UDP or TCP makes no difference.
Beside the data retention, there is also the "normal" lawful
interception in case of a probable cause. But in this case there are
no restrictions what to record, AFAIK. So I don't see why UDP would be
more of a problem for them.
Sven
smime.p7s
Description: S/MIME cryptographic signature
Re: UDP and data retention
On Fri, Dec 19, 2008 at 02:32:57PM +0100, Sven Anderson wrote: > Since you seem to talk about Germany: Again, data retention does and > will not happen on a per-packet basis and especially not on the > transport layer (TCP/UDP) with the current law. There will "only" be Thanks for the pointer. I'm pretty sure this is going to change rather soon (not that TLAs care much either way about what the local law says). > records which dynamic IP-address was assigned to which customer at > which time. That's it. See Paragraph 4 in [1] (German). > > [1] > http://de.wikipedia.org/wiki/Vorratsdatenspeicherung#Verkehrsdatenspeicherung Thanks. Assuming the Wikipedia interpretation is correct and my ISP cares about the law (from what little I know Tor operators might be well under extra scrutiny from LKA and/or BND) they're not going to log anything relevant for time being, and anyone else is explicitly forbidded by law. That's probably the best piece of news I've had this year. How large a traffic fraction of the usual suspect P2P file-sharing is UDP, anyone knows?
Re: UDP and data retention
On Fri, Dec 19, 2008 at 08:23:40AM -0500, pho...@rootme.org wrote: > On Fri, Dec 19, 2008 at 11:24:01AM +0100, eu...@leitl.org wrote 0.1K bytes in > 3 lines about: > : > : This is off-topic, but isn't UDP making data retention more difficult > : than TCP/IP. > > How would UDP make data retention more difficult? That was posed as a question, but I accidently dropped the question mark. The idea is that UDP is a connectionless protocol, while the bulk of off-shelf lawful interception software and intent behind the data retention legislation as well as ISP-side financial investment into interception infrastructure will be initially mostly focused on HTTP, SMTP, POP3 and its ilk. This might open up a loophole which could take several years to close. That's the hypothesis. What do you think?
Re: UDP and data retention
Am 19.12.2008 um 11:24 schrieb Eugen Leitl: This is off-topic, but isn't UDP making data retention more difficult than TCP/IP. Since you seem to talk about Germany: Again, data retention does and will not happen on a per-packet basis and especially not on the transport layer (TCP/UDP) with the current law. There will "only" be records which dynamic IP-address was assigned to which customer at which time. That's it. See Paragraph 4 in [1] (German). [1] http://de.wikipedia.org/wiki/Vorratsdatenspeicherung#Verkehrsdatenspeicherung Sven smime.p7s Description: S/MIME cryptographic signature
Re: UDP and data retention
On Fri, Dec 19, 2008 at 11:24:01AM +0100, eu...@leitl.org wrote 0.1K bytes in 3 lines about: : : This is off-topic, but isn't UDP making data retention more difficult : than TCP/IP. How would UDP make data retention more difficult? -- Andrew
Re: UDP and data retention
On Fri, Dec 19, 2008 at 01:00:36PM +0100, Eugen Leitl wrote: > On Fri, Dec 19, 2008 at 12:23:25PM +0100, slush wrote: > > > >No. > > This monosyllabic answer no doubt comes from in-depth knowledge > of legal requirements in regards to data retention for ISPs in Germany? > Yes. > >2008/12/19 Eugen Leitl <[1]eu...@leitl.org> > > > > This is off-topic, but isn't UDP making data retention more > > difficult > > than TCP/IP. (Sorry couldn't help myself. And sorry I don't have a substantive answer to your question. -Paul)
Re: UDP and data retention
On Fri, Dec 19, 2008 at 12:23:25PM +0100, slush wrote: > >No. This monosyllabic answer no doubt comes from in-depth knowledge of legal requirements in regards to data retention for ISPs in Germany? >2008/12/19 Eugen Leitl <[1]eu...@leitl.org> > > This is off-topic, but isn't UDP making data retention more > difficult > than TCP/IP.
Re: UDP and data retention
No. 2008/12/19 Eugen Leitl > > This is off-topic, but isn't UDP making data retention more difficult > than TCP/IP. >
UDP and data retention
This is off-topic, but isn't UDP making data retention more difficult than TCP/IP.
