eliminating bogus port 43 exits
A bit over a month ago, I posted here some exit statistics by port number.
One major oddity among them was the count of port 43 (whois) exits, which
seemed extraordinarily large, especially in relation to the counts for other,
more expectedly popular port numbers. Some of the comments I got in response
gave me an idea. In the what follows here, keep in mind that the second most
frequently occurring exit port number in the statistics previously reported
was 443 (https), and that the count of port 43 exits was in the millions when
the count of port 443 exits was several hundred thousand. It is important to
note that my node's exit policy regarding port 80 (http) is highly restrictive,
resulting in very low exit counts for that port. Keeping that in mind, the
exit counts for almost all other ports were not and are not similarly
restricted.
I replaced the ExitPolicy accept *:43 in my torrc file with the
following:
###---Limited list of allowed whois exit addresses
ExitPolicy accept 192.103.19.12:43 # whois access to whois.6bone.net
ExitPolicy accept 192.149.252.44:43 # whois access to whois.arin.net
ExitPolicy accept 193.0.0.135:43# whois access to whois.ripe.net
ExitPolicy accept 194.85.119.77:43 # whois access to whois.ripn.net
ExitPolicy accept 196.216.2.1:43# whois access to whois.afrinic.net
ExitPolicy accept 198.108.0.18:43 # whois access to whois.ra{,db}.net
ExitPolicy accept 199.7.51.74:43# whois access to whois.crsnic.net
ExitPolicy accept 199.7.55.74:43# whois access to whois.internic.net
ExitPolicy accept 199.43.0.144:43 # whois access to whois.arin.net
ExitPolicy accept 200.160.2.3:43# whois access to whois.registro.br
ExitPolicy accept 200.160.2.15:43 # whois access to whois.lacnic.net
ExitPolicy accept 202.12.29.13:43 # whois access to whois.apnic.net
ExitPolicy accept 202.30.50.120:43 # whois access to whois.krnic.net
ExitPolicy accept 205.178.188.12:43 # whois access to
whois.networksolutions.com
ExitPolicy accept 206.51.224.229:43 # whois access to whois.nic.gov
ExitPolicy accept 208.77.188.18:43 # whois access to whois.icann.org
ExitPolicy accept 208.77.188.87:43 # whois access to whois.iana.org
ExitPolicy reject *:43 # nicname whois
###---End of whois exit policy specifications
The relationship now between the exit counts for ports 43 and 443 in the
last few days since I switched to 0.2.1.15-rc with Nick's patch applied looks
like this:
439 Exit to port 43
72052 Exit to port 443
In other words, by restricting just port 43 exits to only the legitimate whois
IP addresses, I eliminated at least 70% of *all* exits through my tor node,
which suggests to me that the vast, overwhelming majority of exits from the
tor network are illegitimate and place a terribly taxing load upon the tor
network as a whole. This apparent fact, in turn, suggests that if a) all
tor nodes with an explicit exit policy were to restrict port 443 exits to
just the legitimate port 43 IP addresses and b) the tor default exit policy
did the same, a huge and illegitimate load would be lifted from the tor network
overall. If no relays offer exits to port 43 that don't go to the NICs' whois
servers, well over half of all tor exits, which are illegitimate and
undeserving of service in the first place, will be eliminated (not counting
typical port 80 (http) traffic, of course).
Because my node's exit policy for port 80 (http) is not wide open, it is
hard for me to estimate the relative importance of bogus port 43 requests
w.r.t. legitimate port 80 (http) requests. Because of my node's limited port
80 exit policy, I would be *very* interested in seeing exit counts for nodes
with unrestricted exit policies for the combination of ports 43, 80, and 443
in order to get a better idea of their relative importances.
Nevertheless, the impact of eliminating those exit opportunities can be
expected to be quite significant in terms of performance of the network
overall, particularly because circuits will not need to be built in the first
place for such requests. If even a few relays continue to offer unrestricted
exits for port 43, they will get so badly hammered by all the bogus exit
requests that they will cease to be important to normal operations of the tor
network until such time as they may modify their exit policies to be more in
tune with valid use of the tor network, rather than use by some sort of port
scanner or whatever junk software is currently consuming so much of the tor
network's resources, except to the extent that such non-conforming nodes would
be incurring the cost of the circuits to reach them for the exit service.
Please note also that changing the default exit policy and most tor node's
explicit exit policies to the above specification would not prevent tor exit
node operators from adding other legitimate whois servers' IP addresses to
their exit policies.
Re: eliminating bogus port 43 exits
Hi Scott,
Got a couple of questions.
- Have you looked deeper into the request for port 43, using tcpdump or
Wireshark?
- Do you KNOW that it is a WHOIS request, not OpenVPN or something else
running on the WHOIS port?
- Have you logged what IP's are being connected to?
I just curious, as this seems to be really odd to me that so many WHOIS
request are going through Tor.
I'm almost curious enough to run a exit node now just to see what might be
going on.
- Kyle
On Fri, Jun 12, 2009 at 12:29 AM, Scott Bennett benn...@cs.niu.edu wrote:
A bit over a month ago, I posted here some exit statistics by port
number.
One major oddity among them was the count of port 43 (whois) exits, which
seemed extraordinarily large, especially in relation to the counts for
other,
more expectedly popular port numbers. Some of the comments I got in
response
gave me an idea. In the what follows here, keep in mind that the second
most
frequently occurring exit port number in the statistics previously reported
was 443 (https), and that the count of port 43 exits was in the millions
when
the count of port 443 exits was several hundred thousand. It is important
to
note that my node's exit policy regarding port 80 (http) is highly
restrictive,
resulting in very low exit counts for that port. Keeping that in mind, the
exit counts for almost all other ports were not and are not similarly
restricted.
I replaced the ExitPolicy accept *:43 in my torrc file with the
following:
###---Limited list of allowed whois exit addresses
ExitPolicy accept 192.103.19.12:43 # whois access to whois.6bone.net
ExitPolicy accept 192.149.252.44:43 # whois access to whois.arin.net
ExitPolicy accept 193.0.0.135:43# whois access to whois.ripe.net
ExitPolicy accept 194.85.119.77:43 # whois access to whois.ripn.net
ExitPolicy accept 196.216.2.1:43# whois access to
whois.afrinic.net
ExitPolicy accept 198.108.0.18:43 # whois access to
whois.ra{,db}.net
ExitPolicy accept 199.7.51.74:43# whois access to whois.crsnic.net
ExitPolicy accept 199.7.55.74:43# whois access to
whois.internic.net
ExitPolicy accept 199.43.0.144:43 # whois access to whois.arin.net
ExitPolicy accept 200.160.2.3:43# whois access to
whois.registro.br
ExitPolicy accept 200.160.2.15:43 # whois access to whois.lacnic.net
ExitPolicy accept 202.12.29.13:43 # whois access to whois.apnic.net
ExitPolicy accept 202.30.50.120:43 # whois access to whois.krnic.net
ExitPolicy accept 205.178.188.12:43 # whois access to
whois.networksolutions.com
ExitPolicy accept 206.51.224.229:43 # whois access to whois.nic.gov
ExitPolicy accept 208.77.188.18:43 # whois access to whois.icann.org
ExitPolicy accept 208.77.188.87:43 # whois access to whois.iana.org
ExitPolicy reject *:43 # nicname whois
###---End of whois exit policy specifications
The relationship now between the exit counts for ports 43 and 443 in
the
last few days since I switched to 0.2.1.15-rc with Nick's patch applied
looks
like this:
439 Exit to port 43
72052 Exit to port 443
In other words, by restricting just port 43 exits to only the legitimate
whois
IP addresses, I eliminated at least 70% of *all* exits through my tor node,
which suggests to me that the vast, overwhelming majority of exits from the
tor network are illegitimate and place a terribly taxing load upon the tor
network as a whole. This apparent fact, in turn, suggests that if a) all
tor nodes with an explicit exit policy were to restrict port 443 exits to
just the legitimate port 43 IP addresses and b) the tor default exit policy
did the same, a huge and illegitimate load would be lifted from the tor
network
overall. If no relays offer exits to port 43 that don't go to the NICs'
whois
servers, well over half of all tor exits, which are illegitimate and
undeserving of service in the first place, will be eliminated (not counting
typical port 80 (http) traffic, of course).
Because my node's exit policy for port 80 (http) is not wide open, it
is
hard for me to estimate the relative importance of bogus port 43 requests
w.r.t. legitimate port 80 (http) requests. Because of my node's limited
port
80 exit policy, I would be *very* interested in seeing exit counts for
nodes
with unrestricted exit policies for the combination of ports 43, 80, and
443
in order to get a better idea of their relative importances.
Nevertheless, the impact of eliminating those exit opportunities can be
expected to be quite significant in terms of performance of the network
overall, particularly because circuits will not need to be built in the
first
place for such requests. If even a few relays continue to offer
unrestricted
exits for port 43, they will get so badly hammered by all the bogus exit
requests that they will cease to be important to normal operations of the
tor
network until such
Re: eliminating bogus port 43 exits
On Fri, 12 Jun 2009 00:44:19 -0700 Kyle Williams kyle.kwilli...@gmail.com top-posted: Please stop doing that. It is terribly rude. Got a couple of questions. - Have you looked deeper into the request for port 43, using tcpdump or Wireshark? No, of course not! - Do you KNOW that it is a WHOIS request, not OpenVPN or something else running on the WHOIS port? I have not regarded that as an important issue. Please see http://www.iana.org/assignments/port-numbers - Have you logged what IP's are being connected to? Absolutely not! Please either justify your insults or else refrain from further aspersions. I just curious, as this seems to be really odd to me that so many WHOIS request are going through Tor. That was my reaction as well and part of the reason I posted the earlier round of exit statistics. As I mentioned before, the responses I received prompted the change in exit policy and now the posting of relevant results from the change. I'm almost curious enough to run a exit node now just to see what might be going on. I hope you have more scruples with regard to such action than you have apparently assumed that I might have. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at cs.niu.edu * ** * A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army. * *-- Gov. John Hancock, New York Journal, 28 January 1790 * **
Re: eliminating bogus port 43 exits
On 6/12/2009 3:29 AM, Scott Bennett wrote: In other words, by restricting just port 43 exits to only the legitimate whois IP addresses, I eliminated at least 70% of *all* exits through my tor node, which suggests to me that the vast, overwhelming majority of exits from the tor network are illegitimate and place a terribly taxing load upon the tor network as a whole. Scott, Thanks for your continued analysis, this is interesting information. However, the list of WHOIS servers you mentioned (and I snipped for brevity) is by no means a complete set of the legitimate WHOIS IP addresses. In fact, it's much much too small to draw any significant conclusions, for at least two major reasons: 1) Any .com or .net WHOIS queries that hit whois.verisign-grs.com (aka whois.internic.net in your list) with a legitimate domain name will result in a referral to an individual registrar's WHOIS server, which will often be followed by the client, and would not be allowed by your exit policy. There are potentially tens of thousands of these registrar WHOIS servers out there. 2) Your list significantly excludes all ccTLD WHOIS servers. While the numbers of domains registered in ccTLDs are not significant compared to .com/.net, their use is quite popular in a number of places, particularly in some where Tor is also quite popular, ie Germany. I'd be interested in seeing a comparison done with a more significantly complete list. I understand you feel very strongly about sampling the contents of the traffic, and that's perfectly understandable and appropriate, but it is probably the only way to actually make a firm determination of how much of this exit traffic really is WHOIS, without crafting a VERY large Exit policy. It may be possible, with appropriately engineered tools, to sample the traffic in a suitably anonymous way but still draw some conclusions, perhaps by simply attempting to determine if the TCP session involves mostly text or binary data. That may still be a bit too intrusive, so I suppose we might just never know. Given these shortcomings in the list, I definitely wouldn't suggest that such a list be considered a default, as you'll be blocking a potentially significant amount of legitimate WHOIS traffic. If you do attempt to dig up a more complete list of WHOIS servers, I'd certainly be interested to see what you come up with, but of course understand you're doing this all on your own time and dime, and would never suggest that you're by any means obligated to do so. :) Best Regards, Tim
Re: Tor bridge not generating any traffic
On 2009.06.11 15:19, Scott Bennett wrote: On Thu, 11 Jun 2009 21:23:33 +0200 Johannes Nitsche nitsc...@rambler.ru top-posted (please learn not to do that): Thanks for all the answers. It seems my view of what a bridge is was wrong. I thought a bridge is a link point between tor nodes which forwards traffic from an entry node to an exit node. But as I understand it now a bridge is kind of a secret entry point into the tor network. As I read in the FAQ the standard path lenght is hardcoded at 3. So how would I have to configure it to run as a node that operates as an entry point(not secret necessarily), a forwarding node but not as an exit node? ExitPolicy reject *:* Thanks, it works.
Re: eliminating bogus port 43 exits
While node operators are certainly welcome to characterize and define both traffic and policy as deemed fit for their own purposes... I might suggest that node operators examine things more fully in order to make better policy decisions overall. 1 - The use of any given TCP port alone is not sufficient to qualify traffic on it as illegitimate, bogus, undeserving, invalid, scanner, junk, etc. For all anyone knows, such traffic could be saving the world, over ports otherwise unavailable to the client, EXACTLY as per one of Tor's very legitimate use cases. 2 - Impact can be defined as number of connections over time and/or bandwidth used. The operator would be well served by deploying and using netflow analysis to better understand their exit's traffic. 2a - Tor itself should be intrumented with stats about attempted circuit construction that fails due to exit policy. 3 - Further, there needs to be an understanding of what the traffic ACTUALLY IS. Operators should be using tools such as wireshark, tcpdump, bro, etc to determine the content. And if it turns out to be encrypted to destinations and services unknown, NO such determination can be made. The only thing left to go on is impact as in #2 above. 4 - /etc/services is defunct law and means very little these days. App developers don't write to it, they just include a -p port knob and a seemingly unused default. Clients use that -p knob to avoid app conflicts, fix bum network policy, or simply to find a hole that works for their perfectly legitimate uses, such as VPN's. 5 - There are many more whois servers than those listed, particularly referral/delegated whois servers. The list is in permanent flux. 6 - Clients may have chosen certain exits to '[ab]use' for certain ports, destinations, or activities, skewing the results for any single exit or set of exits. 7 - It is well established practice at ISP's, corporations, institutions, etc... that network admins may observe content in order to determine policy direction, protect their network, and in general, figure out what's up. Disclosing that content and/or acting specifically on, or against, users, when the traffic was not collected for that purpose, is entirely different and governed by strict laws, at least in the US. Tor is no different than being a mini ISP, do as any ISP would. Bottom line, one must either: 1 - Take the corporate, block all but known stance. Hopefully know that people will still jam their unknowns through your knowns. 2 - Block/allow based on reasonably thorough analysis of content, risk, load, etc. 3 - Block on whim, gut feel, religion, sunspots, etc. 4 - Allow all. Unfortunately, #2 is usually the last to be chosen because it requires the largest investment in time, knowledge, etc. And lastly, as food for Tor development thought... there is no interaction between Tor and kernel level packet filters. Most network admins use such packet filters as their primary point of wizardry. Tor could be enhanced with a mode of operation that interfaces in real time with such filters to determine when to create circuits.
JanusVM tests
Hi, if I go to the JanusVM deanonimyzer test without being connected through Tor, the test passes (of course), but I get info, that I have very good anonymizer service: If you do not see your real IP address in the report, then CONGRATULATIONS! This means that you have a very good anonymity service, that's if you are using one. This information is true, but a bit misledaing. My proposal (for JanusVM test owner) is to explain a little bit more what means if user passes test and to also check if user is using Tor exit point and then warn user that he passed the test, but is not using Tor. (I just wrote an article for one slovenian IT portal and some users are commenting that this test is misleading...) JanusVM test is here: http://www.janusvm.com/deanonymizer/scan.html byr, Matej
Re: eliminating bogus port 43 exits
Hey Scott, On Fri, Jun 12, 2009 at 9:29 AM, Scott Bennettbenn...@cs.niu.edu wrote: I replaced the ExitPolicy accept *:43 in my torrc file with the following: ###---Limited list of allowed whois exit addresses ExitPolicy accept 192.103.19.12:43 # whois access to whois.6bone.net ExitPolicy accept 192.149.252.44:43 # whois access to whois.arin.net etc I would like to suggest a functionality to make this a bit more easy: Add support for dnsbl's to determine the exitpolicy. I know this would be non-trivial to implement, especially given the possible abuse scenario's and the fact that exitpolicy's are currently published in the directory, but it would be great to have universal known good and known bad destinations that exitnodes could use. Greets, Nils -- Simple guidelines to happiness: Work like you don't need the money, Love like your heart has never been broken and Dance like no one can see you.
trying to use specific country for node selection in tor
Hello All
I am trying to use specific country selection in the torrc as per the
FAQs in the Tor website. The following is a snippet from my torrc
...
EntryNodes {us}
ExitNodes {us}
StrictEntryNode 1
StrictExitNode 1
However when I run tor - 0.2.1.15 with this configuration , I get the
following error message :-
Jun 13 01:49:15.860 [warn] Failed to parse/validate config: IPs or
countries are not yet supported in EntryNodes.
Am I doing something wrong ? Is there some other way to specify which
countries to use for entry,middleman and exit node selection ?
Thanks
Sambuddho
Re: eliminating bogus port 43 exits
Being familiar with ISP practice in this area, it is why you examine the content and what you do with the knowledge of the content observed, be it stored in your head or on disk, that matters. It's pretty well established that one may monitor traffic in a general way in order to figure out what's up, make and enforce policy and so on. One cannot monitor/record a particular users's traffic and then disclose that traffic or use it for/against them or oneself. A few examples. Think about what is or is not legal and why. 1a - Any content based IDS such as bro. 1b - Any content based traffic shapers, balancers, etc. 1c - Any mail virus scanner, NetNanny, etc. 1d - Nagios, OpenNMS, netflow, HPOpenview, and so on. You can sure bet the purveyors of these products do not develop these systems in a pristine air gap lab environment using only traffic they generated. And they are deployed on real data. 2 - Any ISP trying to figure out why their traffic just trended up by 50% the last month. Any LAN admin trying to figure out why their T1 is saturated. 3a - Any network research group, whether private, institutional or white/gray/black. Bugtraq/FullDisclosure, Defcon presentations, live demos, etc. 3b - That guy who snooped Tor and published embassy passwords. 4a - Employer x, checking up on adherance to corporate email policy, reading random mails in the process. 4b - Finding out that you enjoy watching the mating habits of penuins on PBS and then wondering why you have one or more fewer friends in the lunchroom at work the next day. 5a - Social networking sites selling 'demographic and statistical' data to places like Intelius. 5b - Google trolling your email to display targeted ads and do who knows what else with. 5c - This call may be monitored or recorded. These are all black areas that are hard to get internal facts about unless you work deep inside where it happens. Some is ok, some is untrustworthy, some is evil. 6 - The US govt itself, and other countries, with their tap the entire internet projects. Some of this, and the handling of product from it, is known to be illegal, it's just so black that no one has been able to prove it yet. 7 - The thousands of networked entities that use netflow and other statistical and content analysis tools 24x7x365 without concern. 8 - Public records requests for netflow data from public institutions. Yes, they have had to disclose them. It's safe to snoop port 43 for this purpose and say I found: 200 whois queries to known public servers x, y and z. 53 HTTP GET's 34 plaintext irc sessions to these public ircnets. 22 initial ssh fingerprints 16 encrypted sessions to somewhere inside the pentagon. But not safe to say: 200 whois queries for these domains, some of which sent their domain passwords over port 80 to the registrar, here's the tokens. 53 HTTP GET's to a hapless bank x, here's Tony's info 34 irc sessions of Linda and Mark cybering, check out this conversation. 38 encrypted sessions where I further MITM'd them and here's their contents. For the most part, in the US, an exit node operator is an ISP. They are subject to common carrier, DMCA, ECPA and so on as it applies to their role as an ISP. And ISP's also have the right to protect, monitor, price and modify their nets as is standard industry practice. And to shield themselves from potential liability or legal expenditure and entanglement by dropping traffic that is too risky to handle, so long as it's done agnostically. If I were running an exit in the US, I'd be VERY happy to distill any amount of stats, be it IP or content based, and post them here. Including the number of times I saw the phrase 'I eat boogers' on my wire. It's just stats. And heck no, I'd never save or post the raw content, that's nuts. IANAL, jail may occur, subscribe to NANOG, your lawyer, EFF, etc.
Re: eliminating bogus port 43 exits
Well. I see that there has been moderately vigorous discussion going on since I posted my new information regarding port 43 exit statistics, which is just what I had hoped for. :-) I don't have responses for all of the points raised in the followups so far, but I can comment on some of them. On Fri, 12 Jun 2009 07:54:55 -0400 Tim Wilde t...@krellis.org wrote: On 6/12/2009 3:29 AM, Scott Bennett wrote: In other words, by restricting just port 43 exits to only the legitimate whois IP addresses, I eliminated at least 70% of *all* exits through my tor node, which suggests to me that the vast, overwhelming majority of exits from the tor network are illegitimate and place a terribly taxing load upon the tor network as a whole. Scott, Thanks for your continued analysis, this is interesting information. However, the list of WHOIS servers you mentioned (and I snipped for brevity) is by no means a complete set of the legitimate WHOIS IP addresses. In fact, it's much much too small to draw any significant conclusions, for at least two major reasons: 1) Any .com or .net WHOIS queries that hit whois.verisign-grs.com (aka whois.internic.net in your list) with a legitimate domain name will result in a referral to an individual registrar's WHOIS server, which will often be followed by the client, and would not be allowed by your exit policy. There are potentially tens of thousands of these registrar WHOIS servers out there. I'm not at all sure that that is happening in this case. My node's exit policy leaves port 4321 (rwhois) wide open, yet the exit count for the same time period covered in the statistics I posted last night is only 22. 2) Your list significantly excludes all ccTLD WHOIS servers. While the Drat. You're quite right. I forgot all about those. However, a quick check shows that an awful lot of those are at the same IP addresses for which I currently allow port 43 exits. In other words, the whois servers I've listed in my exit policy are also covering many of those ccTLDs. numbers of domains registered in ccTLDs are not significant compared to .com/.net, their use is quite popular in a number of places, particularly in some where Tor is also quite popular, ie Germany. I'd be interested in seeing a comparison done with a more significantly complete list. I understand you feel very strongly about sampling the I agree. I'll try to add the ones I can find that are at IP addresses distinct from the ones already allowed. contents of the traffic, and that's perfectly understandable and appropriate, but it is probably the only way to actually make a firm determination of how much of this exit traffic really is WHOIS, without crafting a VERY large Exit policy. It may be possible, with appropriately engineered tools, to sample the traffic in a suitably anonymous way but still draw some conclusions, perhaps by simply attempting to determine if the TCP session involves mostly text or binary data. That may still be a bit too intrusive, so I suppose we might just never know. Well, I see the situation a bit differently. First off, I just find it very hard to understand how there could be five, ten, or more times as many legitimate whois connections as https connections. My own usage of whois lookups is generally fewer than ten per week, mainly in tracking down information about sources of junk mail, whereas I do untold numbers of https web page fetches per week. Given these shortcomings in the list, I definitely wouldn't suggest that such a list be considered a default, as you'll be blocking a potentially significant amount of legitimate WHOIS traffic. An alternative approach would be to treat a default for port 43 just like the default treats port 25, I suppose. If you do attempt to dig up a more complete list of WHOIS servers, I'd certainly be interested to see what you come up with, but of course understand you're doing this all on your own time and dime, and would never suggest that you're by any means obligated to do so. :) As noted above, I'll get to the additions when I find an hour or so free to do it. I'll provide another update to the list once I've accumulated more data with the expanded list. However, I suspect at this point anyway that the expanded list is unlikely to result in drastically different exit counts relative to the counts for other ports. As you say, though, the truth will be in the data, not in my suspicions. Scott Bennett, Comm. ASMELG, CFIAG ** * Internet: bennett at cs.niu.edu * ** * A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army. * *-- Gov. John
