RE: recursive DNS servers DDoS as a growing DDoS problem
They don't need more servers, just better software. If you think open recursion (DNS DoS amplification) is an issue ISPs can ignore, I suggest you look at the history of open SMTP relays and networks supporting/allowing directed broadcast. I'll address the ignore part. I don't think closing recursive dns servers is going to make squat difference for dns based flooding just like closing SMTP relays didn't make squat difference for the spam problem. The spam continues to flow today.. Closing SMTP relays solved another problem, server capacity for the ISP, so it was in their interest to close the relays because it ate up their bandwidth and mail server capacity. Has anyone being used for a dns flood noticed they were being used? As to the issue of dns flooding, it doesn't require open recursive servers. I can point the whole domain to someone's website without even having a DNS server of my own simply by using www.domain.com and the target's IP address as one of the authorative name servers listed with the registrar and target someone that way. All I need to do then is generate queries for a bunch of random.domain.com names, I don't even need to spoof, 20,000 bots talking to their authorized recursive servers should work just fine. Heck for that matter I don't even need bots, I could just spam the planet and use [EMAIL PROTECTED] as the return address. (that might even give the amplification required) What is closing an open recursive server going to do for the ISP hosting it? I haven't heard anyone screaming that these floods were even noticable by the folks running the recursive dns servers. Where is the motivation for the ISP, ISP customer, corporation, university, etc. to do anything? Yeah, I think they can ignore it until someone decides to target them. Geo.
Re: recursive DNS servers DDoS as a growing DDoS problem
Geo. wrote: We have done just this (block inbound udp/53) to certain subnets due to a rash of CPEs that happily proxy DNS, including recursive queries, from their WAN side. What devices? Is this a default or something customers are configuring? Just about every Siemens/Efficient *DSL router (5851, 5871, 5890, some others) I have seen does this by default. During one attack I logged into a router and changed the DNS servers used for DHCP from both, to one, to the other, and watched its traffic flow exactly where I pointed the DNS servers. These routers also have a bad habit of hijacking DNS requests for anything in the form of routername.anydomain.com. Even if the DNS request is sent to an ISP name server, it will still return a record pointing to its own LAN IP when queried for, i.e. foo.bork.com and foo.microsoft.com if you named the router foo. It's not only done by default, it's not a configurable behavior as far as I've been able to find. The fix we were supplied with was to block inbound udp/53 using the router's internal IP filtering, and to name the router something unique (i.e. a customer's order number). Jim
Re: recursive DNS servers DDoS as a growing DDoS problem
Geo. wrote: Really? Ok educate me, how do you do this with Windows 2000 running MS dns? (telling people to use another server is not acceptable) If Microsoft's products are broken, why souldn't I tell people to use something else? You tell them whatever you like, they aren't going to switch just like they didn't switch from exchange just because it installed as an open relay. The whack-a-mole problem didn't go away with open relays until MS changed the default to disable relay and it's not going to go away with open recursive dns servers until they change the default there as well. Geo. Geo, the default is bad. However, it is not a Microsoft issue, this is a spoofing issue. Many like to bash Microsoft, some hate them. Myself I am known as a Microsoft critic at times. It is true Microsoft has recursion set on default, but so does bind. That is how DNS is general is set all around the world. Hopefully the very clued guys at both will change it. Why are we arguing on the colour of bytes when we could be discussing making trivial spoofing a thing of the past? Gadi.
Re: recursive DNS servers DDoS as a growing DDoS problem
Geo, the default is bad. However, it is not a Microsoft issue, this is a spoofing issue. Many like to bash Microsoft, some hate them. Myself I am known as a Microsoft critic at times. Please don't misunderstand me, I'm not bashing MS or even being a critic (although I have been at times), heck I played sysop on CIS:WINNT for 7 years and was an NT MVP for at least that long so just recognize my comments are coming from an intimate understanding of the MS user base and product default values. (enough on MS) Why are we arguing on the colour of bytes when we could be discussing making trivial spoofing a thing of the past? I agree, and ingress/egress filters for all except major backbones are really quite easy to put in place. I don't even see resistance to the idea, just a lot of people who don't know it needs to be done or why. Certainly Paul Vixie's interview the other day will help (he did a very nice job of getting the word out) but this really needs a big flood that makes all the news casts and then the media has to mention the fix. Geo.
Re: recursive DNS servers DDoS as a growing DDoS problem
If your goal is to eliminate the recursive resolution reflection
amplification, then you must disable it for all but trusted subnets.
This also defends the server from the more trivial of cache poisoning
attacks (assuming your own systems use the resolver as well).
I know this is a more generic problem, and not everyone runs bind/named,
but for those who do, is it sufficient to simply do this in named.conf:
acl goodguys {
(list of trusted peers who can request your zone files)
};
acl locals {
127.0.0.0/8;
(list of your subnets);
(list of TRUSTED hosts outside your network);
};
options {
allow-transfer { goodguys; };
allow-query { locals; };
allow-recursion { locals; };
};
then in each zone you are authorative for:
zone mydomain.com { type master;
file zone.mydomain.com;
allow-query { any; };
};
(repeat for each authorative zone)
This lets anyone on your network, and others you might trust, full
recursive lookups, while simply denying recursion for everyone else, but
allows others to query your nameserver for domains YOU are authorative
for? Or am I missing something obvious... because this is how we've been
doing it for years.
RossW
Re: recursive DNS servers DDoS as a growing DDoS problem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Tim wrote: All it takes is to throttle traffic from the resovers to outside the ISP network to a reasonably low value. Depending on the ISP this is usually in the low Kbits. All it takes is a moderate amount of competence in the ISP: I don't believe this would help the problem. One of the notable features of many reflected attacks is that no single reflector is hit with a large amount of traffic. It is spread out amongst many many reflectors so that the reflectors don't notice the issue, and so that the victim has a harder time filtering the traffic. Correct. By itself it will not. As a matter of fact no approach will do it by itself alone. Still, let us suppose that antispoofing is largely reduced (there are too many access designs out there where it cannot be fully eliminated) and running open recursive DNS servers on a customer machine is treated the same way we treat open SMTP relays nowdays. This still leaves larger ISPs and Telcos who have large DNS resolver installations that have to serve customers. It will be possible to use these for aplification for years to come unless something is done. If your goal is to eliminate the recursive resolution reflection amplification, then you must disable it for all but trusted subnets. This also defends the server from the more trivial of cache poisoning attacks (assuming your own systems use the resolver as well). This is feasible only for corporate networks where the allocations are constant and change once in a few years. It is not feasible in any ISP/Telco above a certain size. In fact, considering the consolidation over the recent years it is not feasible for most ISPs or Telcos. In an ISP you will have to provision and reprovision the nameserver ACLs on a daily basis to match your current customer allocations and reload it like there is no tomorrow. One mistake in provisioning and you will have a large chunk of customers shouting down the support line why their internet does not work. It becomes even more entertaining if you use RFC3258 or clustering to load balance DNS traffic. In that case you often end up with a lottery where one server replies, other servers deny or vice versa. Debugging that is even more entertaining. Frankly, expecting any large ISP to deploy anything like this is not realistic. Using QoS to limit queries coming from the outside world can be done in a manner where it does not require any extra provisioning and modification to the nameserver config. On top of that, for most well designed large ISP/Telco DNS server deployments this is just a simple config change. Once it has been rolled out it maintains itself. After all, if your customers have no network access having or not having DNS is largely irrelevant. By the way, nothing prevents you from putting the limit at 0 or filtering based on route tag so that you completely block recursive queries from outside your network. Personally, I would not do it (there are failure cases which will not be covered), but nothing prevents an oversealous BOFH from doing it. - -- A. R. Ivanov E-mail: [EMAIL PROTECTED] WWW: http://www.sigsegv.cx/ pub 1024D/DDE5E715 2002-03-03 Anton R. Ivanov [EMAIL PROTECTED] Fingerprint: C824 CBD7 EE4B D7F8 5331 89D5 FCDA 572E DDE5 E715 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFEMhV3/NpXLt3l5xURAkeQAJ9S1IUxoJYDEAlqsJ5nPq6xZELoCwCgnHDJ BnUYwhfLTD6eU20cF09aA/U= =hL61 -END PGP SIGNATURE-
RE: recursive DNS servers DDoS as a growing DDoS problem
We have done just this (block inbound udp/53) to certain subnets due to a rash of CPEs that happily proxy DNS, including recursive queries, from their WAN side. What devices? Is this a default or something customers are configuring? Ingress/Egress filtering did not help because the traffic coming to the name server was not spoofed to appear like it was coming from our network, it really was. Ingress/Egress filtering really needs to be addressed by router manufacturers so it's a default when the router is configured. If every dsl router did *gress filtering most of the spoofing issues would go away overnight. It's the same sort of thing as Exchange finally installing with relay disabled or the patch for smurf ping replies. In the case where a router is located someplace that *gress filtering just isn't a viable option the people configuring those routers should be smart enough to be able to figure out how to disable it so enabled by default really should not be a change that is an issue for router manufacturers. Geo.
RE: recursive DNS servers DDoS as a growing DDoS problem
--On den 30 mars 2006 16.08.51 -0500 Geo. [EMAIL PROTECTED] wrote: Don't you think creating a control point like that is dangerous? Especially dangerous when it's DNS which runs virtually every function on the internet? The control point is there already, as has been demonstrated by several attacks on DNS data. If we were to ignore the problem of open recursive servers, there would be a rush to implement draconian, ineffective countermeasures like packet filters. It would be a very bad thing, just like the stupid port 25 blocks are. That must be avoided and the end-to-end capability of the Internet must be preserved. (I think we are in agreement here.) It's not a conspiracy theory, it's fact, if you create a control like that someone is going to want to control it. I suggest only that we consider this along with everything else. If you have these concerns, I suggest you work with the available, standardised, implemented methods of verifying that DNS data is correct (ie. DNSSEC and TSIG for various parts of the infrastructure.) instead of pushing your head down in the sand even further believing TCP or open access to the rest of the net would make some difference in credibility for DNS messages. At least one TLD (se) has DNSSEC in production, large amounts of european IP address space have it, courtesy of RIPE. -- Måns Nilsson Systems Specialist +46 70 681 7204 cell KTHNOC +46 8 790 6518 office MN1334-RIPE Inside, I'm already SOBBING! pgpiov9uw6Iyk.pgp Description: PGP signature
Re: recursive DNS servers DDoS as a growing DDoS problem
Hello Anton, This is feasible only for corporate networks where the allocations are constant and change once in a few years. It is not feasible in any ISP/Telco above a certain size. In fact, considering the consolidation over the recent years it is not feasible for most ISPs or Telcos. In an ISP you will have to provision and reprovision the nameserver ACLs on a daily basis to match your current customer allocations and reload it like there is no tomorrow. One mistake in provisioning and you will have a large chunk of customers shouting down the support line why their internet does not work. It becomes even more entertaining if you use RFC3258 or clustering to load balance DNS traffic. In that case you often end up with a lottery where one server replies, other servers deny or vice versa. Debugging that is even more entertaining. Frankly, expecting any large ISP to deploy anything like this is not realistic. Are you sure this difficulty is due to the real problem at hand, or due to poorly designed/implemented software to manage DNS? Leaving a single ISP's recursive resolution open to the world is a minor disservice to the Internet community, but a *major* disservice to it's own customers. Cache poisoning really isn't that hard when you can dictate which records you want to poison and when you want to do it, from the outside. Especially with certain softwares' lack of source port randomization. An attacker can just wait until the next time a remotely exploitable IE hole comes out and then poison the records of a popular website, and *bam* your users are 0wned. Using QoS to limit queries coming from the outside world can be done in a manner where it does not require any extra provisioning and modification to the nameserver config. On top of that, for most well designed large ISP/Telco DNS server deployments this is just a simple config change. Once it has been rolled out it maintains itself. After all, if your customers have no network access having or not having DNS is largely irrelevant. Um... I guess I'm missing something. If it isn't difficult to limit _recursive_ query rates from the outside world, how would it be difficult to disallow them? This seems like an artifical limitation of the DNS software in use. With that said, I've never ran a very large DNS infrastructure, but I do know there's a lot of terrible DNS software out there... cheers, tim signature.asc Description: Digital signature
Re: recursive DNS servers DDoS as a growing DDoS problem
On Thu, 30 Mar 2006, Geo. wrote: Don't you think creating a control point like that is dangerous? Especially dangerous when it's DNS which runs virtually every function on the internet? Yeah, it could be indeed... It's not directly related to the discussion topic, but i just wanted to inform you about the current Internet censorship campaign promoted by the Italian government since last February 20th: a list of web sites offering (illegal) on-line gambling services is blocked through DNS records hijacking performed at the ISP level. In other words, the Italian government is forcing all the major ISPs to break regular DNS functionality and override these censored records: http://www.aams.it/site.php?page=20060213093814964op=download Italy is the first democratic country to do something like that, AFAIK. Just my 2 euro-cents, -- Marco Ivaldi Antifork Research, Inc. http://0xdeadbeef.info/ 3B05 C9C5 A2DE C3D7 4233 0394 EF85 2008 DBFD B707
Re: recursive DNS servers DDoS as a growing DDoS problem
Geo. wrote: What is stopping you from running your own local DNS server? What is stopping you from running your own SMTP server? A port 25 block? Well if an ISP doesn't want to play whack-a-mole with unsecured dns servers popping up every day do you not think it likely that they will resort to the same techniques they used for smtp? Granted a port 53 inbound block would make more sense for the current example but just like bots started running their own SMTP engines I see the dns flood model changing to fit the new landscape. We have done just this (block inbound udp/53) to certain subnets due to a rash of CPEs that happily proxy DNS, including recursive queries, from their WAN side. They DoS their own circuits more effectively than the intended DoS targets. Ingress/Egress filtering did not help because the traffic coming to the name server was not spoofed to appear like it was coming from our network, it really was. The attack reflected off of the routers and because they were local to our name servers, they got replies to the recursive queries despite our rejecting them from outside our network. And of course once it was cached, it was open for public queries. Broken/misconfigured/buggy routers appear to look just like open DNS servers, and are likely to be much higher in numbers. Jim
Re: recursive DNS servers DDoS as a growing DDoS problem
gboyce wrote: I haven't heard anyone talk about requiring that users use their ISP's DNS server. Just that they should not be able to use any random DNS server on the internet. What is stopping you from running your own local DNS server? My system at home runs named in a configuration that allows any systems on my local network to query from it. That's what I use as my authoritative nameserver. -- Greg People tend to forget that authoritative nameservers can also be used if you know what zone to query for. How about sending spoofed packets to ns1.msft.net asking for microsoft.com? Amplification attacks can be performed on any UDP service that responds with a reply packet larger than the initial query packet. All DNS servers are at risk, some servers will produce lower amplification factors than others. Simon
RE: recursive DNS servers DDoS as a growing DDoS problem
What is stopping you from running your own local DNS server? What is stopping you from running your own SMTP server? A port 25 block? Well if an ISP doesn't want to play whack-a-mole with unsecured dns servers popping up every day do you not think it likely that they will resort to the same techniques they used for smtp? Granted a port 53 inbound block would make more sense for the current example but just like bots started running their own SMTP engines I see the dns flood model changing to fit the new landscape. Geo.
Re: recursive DNS servers DDoS as a growing DDoS problem
1. Resolvers and Authoritative nameservers must be separate and authoritative nameservers must have recursion turned off. Otherwise there is no way to throttle only recursive queries. Great, for small ISP's you just doubled the number of machines they need to dedicate to DNS. 2. In a smaller ISP the nameservers themselves can get an aggregate of the ISP routing table and have internal routes tagged accordingly so that the DNS server can throttle them. No rocket science there, the provisions are already available in every single OS in use as a DNS server in ISPs/Telcos. All this requires is a moderate level of competence in the person who has designed the service. Really? Ok educate me, how do you do this with Windows 2000 running MS dns? (telling people to use another server is not acceptable) Geo.
Re: recursive DNS servers DDoS as a growing DDoS problem
[snip] I haven't heard anyone talk about requiring that users use their ISP's DNS server. Just that they should not be able to use any random DNS server on the internet. This is standard practice in Wireless and other ISPs which operate pay as you go service (hotels, conferences, hotspots of all varieties and aggregators that serve them). The reason is that people ran OpenVPN and other VPN stacks on port udp/53 without logging in and paying. Frankly - nothing but a knee jerk reaction. DNS is one of the services that easiest to transparently proxy. The same result could have been achieved if any traffic to udp/53 was redirected to the local name server (at about the same performance cost as a filter entry). At the same time people would have had the service working even if they had a remote DNS server configured in for administrative reasons. Similarly, as far as ISP servers are concerned it is possible to mitigate the recursion amplification problem completely without denying the service to a limited number of people outside the ISP who want to query the ISP resolvers for debugging and other purposes. All it takes is to throttle traffic from the resovers to outside the ISP network to a reasonably low value. Depending on the ISP this is usually in the low Kbits. All it takes is a moderate amount of competence in the ISP: 1. Resolvers and Authoritative nameservers must be separate and authoritative nameservers must have recursion turned off. Otherwise there is no way to throttle only recursive queries. 2. In a smaller ISP the nameservers themselves can get an aggregate of the ISP routing table and have internal routes tagged accordingly so that the DNS server can throttle them. No rocket science there, the provisions are already available in every single OS in use as a DNS server in ISPs/Telcos. All this requires is a moderate level of competence in the person who has designed the service. 3. Similarly, a larger ISP which has more resources can isolate the servers in something like a modified RFC2270 leaf network receiving default + local instead of default only and do the same on the router in front of them. Once again all this requires is the person who has done the DNS design to possess some moderate level of network competence instead of engaging in massive clusterf^Hing. 4. While implementing both 2 and 3 costs money both of them save resources so on the balance of things they are worth it even without getting into the aspects of mitigating DDOS attacks. They are not that hard to implement either. [snip] -- A. R. Ivanov E-mail: [EMAIL PROTECTED] WWW: http://www.sigsegv.cx/ pub 1024D/DDE5E715 2002-03-03 Anton R. Ivanov [EMAIL PROTECTED] Fingerprint: C824 CBD7 EE4B D7F8 5331 89D5 FCDA 572E DDE5 E715
Re: recursive DNS servers DDoS as a growing DDoS problem
Geo. wrote: The flood is a flood of answers not queries, you spoof the source address of a query with the address of your target, the target gets the response from the dns server. A cache on the dns server just makes it a more efficient response. Queries are bad enough. This can be played with from any point in the chain. I.e. from the amount of querying clients, the number of so-called relaying servers, possible /effective/ amplification and the size of the TXT SOA record. Increase any of these and you somewhat increase the attack if one of the others is controlled. So treating just one is not really a solution. We provided with an equation of sort to this effect in the paper we pre-released for the community when the FUD started (was supposed to eventually be an academic paper): http://www.isotf.org/news/DNS-Amplification-Attacks.pdf Yes, in my opinion recursion should be put under better control, but it's what-a-mole all over again if we do it by running after servers. The problem here (if we are to ignore just for a moment other UDP/DNS attacks) is with the attack vector - spoofing. Many networks today allow spoofing. Should recursion by default be disabled? Yes. Is it a problem when done insecurely? Yes. Is it what we should run after on the client-side servers? No, or maybe not Yes but not Only. I have not seen a perfect solution yet, at best the solutions I've seen mentioned eliminate this one flood vector. I would suggest that when considering which one to choose we look at what we lose with each choice. Eliminate spoofing and you lose virtually nothing, eliminate open recursive I am with you so far. Spoofing is not the only problem of the Internet by far, but is is a problem mostly ignored thus far as it was not actively exploited *sniff* servers and you have just created a really powerful control mechanism for entities to control large sections of the internet since folks from those sections won't be able to use anyone else's DNS servers or even run their own (much like port 25 blocking limits who can run a mail server today). He who controls dns controls the network. Is this like when you brought up Government conspiracies in this regard on another list? I can understand discussion going on with different lists, I sin with that myself recently. There were no direct lists to handle DNS or botnets issues until not long ago, still - should we just skip a list whenever you are disagreed with? Gadi.
Re: recursive DNS servers DDoS as a growing DDoS problem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Stephen Samuel wrote: | To put it another way: UDP as a purely connectionless | protocol is fast becoming a liability in situations where | significant amplification is possible. My thoughts exactly. This attack is possible because of a design limitation of UDP (and other connectionless protocols). I realise that DNS is receiving attention because of it's ubiquity on the Internet, but this type of attack can leverage any service that responds with a reply packet larger than the initial query packet where the protocol used is connectionless. TCP/IP is showing it's age (again). Thanks, Paul -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFELIV14qOLghPAuV0RAvrVAKCZBb85yHnwB3+RAOyqvDocbUQYPwCgzSpV 1nnSrOO74gRDzoZSnNxd2jI= =Rl7W -END PGP SIGNATURE-
RE: recursive DNS servers DDoS as a growing DDoS problem
servers and you have just created a really powerful control mechanism for entities to control large sections of the internet since folks from those sections won't be able to use anyone else's DNS servers or even run their own (much like port 25 blocking limits who can run a mail server today). He who controls dns controls the network. Is this like when you brought up Government conspiracies in this regard on another list? It's a security issue. He who controls the dns server controls you, yes? Ok we are talking about locking down DNS like we locked down smtp relay. So if you want to send a mail today can you just use any smtp server you want or are you severly limited, possibly by a port 25 block which forces you to use just your ISP's mail smtp server? Don't you think creating a control point like that is dangerous? Especially dangerous when it's DNS which runs virtually every function on the internet? It's not a conspiracy theory, it's fact, if you create a control like that someone is going to want to control it. I suggest only that we consider this along with everything else. Geo.
Re: recursive DNS servers DDoS as a growing DDoS problem
If you have a 20,000 bot botnet and each bot has 2 defined recursive dns servers that it is allowed to use and these bots are on the local subnet (ie BCP38 is implimented at the gateway but not at every router) then how exactly is locking down recursive servers so you can only use yours going to solve anything? uh... caching maybe?... the second field of your answer section when using dig.. To fix DNS we would have to eliminate it's use of UDP which means pretty much all internet software would need to be rewritten, that is an unrealistic goal. no, its not, and it doesnt require that all internet sofware be rewritten, it means that they either need to be recompiled, or just have the dynamicly linked library they use for dns resolution to be replaced.. Locking down recursive servers may increase the number of machines required to create a flood but again a large botnet will have no problem so that's no solution either. BCP38 will accomplish the same ineffective goal but at least has the added potential to reduce non DNS related spoofed attacks at the same time making it easier to at least track down the sources of a distributed flood at least to the provider level if not to the exact IP. i think its pretty obvious that locking down dns servers at least brings the DNS attacks down to the same problem weve always had.. that being good old fashioned udp packeting. and at that point.. why bother using dns.. -phar
Re: recursive DNS servers DDoS as a growing DDoS problem
On Sun, 26 Mar 2006, Geo. wrote: Spoofing is indeed the attack vector and it can also be utilized for NTP, ICMP, etc. It is to blame. Still, DNS is what's being exploited and in my opinion a broken feature being exploited needs fixing, or it will be exploited. What feature of DNS is being exploited, UDP or the fact that there are a lot of dns servers you can use? If you have a 20,000 bot botnet and each bot has 2 defined recursive dns servers that it is allowed to use and these bots are on the local subnet (ie BCP38 is implimented at the gateway but not at every router) then how exactly is locking down recursive servers so you can only use yours going to solve anything? Right now you don't need the 20,000 bot botnet. You can find plenty of recursive nameservers to send large responses to your victim. Even if we moved to a state where a 20,000 bot botnet could take you down, that's still better than anyone on a cable modem can take you down. However, properly fixed the 20,000 bot botnet isn't able to perform this sort of attack unless all 20,000 bots are on your ISPs network. Each bot has 2 nameservers. Properly configured, those nameservers should only send responses to the customers of the ISP in question (this would require dns server changes, not firewall rules blocking the requests). If the victim of the attack is not on the list of allowed clients, then the bot would send the request to its own nameservers, and the nameservers would refuse to respond since the victim is not an allowed IP. Greg
Re: recursive DNS servers DDoS as a growing DDoS problem
Geo. wrote: What feature of DNS is being exploited, UDP or the fact that there are a lot of dns servers you can use? I think that this is probably a better point than you think. It's almost impossible to change the design of the DNS protocol now but, going foreward, I think that we do need to add to the best-practices list that any UDP based protocol that has an ability to produce packet size amplification, and that is likely to be available to the public (i.e. not firewalled off just on principle) should be modified so that, before large packets get sent back to a client, that the service have some sort of 'hello' type protocol that requires that the initiating machine can prove that it's actually able to receive the packets that it's causing to be produced. Even something as simple as syn cookies would probably make amplification difficult for most attackers. To put it another way: UDP as a purely connectionless protocol is fast becoming a liability in situations where significant amplification is possible. -- Stephen Samuel +1(778)861-7641 [EMAIL PROTECTED] http://www.bcgreen.com/ Powerful committed communication. Transformation touching the jewel within each person and bringing it to light.
Re: recursive DNS servers DDoS as a growing DDoS problem
BCP38 is implimented at the gateway but not at every router) then how exactly is locking down recursive servers so you can only use yours going to solve anything? uh... caching maybe?... the second field of your answer section when using dig.. The flood is a flood of answers not queries, you spoof the source address of a query with the address of your target, the target gets the response from the dns server. A cache on the dns server just makes it a more efficient response. no, its not, and it doesnt require that all internet sofware be rewritten, it means that they either need to be recompiled, or just have the dynamicly linked library they use for dns resolution to be replaced.. Ok fine, it would just mean that everyone on the planet would need to replace all their internet enabled software with new versions. That'll happen overnight.. I think it's still an unrealistic goal. i think its pretty obvious that locking down dns servers at least brings the DNS attacks down to the same problem weve always had.. that being good old fashioned udp packeting. and at that point.. why bother using dns.. I have not seen a perfect solution yet, at best the solutions I've seen mentioned eliminate this one flood vector. I would suggest that when considering which one to choose we look at what we lose with each choice. Eliminate spoofing and you lose virtually nothing, eliminate open recursive servers and you have just created a really powerful control mechanism for entities to control large sections of the internet since folks from those sections won't be able to use anyone else's DNS servers or even run their own (much like port 25 blocking limits who can run a mail server today). He who controls dns controls the network. Geo.
Re: recursive DNS servers DDoS as a growing DDoS problem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chris Thompson wrote: Michael Sierchio [EMAIL PROTECTED] writes: Robert Story wrote: VG In the scenario you describe, I cannot see any actual amplification... The amplification isn't in the number of hosts responding, but in packet size. A very small DNS request packet results in a huge response packet. Are you talking about rogue authoritative servers? Otherwise, responses will be limited to 512 bytes, possibly with the truncation bit set. Unless it supports EDNS, in which case it may be persuaded to send larger replies. BIND does currently have you cannot be serious cutoff at 4096 bytes. The reason that it is more awkward to use the method against authoritative-only nameservers is that you have to find a large RRset in the wild (or one that will come with large authority and/or additional sections in the reply) and then use the authoritative nameservers for that RRset, not any old open recursive nameserver (or many of them). You cannot craft your own RRset for the purpose. That is not a problem. As usually MCI at your service. They have switched from RFC 3258 DNS design to having a very long list of name servers each of which is separate. That is at least 345 bytes of extra/authority section instead of the usual 70-100. All you need is to find a domain hosted with them. If you are happy with a 5x amplification you can simply use MCI.com They are not the only ones. It is a general trend in large ISPs/Telcos to exterminate with extreme prejudice any DNS design that requires some networking competence. Once again - transitions from RFC3258 to long lists are only one example. Plenty of others. But you can still get amplification, certainly. The real solution to this problem is people finally starting to enforce antispoofing on access networks. It is the same story as with smurf and broadcast amplification 7 years ago. It is time to put up a name and shame list out there. - -- A. R. Ivanov E-mail: [EMAIL PROTECTED] WWW: http://www.sigsegv.cx/ pub 1024D/DDE5E715 2002-03-03 Anton R. Ivanov [EMAIL PROTECTED] Fingerprint: C824 CBD7 EE4B D7F8 5331 89D5 FCDA 572E DDE5 E715 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFEI5uu/NpXLt3l5xURApliAJ9LzA/Cnan74hSvRhOEKH6B0BI1zwCfe3x2 uDzVwvQTQQ5ugwYdtRdKhbM= =AKsS -END PGP SIGNATURE-
Re: recursive DNS servers DDoS as a growing DDoS problem
Spoofing is indeed the attack vector and it can also be utilized for NTP, ICMP, etc. It is to blame. Still, DNS is what's being exploited and in my opinion a broken feature being exploited needs fixing, or it will be exploited. What feature of DNS is being exploited, UDP or the fact that there are a lot of dns servers you can use? If you have a 20,000 bot botnet and each bot has 2 defined recursive dns servers that it is allowed to use and these bots are on the local subnet (ie BCP38 is implimented at the gateway but not at every router) then how exactly is locking down recursive servers so you can only use yours going to solve anything? To fix DNS we would have to eliminate it's use of UDP which means pretty much all internet software would need to be rewritten, that is an unrealistic goal. Locking down recursive servers may increase the number of machines required to create a flood but again a large botnet will have no problem so that's no solution either. BCP38 will accomplish the same ineffective goal but at least has the added potential to reduce non DNS related spoofed attacks at the same time making it easier to at least track down the sources of a distributed flood at least to the provider level if not to the exact IP. Geo.
Re: recursive DNS servers DDoS as a growing DDoS problem
We discussed recursive DNS servers before (servers which allow to query anything - including what they are not authoritative for, through them). ... One of the problems is obviously the spoofing. ... Maybe I'm misunderstanding the problem here (but I don't think so). It seems to be the issue is not DNS -- recursive or not, but rather a failure to adhere to BCP 38. http://rfc.net/bcp38.html One may get easier/larger amplification from a recursive DNS server, but the same problem exists for a non-recursive server. DNS, then, isn't really where the problem needs to be fixed.
Re: recursive DNS servers DDoS as a growing DDoS problem
MaddHatter wrote: We discussed recursive DNS servers before (servers which allow to query anything - including what they are not authoritative for, through them). ... One of the problems is obviously the spoofing. ... Maybe I'm misunderstanding the problem here (but I don't think so). It seems to be the issue is not DNS -- recursive or not, but rather a failure to adhere to BCP 38. http://rfc.net/bcp38.html One may get easier/larger amplification from a recursive DNS server, but the same problem exists for a non-recursive server. DNS, then, isn't really where the problem needs to be fixed. You are right, to a degree. Spoofing is indeed the attack vector and it can also be utilized for NTP, ICMP, etc. It is to blame. Still, DNS is what's being exploited and in my opinion a broken feature being exploited needs fixing, or it will be exploited.
Re: recursive DNS servers DDoS as a growing DDoS problem
Robert Story wrote: VG In the scenario you describe, I cannot see any actual amplification... The amplification isn't in the number of hosts responding, but in packet size. A very small DNS request packet results in a huge response packet. Are you talking about rogue authoritative servers? Otherwise, responses will be limited to 512 bytes, possibly with the truncation bit set.
Re: recursive DNS servers DDoS as a growing DDoS problem
On Tue, 7 Mar 2006 19:26:19 +0200 Ventsislav wrote: VG Are you sure about that amplification process?? Yes. VG In the scenario you describe, I cannot see any actual amplification... The amplification isn't in the number of hosts responding, but in packet size. A very small DNS request packet results in a huge response packet.
Re: recursive DNS servers DDoS as a growing DDoS problem
Are you sure about that amplification process?? Actually if the packet reaches huge sizes it will be fragmented at the attacker's own place cuz of the network equipment's mtu... or won't be transmitted at all... The concept of the smurf attack is in sending large amount of spoofed packets to the sub-network's broadcast address. Thus resulting in a enormous amplified traffic back to the spoofed address, which happens to be the victim. In the scenario you describe, I cannot see any actual amplification... Just fragmentation. Either I misunderstood your words, of which I apologise if the case, or you've got something wrong there... Anyway.. i do not question the DDoS attack on the recursive DNS servers, but find it not so scary as some folks may say. Take care and best wishes, Ventsi PS: sorry if duplicated... On 2/28/06, Gadi Evron [EMAIL PROTECTED] wrote: Hi guys. We discussed recursive DNS servers before (servers which allow to query anything - including what they are not authoritative for, through them). The attack currently in the wild is a lot bigger and more complicated than this, but to begin, here is an explanation (by metaphor) of that part: Spoofed ICMP attacks have been around for a while. How many of us still get quite a bit of ICMP echo replies stopped at our borders? These replies come to us due to spoofed attacks using our addresses. Now, imagine it - only bigger: Smurf. Introduce an amplification effect. As bigger UDP packets will be fragmented by the servers, and UDP obviously does not do any handshake and can easily be spoofed... The server receives a large packet, breaks it down to several fragments and moves the query on. That's where the amplification effect _starts_. Both the attacked server and the unwilling participant in the attack, the recursive servers, experience a serious DNS denial of service that keeps getting amplified considerably. One of the problems is obviously the spoofing. Let us, metaphorically and WRONGLY treat it for a minute as the remote exploit. The second part of this problem is the recursive server, which for the moment we will WRONGLY treat as the local exploit. Obviously both need to be fixed. Which is easier I am not so sure. In the past, most network operators refused to implement best practices such as BCP38 (go Fergie!) because they saw no reason for the hassle. Returning back to: if it isn't being exploited right now, why should I worry about it? Well, it is being exploited now, and will be further exploited in the future. Combating spoofing on the Internet is indeed important and now becoming critical. Removing the spoofing part for a second, the attack vector for this can easily be replaced, as one example, with a botnet. A million Trojaned hosts sending in even one packet a minute would cause quite a buzz - and do. Now amplify the effect by the recursive servers and... So, putting the spoofing aside, what do we do about our recursive servers? There are some good URL's for that, here are some: http://www.us-cert.gov/reading_room/DNS-recursion121605.pdf http://cc.uoregon.edu/cnews/winter2006/recursive.htm http://dns.measurement-factory.com/surveys/sum1.html The recursive behaviour is necessary for some authoritative servers, but not for all. As a best practice for organizations, as an example, the server facing the world should not also be the one facing your organization (your users/clients). Limiting this ability to your network space is also a good idea. If you would like to check for yourselves, here is a message from Duane Wessels [1] to the DNS-operations [2] mailing list where this is currently being discussed: - If anyone has the need to test particular addresses for the presence of open resolvers, please feel free to use this tool: http://dns.measurement-factory.com/cgi-bin/openresolvercheck.pl It will send a single recursion desired query to a target address. If that query is forwarded to our authoritative server, the host has an open resolver running at that address. - Dan (DA MAN) Kaminsky and Mike Schiffman have done some impressive work on this subject, outlined in Dan's latest ShmooCon talk. They found ~580K open resolvers: http://deluvian.doxpara.com/, http://www.doxpara.com/ I suggest those of us who need more information or help go to the DNS-operations mailing list from OARC (see below) and ask the experts there, now that this is finally public. Thanks, Gadi. [1] Duane Wessels - DNS genius and among other accomplishments the author of dns top. [2] DNS-operations - http://lists.oarci.net/mailman/listinfo/dns-operations -- http://blogs.securiteam.com/ Out of the box is where I live. -- Cara Starbuck Thrace, Battlestar Galactica.
Re: recursive DNS servers DDoS as a growing DDoS problem
Here are some dns servers I gathered/scanned during the time I researched this months ago(that appear to still be up): 68.1.199.151 68.1.196.116 68.1.195.161 68.1.193.177 Just remember when you test/capture packets that the domain being resolved must NOT exist(ie. x). On Thu, 2 Mar 2006, Gadi Evron wrote: [EMAIL PROTECTED] wrote: While you're on the subject of the potentials of DOSing using DNS servers, I noticed several months ago some possible abuses myself, although I soon lost interest for some reason or another. I noticed that a portion of the worlds DNS servers for some reason or another send back large amounts of duplicate replies if, and only if, the domain being resolved does not exist. The amount of duplicates seems to range between 2 and 24(in steps of 2, 4, 8, 12, 16, 20 and 24), where each reply packet is roughly 2.5x(including IP header) larger than the original request(because of the SOA). So, for example one request to a DNS server that sends 24 dups back would roughly equal 60x(24*2.5) amplification of data. This is very interesting. I don't have any idea why that is happeniong (yet). Can you share packet captures?
Re: recursive DNS servers DDoS as a growing DDoS problem
Looking at this further, it seems to be the same attack with the x60 amplification effect. We will know more when we know more. Gadi.
Re: recursive DNS servers DDoS as a growing DDoS problem
While you're on the subject of the potentials of DOSing using DNS servers, I noticed several months ago some possible abuses myself, although I soon lost interest for some reason or another. I noticed that a portion of the worlds DNS servers for some reason or another send back large amounts of duplicate replies if, and only if, the domain being resolved does not exist. The amount of duplicates seems to range between 2 and 24(in steps of 2, 4, 8, 12, 16, 20 and 24), where each reply packet is roughly 2.5x(including IP header) larger than the original request(because of the SOA). So, for example one request to a DNS server that sends 24 dups back would roughly equal 60x(24*2.5) amplification of data. an example of a random server I found while scanning(12 dups from one request): -term1# host x 68.1.2.3 ... term2# /usr/sbin/tcpdump -n src 68.1.2.3 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 68 bytes 00:04:58.459356 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:04:58.481281 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:04:58.514411 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:05:01.459157 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:05:01.478706 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:05:01.512249 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:05:04.459512 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:05:04.480542 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:05:04.512085 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:05:07.458823 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:05:07.477374 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) 00:05:07.511919 IP 68.1.2.3.53 xxx.xxx.xxx.xxx.1865: 62623 NXDomain 0/1/0 (94) - At the time I noticed this I decided to create a scanner to find out how many DNS servers are susceptible to this, I found no shortage. I ran it only for a few hours starting at 68.0.0.1 and found hundreds of DNS servers that sent back dup replies(mostly 12 and 8 dups). I also created a DOS tool to test the theory at the time, but I see no reason to post that. I still don't know the cause of this, just figured I would attach it on this subject for someone to decypher. For anyone interested in the scanner, which is light on documentation: http://fakehalo.us/dnsdbd-gp.c http://fakehalo.us/dnsdbd.c (the -gp.c version simply stores the ip of the dns server in character form so its easier to read by human eyes)
