Re: [Fwd: Re: DNS DDoS [was: register.com down sev0?]]
On Thu, 26 Oct 2006, virendra rode // wrote: Just curious, any ddos vendors want to share their success stories :-) If you access Cisco as a customer: http://www.cisco.com/en/US/customer/products/ps5887/products_case_study0900aecd80120478.shtml "Rackspace Managed Hosting" - Customer Success Story -Hank Nussbacher http://www.interall.co.il
[Fwd: Re: DNS DDoS [was: register.com down sev0?]]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 We ran into similar attacks (couple days back) coming from non-spoofed address range (being initiated from valid prefixes). In working (w/ a co-worker of mine) on a network attack situation (trace process) for a 30,000 user location (serving 60 other school districts) running BCP38 & rate-limit which got ddos'd w/ about 8mpps. It appears that these attacks were coming from the inside which not only saturated devices along its way but also got amplified into several other networks also causing significant flaps to its peered connection (OC-xx). Besides being distracted with this incredible among of traffic flow our goal number one goal was to prevent this bleeding, thanks to the distributed monitoring sensors (maybe we got lucky) we were able to identify and sink-hole (null route) certain blocks (vlans) while we worked with the network/desktop team to isolate the infected machines. This was certainly a hair-pulling experience. The point that I'm trying to make here is, you can have data coming from a herd of comprised hosts (bots, self-propagating worms, spam-relays,fake http get request, backdoors, etc) that can attack against a well-protected system(s) so any kind of defense mechanism can/will get defeated. Then again, it doesn't mean one wouldn't want to follow well practiced prevention methods. Just curious, any ddos vendors want to share their success stories :-) regards, /virendra - Original Message ---- Subject: Re: DNS DDoS [was: register.com down sev0?] Date: Thu, 26 Oct 2006 17:32:56 + From: [EMAIL PROTECTED] Reply-To: [EMAIL PROTECTED] To: Robert Boyle <[EMAIL PROTECTED]>, [EMAIL PROTECTED],Patrick W. Gilmore <[EMAIL PROTECTED]>, Nanog References: <[EMAIL PROTECTED]><[EMAIL PROTECTED]> <[EMAIL PROTECTED]> The network hardware vendors do need to include the feature to support BCP-38. It'll help us out on a number of fronts especially with some of the recent cyber attacks. We're in process of reaching out to many of the companies and many providers to encourage the implementation of BCP-38. We've gotten a lot of great feedback from many of you and its greatly appreciated. You know who you are :) Especially some of the feedback related to the hardware OS issues. - -Jerry [EMAIL PROTECTED] or [EMAIL PROTECTED] Sent via BlackBerry from Cingular Wireless - -Original Message- From: Robert Boyle <[EMAIL PROTECTED]> Date: Thu, 26 Oct 2006 12:04:03 To:"Patrick W. Gilmore" <[EMAIL PROTECTED]>, nanog@merit.edu Subject: Re: DNS DDoS [was: register.com down sev0?] At 11:21 AM 10/26/2006, you wrote: Unfortunately, as Jared has pointed out, the equipment vendors have >to help the operators support this. So let's all call your favorite >router vendor and ask them when they will have the "ip bcp38" config >option. :) Even better would be the option: "no ip bcp38" Make it so a conscious action is needed to disable it, but PLEASE put that in the release notes so when the config doesn't "change" we know that something really did change... :) R Tellurian Networks - Global Hosting Solutions Since 1995 http://www.tellurian.com | 888-TELLURIAN | 973-300-9211 "Well done is better than well said." - Benjamin Franklin -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFQS8zpbZvCIJx1bcRAn93AKCSF2JcGTbB/bX/NcxxWdOwBXRDagCbBkY4 OBRqFdIvWojOwTK+K6Mlp2U= =LumS -END PGP SIGNATURE-
Re: DNS DDoS [was: register.com down sev0?]
The network hardware vendors do need to include the feature to support BCP-38. It'll help us out on a number of fronts especially with some of the recent cyber attacks. We're in process of reaching out to many of the companies and many providers to encourage the implementation of BCP-38. We've gotten a lot of great feedback from many of you and its greatly appreciated. You know who you are :) Especially some of the feedback related to the hardware OS issues. -Jerry [EMAIL PROTECTED] or [EMAIL PROTECTED] Sent via BlackBerry from Cingular Wireless -Original Message- From: Robert Boyle <[EMAIL PROTECTED]> Date: Thu, 26 Oct 2006 12:04:03 To:"Patrick W. Gilmore" <[EMAIL PROTECTED]>, nanog@merit.edu Subject: Re: DNS DDoS [was: register.com down sev0?] At 11:21 AM 10/26/2006, you wrote: Unfortunately, as Jared has pointed out, the equipment vendors have >to help the operators support this. So let's all call your favorite >router vendor and ask them when they will have the "ip bcp38" config >option. :) Even better would be the option: "no ip bcp38" Make it so a conscious action is needed to disable it, but PLEASE put that in the release notes so when the config doesn't "change" we know that something really did change... :) R Tellurian Networks - Global Hosting Solutions Since 1995 http://www.tellurian.com | 888-TELLURIAN | 973-300-9211 "Well done is better than well said." - Benjamin Franklin
Re: DNS DDoS [was: register.com down sev0?]
At 11:21 AM 10/26/2006, you wrote: Unfortunately, as Jared has pointed out, the equipment vendors have to help the operators support this. So let's all call your favorite router vendor and ask them when they will have the "ip bcp38" config option. :) Even better would be the option: "no ip bcp38" Make it so a conscious action is needed to disable it, but PLEASE put that in the release notes so when the config doesn't "change" we know that something really did change... :) R Tellurian Networks - Global Hosting Solutions Since 1995 http://www.tellurian.com | 888-TELLURIAN | 973-300-9211 "Well done is better than well said." - Benjamin Franklin
DNS DDoS [was: register.com down sev0?]
On Oct 26, 2006, at 1:31 AM, [EMAIL PROTECTED] wrote: It is essentially impossible to distinguish end-user requests from (im)properly created DoS packets (especially until BCP38 is widely adopted - i.e. probably never). Since there is no single place - no 13 places - which can withstand a well crafted DoS, you are guaranteed that some users will not be able to reach any of your listed authorities. Yeah - I know it hard-to-impossible to do that, and it is a tug-of-war between worm writers (to generate queries indistinguishable from real client-resolver-generated queries) and trying-to-detect-malformed- queries (such as duplicated qid, or from IP space that shouldn't be hitting this specific node). You probably dealt with more ddos than rest of us combined, so I bow to your superior knowledge. First, thanx for the nod, but there are some here who have dealt with more than I have. But I think I've seen enough to know something about it. You can try things like "filter IP addresses which should not be going to node X", but what happens if the DDoS changes the network topology enough that you can't be certain users are going where you did not? If the DDoS is large, this is pretty much guaranteed. Worse, suppose the topology changes for reasons unrelated to a DDoS. You could end up DoS'ing end users without an attack! (You could theoretically only put the filters in place when an attack is happening, but that has other problems - which may or may not be worse.) Filtering on things like duplicated query IDs is not possible on router hardware doing 10s of Gbps or millions of PPS. And doing it on the server is not useful if there are more bits / pps than the router can process. Remember, servers can't answer packets that are dropped before they get to the servers. Etc., etc., etc. Overall, we are losing the war. What good providers, like the roots, Ultra, etc., do is to minimize the effect of any attack. If a "miscreant" fires the "DDoS of biblical proportions" and only 5% of users are affected, I consider that a success. Unfortunately, those 5% don't think so, but one can only do what one can do. Besides, if it truly is an attack of biblical proportion, those 5% are probably having much larger problems than name resolution. Couple other comments: From all indications I've seen (and most are not authoritative, but it's all the info I have), this was not a DDoS of "biblical proportions". There were no whole networks to go offline, there were no massive swaths of address space flapping, there were no entire peering points being congested, etc. A few Gbps does not count as "biblical" any more. Whether this attack used spoof-source or not, BCP38 is _VITAL_, IMHO, to helping curb these things. It guarantees, at the very least, that you know where the attack is sourced. Filtering become much easier. Reaching the right operators to help with the problem becomes orders of magnitude easier. And if the miscreants just start using BotNets with real IP address, GOOD. It's not the End All Be All answer, but it is a _huge_ step in the right direction. Unfortunately, as Jared has pointed out, the equipment vendors have to help the operators support this. So let's all call your favorite router vendor and ask them when they will have the "ip bcp38" config option. :) -- TTFN, patrick
