Re: Over a decade of DDOS--any progress yet?
The NIST has proposed a framework for operators to notify botnet victims. The call for comments and article discussing it are described here: https://www.infosecisland.com/blogview/17021-Government-Proposes-ISPs-Notif y-Victims-of-Botnets.html#.TotXA6C-16Q.twitter Comments on the proposed Code of Conduct and botnet reporting initiative are due on or before 5 p.m. EDT, November 4, 2011. Written comments on the proposal may be submitted by mail to the National Institute of Standards and Technology at the U.S. Department of Commerce, 1401 Constitution Avenue, NW., Room 4822, Washington, DC 20230. Submissions may be in any of the following formats: HTML, ASCII, Word, rtf, or pdf. Online comment submissions in electronic form may be sent to consumer_notice_...@nist.gov. Paper submissions should include a compact disc (CD). CDs should be labeled with the name and organizational affiliation of the filer and the name of the word processing program used to create the document. Comments will be posted at http://www.nist.gov/itl/. A list of questions are included in the Request for Information, and can be accessed at the source link below: Source: http://www.federalregister.gov/articles/2011/09/21/2011-24180/models-to-adv ance-voluntary-corporate-notification-to-consumers-regarding-the-illicit-us e-of#p-3 http://www.federalregister.gov/articles/2011/09/21/2011-24180/models-to-ad vance-voluntary-corporate-notification-to-consumers-regarding-the-illicit-u se-of#p-3 IMHO this would go a long way to addressing the underlying root cause (botted machines). Regards, Zachary On 12/14/10 5:34 PM, Joel Jaeggli joe...@bogus.com wrote: On 12/8/10 6:30 AM, Drew Weaver wrote: Yes, but this obviously completes the 'DDoS attack' and sends the signal that the bully will win. it's part of a valid mitigation strategy. shifting the target out from underneath the blackholed address is also part of the activity. that's easier in some cases than others. the bots will move and you play whack a rat with your upstreams. joel -Drew From: alvaro.sanc...@adinet.com.uy [mailto:alvaro.sanc...@adinet.com.uy] Sent: Wednesday, December 08, 2010 8:46 AM To: rdobb...@arbor.net; North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? A very common action is to blackhole ddos traffic upstream by sending a bgp route to the next AS with a preestablished community indicating the traffic must be sent to Null0. The route may be very specific, in order to impact as less as possible. This needs previous coordination between providers. Regards.
Re: Over a decade of DDOS--any progress yet?
On 12/8/10 6:30 AM, Drew Weaver wrote: Yes, but this obviously completes the 'DDoS attack' and sends the signal that the bully will win. it's part of a valid mitigation strategy. shifting the target out from underneath the blackholed address is also part of the activity. that's easier in some cases than others. the bots will move and you play whack a rat with your upstreams. joel -Drew From: alvaro.sanc...@adinet.com.uy [mailto:alvaro.sanc...@adinet.com.uy] Sent: Wednesday, December 08, 2010 8:46 AM To: rdobb...@arbor.net; North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? A very common action is to blackhole ddos traffic upstream by sending a bgp route to the next AS with a preestablished community indicating the traffic must be sent to Null0. The route may be very specific, in order to impact as less as possible. This needs previous coordination between providers. Regards.
Re: Over a decade of DDOS--any progress yet?
The thread made it to both NetworkWorld: http://www.networkworld.com/news/2010/120910-wikileaks-ddos-attacks.html and Slashdot: http://tech.slashdot.org/story/10/12/12/2120254/Has-Progress-Been-Made-In-Fighting-DDoS-Attacks with the usual set of comments :) -Lorand Jakab On 12/12/2010 08:58 AM, Christopher Morrow wrote: On Sun, Dec 12, 2010 at 12:42 AM, Aaron Glenn aaron.gl...@gmail.com wrote: On Sun, Dec 12, 2010 at 12:05 AM, Christopher Morrow morrowc.li...@gmail.com wrote: verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) reasonable, but 'completely self-service' ? how much to have an engineer pump my gas for me (full service)? does that include a windshield wipe down, tire pressure and oil check (old timey full service extras)? end customer sends the right community and mitigation happens... remove the community it stops. no need to call someone and make it happen, just have the NOC/etc at your network follow a simple procedure. you are funny though :) (and I think you can call for free, 1-800 number, and get an engineer to make things happen for you as well...) -Chris
RE: Over a decade of DDOS--any progress yet?
verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) -chris That doesn't sound too unreasonable as long as you are in a market Verizon services and you can find the right Verizon rep who isn't trying to sell transit at $25/mbps. thanks, -Drew
RE: Over a decade of DDOS--any progress yet?
I'm certain there are thresholds to that. Carrier grade mitigation solutions will start low and ramp up to 5, 6, 7, etc. figures depending on the attack and amount of bandwidth to be filtered among other variables. My point was, if you mitigate the attack vs. null routing the target you have to pay for the transit that the attack consumes between your network and the upstream network(s). thanks, -Drew
Re: Over a decade of DDOS--any progress yet?
On Dec 12, 2010, at 12:05 AM, Christopher Morrow wrote: verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) Or just buy a gig-e from cogent at 3$/meg/mo (or is it $4 this month?) to burn for ddos. The problem I've found is that some of the vendors of ddos gear still have significant problems they are working to address. The Cisco (riverhead) guard would have a 1 second delay (for example) for each configuration line one would add. If you dealt with a wildcard rule, it would be 1 second per underlying rule to make the configuration change. The ability to 'paste' something in to a device and have a predictable output seemed to be too high of a bar for them to solve, this could be one of the reasons the product went to the wayside. I'm also not sure that anyone else is much better in this regard. Of course everyone is willing to sell you a seven-figure solution for your problems, but once you actually start talking about the usability, ease of provisioning, and the customer education about the caveats most people start to glaze quickly. Even with the right gear, technology, etc.. the vendors don't make it easy to deliver these solutions. - Jared
Re: Over a decade of DDOS--any progress yet?
On Mon, Dec 13, 2010 at 8:49 AM, Drew Weaver drew.wea...@thenap.com wrote: verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) -chris That doesn't sound too unreasonable as long as you are in a market Verizon services and you can find the right Verizon rep who isn't trying to sell transit at $25/mbps. if you find that guy, maybe they'll also be the mythical unicorn of a sales person who will sell you ipv6 transit too? -chris
Re: Over a decade of DDOS--any progress yet?
On Mon, Dec 13, 2010 at 8:52 AM, Drew Weaver drew.wea...@thenap.com wrote: I'm certain there are thresholds to that. Carrier grade mitigation solutions will start low and ramp up to 5, 6, 7, etc. figures depending on the attack and amount of bandwidth to be filtered among other variables. My point was, if you mitigate the attack vs. null routing the target you have to pay for the transit that the attack consumes between your network and the upstream network(s). so... with a carrier managed solution (or the one ATT/Sprint/VZB sold) the transit of the attack happens inside their networks and isn't charged to the end-customer (the destination, obviously contributing customers get charged :) ) -chris
Re: Over a decade of DDOS--any progress yet?
On 12/13/2010 8:32 AM, Jared Mauch wrote: Or just buy a gig-e from cogent at 3$/meg/mo (or is it $4 this month?) to burn for ddos. *cough* 10G burstable with 1-2G commit. Still cheaper than anything else I have or can get, and more likely to handle those large DDOS cases, where you can just reroute the effected network through the 10G and mitigate with whatever hardware you have. Of course everyone is willing to sell you a seven-figure solution for your problems, but once you actually start talking about the usability, ease of provisioning, and the customer education about the caveats most people start to glaze quickly. Even with the right gear, technology, etc.. the vendors don't make it easy to deliver these solutions. True, but they often will dedicate some time and effort during an attack to make things work. There are many in-house custom solutions you can use, and we've seen public blacklists use many of them over the years. If you want the extra support during the crisis, you pay the 3rd party for their product to get it. Jack
Re: Over a decade of DDOS--any progress yet?
On Dec 13, 2010, at 11:15 AM, Jack Bates wrote: On 12/13/2010 8:32 AM, Jared Mauch wrote: Or just buy a gig-e from cogent at 3$/meg/mo (or is it $4 this month?) to burn for ddos. *cough* 10G burstable with 1-2G commit. Still cheaper than anything else I have or can get, and more likely to handle those large DDOS cases, where you can just reroute the effected network through the 10G and mitigate with whatever hardware you have. my point is, there is this 'middle' space where it's hard to justify spending money on something that isn't used. Of course it's easy to view as insurance and easier to justify *after* an attack (or loss). it is hard to proactively justify this type of expense. If for every 10g of capacity, you had a 40k/year Security surcharge, at what point do you factor this in as part of your regular bandwidth costs vs the current down and to the right pricing trend. Delivering these services is something I have observed it is difficult to ask someone to pay for unless they have experience with it. Most are willing to start off with the self-insure premise until it is too much to bear, then immediately they are willing to pay 'something' to allow capital cost recovery. Of course everyone is willing to sell you a seven-figure solution for your problems, but once you actually start talking about the usability, ease of provisioning, and the customer education about the caveats most people start to glaze quickly. Even with the right gear, technology, etc.. the vendors don't make it easy to deliver these solutions. True, but they often will dedicate some time and effort during an attack to make things work. There are many in-house custom solutions you can use, and we've seen public blacklists use many of them over the years. If you want the extra support during the crisis, you pay the 3rd party for their product to get it. I am talking about those purporting to offer ddos solution hardware either past, present or future. If it's 2010 or 2011 and you experience flow-control like issues with your CLI interface, either slow interactive response or garbled processing (over telnet/ssh) there is something not quite right IMHO. Then again, I'm known for being a bit of an odd character. - Jared
Re: Over a decade of DDOS--any progress yet?
FYI, A single data point on current DDOS traffic levels. An Akamai press release says they handled DDOS attacks peaking at 14Gbps in the Nov. 30 to Dec 2nd time frame. http://finance.yahoo.com/news/Akamai-Shields-Leading-prnews-2768453391.html The majority of attack traffic against the five retailers initiated from distributed IP addresses out of Thailand, Mexico, Philippines, and Brazil and reached peeks of up to 14 Gbps, with some websites experiencing up to 10,000 times above normal daily traffic. Bill Bogstad
Re: Over a decade of DDOS--any progress yet?
On Dec 14, 2010, at 2:04 AM, Bill Bogstad wrote: A single data point on current DDOS traffic levels. In the 2009 Arbor WWISR, the largest attack reported was 49gb/sec. We're currently wrapping up the 2010 WWISR, and the largest attack report was considerably larger. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
The largest attacks we have solid proof on are 20+ Gbps. The only larger ones that i've seen were in company's marketing collateral vs. real life. Jeff On Mon, Dec 13, 2010 at 2:11 PM, Dobbins, Roland rdobb...@arbor.net wrote: On Dec 14, 2010, at 2:04 AM, Bill Bogstad wrote: A single data point on current DDOS traffic levels. In the 2009 Arbor WWISR, the largest attack reported was 49gb/sec. We're currently wrapping up the 2010 WWISR, and the largest attack report was considerably larger. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar. -- Jeffrey Lyon, Leadership Team jeffrey.l...@blacklotus.net | http://www.blacklotus.net Black Lotus Communications - AS32421 First and Leading in DDoS Protection Solutions
Re: Over a decade of DDOS--any progress yet?
Date: Mon, 13 Dec 2010 10:09:16 -0500 From: Christopher Morrow morrowc.li...@gmail.com On Mon, Dec 13, 2010 at 8:49 AM, Drew Weaver drew.wea...@thenap.com wrote: verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) -chris That doesn't sound too unreasonable as long as you are in a market Verizon services and you can find the right Verizon rep who isn't trying to sell transit at $25/mbps. if you find that guy, maybe they'll also be the mythical unicorn of a sales person who will sell you ipv6 transit too? Unless VZB has started accepting prefixes longer than /32, they really don't have real IPv6 transit to sell. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: ober...@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751
Re: Over a decade of DDOS--any progress yet?
On Mon, Dec 13, 2010 at 3:29 PM, Kevin Oberman ober...@es.net wrote: Date: Mon, 13 Dec 2010 10:09:16 -0500 From: Christopher Morrow morrowc.li...@gmail.com if you find that guy, maybe they'll also be the mythical unicorn of a sales person who will sell you ipv6 transit too? Unless VZB has started accepting prefixes longer than /32, they really don't have real IPv6 transit to sell. I did say 'mythical unicorn of a sales person' didn't I? :) -chris
Re: Over a decade of DDOS--any progress yet?
On Dec 14, 2010, at 2:40 AM, Jeffrey Lyon jeffrey.l...@blacklotus.net wrote: The only larger ones that i've seen were in company's marketing collateral vs. real life. Here's a link to last year's Report (previous editions may be downloaded, as well): http://www.arbornetworks.com/report The WWISR is the result of a survey we perform every year of network operators; survey participants fill in their own answers, we collect the data, collate it, analyze it, publish it. We've observed packet-flooding attacks which are considerably larger than what's reported in the WWISR via ATLAS; but as the WWISR is about what operators see and share, we vet, relay comment upon the observations of survey respondents. - Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
Greg Whynott writes: i found it funny how M$ started giving away virus/security software for its OS. it can't fix the leaky roof, so it includes a roof patch kit. (and puts about 10 companies out of business at the same time) I actually like the new arrangement better, where Microsoft provides the security software to its OS customers for free. The previous setup had third parties (anti-virus vendors) profiting from the weaknesses in Microsoft's software. The new arrangement provides better incentives for fixing the security weaknesses at the source, at least as far as Microsoft is concerned. Even for third-party providers of buggy software, Microsoft probably better leverage towards them than the numerous anti-virus vendors. But then maybe my armchair economics are totally wrong. -- Simon.
Re: Over a decade of DDOS--any progress yet?
On Fri, 10 Dec 2010 15:32:10 -0500 Drew Weaver drew.wea...@thenap.com wrote: I should've qualified my question by saying What valid application which traverses the Internet and could be seen at the edge of a network actually uses UDP 80? I'll grant that my response was a bit pedantic: there is no legitimate reason for such traffic to leave a network. I can't imagine there is too much Cisco NAC client for macs carrying on over the Internet, although I have been wrong in the past. I imagine you're right, and that any network that detects any significant amount would be one whose first octet is a common fourth-octet-of-a-gateway (1, 65, 129, etc). mc
Re: Over a decade of DDOS--any progress yet?
On Fri, Dec 10, 2010 at 5:51 PM, Joel Jaeggli joe...@bogus.com wrote: On 12/10/10 12:33 PM, Drew Weaver wrote: Nobody has really driven the point home that yes you can purchase a system from Arbor, RioRey, make your own mitigation system; what-have you, but you still have to pay for the transit to digest the attack, which is probably the main cost right now. or you outsource it and it's still costlier. Paying for DOS mitigation you rarely if ever use is quite expensive. If you use it a lot it's even more expensive, but can at least be rationalized on the basis of known costs e.g. npv calculation on the number and duration of outages... verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) -chris -Drew -Original Message- From: Dobbins, Roland [mailto:rdobb...@arbor.net] Sent: Wednesday, December 08, 2010 11:54 AM To: North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 11:47 PM, Jay Coley wrote: This has been our recent experience as well. I see a link-filling attacks with some regularity; but again, what I'm saying is simply that they aren't as prevalent as they used to be, because the attackers don't *need* to fill links in order to achieve their goals, in many cases. That being said, high-bandwidth DNS reflection/amplification attacks tip the scales, every time. Lastly there is usually always someone at the other end of these attacks watching what is working and what is not This is a very important point - determined attackers will observe and react in order to try and defeat successful countermeasures, so the defenders must watch for shifting attack vectors. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
I'm certain there are thresholds to that. Carrier grade mitigation solutions will start low and ramp up to 5, 6, 7, etc. figures depending on the attack and amount of bandwidth to be filtered among other variables. Jeff On Sun, Dec 12, 2010 at 12:05 AM, Christopher Morrow morrowc.li...@gmail.com wrote: On Fri, Dec 10, 2010 at 5:51 PM, Joel Jaeggli joe...@bogus.com wrote: On 12/10/10 12:33 PM, Drew Weaver wrote: Nobody has really driven the point home that yes you can purchase a system from Arbor, RioRey, make your own mitigation system; what-have you, but you still have to pay for the transit to digest the attack, which is probably the main cost right now. or you outsource it and it's still costlier. Paying for DOS mitigation you rarely if ever use is quite expensive. If you use it a lot it's even more expensive, but can at least be rationalized on the basis of known costs e.g. npv calculation on the number and duration of outages... verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) -chris -Drew -Original Message- From: Dobbins, Roland [mailto:rdobb...@arbor.net] Sent: Wednesday, December 08, 2010 11:54 AM To: North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 11:47 PM, Jay Coley wrote: This has been our recent experience as well. I see a link-filling attacks with some regularity; but again, what I'm saying is simply that they aren't as prevalent as they used to be, because the attackers don't *need* to fill links in order to achieve their goals, in many cases. That being said, high-bandwidth DNS reflection/amplification attacks tip the scales, every time. Lastly there is usually always someone at the other end of these attacks watching what is working and what is not This is a very important point - determined attackers will observe and react in order to try and defeat successful countermeasures, so the defenders must watch for shifting attack vectors. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar. -- Jeffrey Lyon, Leadership Team jeffrey.l...@blacklotus.net | http://www.blacklotus.net Black Lotus Communications - AS32421 First and Leading in DDoS Protection Solutions
Re: Over a decade of DDOS--any progress yet?
On Sun, Dec 12, 2010 at 12:20 AM, Jeffrey Lyon jeffrey.l...@blacklotus.net wrote: I'm certain there are thresholds to that. Carrier grade mitigation solutions will start low and ramp up to 5, 6, 7, etc. figures depending on the attack and amount of bandwidth to be filtered among other variables. nope, the pricing (when I was there, and I don't think it's changed much) is 3250/month for 500mbps or mitigation, though there was ~12gbps available easily before any work had to be done by the ISP... If the plan I/sfouant put in place was followed you could had scaled the capacity to much higher than that. If a customer continuously abused the 'limit' they may have been boosted to the next tier, but... I'd not ever seen that done. 3250/month... easy, peasy. -chris Jeff On Sun, Dec 12, 2010 at 12:05 AM, Christopher Morrow morrowc.li...@gmail.com wrote: On Fri, Dec 10, 2010 at 5:51 PM, Joel Jaeggli joe...@bogus.com wrote: On 12/10/10 12:33 PM, Drew Weaver wrote: Nobody has really driven the point home that yes you can purchase a system from Arbor, RioRey, make your own mitigation system; what-have you, but you still have to pay for the transit to digest the attack, which is probably the main cost right now. or you outsource it and it's still costlier. Paying for DOS mitigation you rarely if ever use is quite expensive. If you use it a lot it's even more expensive, but can at least be rationalized on the basis of known costs e.g. npv calculation on the number and duration of outages... verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) -chris -Drew -Original Message- From: Dobbins, Roland [mailto:rdobb...@arbor.net] Sent: Wednesday, December 08, 2010 11:54 AM To: North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 11:47 PM, Jay Coley wrote: This has been our recent experience as well. I see a link-filling attacks with some regularity; but again, what I'm saying is simply that they aren't as prevalent as they used to be, because the attackers don't *need* to fill links in order to achieve their goals, in many cases. That being said, high-bandwidth DNS reflection/amplification attacks tip the scales, every time. Lastly there is usually always someone at the other end of these attacks watching what is working and what is not This is a very important point - determined attackers will observe and react in order to try and defeat successful countermeasures, so the defenders must watch for shifting attack vectors. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar. -- Jeffrey Lyon, Leadership Team jeffrey.l...@blacklotus.net | http://www.blacklotus.net Black Lotus Communications - AS32421 First and Leading in DDoS Protection Solutions
Re: Over a decade of DDOS--any progress yet?
On Sun, Dec 12, 2010 at 12:05 AM, Christopher Morrow morrowc.li...@gmail.com wrote: verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) reasonable, but 'completely self-service' ? how much to have an engineer pump my gas for me (full service)? does that include a windshield wipe down, tire pressure and oil check (old timey full service extras)?
Re: Over a decade of DDOS--any progress yet?
On Sun, Dec 12, 2010 at 12:42 AM, Aaron Glenn aaron.gl...@gmail.com wrote: On Sun, Dec 12, 2010 at 12:05 AM, Christopher Morrow morrowc.li...@gmail.com wrote: verizon's ddos service was/is 3250/month flat... not extra if there was some sort of incident, and completely self-service for the customer(s). Is 3250/month a reasonable insurance against loss? (40k/yr or there abouts) reasonable, but 'completely self-service' ? how much to have an engineer pump my gas for me (full service)? does that include a windshield wipe down, tire pressure and oil check (old timey full service extras)? end customer sends the right community and mitigation happens... remove the community it stops. no need to call someone and make it happen, just have the NOC/etc at your network follow a simple procedure. you are funny though :) (and I think you can call for free, 1-800 number, and get an engineer to make things happen for you as well...) -Chris
RE: Over a decade of DDOS--any progress yet?
Upstream providers generally have a hard time allowing you to write routes that you don't own into their table(s). thanks, -Drew -Original Message- From: Chris Boyd [mailto:cb...@gizmopartners.com] Sent: Wednesday, December 08, 2010 2:19 PM To: NANOG Subject: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 9:33 AM, Arturo Servin wrote: Yes, but all of them rely on your upstreams or in mirroring your content. If 100 Mbps are reaching your input interface of 10Mbps there is not much that you can do. Hmm. What would be really cool is if you could use Snort, NetFlow/NBAR, or some other sort of DPI tech to find specifically the IP addresses of the DDoS bots, and then pass that information back upstream via BGP communities that tell your peer router to drop traffic from those addresses. That way the target of the traffic can continue to function if the DDoS traffic doesn't closely mimic the normal traffic. Your BGP peer router would need to have lots of memory for /32 or /64 routes though. Anyone heard of such a beast? Or is this how the stuff from places like Arbor Networks do their thing? --Chris
RE: Over a decade of DDOS--any progress yet?
Ah, Honestly we can usually point to the exact cause of the attacks once we have time to triage the situation. Recently it has been stuff like: -Made someone in Asia angry. -Running a runescape server and made someone angry -Made someone on IRC angry It has been pretty rare to see an attack that wasn't just the end result of a pissing contest. and like I said most of the ones I have seen recently are either UDP 80 floods which is probably the result of one of the UDP.PL variants or fragments (UDP DST 0) attacks which kind of indicates at least in part that the 'attacker' simply downloaded the first thing they could find that said 'DDoS' on it and didn't spend too much time worrying about it. This is probably mainly because of how easy it is now to acquire dedicated servers (that arent properly monitored) and have 1Gbps (and now) 10Gbps connections to the Internet. How many organizations are using 10G connections to the Internet these days? -Drew -Original Message- From: Matthew Petach [mailto:mpet...@netflight.com] Sent: Wednesday, December 08, 2010 1:35 PM To: j...@prolexic.com Cc: nanog@nanog.org Subject: Re: Over a decade of DDOS--any progress yet? On Wed, Dec 8, 2010 at 8:47 AM, Jay Coley j...@prolexic.com wrote: On 08/12/2010 16:14, Drew Weaver wrote: I would say that 99% of the attacks that we see are 'link fillers' with 1% being an application attack. thanks, -Drew This has been our recent experience as well. There are some pure app attacks, to be sure, but we many blended attacks also. Bandwidth (UDP/ICMP/SYN Flood) attack to distract with a app attack (GET/PUSH floods) attempting to run underneath the radar. We regularly see SYN floods these days 20 Gb/s. Another thing to be aware of--when you get hit with what seems to be a simple flooding attack aimed at one point of your infrastructure... start checking your logs at _other_ places in your network very, VERY carefully. There seems to be a trend of using larger-scale flooding, or other simple types of attacks to get all the network people at an organization rushing over to throw resources and energy at it...while the real target of the attack is something completely different, on a different subnet, in a different part of the company; and that attack is small, carefully focused at its target, and is designed to be relatively quiet. The big attack is used simply to ensure all the human energy is focused on the wrong place, increasing the chance that what otherwise might caused raised eyebrows and double-checking of logs/IDS alerts, etc. gets missed while everyone is focusing on thebig attack. The thing to bear in mind is that app attacks *are* difficult to detect as they are low bandwidth and make a full TCP connection. As a result many IDS/Firewalls etc regularly miss these attacks. Lastly there is usually always someone at the other end of these attacks watching what is working and what is not. If the attack doesn't work they will simply round up more bots to increase the attack bandwidth or change the attack vector. And, in what seems to be an increasing trend, what they are watching for is *not* necessarily the result of the large botnet attack; they're checking on the results of their targeted probes elsewhere in the network, or on the outbound set of connections from a compromised machine within an organization; after all, during a huge DDoS attack, with everyone focusing on a set of uplinks being flooded with _inbound_ traffic, who is going to notice the (relatively smaller) outbound spike of traffic as the compromised machine sends out a copy of your internal intellectual property to the miscreant recipients? Matt (speaking purely hypothetically, of course, and definitely not on behalf of any institution or entity other than myself)
RE: Over a decade of DDOS--any progress yet?
I should've qualified my question by saying What valid application which traverses the Internet and could be seen at the edge of a network actually uses UDP 80? I can't imagine there is too much Cisco NAC client for macs carrying on over the Internet, although I have been wrong in the past. -Drew -Original Message- From: Michael Costello [mailto:mc3...@columbia.edu] Sent: Wednesday, December 08, 2010 11:59 AM To: nanog@nanog.org Subject: Re: Over a decade of DDOS--any progress yet? On Wed, 8 Dec 2010 11:13:01 -0500 Drew Weaver drew.wea...@thenap.com wrote: The most common attacks that I have seen over the last 12 months, and let's say I have seen a fair share have been easily detectable by the source network. It is either protocol 17 (UDP) dst port 80 or UDP Fragments (dst port 0..) What valid application actually uses UDP 80? The Cisco NAC client for Macs, for the purpose of VLAN change detection, sends UDP/80 packets to the host's reversed default gateway (i.e., if the actual gateway is 1.2.3.4, it sends the packets to 4.3.2.1) once every five seconds. mc
RE: Over a decade of DDOS--any progress yet?
Nobody has really driven the point home that yes you can purchase a system from Arbor, RioRey, make your own mitigation system; what-have you, but you still have to pay for the transit to digest the attack, which is probably the main cost right now. -Drew -Original Message- From: Dobbins, Roland [mailto:rdobb...@arbor.net] Sent: Wednesday, December 08, 2010 11:54 AM To: North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 11:47 PM, Jay Coley wrote: This has been our recent experience as well. I see a link-filling attacks with some regularity; but again, what I'm saying is simply that they aren't as prevalent as they used to be, because the attackers don't *need* to fill links in order to achieve their goals, in many cases. That being said, high-bandwidth DNS reflection/amplification attacks tip the scales, every time. Lastly there is usually always someone at the other end of these attacks watching what is working and what is not This is a very important point - determined attackers will observe and react in order to try and defeat successful countermeasures, so the defenders must watch for shifting attack vectors. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
RE: Over a decade of DDOS--any progress yet?
Yes, and I have no problem with this in theory, I just wish that some of the larger ones could proactively monitor their networks to avoid crushing the smaller ones but maybe this is intentional. I have seen a huge increase in the number of attacks originating from other hosting companies recently. Previously it had mainly been cable modems, etc. It must be much easier to just target IaaS providers to build botnets because each machine there has 1Gbps than to worry about collecting 100 10Mbps cable modem customers. -Drew -Original Message- From: Randy McAnally [mailto:r...@fast-serv.com] Sent: Wednesday, December 08, 2010 11:59 AM To: Drew Weaver; 'Jeffrey Lyon'; Jack Bates Cc: North American Operators' Group Subject: RE: Over a decade of DDOS--any progress yet? Soon several providers will begin offering dedicated servers with a 10Gbps connection to a single machine. -Drew Several already do. -Randy
Re: Over a decade of DDOS--any progress yet?
On Dec 11, 2010, at 5:51 AM, Joel Jaeggli wrote: Paying for DOS mitigation you rarely if ever use is quite expensive. Some operators offer 'Clean Pipes' commercial DDoS mitigation services; they have various fee models, and they charge their end-customers for it. It's positioned as a form of insurance, for the end-customer. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On Wed, Dec 8, 2010 at 8:02 PM, JC Dill jcdill.li...@gmail.com wrote: On 08/12/10 1:38 PM, valdis.kletni...@vt.edu wrote: The second issue is that if you *do* establish a legal precident that software vendors are liable for faults no matter what the contract/EULA says, It doesn't matter what contract an auto maker makes with someone who purchases the car, if the brakes fail and the car hits ME, I can sue the auto maker due to the defective brakes. If they design the car in a way that a 3rd party can easily tamper with the brakes, and then the car hits me, I can also sue the auto maker. They are legally required to take due care in how they design the car to ensure that innocent bystanders aren't injured or killed by a design defect. IMHO, there's no difference in the core responsibility that software makers should be held to, to ensure that their software isn't easily compromised and used to attack and injure 3rd parties. The EULA is a red herring, as it only applies to the purchaser (who agrees to the EULA when they purchase the computer or software), not to 3rd parties who are injured. If the software doesn't work as designed and the purchaser is unhappy, that's between them and the company they bought the software from. But when it injures a 3rd party, that's a whole different ball game. I truly don't understand why ISP's (who bear the brunt of the burden of the fall-out from the compromised software, as they fight spam and have to provide customer support to users who complain that the internet is slow etc.) haven't said ENOUGH. jc If you look at the national vulnerability database listings, though, it's really not clear who you'd need to go after: http://blogs.technet.com/b/security/archive/2008/05/15/q1-2008-client-os-vulnerability-scorecard.aspx Granted, that was two years ago; but it sure seems that just vilifying Microsoft, satisfying though it might be, would be to ignore the breadth of the problem. Matt
RE: Over a decade of DDOS--any progress yet?
If you look at the national vulnerability database listings, though, it's really not clear who you'd need to go after: http://blogs.technet.com/b/security/archive/2008/05/15/q1-2008-client- os-vulnerability-scorecard.aspx Granted, that was two years ago; but it sure seems that just vilifying Microsoft, satisfying though it might be, would be to ignore the breadth of the problem. Matt Is anyone actually using Ubuntu 6.06LTS anymore? That was published for Q1 2008, that was almost three years ago which in internet years is a long time. One also has to wonder (since the link to the original paper seems to be dead) if that was out of the box 6.06LTS or 6.06LTS kept updated with the security releases.
Re: Over a decade of DDOS--any progress yet?
On Wed, Dec 08, 2010 at 07:43:52AM -0800, JC Dill wrote: ISPs are not the source. The source is Microsoft. The source is their buggy OS that is easily compromised to enable the computers to be taken over as part of the botnet. I often disagree vehemently with JC, but not this time. I've been studying bot-generated spam for most of the last decade, and to about 6 nine's, it's all been from Windows boxes. (The rest? A smattering of indeterminate and various 'nix systems including MacOS.) The botnet problem is a Microsoft problem. Now...whether the botnet problem will still be a Microsoft problem in 2015: can't say. Clearly attackers have plenty of reasons to attack other systems and in some cases, they'll be successful. But it appears that to date, the advantages they might accrue from owning a box running one of the superior operating systems are outweighed by the costs of the effort to do so. (With a few rare exceptions, of course.) But you don't have to take my word for this. Turn on passive OS fingerprinting on your MX's and start recording data, including DNS and rDNS, putative sender, recipient, etc. Accumulate a couple years' worth and analyze. This is why some rather effective defensive techniques (not just for spam) can be constructed by differentiating traffic based on the operating system of the host originating that traffic. ---rsk
Re: Over a decade of DDOS--any progress yet?
On Thursday, December 09, 2010 03:43:11 am George Bonser wrote: Is anyone actually using Ubuntu 6.06LTS anymore? That was published for Q1 2008, that was almost three years ago which in internet years is a long time. Yes. I have some desktop users still on 6.06LTS, and they are kept updated. Plans to migrate to CentOS 6 are in the works, with very careful application mapping for the least user retraining, and we should be able to do the migration shortly after CentOS 6 is out, which could be a little while (I would guess February or March timeframes for final C6 release, personally, press reports notwithstanding). So we're taking our time doing that Further, I know of RH9 and RH8.0 systems still in production, and have a Red Hat Linux 5.2 box still in (not connected to the Internet) production, where it's run for the last 12 years, with a few hardware repairs and upgrades of the years. It wouldn't be wise to run that box on an open Internet connection; but for the application it serves it works, and retooling the app to run on something later isn't currently an option (the app uses libc5, and the version in Red Hat Linux 6 doesn't get along with the app very well). It will soon be time to virtualize it, and, like COBOL and FORTRAN apps of yesteryear, it will live on and on and on and on...
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 10:10 PM, Thomas Mangin wrote: Until this is sorted I believe flowspec will be a marginal solution. We're seeing a significant uptick in flowspec interest, actually, and S/RTBH has been around for ages. Great to hear :) But my point is still valid [...] After some offline discussion with Pedro Marques, I now realise that I misunderstood the flow rule validation process, which mean that my complain is really irrelevant, which is good news as it mean that inter ISP flow route exchange really have no technical obstacle that I can now think off. Thomas
Re: Over a decade of DDOS--any progress yet?
On 12/8/2010 3:04 PM, Seth Mattinen wrote: On 12/8/2010 08:06, Jack Bates wrote: I call BS. Windows has it's problems, but it is the most common exploited as it holds the largest market share. Many Windows infections I've seen occur not due to the OS, but due to lack of patching of applications on the OS. The system does as much as it can. And end users clicking/running every shiny thing they come across, consequences be damned. ActiveX is the problem. Its got about as much security as a piece of swiss cheese.
Re: Over a decade of DDOS--any progress yet?
i found it funny how M$ started giving away virus/security software for its OS. it can't fix the leaky roof, so it includes a roof patch kit. (and puts about 10 companies out of business at the same time) Many Windows infections I've seen occur not due to the OS, but due to lack of patching of applications on the OS. The system does as much as it can. which applications are home users using which are exploited more than RPC and friends? -g -- This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.
Re: Over a decade of DDOS--any progress yet?
On Thu, Dec 9, 2010 at 3:45 AM, Rich Kulawiec r...@gsp.org wrote: On Wed, Dec 08, 2010 at 07:43:52AM -0800, JC Dill wrote: ISPs are not the source. The source is Microsoft. The source is their buggy OS that is easily compromised to enable the computers to be taken over as part of the botnet. I often disagree vehemently with JC, but not this time. I've been studying bot-generated spam for most of the last decade, and to about 6 nine's, it's all been from Windows boxes. (The rest? A smattering of indeterminate and various 'nix systems including MacOS.) The botnet problem is a Microsoft problem. OK. People took exception to my last message, as the data from it was 2 years old. Here's data from 2010, which shows that the problem isn't the MSFT OS itself; it's the third-party apps that people happily double click on and install willy-nilly: http://blogs.computerworld.com/16575/security_firm_says_apple_has_more_security_holes_than_anyone (yes, you have to read past some apple bashing at the beginning; get past that, and you hit the real aspect, which is that the major security vulnerabilities exist in third party applications, rather than the OS itself.) So, as much as I love Microsoft bashing as much as the next person (and the folks here know there's definite reasons why I'll usually be one of the first in line to bash them, when the situation calls for it), in this case, putting the thumbscrews to Microsoft isn't going to fix buggy Acrobat Reader software, and all those other third party apps that people use to exploit the platform. Now...whether the botnet problem will still be a Microsoft problem in 2015: can't say. Clearly attackers have plenty of reasons to attack other systems and in some cases, they'll be successful. But it appears that to date, the advantages they might accrue from owning a box running one of the superior operating systems are outweighed by the costs of the effort to do so. (With a few rare exceptions, of course.) The sheer volume of bots may still be Windows boxes, yes; but that doesn't mean the initial vulnerability and exploit happened anywhere in the Microsoft code base. Look at how many vulnerabilities have been listed for Adobe Acrobat Reader, for example: https://secunia.com/advisories/product/19237/ 159 vulnerabilities in Adobe Reader, vs 69 in Windows 7: https://secunia.com/advisories/product/27467/ But you don't have to take my word for this. Turn on passive OS fingerprinting on your MX's and start recording data, including DNS and rDNS, putative sender, recipient, etc. Accumulate a couple years' worth and analyze. This is why some rather effective defensive techniques (not just for spam) can be constructed by differentiating traffic based on the operating system of the host originating that traffic. Sure, there's more windows boxes out there than any other OS. But that doesn't mean the weakness and vulnerabilities being exploited are *part of the native OS*. If the OS is 100% bulletproof, but users are still installing insecure third party apps that are riddled with holes, you're still going to see more botnet machines with that OS fingerprint than any other, simply based on their overall percentage representation out of the total count of computers; but hammering on the OS vendor isn't going to do *anything* to slow down the rate of infection--there isn't anything more they can do. So--as much as I dislike Microsoft, beating on them isn't the answer here. Tell people to stop installing buggy software like Adobe Acrobat Reader, and you'll get closer to stemming the tide of infections. Matt
Re: Over a decade of DDOS--any progress yet?
On Thu, 09 Dec 2010 06:45:45 EST, Rich Kulawiec said: I've been studying bot-generated spam for most of the last decade, and to about 6 nine's, it's all been from Windows boxes. (The rest? A smattering of indeterminate and various 'nix systems including MacOS.) The botnet problem is a Microsoft problem. If it's a Flash exploit, and the miscreants only do a Windows version because that gets them 85% of the targets and they feel the effort of creating a Mac/ Linux version isn't worth the incremental 15%, then you'll only see hits from Windows boxes. But how does that make it a Microsoft problem? You don't see spam from many Linux boxes because there aren't enough Linux boxes to make it cost-effective to develop malware for. If you need 5,000 bots, it's easier to find 5,000 Windows targets than finding 5,000 Linux targets. And the reason you don't see worms that target Z/OS or VMS or Irix isn't because of their inherent security. The only way you'll get it to be a non-Microsoft problem is by changing the playing field enough so that OSX and Linux and others have enough market share that targeting just Windows is a losing strategy. Good luck with that. Meanwhile, ponder what I mentioned in a previous mail - Windows is *already* close to as secure as you can sell to an end user. Consider these Google results for SELinux: SELinux howto - about 96,900 results SELInux disable - about 178,000 results SELinux turn off - about 199,000 results It's pretty obvious that there is a point where most users won't put up with the inconvenience of security, and SELinux is already on the far side of it, even for the probably-more-technical users of Linux. How are you going to sell similar hardening to Joe Sixpack, given that most of the hardening will result in either additional are you sure? pop-ups or breakage of things they bought the computer to do? The first time a user gets fragged in WoW or other game because the security threw up a pop-up at an inopportune time, that user *will* look for a way to turn the security off. pgpl1uZ2bwYel.pgp Description: PGP signature
Re: Over a decade of DDOS--any progress yet?
actually, botnets are an artifact. claiming that the tool is the problem might be a bit short sighted. with the evolution of Internet technologies (IoT) i suspect botnet-like structures to become much more prevelent and useful for things other than coordinated attacks. just another PoV. --bill On Wed, Dec 08, 2010 at 04:46:13AM +, Dobbins, Roland wrote: On Dec 8, 2010, at 11:26 AM, Sean Donelan wrote: Other than trying to hide your real address, what can be done to prevent DDOS in the first place. DDoS is just a symptom. The problem is botnets. Preventing hosts from becoming bots in the first place and taking down existing botnets is the only way to actually *prevent* DDoS attacks. Note that prevention is distinct from *defending* oneself against DDoS attacks. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 5:58 PM, bmann...@vacation.karoshi.com wrote: actually, botnets are an artifact. claiming that the tool is the problem might be a bit short sighted. with the evolution of Internet technologies (IoT) i suspect botnet-like structures to become much more prevelent and useful for things other than coordinated attacks. I'm a big advocate of distributed/agile computing models with swarming/flocking behaviors - see slide 32 of this preso for an example: https://files.me.com/roland.dobbins/c07vk1 When these things are harnessed together in order to launch DDoS attacks and steal financial information and intellectual property and so forth, we call them 'botnets'. They're a force-multiplier which allow the attacker to avoid the von Clausewitzian friction of conflict, and which give him a comfortable degree of anonymity, not to mention highly asymmetrical force projection capabilities and global presence. 'Botnet-like structures' = botnets, for purposes of this discussion. Semantic hair-splitting. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 7:28 PM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. The technology exists to detect and classify this attack traffic, and is deployed in production networks today. And of course, the legitimate owners of the botted hosts are generally unaware that their machine is being used for nefarious purposes. In the other hand the target of a DDoS cannot do anything to stop to attack besides adding more BW or contacting one by one the whole path of providers to try to minimize the effect. Actually, there're lots of things they can do. I know that this has many security concerns, but would it be good a signalling protocol between ISPs to inform the sources of a DDoS attack in order to take semiautomatic actions to rate-limit the traffic as close as the source? Of course that this is more complex that these three or two lines, but I wonder if this has been considerer in the past. It already exists. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
RE: Over a decade of DDOS--any progress yet?
Yes, but this obviously completes the 'DDoS attack' and sends the signal that the bully will win. -Drew -Original Message- From: alvaro.sanc...@adinet.com.uy [mailto:alvaro.sanc...@adinet.com.uy] Sent: Wednesday, December 08, 2010 8:46 AM To: rdobb...@arbor.net; North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? A very common action is to blackhole ddos traffic upstream by sending a bgp route to the next AS with a preestablished community indicating the traffic must be sent to Null0. The route may be very specific, in order to impact as less as possible. This needs previous coordination between providers. Regards. Mensaje original De: rdobb...@arbor.net Fecha: 08/12/2010 10:53 Para: North American Operators' Groupnanog@nanog.org Asunto: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 7:28 PM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. The technology exists to detect and classify this attack traffic, and is deployed in production networks today. And of course, the legitimate owners of the botted hosts are generally unaware that their machine is being used for nefarious purposes. In the other hand the target of a DDoS cannot do anything to stop to attack besides adding more BW or contacting one by one the whole path of providers to try to minimize the effect. Actually, there're lots of things they can do. I know that this has many security concerns, but would it be good a signalling protocol between ISPs to inform the sources of a DDoS attack in order to take semiautomatic actions to rate-limit the traffic as close as the source? Of course that this is more complex that these three or two lines, but I wonder if this has been considerer in the past. It already exists. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
+1 On Wed, Dec 8, 2010 at 10:30 AM, Drew Weaver drew.wea...@thenap.com wrote: Yes, but this obviously completes the 'DDoS attack' and sends the signal that the bully will win. -Drew -Original Message- From: alvaro.sanc...@adinet.com.uy [mailto:alvaro.sanc...@adinet.com.uy] Sent: Wednesday, December 08, 2010 8:46 AM To: rdobb...@arbor.net; North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? A very common action is to blackhole ddos traffic upstream by sending a bgp route to the next AS with a preestablished community indicating the traffic must be sent to Null0. The route may be very specific, in order to impact as less as possible. This needs previous coordination between providers. Regards. Mensaje original De: rdobb...@arbor.net Fecha: 08/12/2010 10:53 Para: North American Operators' Groupnanog@nanog.org Asunto: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 7:28 PM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. The technology exists to detect and classify this attack traffic, and is deployed in production networks today. And of course, the legitimate owners of the botted hosts are generally unaware that their machine is being used for nefarious purposes. In the other hand the target of a DDoS cannot do anything to stop to attack besides adding more BW or contacting one by one the whole path of providers to try to minimize the effect. Actually, there're lots of things they can do. I know that this has many security concerns, but would it be good a signalling protocol between ISPs to inform the sources of a DDoS attack in order to take semiautomatic actions to rate-limit the traffic as close as the source? Of course that this is more complex that these three or two lines, but I wonder if this has been considerer in the past. It already exists. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On 6 Dec 2010, at 15:34, David Ulevitch wrote: On Mon, Dec 6, 2010 at 6:10 AM, Patrick W. Gilmore patr...@ianai.net wrote: On Dec 6, 2010, at 4:07 AM, Jonas Frey (Probe Networks) wrote: Besides having *alot* of bandwidth theres not really much you can do to mitigate. Once you have the bandwidth you can filter (w/good hardware). Even if you go for 802.3ba with 40/100 Gbps...you'll need alot of pipes. There is a variation on that theme. Using a distributed architecture (anycast, CDN, whatever), you can limit the attack to certain nodes. If you have 20 nodes and get attacked from a botnet China, only the users on the same node as the Chinese use will be down. The other 95% of your users will be fine. This is true even if you have 1 Gbps per node, and the attack is 100 Gbps strong. I think this is only true if you run your BGP session on a different path (or have your provider pin down a static route). If you are using BGP and run it on the same path, the 100Gbps will cause massive packet loss and likely cause your BGP session to drop which will just move the attack to another site, rinse / repeat. I don't think very many people run BGP over a separate circuit, but for some folks, it might be appropriate. Running BGP over a different circuit will cause some blackholing of the traffic if the real link is down but not the BGP path. So IIMHO the best way is still a good router with some basic QOS to protect BGP on the link. Thomas
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 10:04 PM, Thomas Mangin wrote: So IIMHO the best way is still a good router with some basic QOS to protect BGP on the link. iACLs and GTSM are your friends. ; --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
A less common action is to use flowspec (if you have some Juniper gear) to drop only the attack and hopefully not any legitimate traffic. What is really missing atm is a way to filter flowspec announcements (limit the number and make sure they are for routes the peer is announcing). Until this is sorted I believe flowspec will be a marginal solution. Thomas PLUG: http://code.google.com/p/exabgp/ On 8 Dec 2010, at 13:46, alvaro.sanc...@adinet.com.uy wrote: A very common action is to blackhole ddos traffic upstream by sending a bgp route to the next AS with a preestablished community indicating the traffic must be sent to Null0. The route may be very specific, in order to impact as less as possible. This needs previous coordination between providers. Regards. Mensaje original De: rdobb...@arbor.net Fecha: 08/12/2010 10:53 Para: North American Operators' Groupnanog@nanog.org Asunto: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 7:28 PM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. The technology exists to detect and classify this attack traffic, and is deployed in production networks today. And of course, the legitimate owners of the botted hosts are generally unaware that their machine is being used for nefarious purposes. In the other hand the target of a DDoS cannot do anything to stop to attack besides adding more BW or contacting one by one the whole path of providers to try to minimize the effect. Actually, there're lots of things they can do. I know that this has many security concerns, but would it be good a signalling protocol between ISPs to inform the sources of a DDoS attack in order to take semiautomatic actions to rate-limit the traffic as close as the source? Of course that this is more complex that these three or two lines, but I wonder if this has been considerer in the past. It already exists. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 10:10 PM, Thomas Mangin wrote: Until this is sorted I believe flowspec will be a marginal solution. We're seeing a significant uptick in flowspec interest, actually, and S/RTBH has been around for ages. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On 8 Dec 2010, at 13:12, nanog-requ...@nanog.org wrote: Date: Wed, 8 Dec 2010 12:53:51 + From: Dobbins, Roland rdobb...@arbor.net Subject: Re: Over a decade of DDOS--any progress yet? To: North American Operators' Group nanog@nanog.org Message-ID: bf571ad7-1122-407b-b7fa-77b9bbac4...@arbor.net Content-Type: text/plain; charset=us-ascii On Dec 8, 2010, at 7:28 PM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. The technology exists to detect and classify this attack traffic, and is deployed in production networks today. Yes, they do exist. But, is people really filtering out attacks or just watching the attacks going out? And of course, the legitimate owners of the botted hosts are generally unaware that their machine is being used for nefarious purposes. In the other hand the target of a DDoS cannot do anything to stop to attack besides adding more BW or contacting one by one the whole path of providers to try to minimize the effect. Actually, there're lots of things they can do. Yes, but all of them rely on your upstreams or in mirroring your content. If 100 Mbps are reaching your input interface of 10Mbps there is not much that you can do. I know that this has many security concerns, but would it be good a signalling protocol between ISPs to inform the sources of a DDoS attack in order to take semiautomatic actions to rate-limit the traffic as close as the source? Of course that this is more complex that these three or two lines, but I wonder if this has been considerer in the past. It already exists. If you have an URL would be good. I only found a few research papers on the topic and RSVP documents but nothing really concrete. Regards, -as
Re: Over a decade of DDOS--any progress yet?
On 8 Dec 2010, at 15:12, Dobbins, Roland wrote: On Dec 8, 2010, at 10:10 PM, Thomas Mangin wrote: Until this is sorted I believe flowspec will be a marginal solution. We're seeing a significant uptick in flowspec interest, actually, and S/RTBH has been around for ages. Great to hear :) But my point is still valid, Flowspec is great if you are are a backbone and are performing the filtering, or if you want to filter outgoing traffic. If you are a smaller network, you need the filtering to be performed by your transit provider, as your uplink will otherwise be congested. So I will stand by my comment that flowspec would see a bigger uptake if T1 could accept the flowspec routes, which they will only do once they can filter them (to insure correctness and resource protection). Thomas PS : Someone need to add IPv6 support to the RFC :p
Re: Over a decade of DDOS--any progress yet?
We have seen a recent trend of attackers legitimately purchasing servers to use for attacks. They'll setup a front company, attempt to make the traffic look legitimate, and then launch attacks from their legitimate botnet. Jeff On Wed, Dec 8, 2010 at 10:33 AM, Arturo Servin arturo.ser...@gmail.com wrote: On 8 Dec 2010, at 13:12, nanog-requ...@nanog.org wrote: Date: Wed, 8 Dec 2010 12:53:51 + From: Dobbins, Roland rdobb...@arbor.net Subject: Re: Over a decade of DDOS--any progress yet? To: North American Operators' Group nanog@nanog.org Message-ID: bf571ad7-1122-407b-b7fa-77b9bbac4...@arbor.net Content-Type: text/plain; charset=us-ascii On Dec 8, 2010, at 7:28 PM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. The technology exists to detect and classify this attack traffic, and is deployed in production networks today. Yes, they do exist. But, is people really filtering out attacks or just watching the attacks going out? And of course, the legitimate owners of the botted hosts are generally unaware that their machine is being used for nefarious purposes. In the other hand the target of a DDoS cannot do anything to stop to attack besides adding more BW or contacting one by one the whole path of providers to try to minimize the effect. Actually, there're lots of things they can do. Yes, but all of them rely on your upstreams or in mirroring your content. If 100 Mbps are reaching your input interface of 10Mbps there is not much that you can do. I know that this has many security concerns, but would it be good a signalling protocol between ISPs to inform the sources of a DDoS attack in order to take semiautomatic actions to rate-limit the traffic as close as the source? Of course that this is more complex that these three or two lines, but I wonder if this has been considerer in the past. It already exists. If you have an URL would be good. I only found a few research papers on the topic and RSVP documents but nothing really concrete. Regards, -as -- Jeffrey Lyon, Leadership Team jeffrey.l...@blacklotus.net | http://www.blacklotus.net Black Lotus Communications - AS32421 First and Leading in DDoS Protection Solutions
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 10:36 PM, Thomas Mangin wrote: If you are a smaller network, you need the filtering to be performed by your transit provider, as your uplink will otherwise be congested. Actually, most DDoS attacks aren't link-flooding attacks - this hasn't been true for the last ~7 years or so. I'm not saying it doesn't happen, because it does, and sometimes quite spectacularly - but in most cases, the attackers don't have to flood the link to achieve their desired goal. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On 08/12/10 4:28 AM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. ISPs are not the source. The source is Microsoft. The source is their buggy OS that is easily compromised to enable the computers to be taken over as part of the botnet. Why isn't ANYONE going after Microsoft over this? If Microsoft were held accountable for the spam and DDOSs that spew from their crappy software, they would find a way to stop the problem. I've raised this issue before, IMHO Windows OSs are attractive nuisances and that legal argument can be used to hold Microsoft responsible for not putting an adequate fence around their attractive nuisance. If all the big ISPs banded together to file suit against Microsoft, they could share the cost (and pain) of the lawsuit. Instead, you each individually keep trying to implement in-house solutions to filter/block spam and DDOSs. How's that working for ya? jc
Re: Over a decade of DDOS--any progress yet?
And those are much more complex to detect than SYN attacks or simple flood attacks with ICMP. But even for simple flood attacks, I still think that the target has very few defence mechanisms, and those that exists require a complex coordination with upstreams. Cheers, .as On 8 Dec 2010, at 13:39, Jeffrey Lyon wrote: We have seen a recent trend of attackers legitimately purchasing servers to use for attacks. They'll setup a front company, attempt to make the traffic look legitimate, and then launch attacks from their legitimate botnet. Jeff On Wed, Dec 8, 2010 at 10:33 AM, Arturo Servin arturo.ser...@gmail.com wrote: On 8 Dec 2010, at 13:12, nanog-requ...@nanog.org wrote: Date: Wed, 8 Dec 2010 12:53:51 + From: Dobbins, Roland rdobb...@arbor.net Subject: Re: Over a decade of DDOS--any progress yet? To: North American Operators' Group nanog@nanog.org Message-ID: bf571ad7-1122-407b-b7fa-77b9bbac4...@arbor.net Content-Type: text/plain; charset=us-ascii On Dec 8, 2010, at 7:28 PM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. The technology exists to detect and classify this attack traffic, and is deployed in production networks today. Yes, they do exist. But, is people really filtering out attacks or just watching the attacks going out? And of course, the legitimate owners of the botted hosts are generally unaware that their machine is being used for nefarious purposes. In the other hand the target of a DDoS cannot do anything to stop to attack besides adding more BW or contacting one by one the whole path of providers to try to minimize the effect. Actually, there're lots of things they can do. Yes, but all of them rely on your upstreams or in mirroring your content. If 100 Mbps are reaching your input interface of 10Mbps there is not much that you can do. I know that this has many security concerns, but would it be good a signalling protocol between ISPs to inform the sources of a DDoS attack in order to take semiautomatic actions to rate-limit the traffic as close as the source? Of course that this is more complex that these three or two lines, but I wonder if this has been considerer in the past. It already exists. If you have an URL would be good. I only found a few research papers on the topic and RSVP documents but nothing really concrete. Regards, -as -- Jeffrey Lyon, Leadership Team jeffrey.l...@blacklotus.net | http://www.blacklotus.net Black Lotus Communications - AS32421 First and Leading in DDoS Protection Solutions
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 10:47 PM, Arturo Servin wrote: But even for simple flood attacks, I still think that the target has very few defence mechanisms, and those that exists require a complex coordination with upstreams. This is demonstrably incorrect. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On 12/8/2010 9:43 AM, JC Dill wrote: Why isn't ANYONE going after Microsoft over this? If Microsoft were held accountable for the spam and DDOSs that spew from their crappy software, they would find a way to stop the problem. I've raised this issue before, IMHO Windows OSs are attractive nuisances and that legal argument can be used to hold Microsoft responsible for not putting an adequate fence around their attractive nuisance. I call BS. Windows has it's problems, but it is the most common exploited as it holds the largest market share. Many Windows infections I've seen occur not due to the OS, but due to lack of patching of applications on the OS. The system does as much as it can. I've seen plenty of webmail/php/cgi hacks to not blame M$ for having market share. Jack
RE: Over a decade of DDOS--any progress yet?
The most common attacks that I have seen over the last 12 months, and let's say I have seen a fair share have been easily detectable by the source network. It is either protocol 17 (UDP) dst port 80 or UDP Fragments (dst port 0..) What valid application actually uses UDP 80? You could literally wipe out a large amount of these attacks by simply filtering this. -Drew -Original Message- From: Arturo Servin [mailto:arturo.ser...@gmail.com] Sent: Wednesday, December 08, 2010 10:48 AM To: Jeffrey Lyon Cc: nanog@nanog.org Subject: Re: Over a decade of DDOS--any progress yet? And those are much more complex to detect than SYN attacks or simple flood attacks with ICMP. But even for simple flood attacks, I still think that the target has very few defence mechanisms, and those that exists require a complex coordination with upstreams. Cheers, .as On 8 Dec 2010, at 13:39, Jeffrey Lyon wrote: We have seen a recent trend of attackers legitimately purchasing servers to use for attacks. They'll setup a front company, attempt to make the traffic look legitimate, and then launch attacks from their legitimate botnet. Jeff On Wed, Dec 8, 2010 at 10:33 AM, Arturo Servin arturo.ser...@gmail.com wrote: On 8 Dec 2010, at 13:12, nanog-requ...@nanog.org wrote: Date: Wed, 8 Dec 2010 12:53:51 + From: Dobbins, Roland rdobb...@arbor.net Subject: Re: Over a decade of DDOS--any progress yet? To: North American Operators' Group nanog@nanog.org Message-ID: bf571ad7-1122-407b-b7fa-77b9bbac4...@arbor.net Content-Type: text/plain; charset=us-ascii On Dec 8, 2010, at 7:28 PM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. The technology exists to detect and classify this attack traffic, and is deployed in production networks today. Yes, they do exist. But, is people really filtering out attacks or just watching the attacks going out? And of course, the legitimate owners of the botted hosts are generally unaware that their machine is being used for nefarious purposes. In the other hand the target of a DDoS cannot do anything to stop to attack besides adding more BW or contacting one by one the whole path of providers to try to minimize the effect. Actually, there're lots of things they can do. Yes, but all of them rely on your upstreams or in mirroring your content. If 100 Mbps are reaching your input interface of 10Mbps there is not much that you can do. I know that this has many security concerns, but would it be good a signalling protocol between ISPs to inform the sources of a DDoS attack in order to take semiautomatic actions to rate-limit the traffic as close as the source? Of course that this is more complex that these three or two lines, but I wonder if this has been considerer in the past. It already exists. If you have an URL would be good. I only found a few research papers on the topic and RSVP documents but nothing really concrete. Regards, -as -- Jeffrey Lyon, Leadership Team jeffrey.l...@blacklotus.net | http://www.blacklotus.net Black Lotus Communications - AS32421 First and Leading in DDoS Protection Solutions
Re: Over a decade of DDOS--any progress yet?
On 12/8/2010 9:52 AM, Dobbins, Roland wrote: On Dec 8, 2010, at 10:47 PM, Arturo Servin wrote: But even for simple flood attacks, I still think that the target has very few defence mechanisms, and those that exists require a complex coordination with upstreams. This is demonstrably incorrect. +1 For IPs that don't matter, automated /32 blackholes are usually supported by most providers. For critical infrastructure, I've not had a problem with the security/abuse/noc departments working with me to resolve the issue. The first step to DOS mitigation is being able to shut down the attack vector. If they hit an IP, shut it down, let the 50 other distributed systems take care of it. It's all a matter of perspective, and it has to be handled on a case by case basis. I had a dialup modem bank IP get DOS's due to a customer off it. Well, the modem bank itself doesn't need to talk to the outside world (outside of traceroutes), so a quick blackhole of it stopped the DDOS (which was a small 300mb/s). I've talked with several providers who will gladly redirect a subset of IP's through their high end filters, so in event of DOS, I can drop that /24 down to 1 transit peer, have them redirect it through their filter servers, and get clean traffic back to my network. Jack
RE: Over a decade of DDOS--any progress yet?
I would say that 99% of the attacks that we see are 'link fillers' with 1% being an application attack. thanks, -Drew -Original Message- From: Dobbins, Roland [mailto:rdobb...@arbor.net] Sent: Wednesday, December 08, 2010 10:41 AM To: North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 10:36 PM, Thomas Mangin wrote: If you are a smaller network, you need the filtering to be performed by your transit provider, as your uplink will otherwise be congested. Actually, most DDoS attacks aren't link-flooding attacks - this hasn't been true for the last ~7 years or so. I'm not saying it doesn't happen, because it does, and sometimes quite spectacularly - but in most cases, the attackers don't have to flood the link to achieve their desired goal. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On 12/8/2010 10:13 AM, Drew Weaver wrote: The most common attacks that I have seen over the last 12 months, and let's say I have seen a fair share have been easily detectable by the source network. It is either protocol 17 (UDP) dst port 80 or UDP Fragments (dst port 0..) What valid application actually uses UDP 80? You could literally wipe out a large amount of these attacks by simply filtering this. -Drew You mean silly things like: Warning, it is an 87160 line flow capture. http://www.brightok.net/~abuse/ddos/flows.txt Jack
Re: Over a decade of DDOS--any progress yet?
May be. Anyway, under ddos attack, your links may be congested, and you need to recover them. You have small margin to move. The farther upstream the attack is repelled, the better chances you have for restoring connectivity. Mensaje original De: deles...@gmail.com Fecha: 08/12/2010 12:31 Para: Drew Weaverdrew.wea...@thenap.com CC: alvaro.sanc...@adinet.com.uyalvaro.sanc...@adinet.com.uy, rdobb...@arbor.netrdobb...@arbor.net, North American Operators' Groupnanog@nanog.org Asunto: Re: Over a decade of DDOS--any progress yet? +1 On Wed, Dec 8, 2010 at 10:30 AM, Drew Weaver drew.wea...@thenap.com wrote: Yes, but this obviously completes the 'DDoS attack' and sends the signal that the bully will win. -Drew -Original Message- From: alvaro.sanc...@adinet.com.uy [mailto:alvaro.sanc...@adinet. com.uy] Sent: Wednesday, December 08, 2010 8:46 AM To: rdobb...@arbor.net; North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? A very common action is to blackhole ddos traffic upstream by sending a bgp route to the next AS with a preestablished community indicating the traffic must be sent to Null0. The route may be very specific, in order to impact as less as possible. This needs previous coordination between providers. Regards. Mensaje original De: rdobb...@arbor.net Fecha: 08/12/2010 10:53 Para: North American Operators' Groupnanog@nanog.org Asunto: Re: Over a decade of DDOS--any progress yet? On Dec 8, 2010, at 7:28 PM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. The technology exists to detect and classify this attack traffic, and is deployed in production networks today. And of course, the legitimate owners of the botted hosts are generally unaware that their machine is being used for nefarious purposes. In the other hand the target of a DDoS cannot do anything to stop to attack besides adding more BW or contacting one by one the whole path of providers to try to minimize the effect. Actually, there're lots of things they can do. I know that this has many security concerns, but would it be good a signalling protocol between ISPs to inform the sources of a DDoS attack in order to take semiautomatic actions to rate-limit the traffic as close as the source? Of course that this is more complex that these three or two lines, but I wonder if this has been considerer in the past. It already exists. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks. com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On Mon, Dec 6, 2010 at 2:50 AM, Sean Donelan s...@donelan.com wrote: What progress has been made during the last decade at stopping DDOS attacks? Observing Mastercard today, apparently none. Can't blame stupid users or Microsoft for this one, either. The 'attackers' are using a .NET tool which I'm sure all of us are familiar with, LOIC. It voluntarily (with user's consent!) adds their machine to a botnet controlled by somebody from 4chan over IRC. Because that can end well. Blaming Microsoft for DoS attacks and spam is so passé. These mouthbreathers are the bigger threat, I think. J
Re: [nanog] Re: Over a decade of DDOS--any progress yet?
Hello: On 12/8/10 10:43 AM, JC Dill wrote: On 08/12/10 4:28 AM, Arturo Servin wrote: One big problem (IMHO) of DDoS is that sources (the host of botnets) may be completely unaware that they are part of a DDoS. I do not mean the bot machine, I mean the ISP connecting those. ISPs are not the source. The source is Microsoft. The source is their buggy OS that is easily compromised to enable the computers to be taken over as part of the botnet. Many third party vendors like Adobe, Sun and others are just as culpable in this sense, if not more. A large majority of the vulnerabilities leveraged to deploy modern malware / botnets come from these client-side applications (e.g. flash, reader, java, etc) and not the OS specifically. It's beyond the point that we can blame just Microsoft. Yes, they can get better, but they've actually made great strides in software security in the last few years. Now that the other vendors are starting to feel the pain, hopefully they'll start to follow suit. Aaron
Re: Over a decade of DDOS--any progress yet?
On 12/8/2010 10:28 AM, Dobbins, Roland wrote: Application-layer attacks aside, most packet-flooding attacks these days don't completely fill links, as there's no need for the attacker to do so. I think the difference here is scale. packet-flooding attacks often do fill links; if the links drop to 155mb/s or below. I've seen some gig+ DOS, but that is less common. The DOS I posted a flow capture link for wasn't that large, but enough to flood out the little DS3 going to the small town where the target DSL customers was. Jack
Re: Over a decade of DDOS--any progress yet?
1 Gbps attacks used to be standard issue but as of the past 90 days we have been seeing 2 - 8 Gbps a lot more frequently. Jeff On Wed, Dec 8, 2010 at 11:38 AM, Jack Bates jba...@brightok.net wrote: On 12/8/2010 10:28 AM, Dobbins, Roland wrote: Application-layer attacks aside, most packet-flooding attacks these days don't completely fill links, as there's no need for the attacker to do so. I think the difference here is scale. packet-flooding attacks often do fill links; if the links drop to 155mb/s or below. I've seen some gig+ DOS, but that is less common. The DOS I posted a flow capture link for wasn't that large, but enough to flood out the little DS3 going to the small town where the target DSL customers was. Jack -- Jeffrey Lyon, Leadership Team jeffrey.l...@blacklotus.net | http://www.blacklotus.net Black Lotus Communications - AS32421 First and Leading in DDoS Protection Solutions
Re: Over a decade of DDOS--any progress yet?
On 08/12/2010 16:14, Drew Weaver wrote: I would say that 99% of the attacks that we see are 'link fillers' with 1% being an application attack. thanks, -Drew This has been our recent experience as well. There are some pure app attacks, to be sure, but we many blended attacks also. Bandwidth (UDP/ICMP/SYN Flood) attack to distract with a app attack (GET/PUSH floods) attempting to run underneath the radar. We regularly see SYN floods these days 20 Gb/s. The thing to bear in mind is that app attacks *are* difficult to detect as they are low bandwidth and make a full TCP connection. As a result many IDS/Firewalls etc regularly miss these attacks. Lastly there is usually always someone at the other end of these attacks watching what is working and what is not. If the attack doesn't work they will simply round up more bots to increase the attack bandwidth or change the attack vector. Best, --J --- Jay Coley Prolexic Technologies
Re: Over a decade of DDOS--any progress yet?
On 12/8/2010 10:41 AM, Jeffrey Lyon wrote: 1 Gbps attacks used to be standard issue but as of the past 90 days we have been seeing 2 - 8 Gbps a lot more frequently. That may well be true. I'm an eyeball network and I can usually point at a user pissing someone off on IRC/Forums for DOS instigating. I probably deal with 1 large scale attack per year at most, though most likely my attacks are from smaller botnet owners. Jack
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 11:38 PM, Jack Bates wrote: I think the difference here is scale. packet-flooding attacks often do fill links; if the links drop to 155mb/s or below. I'm not saying that link-flooding attacks don't happen; they certainly do, and on very big links, sometimes. But in the scheme of things, they don't happen nearly as often as they used to, as the attackers simply don't need to fill the links in order to accomplish their goals, in most cases. It's also important to note that a lot of DDoS isn't directly perpetrated by those who wish the DDoS performed, but rather is hired out to botmasters who're paid to execute the attacks. Even if the person who is the motivating force behind the attack is paying in stolen credit cards or whatever, he doesn't want to pay for more than is needed to accomplish his goal. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
RE: Over a decade of DDOS--any progress yet?
You can get a dedicated server for $80 with a 1Gbps connection to the Internet without looking that hard. It is pretty easy/cheap to kill a 1Gbps connection now a days. Soon several providers will begin offering dedicated servers with a 10Gbps connection to a single machine. -Drew -Original Message- From: Jeffrey Lyon [mailto:jeffrey.l...@blacklotus.net] Sent: Wednesday, December 08, 2010 11:42 AM To: Jack Bates Cc: North American Operators' Group Subject: Re: Over a decade of DDOS--any progress yet? 1 Gbps attacks used to be standard issue but as of the past 90 days we have been seeing 2 - 8 Gbps a lot more frequently. Jeff On Wed, Dec 8, 2010 at 11:38 AM, Jack Bates jba...@brightok.net wrote: On 12/8/2010 10:28 AM, Dobbins, Roland wrote: Application-layer attacks aside, most packet-flooding attacks these days don't completely fill links, as there's no need for the attacker to do so. I think the difference here is scale. packet-flooding attacks often do fill links; if the links drop to 155mb/s or below. I've seen some gig+ DOS, but that is less common. The DOS I posted a flow capture link for wasn't that large, but enough to flood out the little DS3 going to the small town where the target DSL customers was. Jack -- Jeffrey Lyon, Leadership Team jeffrey.l...@blacklotus.net | http://www.blacklotus.net Black Lotus Communications - AS32421 First and Leading in DDoS Protection Solutions
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 11:47 PM, Jay Coley wrote: This has been our recent experience as well. I see a link-filling attacks with some regularity; but again, what I'm saying is simply that they aren't as prevalent as they used to be, because the attackers don't *need* to fill links in order to achieve their goals, in many cases. That being said, high-bandwidth DNS reflection/amplification attacks tip the scales, every time. Lastly there is usually always someone at the other end of these attacks watching what is working and what is not This is a very important point - determined attackers will observe and react in order to try and defeat successful countermeasures, so the defenders must watch for shifting attack vectors. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On Wed, 8 Dec 2010 11:13:01 -0500 Drew Weaver drew.wea...@thenap.com wrote: The most common attacks that I have seen over the last 12 months, and let's say I have seen a fair share have been easily detectable by the source network. It is either protocol 17 (UDP) dst port 80 or UDP Fragments (dst port 0..) What valid application actually uses UDP 80? The Cisco NAC client for Macs, for the purpose of VLAN change detection, sends UDP/80 packets to the host's reversed default gateway (i.e., if the actual gateway is 1.2.3.4, it sends the packets to 4.3.2.1) once every five seconds. mc
RE: Over a decade of DDOS--any progress yet?
Soon several providers will begin offering dedicated servers with a 10Gbps connection to a single machine. -Drew Several already do. -Randy
Re: Over a decade of DDOS--any progress yet?
On 8 Dec 2010, at 15:40, Dobbins, Roland wrote: On Dec 8, 2010, at 10:36 PM, Thomas Mangin wrote: If you are a smaller network, you need the filtering to be performed by your transit provider, as your uplink will otherwise be congested. Actually, most DDoS attacks aren't link-flooding attacks - this hasn't been true for the last ~7 years or so. I'm not saying it doesn't happen, because it does, and sometimes quite spectacularly - but in most cases, the attackers don't have to flood the link to achieve their desired goal. Fair point. I never had to face any intelligent type of DDOS ... lucky me :) Thomas
Re: Over a decade of DDOS--any progress yet?
On Wed, Dec 8, 2010 at 8:47 AM, Jay Coley j...@prolexic.com wrote: On 08/12/2010 16:14, Drew Weaver wrote: I would say that 99% of the attacks that we see are 'link fillers' with 1% being an application attack. thanks, -Drew This has been our recent experience as well. There are some pure app attacks, to be sure, but we many blended attacks also. Bandwidth (UDP/ICMP/SYN Flood) attack to distract with a app attack (GET/PUSH floods) attempting to run underneath the radar. We regularly see SYN floods these days 20 Gb/s. Another thing to be aware of--when you get hit with what seems to be a simple flooding attack aimed at one point of your infrastructure... start checking your logs at _other_ places in your network very, VERY carefully. There seems to be a trend of using larger-scale flooding, or other simple types of attacks to get all the network people at an organization rushing over to throw resources and energy at it...while the real target of the attack is something completely different, on a different subnet, in a different part of the company; and that attack is small, carefully focused at its target, and is designed to be relatively quiet. The big attack is used simply to ensure all the human energy is focused on the wrong place, increasing the chance that what otherwise might caused raised eyebrows and double-checking of logs/IDS alerts, etc. gets missed while everyone is focusing on thebig attack. The thing to bear in mind is that app attacks *are* difficult to detect as they are low bandwidth and make a full TCP connection. As a result many IDS/Firewalls etc regularly miss these attacks. Lastly there is usually always someone at the other end of these attacks watching what is working and what is not. If the attack doesn't work they will simply round up more bots to increase the attack bandwidth or change the attack vector. And, in what seems to be an increasing trend, what they are watching for is *not* necessarily the result of the large botnet attack; they're checking on the results of their targeted probes elsewhere in the network, or on the outbound set of connections from a compromised machine within an organization; after all, during a huge DDoS attack, with everyone focusing on a set of uplinks being flooded with _inbound_ traffic, who is going to notice the (relatively smaller) outbound spike of traffic as the compromised machine sends out a copy of your internal intellectual property to the miscreant recipients? Matt (speaking purely hypothetically, of course, and definitely not on behalf of any institution or entity other than myself)
Re: Over a decade of DDOS--any progress yet?
On Dec 9, 2010, at 1:34 AM, Matthew Petach wrote: There seems to be a trend of using larger-scale flooding, or other simple types of attacks to get all the network people at an organization rushing over to throw resources and energy at it. Concur, the more serious attackers use diversionary attacks or 'demonstrations' like this from time to time, absolutely. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 9:33 AM, Arturo Servin wrote: Yes, but all of them rely on your upstreams or in mirroring your content. If 100 Mbps are reaching your input interface of 10Mbps there is not much that you can do. Hmm. What would be really cool is if you could use Snort, NetFlow/NBAR, or some other sort of DPI tech to find specifically the IP addresses of the DDoS bots, and then pass that information back upstream via BGP communities that tell your peer router to drop traffic from those addresses. That way the target of the traffic can continue to function if the DDoS traffic doesn't closely mimic the normal traffic. Your BGP peer router would need to have lots of memory for /32 or /64 routes though. Anyone heard of such a beast? Or is this how the stuff from places like Arbor Networks do their thing? --Chris
Re: Over a decade of DDOS--any progress yet?
On Dec 9, 2010, at 2:19 AM, Chris Boyd wrote: Your BGP peer router would need to have lots of memory for /32 or /64 routes though. Any modern router can handle this. Anyone heard of such a beast? Or is this how the stuff from places like Arbor Networks do their thing? This can be done with open-source tools or with some commercial tools. [Full disclosure - I work for a vendor which produces commercial tools in this category.] --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On 12/8/2010 08:06, Jack Bates wrote: I call BS. Windows has it's problems, but it is the most common exploited as it holds the largest market share. Many Windows infections I've seen occur not due to the OS, but due to lack of patching of applications on the OS. The system does as much as it can. And end users clicking/running every shiny thing they come across, consequences be damned. ~Seth
Re: Over a decade of DDOS--any progress yet?
On Wed, 08 Dec 2010 07:43:52 PST, JC Dill said: Why isn't ANYONE going after Microsoft over this? If Microsoft were held accountable for the spam and DDOSs that spew from their crappy software, they would find a way to stop the problem. I've raised this issue before, IMHO Windows OSs are attractive nuisances and that legal argument can be used to hold Microsoft responsible for not putting an adequate fence around their attractive nuisance. Unfortunately, this is one you really don't want to do. Microsoft's current offerings are about as hardened as the competition (Apple and Linux, mostly) right out of the box. And it's not clear that you can *make* a system much harder and still sell it to consumers (try using a Linux box with SELinux turned on in full MLS/MCS mode - quite secure, but *not* the easiest thing in the world to admin, especially if you ever add a third-party program that doesn't have a suitable MLS security policy description already). If all the big ISPs banded together to file suit against Microsoft, they could share the cost (and pain) of the lawsuit. And if you win the lawsuit, what does that get you? Microsoft goes broke, quits shipping security updates to everybody - and things are even worse than before, because now *everybody* is unpatched. The second issue is that if you *do* establish a legal precident that software vendors are liable for faults no matter what the contract/EULA says, you're going to see pretty much all the open-source projects pack up and go home unless they find a way to protect themselves. Quite likely some commercial software vendors will bail as well, or charge a *lot* more for their stuff. Be careful what you ask for, for you may surely get it. pgpDzJOyNsZnR.pgp Description: PGP signature
Re: Over a decade of DDOS--any progress yet?
On 08/12/10 1:38 PM, valdis.kletni...@vt.edu wrote: The second issue is that if you *do* establish a legal precident that software vendors are liable for faults no matter what the contract/EULA says, It doesn't matter what contract an auto maker makes with someone who purchases the car, if the brakes fail and the car hits ME, I can sue the auto maker due to the defective brakes. If they design the car in a way that a 3rd party can easily tamper with the brakes, and then the car hits me, I can also sue the auto maker. They are legally required to take due care in how they design the car to ensure that innocent bystanders aren't injured or killed by a design defect. IMHO, there's no difference in the core responsibility that software makers should be held to, to ensure that their software isn't easily compromised and used to attack and injure 3rd parties. The EULA is a red herring, as it only applies to the purchaser (who agrees to the EULA when they purchase the computer or software), not to 3rd parties who are injured. If the software doesn't work as designed and the purchaser is unhappy, that's between them and the company they bought the software from. But when it injures a 3rd party, that's a whole different ball game. I truly don't understand why ISP's (who bear the brunt of the burden of the fall-out from the compromised software, as they fight spam and have to provide customer support to users who complain that the internet is slow etc.) haven't said ENOUGH. jc
Re: Over a decade of DDOS--any progress yet?
On Dec 7, 2010, at 11:26 PM, Sean Donelan wrote: On Mon, 6 Dec 2010, Patrick W. Gilmore wrote: But as you and others have pointed out, not a lot of defense against DDoS these days besides horsepower and anycast. :-) Not just anycast. I said distributed architecture. There are more ways to distribute than anycast. The content-side can be duplicated, replicated, distributed. On the eyeball-side its not as easy to replicate things. DDOS against user networks doesn't generate as much publicity, outside of the gammer world, but is also a problem. Other than trying to hide your real address, what can be done to prevent DDOS in the first place. Don't piss people off on IRC? :) -- TTFN, patrick
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 11:26 AM, Sean Donelan wrote: Other than trying to hide your real address, what can be done to prevent DDOS in the first place. DDoS is just a symptom. The problem is botnets. Preventing hosts from becoming bots in the first place and taking down existing botnets is the only way to actually *prevent* DDoS attacks. Note that prevention is distinct from *defending* oneself against DDoS attacks. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
Botnets are the symptom. The real problem is people. Adrian On Wed, Dec 08, 2010, Dobbins, Roland wrote: On Dec 8, 2010, at 11:26 AM, Sean Donelan wrote: Other than trying to hide your real address, what can be done to prevent DDOS in the first place. DDoS is just a symptom. The problem is botnets. Preventing hosts from becoming bots in the first place and taking down existing botnets is the only way to actually *prevent* DDoS attacks. Note that prevention is distinct from *defending* oneself against DDoS attacks. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar. -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $24/pm+GST entry-level VPSes w/ capped bandwidth charges available in WA -
Re: Over a decade of DDOS--any progress yet?
On Dec 8, 2010, at 11:52 AM, Adrian Chadd wrote: The real problem is people. Well, yes - but short of mass bombardment, eliminating people doesn't scale very well, and is generally frowned upon. ; --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On Wed, Dec 08, 2010, Dobbins, Roland wrote: On Dec 8, 2010, at 11:52 AM, Adrian Chadd wrote: The real problem is people. Well, yes - but short of mass bombardment, eliminating people doesn't scale very well, and is generally frowned upon. ; I think history can conclusively state we're much, much better at eliminating people then we are hacked boxes; that politicians seem much happier somehow about the former than the latter; and our collective clue at being able to do so is growing much faster than our electronic toolkits. :-) (Oh god. :-) Adrian
Re: Over a decade of DDOS--any progress yet?
On Mon, Dec 6, 2010 at 1:50 AM, Sean Donelan s...@donelan.com wrote: February 2000 weren't the first DDOS attacks, but the attacks on multiple Other than buying lots of bandwidth and scrubber boxes, have any other DDOS attack vectors been stopped or rendered useless during the last decade? Very little, no, and no. Not counting occasional application bugs that are quickly fixed. Even TCP weaknesses that can facilitate attack are still present in the protocol. New vectors and variations of those old vectors emerged since the 1990s. So there is an increase in the number of attack vectors to be concerned about, not a reduction. SYN and Smurf are Swords and spears after someone came up with atomic weaponry. The atomic weaponry named bot net. Which is why there is less concern about the former types of single-real-origin-spoofed-source attacks. Botnet-based DDoS is just Smurf where amplification nodes are obtained by system compromise, instead of router misconfiguration, and a minor variation on the theme where the chain reaction is not started by sending spoofed ICMP ECHOs. Since 2005 there are new beasts such as Slowloris and DNS Reflection. DNS Reflection attacks are a more direct successor to smurf; true smurf broadcast amplification points are rare today, diminishing returns for the attacker, trying to find the 5 or 6 misconfigured gateways out there, but that doesn't diminish the vector of spoofed small request large response attacks. Open DNS servers are everywhere. SYN attacks traditionally come from a small number of sources and rely on spoofing to attack limitations on available number of connection slots for success. New vectors that became most well-known in the late 90s utilize botnets, and an attacker can make full connections therefore requiring zero spoofing, negating the benefit of SYN cookies. In other words, SYN floods got supplanted by TCP_Connect floods. -- -JH
Re: Over a decade of DDOS--any progress yet?
On Mon, Dec 6, 2010 at 01:50, Sean Donelan s...@donelan.com wrote: February 2000 weren't the first DDOS attacks, but the attacks on multiple well-known sites did raise DDOS' visibility. What progress has been made during the last decade at stopping DDOS attacks? SMURF attacks creating a DDOS from directed broadcast replies seems to have been mostly mitigated by changing defaults in major router OS's. TCP SYN attacks creating a DDOS from leaving many half-open connections seems to have been mostly mitigated with SYN Cookies or similar OS changes. Other than buying lots of bandwidth and scrubber boxes, have any other DDOS attack vectors been stopped or rendered useless during the last decade? Spoofing? Bots? Protocol quirks? If anything, the potential is worse now than it ever has been unless you have just ridiculous amounts of bandwidth, as the ratios between leaf user connectivity and data center drops have continued to close. The finger of packety death may be rare, but it is more powerful than ever, just ask Wikileaks, I believe that they were subject to 10Gbit+ at times. At least the frequency has dropped in recent years, if not the amplitude, and I am thankful for that, due to in no small part to what you list above, as it mostly requires compromised bots to preform major attacks now, instead of having many available unwitting non-compromised assists spread across the internet like previously.
Re: Over a decade of DDOS--any progress yet?
Besides having *alot* of bandwidth theres not really much you can do to mitigate. Once you have the bandwidth you can filter (w/good hardware). Even if you go for 802.3ba with 40/100 Gbps...you'll need alot of pipes. Spoofed attacks have reduced significally probably because the use of RPF. However we still see these from time to time. TCP SYN attacks are still quite frequent...these can push alot of pps at times. The attack vectors have changed. Years ago people used hacked *nix boxes with big pipes to start their attacks as only these had enough bandwidth. Nowadays the consumers have alot more bandwidth and its easier than ever to setup your own botnet by infecting users with malware and alike. Even tho end users usually have less than 2mbps upstream the pure amount of infected users makes it worse than ever. Most of the time (depending on the attack) its also hard to differentiate which IP addresse are attacking and which are legitimate users. I do not see a real solution to this problem right now...theres not much you can do about the unwilligness of users to keep their software/OS up2date and deploy anti-virus/anti-malware software (and keep it up2date). Some approaches have been made like cutting of internet access for users which have been identified by ISPs for beeing member of some botnet/beeing infected. This might be the only long-term solution to this probably. There is just no patch for human stupidity. Am Montag, den 06.12.2010, 02:50 -0500 schrieb Sean Donelan: February 2000 weren't the first DDOS attacks, but the attacks on multiple well-known sites did raise DDOS' visibility. What progress has been made during the last decade at stopping DDOS attacks? SMURF attacks creating a DDOS from directed broadcast replies seems to have been mostly mitigated by changing defaults in major router OS's. TCP SYN attacks creating a DDOS from leaving many half-open connections seems to have been mostly mitigated with SYN Cookies or similar OS changes. Other than buying lots of bandwidth and scrubber boxes, have any other DDOS attack vectors been stopped or rendered useless during the last decade? Spoofing? Bots? Protocol quirks? signature.asc Description: This is a digitally signed message part
Re: Over a decade of DDOS--any progress yet?
On Dec 6, 2010, at 2:50 PM, Sean Donelan wrote: Other than buying lots of bandwidth and scrubber boxes, have any other DDOS attack vectors been stopped or rendered useless during the last decade? These .pdf presos pretty much express my view of the situation, though I do need to rev the first one: https://files.me.com/roland.dobbins/y4ykq0 https://files.me.com/roland.dobbins/k54qkv https://files.me.com/roland.dobbins/j0a4sk The bottom line is that there are BCPs that help, but which many folks don't seem to deploy, and then there's little or no thought at all given to maintaining availability when it comes to server/service/app architecture and operations, except by the major players who'd been through the wringer and invest the time and resources to increase their resilience to attack. Of course, the fundamental flaws in the quarter-century old protocol stack we're running, with all the same problems plus new ones carried over into IPv6, are still there. Couple that with the brittleness, fragility, and insecurity of the DNS BGP, and the fact that the miscreants have near-infinite resources at their disposal, and the picture isn't pretty. And nowadays, the attackers are even more organized and highly motivated (OC, financial/ideological) and therefore more highly incentivized to innovate, the tools are easy enough for most anyone to make use of them, and tthe services/apps they attack are now of real importance to ordinary people. So, while the state of the art in defense has improved, the state of the art and resources available to the attackers have also dramatically improved, and the overall level of indifference to the importance of maintaining availability is unchanged - so the overall situation itself is considerably worse, IMHO. The only saving grace is that the bad guys often make so much money via identity theft, click-fraud, spam, and corporate/arm's-length governmental espionage that they'd rather keep the networks/services/servers/apps/endpoints up and running so that they can continue to monetize them in other ways. --- Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com Sell your computer and buy a guitar.
Re: Over a decade of DDOS--any progress yet?
On Mon, Dec 6, 2010 at 6:10 AM, Patrick W. Gilmore patr...@ianai.net wrote: On Dec 6, 2010, at 4:07 AM, Jonas Frey (Probe Networks) wrote: Besides having *alot* of bandwidth theres not really much you can do to mitigate. Once you have the bandwidth you can filter (w/good hardware). Even if you go for 802.3ba with 40/100 Gbps...you'll need alot of pipes. There is a variation on that theme. Using a distributed architecture (anycast, CDN, whatever), you can limit the attack to certain nodes. If you have 20 nodes and get attacked from a botnet China, only the users on the same node as the Chinese use will be down. The other 95% of your users will be fine. This is true even if you have 1 Gbps per node, and the attack is 100 Gbps strong. I think this is only true if you run your BGP session on a different path (or have your provider pin down a static route). If you are using BGP and run it on the same path, the 100Gbps will cause massive packet loss and likely cause your BGP session to drop which will just move the attack to another site, rinse / repeat. I don't think very many people run BGP over a separate circuit, but for some folks, it might be appropriate. I also recommend folks anycast with a /22 or /23 and then use BGP for the /23 or /24 announcements and have their provider pin down the /22 at a few sites so that if all hell breaks loose and the /23 or /24 is flapping and being dampened then you still have reachability with the covering prefix. It also lets you harden and strengthen a few smaller sites that have the /22 statically pinned down. I'm not sure if people think the cost of doing this is worth it, jury still out for us. But as you and others have pointed out, not a lot of defense against DDoS these days besides horsepower and anycast. :-) -David
Re: Over a decade of DDOS--any progress yet?
On Dec 6, 2010, at 10:34 AM, David Ulevitch da...@ulevitch.com wrote: On Mon, Dec 6, 2010 at 6:10 AM, Patrick W. Gilmore patr...@ianai.net wrote: On Dec 6, 2010, at 4:07 AM, Jonas Frey (Probe Networks) wrote: Besides having *alot* of bandwidth theres not really much you can do to mitigate. Once you have the bandwidth you can filter (w/good hardware). Even if you go for 802.3ba with 40/100 Gbps...you'll need alot of pipes. There is a variation on that theme. Using a distributed architecture (anycast, CDN, whatever), you can limit the attack to certain nodes. If you have 20 nodes and get attacked from a botnet China, only the users on the same node as the Chinese use will be down. The other 95% of your users will be fine. This is true even if you have 1 Gbps per node, and the attack is 100 Gbps strong. I think this is only true if you run your BGP session on a different path (or have your provider pin down a static route). You are assuming many things - such as the fact bgp is used at all. But yes, of course you have to ensure the attack traffic does not move when you get attacked or you end up with a domino effect that takes out your entire infrastructure. But as you and others have pointed out, not a lot of defense against DDoS these days besides horsepower and anycast. :-) Not just anycast. I said distributed architecture. There are more ways to distribute than anycast. Not everything is limited to 13 IP addresses at the GTLDs, David. :-) -- TTFN, patrick