[twitter-dev] Re: 302s are NOT the solution
With a DOS attack, you are sending requests to the server in order to tie up resources. The reason for the sequence numbers in TCP is order to make sure that data can flow to and from client and service. With HTTP, a TCP packet is sent, and a response is sent. However for a DOS you don't need the return packets. The request is made, the server processes the request, and the response is generated. It's the server processing you want done, and even waiting for a response is time wasted for an attack. The DOS attack will send request, drop connection, send request, drop connection... Generally you target something that you know will eat up at least some resources on the server. By moving those requests to send a 302, you cut down the amount of processing for false requests using a spoof IP. The service only expends enough resources to generate the 302, and because the IP is either spoofed, or not checking the response, the request to the resource which generates more load never happens. Hope that makes sense. On Aug 9, 8:30 pm, Kyle Mulka wrote: > From Wikipedia: > "Some upper layer protocols provide their own defense against IP > spoofing. For example, Transmission Control Protocol (TCP) uses > sequence numbers negotiated with the remote machine to ensure that > arriving packets are part of an established connection. Since the > attacker normally can't see any reply packets, he has to guess the > sequence number in order to hijack the connection. The poor > implementation in many older operating systems and network devices, > however, means that TCP sequence numbers can be predicted." > > This seems to say that TCP could be used instead of HTTP 302s. Is > there something I'm missing for why 302s are necessary here? > > -- > Kyle Mulkahttp://twilk.com > > On Aug 8, 10:45 pm, John Kalucki wrote: > > > In a simplified sense, the redirect nullifies a pernicious class of > > attack where the source IP address is forged. A redirect cannot be > > followed with a false source address. The attacks that remain are > > those where the source IP address is valid. You can then imagine other > > techniques that than can be applied against valid IP addresses. And so > > the problem is divided and ameliorated, but never fully solved. > > > I'm going to push back for a second with some food for thought for > > developers: The API is via HTTP. HTTP is a well defined protocol. 302 > > redirects are a valid and well worn part of the HTTP protocol. > > Consider why applications are not built using fully HTTP compliant > > libraries. This doesn't address all the problems that we're all > > having, but it does address some. > > > -John Kaluckihttp://twitter.com/jkalucki > > Services, Twitter Inc. > > > On Aug 8, 8:53 am, Kyle Mulka wrote: > > > > An attacker can just as easily follow a 302 as can a legitimate API > > > developer or user of Twitter. I don't understand why Twitter thinks > > > this is a solution to the problem. Please stop 302ing. > > > > Thanks, > > > > -- > > > Kyle Mulkahttp://twilk.com
[twitter-dev] Re: 302s are NOT the solution
TCP/IP is the protocol underneath HTTP, is not a web service protocol and requires a whole different method to manage and use connections. Think of it as the raw data pipe by which the HTTP protocol is used to communicate between a client program (i.e. a web broswer) and the server program (i.e. a web server). It can not be used in the way that I seem to think you are intending it to be used. - h On Sun, Aug 9, 2009 at 17:30, Kyle Mulka wrote: > > From Wikipedia: > "Some upper layer protocols provide their own defense against IP > spoofing. For example, Transmission Control Protocol (TCP) uses > sequence numbers negotiated with the remote machine to ensure that > arriving packets are part of an established connection. Since the > attacker normally can't see any reply packets, he has to guess the > sequence number in order to hijack the connection. The poor > implementation in many older operating systems and network devices, > however, means that TCP sequence numbers can be predicted." > > This seems to say that TCP could be used instead of HTTP 302s. Is > there something I'm missing for why 302s are necessary here? > > -- > Kyle Mulka > http://twilk.com > > On Aug 8, 10:45 pm, John Kalucki wrote: > > In a simplified sense, the redirect nullifies a pernicious class of > > attack where the source IP address is forged. A redirect cannot be > > followed with a false source address. The attacks that remain are > > those where the source IP address is valid. You can then imagine other > > techniques that than can be applied against valid IP addresses. And so > > the problem is divided and ameliorated, but never fully solved. > > > > I'm going to push back for a second with some food for thought for > > developers: The API is via HTTP. HTTP is a well defined protocol. 302 > > redirects are a valid and well worn part of the HTTP protocol. > > Consider why applications are not built using fully HTTP compliant > > libraries. This doesn't address all the problems that we're all > > having, but it does address some. > > > > -John Kaluckihttp://twitter.com/jkalucki > > Services, Twitter Inc. > > > > On Aug 8, 8:53 am, Kyle Mulka wrote: > > > > > An attacker can just as easily follow a 302 as can a legitimate API > > > developer or user of Twitter. I don't understand why Twitter thinks > > > this is a solution to the problem. Please stop 302ing. > > > > > Thanks, > > > > > -- > > > Kyle Mulkahttp://twilk.com >
[twitter-dev] Re: 302s are NOT the solution
>From Wikipedia: "Some upper layer protocols provide their own defense against IP spoofing. For example, Transmission Control Protocol (TCP) uses sequence numbers negotiated with the remote machine to ensure that arriving packets are part of an established connection. Since the attacker normally can't see any reply packets, he has to guess the sequence number in order to hijack the connection. The poor implementation in many older operating systems and network devices, however, means that TCP sequence numbers can be predicted." This seems to say that TCP could be used instead of HTTP 302s. Is there something I'm missing for why 302s are necessary here? -- Kyle Mulka http://twilk.com On Aug 8, 10:45 pm, John Kalucki wrote: > In a simplified sense, the redirect nullifies a pernicious class of > attack where the source IP address is forged. A redirect cannot be > followed with a false source address. The attacks that remain are > those where the source IP address is valid. You can then imagine other > techniques that than can be applied against valid IP addresses. And so > the problem is divided and ameliorated, but never fully solved. > > I'm going to push back for a second with some food for thought for > developers: The API is via HTTP. HTTP is a well defined protocol. 302 > redirects are a valid and well worn part of the HTTP protocol. > Consider why applications are not built using fully HTTP compliant > libraries. This doesn't address all the problems that we're all > having, but it does address some. > > -John Kaluckihttp://twitter.com/jkalucki > Services, Twitter Inc. > > On Aug 8, 8:53 am, Kyle Mulka wrote: > > > An attacker can just as easily follow a 302 as can a legitimate API > > developer or user of Twitter. I don't understand why Twitter thinks > > this is a solution to the problem. Please stop 302ing. > > > Thanks, > > > -- > > Kyle Mulkahttp://twilk.com
[twitter-dev] Re: 302s are NOT the solution
On Sat, 8 Aug 2009 16:11:29 -0700 (PDT) Fawkes wrote: > They can, but apparently they don't, otherwise Twitter wouldn't have > used it as a tactic. They're going through a very difficult time, we > need to be patient and supportive of them! In order for an attacker to respond to a 302, they have to receive it. In order to receive it, they have to be giving their real IP address when they connect. If every legitimate app handled the 302 properly then those spoofing IP addresses would stand out very clearly. Even if attackers monitored the IP addresses that they were spoofing in order to try to escape detection by following the 302 responses, they still behave differently than legitimate users. The difference is that now security personel can identify repeated connections from the IP addresses that are being monitored by the attackers and tarpit those connections. Supplying a valid IP address from the attacking machines still slows down the attacker even if that IP address is not the actual point of origin for the attack. The moral of the story is "Apps that are good netizens contribute to the stability of the server." Chris Babcock
[twitter-dev] Re: 302s are NOT the solution
A forged source IP address is a good reason for doing 302s. Thanks for the explanation. Now... if only OAuth worked... -- Kyle Mulka http://twilk.com On Aug 8, 10:45 pm, John Kalucki wrote: > In a simplified sense, the redirect nullifies a pernicious class of > attack where the source IP address is forged. A redirect cannot be > followed with a false source address. The attacks that remain are > those where the source IP address is valid. You can then imagine other > techniques that than can be applied against valid IP addresses. And so > the problem is divided and ameliorated, but never fully solved. > > I'm going to push back for a second with some food for thought for > developers: The API is via HTTP. HTTP is a well defined protocol. 302 > redirects are a valid and well worn part of the HTTP protocol. > Consider why applications are not built using fully HTTP compliant > libraries. This doesn't address all the problems that we're all > having, but it does address some. > > -John Kaluckihttp://twitter.com/jkalucki > Services, Twitter Inc. > > On Aug 8, 8:53 am, Kyle Mulka wrote: > > > An attacker can just as easily follow a 302 as can a legitimate API > > developer or user of Twitter. I don't understand why Twitter thinks > > this is a solution to the problem. Please stop 302ing. > > > Thanks, > > > -- > > Kyle Mulkahttp://twilk.com
[twitter-dev] Re: 302s are NOT the solution
A secret key will help at application level. But the first defense in DOS is at network gear level where you cannot check secret keys against db tables. On Aug 9, 12:01 am, Scott Haneda wrote: > Can someone point me to the details on the attack? I am a little out > of the loop. I've heard Twitter only uses around 200Mbit/s of data. > From a net ops perspective, why is this challenging to detect and > block? > > I'm not trying to degrade the efforts of the engineers, this is a > genuine question of curiosity. > > I would imagine a detection system is in place, so why not block off > at the upstream the offending attack? > > As far as the API is concerned, I'm not sure I see why this can't be > prevented in the future. If every Twitter app had to get an API key, > which I believe is the case, those get whitelisted, all else are > blocked. > > Create a test sandbox for easy non key based testing of new developers > who want to play. There are a few thousand third party apps, whitelist > their secret keys and how is this not solved for API reliability? > -- > Scott > Iphone says hello. > > On Aug 8, 2009, at 5:09 PM, Howard Siegel wrote: > > > > > I support them wholeheartedly and appreciate everything they've done > > to thwart the DDOS attack. > > > While it is true that many of the tools used in the attack do not > > appear to follow the 302s right now, you can be your bottom dollar > > that they will very quickly be updated to do just that, perhaps even > > quicker than Twitter can finish recovering from the attack and put > > in to place measures to better survive future attacks. > > > At best it is a stopgap to get over the current attack.
[twitter-dev] Re: 302s are NOT the solution
Perhaps someone should set up a wiki page for this with basic info we can all collaborate on so we can know how to adapt to the new changes in our own language. I'm sure that's something we can all work together on. Does Twitter want to take the initiative to at least just start this so we can all continue the collaboration on where things stand in our own languages there? I'm sure that would save Twitter repeated answers on the mailing list. Jesse On Sat, Aug 8, 2009 at 11:01 PM, Scott Haneda wrote: > > Can someone point me to the details on the attack? I am a little out of the > loop. I've heard Twitter only uses around 200Mbit/s of data. From a net ops > perspective, why is this challenging to detect and block? > > I'm not trying to degrade the efforts of the engineers, this is a genuine > question of curiosity. > > I would imagine a detection system is in place, so why not block off at the > upstream the offending attack? > > As far as the API is concerned, I'm not sure I see why this can't be > prevented in the future. If every Twitter app had to get an API key, which I > believe is the case, those get whitelisted, all else are blocked. > > Create a test sandbox for easy non key based testing of new developers who > want to play. There are a few thousand third party apps, whitelist their > secret keys and how is this not solved for API reliability? > -- > Scott > Iphone says hello. > > > On Aug 8, 2009, at 5:09 PM, Howard Siegel wrote: > > I support them wholeheartedly and appreciate everything they've done to >> thwart the DDOS attack. >> >> While it is true that many of the tools used in the attack do not appear >> to follow the 302s right now, you can be your bottom dollar that they will >> very quickly be updated to do just that, perhaps even quicker than Twitter >> can finish recovering from the attack and put in to place measures to better >> survive future attacks. >> >> At best it is a stopgap to get over the current attack. >> >
[twitter-dev] Re: 302s are NOT the solution
Can someone point me to the details on the attack? I am a little out of the loop. I've heard Twitter only uses around 200Mbit/s of data. From a net ops perspective, why is this challenging to detect and block? I'm not trying to degrade the efforts of the engineers, this is a genuine question of curiosity. I would imagine a detection system is in place, so why not block off at the upstream the offending attack? As far as the API is concerned, I'm not sure I see why this can't be prevented in the future. If every Twitter app had to get an API key, which I believe is the case, those get whitelisted, all else are blocked. Create a test sandbox for easy non key based testing of new developers who want to play. There are a few thousand third party apps, whitelist their secret keys and how is this not solved for API reliability? -- Scott Iphone says hello. On Aug 8, 2009, at 5:09 PM, Howard Siegel wrote: I support them wholeheartedly and appreciate everything they've done to thwart the DDOS attack. While it is true that many of the tools used in the attack do not appear to follow the 302s right now, you can be your bottom dollar that they will very quickly be updated to do just that, perhaps even quicker than Twitter can finish recovering from the attack and put in to place measures to better survive future attacks. At best it is a stopgap to get over the current attack.
[twitter-dev] Re: 302s are NOT the solution
In a simplified sense, the redirect nullifies a pernicious class of attack where the source IP address is forged. A redirect cannot be followed with a false source address. The attacks that remain are those where the source IP address is valid. You can then imagine other techniques that than can be applied against valid IP addresses. And so the problem is divided and ameliorated, but never fully solved. I'm going to push back for a second with some food for thought for developers: The API is via HTTP. HTTP is a well defined protocol. 302 redirects are a valid and well worn part of the HTTP protocol. Consider why applications are not built using fully HTTP compliant libraries. This doesn't address all the problems that we're all having, but it does address some. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Aug 8, 8:53 am, Kyle Mulka wrote: > An attacker can just as easily follow a 302 as can a legitimate API > developer or user of Twitter. I don't understand why Twitter thinks > this is a solution to the problem. Please stop 302ing. > > Thanks, > > -- > Kyle Mulkahttp://twilk.com
[twitter-dev] Re: 302s are NOT the solution
Well I must be lucky then. :) I'll probably add redirect support into my library anyway. Shouldn't be too hard to implement. On Sat, Aug 8, 2009 at 9:25 PM, Chad Etzel wrote: > > No, they are not limited to only oauth related calls. > -Chad > > On Sat, Aug 8, 2009 at 10:22 PM, Josh Roesslein > wrote: > > Are the redirects only occuring with oauth? I've yet to run into them, > but > > I'm not really using oauth much so that might be why. > > > > On Sat, Aug 8, 2009 at 7:09 PM, Howard Siegel wrote: > >> > >> I support them wholeheartedly and appreciate everything they've done to > >> thwart the DDOS attack. > >> > >> While it is true that many of the tools used in the attack do not appear > >> to follow the 302s right now, you can be your bottom dollar that they > will > >> very quickly be updated to do just that, perhaps even quicker than > Twitter > >> can finish recovering from the attack and put in to place measures to > better > >> survive future attacks. > >> > >> At best it is a stopgap to get over the current attack. > >> > >> - h > >> > >> On Sat, Aug 8, 2009 at 16:11, Fawkes wrote: > >>> > >>> They can, but apparently they don't, otherwise Twitter wouldn't have > >>> used it as a tactic. They're going through a very difficult time, we > >>> need to be patient and supportive of them! > >>> > >>> Dave > >>> http://twitter.com/DavidHaber > >>> > >>> On Aug 8, 8:53 am, Kyle Mulka wrote: > >>> > An attacker can just as easily follow a 302 as can a legitimate API > >>> > developer or user of Twitter. I don't understand why Twitter thinks > >>> > this is a solution to the problem. Please stop 302ing. > >>> > > >>> > Thanks, > >>> > > >>> > -- > >>> > Kyle Mulkahttp://twilk.com > >> > > > > > > > > -- > > Josh > > > -- Josh
[twitter-dev] Re: 302s are NOT the solution
No, they are not limited to only oauth related calls. -Chad On Sat, Aug 8, 2009 at 10:22 PM, Josh Roesslein wrote: > Are the redirects only occuring with oauth? I've yet to run into them, but > I'm not really using oauth much so that might be why. > > On Sat, Aug 8, 2009 at 7:09 PM, Howard Siegel wrote: >> >> I support them wholeheartedly and appreciate everything they've done to >> thwart the DDOS attack. >> >> While it is true that many of the tools used in the attack do not appear >> to follow the 302s right now, you can be your bottom dollar that they will >> very quickly be updated to do just that, perhaps even quicker than Twitter >> can finish recovering from the attack and put in to place measures to better >> survive future attacks. >> >> At best it is a stopgap to get over the current attack. >> >> - h >> >> On Sat, Aug 8, 2009 at 16:11, Fawkes wrote: >>> >>> They can, but apparently they don't, otherwise Twitter wouldn't have >>> used it as a tactic. They're going through a very difficult time, we >>> need to be patient and supportive of them! >>> >>> Dave >>> http://twitter.com/DavidHaber >>> >>> On Aug 8, 8:53 am, Kyle Mulka wrote: >>> > An attacker can just as easily follow a 302 as can a legitimate API >>> > developer or user of Twitter. I don't understand why Twitter thinks >>> > this is a solution to the problem. Please stop 302ing. >>> > >>> > Thanks, >>> > >>> > -- >>> > Kyle Mulkahttp://twilk.com >> > > > > -- > Josh >
[twitter-dev] Re: 302s are NOT the solution
Are the redirects only occuring with oauth? I've yet to run into them, but I'm not really using oauth much so that might be why. On Sat, Aug 8, 2009 at 7:09 PM, Howard Siegel wrote: > I support them wholeheartedly and appreciate everything they've done to > thwart the DDOS attack. > > While it is true that many of the tools used in the attack do not appear to > follow the 302s right now, you can be your bottom dollar that they will very > quickly be updated to do just that, perhaps even quicker than Twitter can > finish recovering from the attack and put in to place measures to better > survive future attacks. > > At best it is a stopgap to get over the current attack. > > - h > > > On Sat, Aug 8, 2009 at 16:11, Fawkes wrote: > >> >> They can, but apparently they don't, otherwise Twitter wouldn't have >> used it as a tactic. They're going through a very difficult time, we >> need to be patient and supportive of them! >> >> Dave >> http://twitter.com/DavidHaber >> >> On Aug 8, 8:53 am, Kyle Mulka wrote: >> > An attacker can just as easily follow a 302 as can a legitimate API >> > developer or user of Twitter. I don't understand why Twitter thinks >> > this is a solution to the problem. Please stop 302ing. >> > >> > Thanks, >> > >> > -- >> > Kyle Mulkahttp://twilk.com >> > > -- Josh
[twitter-dev] Re: 302s are NOT the solution
I support them wholeheartedly and appreciate everything they've done to thwart the DDOS attack. While it is true that many of the tools used in the attack do not appear to follow the 302s right now, you can be your bottom dollar that they will very quickly be updated to do just that, perhaps even quicker than Twitter can finish recovering from the attack and put in to place measures to better survive future attacks. At best it is a stopgap to get over the current attack. - h On Sat, Aug 8, 2009 at 16:11, Fawkes wrote: > > They can, but apparently they don't, otherwise Twitter wouldn't have > used it as a tactic. They're going through a very difficult time, we > need to be patient and supportive of them! > > Dave > http://twitter.com/DavidHaber > > On Aug 8, 8:53 am, Kyle Mulka wrote: > > An attacker can just as easily follow a 302 as can a legitimate API > > developer or user of Twitter. I don't understand why Twitter thinks > > this is a solution to the problem. Please stop 302ing. > > > > Thanks, > > > > -- > > Kyle Mulkahttp://twilk.com >
[twitter-dev] Re: 302s are NOT the solution
They can, but apparently they don't, otherwise Twitter wouldn't have used it as a tactic. They're going through a very difficult time, we need to be patient and supportive of them! Dave http://twitter.com/DavidHaber On Aug 8, 8:53 am, Kyle Mulka wrote: > An attacker can just as easily follow a 302 as can a legitimate API > developer or user of Twitter. I don't understand why Twitter thinks > this is a solution to the problem. Please stop 302ing. > > Thanks, > > -- > Kyle Mulkahttp://twilk.com
