As many of you have since learned, we deployed an unannounced security fix that has resulted in unexpected failures for quite a few developers using OAuth. A developer reported to us that OAuth signatures were not being verified on our side. The fix was implemented and pushed on Sunday then deployed yesterday. Once the fix was in the wild many applications started returning Invalid Signature errors. Our various successful tests seem to indicate that the signature verification is implemented correctly and yet many people are experiencing errors. So what's going on?
One of the main problems seems to be that many OAuth libraries appear to not be generating correctly signed requests. It's likely that Twitter's implementation was used to test out many of these libraries when they were being implemented. Without the correct signature check, it appeared to these developers that their libraries were implementing the spec correctly. For this confusion we must take at least partial blame. The following email from Simon in the UK seems to indicate absent url encoding and decoding is a likely culprit in many of these libraries: "I don't myself think that twitter are doing anything more than enforcing the standard. I spent 3 hours 'fixing' my code for our application (uses Shannon Whitley's c# library as a base); I only found two bugs in thelibrary that caused any problem... the use of httputility.urlencode in place of the modified urlencode already implemented for use with Oauth, as required by the spec... the library just wasn't using it in two places. Following that, I found I was passing some bad strings to the library... so the methods in it were not urldecoding and re- urlencoding the postdata as required to get it to meet the spec. Once I fixed that also, all seems good. So if our app now posts all data fine over Oauth, it suggests that twitter's interface is OK?" There are at least several things we could have done better in dealing with this: * We should have, it goes without saying, had more extensive test coverage of our implementation ensuring that we were fully implementing the spec so that the whole situation would have been avoided in the first place. * We should have had an email prepared to send out immediately following the deploy explaining the vulnerability and the change that was deployed, encouraging developers to double check that their signatures were in fact being generated correctly. We left a lot of people guessing for half a day, in many cases probably frantically trying to fix mysterious failures in their apps. For that we must admit guilt. We hadn't anticipated that so many libraries might have not been generating signatures correctly. As a result it didn't occur to us to urgently send out details, assuming fully implemented libraries would continue to work transparently and malicious requests would start to fail. We had been focusing our efforts first and foremost on getting the fix implemented and deployed to protect everyone. Lesson learned. We'll take this experience with us and bring it to bear next time such a situation arises. We're going to do a post-mortem on our side to identify all the things we should have done better. We've read all of your feedback about how this could have been done better. To everyone who has chimed into this thread offering details and help, we extend our thanks. On Tue, Jul 28, 2009 at 10:28 AM, Abraham Williams <4bra...@gmail.com>wrote: > If you are encoding properly according to: > http://oauth.net/core/1.0a#encoding_parameters and it still fails to > update post and update to > http://code.google.com/p/twitter-api/issues/entry and make Twitter fix it. > > I've not double checked to verify but "!" should encode to "%21" and then > it should work. > > Abraham > > > On Tue, Jul 28, 2009 at 10:06, Duane Roelands <duane.roela...@gmail.com>wrote: > >> >> Yeah, that's what I'm doing as well. >> >> I wish Twitter would give us some answers. >> >> I can post a tweet is the text is "test" >> If I try to post "test!", it fails. Something about the encoding of >> non-alphanumeric characters is part of the problem. But, because >> Twitter isn't talking, all we can do is guess. >> >> On Jul 28, 12:52 pm, Cristovão Morgado <cristovao.morg...@gmail.com> >> wrote: >> > I use this implementation: >> http://www.codingthewheel.com/archives/codingthetweet >> > >> > works flawlessly! >> > >> > On Tue, Jul 28, 2009 at 5:47 PM, Duane Roelands < >> duane.roela...@gmail.com>wrote: >> > >> > >> > >> > > My stuff is based on Shannon Whitley's as well. Do you mind sharing >> > > where specifically you had to make the changes? >> > >> > > On Jul 28, 11:40 am, Zaudio <si...@z-audio.co.uk> wrote: >> > > > I don't myself think that twitter are doing anything more than >> > > > enforcing the standard. I spent 3 hours 'fixing' my code for our >> > > > application (uses Shannon Whitley's c# library as a base); I only >> > > > found two bugs in thelibrary that caused any problem... the use of >> > > > httputility.urlencode in place of the modified urlencode already >> > > > implemented for use with Oauth, as required by the spec... the >> library >> > > > just wasn't using it in two places. >> > >> > -- >> > Cristovao Morgado >> > aka Saintrhttp://www.oMeuJogoUsado.comhttp://www.TweetaPorSMS.comhttp:// >> twitter.com/TheSaintr >> > > > > -- > Abraham Williams | Community Evangelist | http://web608.org > Hacker | http://abrah.am | http://twitter.com/abraham > Project | http://fireeagle.labs.poseurtech.com > This email is: [ ] blogable [x] ask first [ ] private. > -- Marcel Molina Twitter Platform Team http://twitter.com/noradio
