I just wanted to add to this. The 420s have let up for the most part and I'm no longer seeing rate limiting behavior significantly different from the norm.
I've noticed that many result pages are coming back with empty results but if I re-request the same page (after a couple second delay), I can often get results for that page. These are for queries with very low tweet velocity, so it's not like these are new results coming in. Is this related to http://status.twitter.com/post/3785043723/slow-searches ? Thanks. Hayes On Mon, Mar 21, 2011 at 6:13 PM, Matt Harris <thematthar...@twitter.com>wrote: > In many cases we are forced to change the rate limits in response to > a significant increase in requests, which means it isn't always possible to > give advanced notice of rate limit changes. > > For some of you it sounds like your code that handles rate limiting didn't > react appropriately. When receiving a 420 response we recommend you stop > making requests and then after the retry-after, slowly build up the number > of requests you make. Put another way it isn't a good idea to make requests > to the API at the velocity that caused the 420 response before. > > As always, the rate limits are there to ensure the system is responsive and > available to as many users as possible. This means it is necessary to reduce > the number of queries you can make without notice. > > The best place to stay informed about issues like this are posted through > @twitterapi and published on the Twitter status blog: > http://status.twitter.com/post/3785043723/slow-searches > > Best, > @themattharris > Developer Advocate, Twitter > http://twitter.com/themattharris > > > > On Mon, Mar 21, 2011 at 7:41 AM, Colin Surprenant < > colin.surpren...@gmail.com> wrote: > >> By adjusting the rate limits to reduce the stress on your search api >> without notice you have significantly increased the stress level on >> our end :P Seriously, advanced notice of the situation would have been >> welcome. >> >> In particular what created lots of confusion on our end is that even >> after pausing for the specified "retry_after" delay we would >> immediately get repeated 420s at which point we started to assume our >> IPs were banned (which also contributed to increase the stress level). >> >> Colin >> >> On Mar 21, 9:12 am, Jeffrey Greenberg <jeffreygreenb...@gmail.com> >> wrote: >> > Taylor, >> > Yeah this was definitely NOT good. In the past, when there is a >> > service disruption, your api group would post something on your status >> > page and tweet about it... Instead, I'm finding out about this from my >> > customers... >> > >> > Did y'all tweet about this or present this somewhere where I could find >> it? >> > >> > Jeffrey >> > Tweettronics.com >> > >> > On Sun, Mar 20, 2011 at 3:14 PM, Waldron Faulkner >> > >> > >> > >> > >> > >> > >> > >> > <waldronfaulk...@gmail.com> wrote: >> > > Without prior notice, I can understand (circumstances), but without >> > > any kind of subsequent announcement?? Means we have to discover issues >> > > ourselves, verify that they're Twitter related (and not internal), >> > > then search around for existing discussion on the topic. Saves us a >> > > lot of time and headaches if Twitter would just announce stuff like >> > > this. >> > >> > > On Mar 18, 2:51 pm, Taylor Singletary <taylorsinglet...@twitter.com> >> > > wrote: >> > >> We're working to reinstate the usual limits on the Search API; due to >> the >> > >> impact of the Japanese earthquake and resultant query increase >> against the >> > >> Search API, some rates were adjusted to cope & better serve queries. >> Will >> > >> give everyone an update with the various limits are adjusted. >> > >> > >> @episod <http://twitter.com/episod> - Taylor Singletary - Twitter >> Developer >> > >> Advocate >> > >> > >> On Fri, Mar 18, 2011 at 11:39 AM, Hayes Davis <ha...@appozite.com> >> wrote: >> > >> > Hi, >> > >> > >> > We're seeing this as well starting at approximately the same time >> as >> > >> > described. We've backed off on searching but are seeing no >> reduction in the >> > >> > sporadic limiting. It also appears that the amount of results >> returned on >> > >> > successful queries is severely limited. Some queries that often >> have 1500 >> > >> > tweets from the last 5 days are returning far fewer results from >> only the >> > >> > last day. >> > >> > >> > Could we get an update on this? >> > >> > >> > Hayes >> > >> > >> > On Fri, Mar 18, 2011 at 10:13 AM, Eric <e...@telvetto.com> wrote: >> > >> > >> >> We're also seeing 400s on different boxes across different IP >> > >> >> addresses with different queries (so it does not appear to be >> server >> > >> >> or query specific). These began on all boxes at 2 a.m. UTC. We've >> > >> >> backed off on both number and rate of queries with no effect. >> We've >> > >> >> also noticed an increase in sporadic fail whales via browser based >> > >> >> search (atom and html) from personal accounts, although we haven't >> > >> >> attempted to quantify it. >> > >> > >> >> On Mar 18, 7:40 am, zaver <zave...@hotmail.com> wrote: >> > >> >> > Hello, >> > >> > >> >> > After the latest performance issues with the search api i have >> been >> > >> >> > seeing a lot of 420 response codes.From yesterday until now i >> only get >> > >> >> > 420 responses on the every search i make. In particular, i >> search for >> > >> >> > about 100 keywords simultaneously every 6 mins. Why is this >> happening? >> > >> >> > Was there any change on the Search API limit? >> > >> > >> >> > Any help is greatly appreciated. >> > >> > >> >> > Thanks, >> > >> >> > Zaver >> > >> > >> >> -- >> > >> >> Twitter developer documentation and resources: >> http://dev.twitter.com/doc >> > >> >> API updates via Twitter:http://twitter.com/twitterapi >> > >> >> Issues/Enhancements Tracker: >> > >> >>http://code.google.com/p/twitter-api/issues/list >> > >> >> Change your membership to this group: >> > >> >>http://groups.google.com/group/twitter-development-talk >> > >> > >> > -- >> > >> > Twitter developer documentation and resources: >> http://dev.twitter.com/doc >> > >> > API updates via Twitter:http://twitter.com/twitterapi >> > >> > Issues/Enhancements Tracker: >> > >> >http://code.google.com/p/twitter-api/issues/list >> > >> > Change your membership to this group: >> > >> >http://groups.google.com/group/twitter-development-talk >> > >> > > -- >> > > Twitter developer documentation and resources: >> http://dev.twitter.com/doc >> > > API updates via Twitter:http://twitter.com/twitterapi >> > > Issues/Enhancements Tracker: >> http://code.google.com/p/twitter-api/issues/list >> > > Change your membership to this group: >> http://groups.google.com/group/twitter-development-talk >> >> -- >> Twitter developer documentation and resources: http://dev.twitter.com/doc >> API updates via Twitter: http://twitter.com/twitterapi >> Issues/Enhancements Tracker: >> http://code.google.com/p/twitter-api/issues/list >> Change your membership to this group: >> http://groups.google.com/group/twitter-development-talk >> > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk > -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
