Another data point: I also run a Python app on Google App Engine. It hits the Twitter search API 3 times per minute, with different search parameters. About 20% of my app's search requests get a 503L error code, and the other 80% return search results as expected. There is no clear pattern or grouping of these errors. Other calls to the Twitter API that are not search-related, like users/show, fail only about 0.1% of the time.
Anecdotal evidence: I seem to remember the success rate of my app's search requests was higher before the DDOS attacks two weeks ago. As I haven't saved the logs, I can't be sure. It would be better if the failure rate for searches were lower. But 20% failure is not a big deal for me as the searches run as cron jobs, which can always retry later. /Martin On Aug 23, 7:51 am, Andrew Badera <and...@badera.us> wrote: > 150 is a per-user rate, not a per-IP rate, to begin with, isn't it? > The issue here is whitelisted IPs sharing 20k total, right? > > ∞ Andy Badera > ∞ This email is: [ ] bloggable [x] ask first [ ] private > ∞ Google me:http://www.google.com/search?q=(andrew+badera)+OR+(andy+badera) > > > > On Sun, Aug 23, 2009 at 10:17 AM, boaz<sapirb...@gmail.com> wrote: > > > Hello everyone, > > > I checked the behavior on an AWS instance _without_ static IP (which > > is called by Amazon elastic IP) and I do not see any problem with the > > limits. The "limit status" shows that I have exactly 150 calls left > > minus the ones I have explicitly used. I do not obeserve any behvior > > where my limits are affected by other users with which I share the > > resource. > > Am I missing something? Could it be just a matter of luck/random > > behavior? > > > Thank you, > > Boaz > > > On Aug 22, 12:03 am, "Darren Bounds (Cliqset)" <dbou...@gmail.com> > > wrote: > >> Hello Chad, > > >> Can you confirm that this is not the case for AWS elastic IPs which > >> had been previously whitelisted by Twitter? > > >> Thanks, > >> Darren > > >> On Aug 21, 4:35 pm, Chad Etzel <jazzyc...@gmail.com> wrote: > > >> > Hello, > > >> > I have replied to Jud off-list, but for everyone's benefit we'd like > >> > to reiterate that AWS and GAE are shared resources and therefore share > >> > the rate limit across applications. A dedicated IP and unique UA will > >> > guarantee the maximum API limits. There are several cheap and reliable > >> > VPS hosting services available which can provide a dedicated IP > >> > address and full control over the server. > > >> > Thanks, > >> > -Chad > > >> > On Fri, Aug 21, 2009 at 12:48 PM, Jud<jvale...@gmail.com> wrote: > > >> > > I've got a python app running on Google App Engine (appspot hosted) > >> > > that querieshttp://search.twitter.com/search.atom?q=forsimple > >> > > queries (e.g. "foo OR bar"), and it's being severely throttled (e.g. > >> > > can't get a successful request through (response 200 w/ data) more > >> > > than a couple of times per _hour_). > > >> > > - I'm setting the UA string to something unique/identifiable (e.g. my > >> > > company name) > >> > > - I'm respecting the retry-after header coming back when I see a 503 > >> > > (average retry-after duration is ~750) > >> > > - GAE turns the IP address behind the app over ~ every 6 hours > >> > > - app hits tries to hit search.twitter.com every 5 minutes. > > >> > > I've successfully polled the endpoint at much higher rates (in > >> > > completely different IP address ranges) in the past, without issue. > >> > > Unclear what's going on. Any suggestions would be greatly appreciated.
