Hi Chad, I am sorry but that doesn't even help in the slightest.
You are essentially saying that we shouldn't develop on the App Engine, since would now have to also buy a proxy. Which is completely unfeasible and defeats the purpose of why people are using the app engine. I understand that this might also be an App Engine issue - for instance they could have reduced the number of IP addresses they pool from to make external requests. This is a very noticeable change in rate limiting in the last few weeks. For instance I could run roughly 2 searches a second, then all of a sudden I would be lucky to run 2 every 15 seconds. User-Agent strings were supposed to allievate this issue. There are more than enough pieces of meta data on an App Engine request that Identify the exact application that is making the requests - I guess it is too much effort to take these into account. I am in the fortunate position that allowed me to set up a nginx proxy quickly, but I suspect a lot of other people couldn't do that. I hope something can be sorted for the large number of GAE based Twitter apps. Paul Kinlan On 6 Oct 2009, at 17:50, Chad Etzel <c...@twitter.com> wrote: > > Hi All, > > GAE sites are problematic for the Twitter/Search API because the IPs > making outgoing requests are fluid and cannot as such be easily > allowed for access. Also, since most IPs are shared, other > applications on the same IPs making requests mean that fewer requests > per app get through. > > One work around would be to spin up a server in EC2 or Rackspace Cloud > or something and use it as a proxy for your requests. That way you > have a dedicated IP that will have its full share of resources talking > with the Twitter servers. > > HTH, > -Chad > > On Tue, Oct 6, 2009 at 12:45 PM, Martin Omander > <moman...@google.com> wrote: >> >> >> Same here; my app runs on Google App Engine and 40% of the requests >> to >> the Twitter Search API get the 503 error message indicating rate >> limiting. >> >> Is there anything we as app authors can do on our side to alleviate >> the problem? >> >> /Martin >> >> >> On Oct 5, 1:53 pm, Paul Kinlan <paul.kin...@gmail.com> wrote: >>> I am pretty sure there are custom headers on the App Engine that >>> indicate >>> the application that is sending the request. >>> >>> 2009/10/5 elkelk <danielshaneup...@gmail.com> >>> >>> >>> >>> >>> >>>> Hi all, >>> >>>> I am having the same issue. I have tried setting a custom user- >>>> agent, >>>> but this doesn't seem to affect the fact that twitter is limiting >>>> based on I.P. address. I'm only making about 5 searches an hour >>>> and >>>> 80% of them are failing on app engine due to a 503 rate limit. >>>> Twitter needs to determine a better way to let cloud clients access >>>> their search API. It seems like they have really started blocking >>>> search requests in the last week or so. >>> >>>> If anyone has any idea about how to better identify my app engine >>>> app >>>> please let let me know. >>> >>>> On Oct 5, 2:59 am, steel <steel...@gmail.com> wrote: >>>>> Hi. I have this problem too. >>>>> My application does two request per hour and it get "rate limit". >>>>> What is wrong? I think it is twitter's problems.... >>> >>>>> On 1 окт, 01:45, Paul Kinlan <paul.kin...@gmail.com> wrote: >>> >>>>>> Hi Guys, >>>>>> I have an app on the App engine using the search API and it is >>>>>> getting >>>>>> heavily rate limited again this past couple of days. >>> >>>>>> I know that we are on a shared set of IP addresses and someone >>>>>> else >>>> could be >>>>>> hammering the system, but it seems to run for weeks without >>>>>> seeing the >>>> rate >>>>>> limit being hit and then all of a sudden only about 60% of the >>>>>> searches >>>>>> I perform will be rate limited. This seems to occur every two >>>>>> months >>>> or so. >>> >>>>>> Has something changed recently? >>> >>>>>> Paul >>
