@Dewald Pretorius <<For my site, TweetLater.com, it would mean I have an effective hourly rate limit, per IP address, of 2 BILLION IP GET calls per hour!>>
I believe 20k limit per user is the desirable behavior, but i don't think twitter will allow you to make infinite calls in which case they will black list you. << I have not checked whether it is actually fixed. But, it's easy to check. Just do a GET call from a whitelisted IP with one user's credentials, check the remaining rate limit number, and then do the same call with another user's credentials. If each call gives you 19,999 remaining, then you know the bug still exists, and consequently no IP rate limiting is currently being done.>> You can verify this here http://twxlate.com This bug was closed very recently (about a month and a half ago) as working as intended http://code.google.com/p/twitter-api/issues/detail?id=617 <<That would be the same as having no rate limit at all, because really, which app would beed to make 20,000 GET calls per hour on one Twitter account?>> we dont know the rationale behind that number but if the limit is per IP then your app is easily susceptible to DOS attacks. I believe there are many apps (not whitelisted) out there which make more than 20k calls/hour (150 users /hour and 150 calls) The limit should always be per user(whether IP is whitelisted or not) On Thu, Aug 6, 2009 at 10:54 PM, Dewald Pretorius <[email protected]> wrote: > > Just some background. I talked with Doug about this a few months ago, > because I observed in the Rate Limit Header of get calls that the > 20,000 number decremented by user, not by IP address in aggregate. > > Doug informed me that he was going to hand the issue over to Matt, who > was on vacation at that point, to look into when he got back from > vacation. > > Doug specifically said that the intended behavior was for the 20,000 > rate limit to be by IP address only. > > So, the point I'm trying to make is, at one point the API did count > the 20,000 rate limit per IP address per user, but that was a bug that > should have been fixed. > > I have not checked whether it is actually fixed. But, it's easy to > check. Just do a GET call from a whitelisted IP with one user's > credentials, check the remaining rate limit number, and then do the > same call with another user's credentials. If each call gives you > 19,999 remaining, then you know the bug still exists, and consequently > no IP rate limiting is currently being done. > > Dewald > > On Aug 6, 2:04 pm, Chad Etzel <[email protected]> wrote: > > Hi Dewald, > > > > I asked "The Powers That Be" about it, and that was the response I > > got. However, I am double and triple checking because that does sound > > too good to be true :) > > > > -Chad > > > > On Thu, Aug 6, 2009 at 1:01 PM, Dewald Pretorius<[email protected]> > wrote: > > > > > Chad, > > > > > Are you 100% sure of that? > > > > > I mean, in terms of rate limiting that simply does not make sense. > > > > > For my site, TweetLater.com, it would mean I have an effective hourly > > > rate limit, per IP address, of 2 BILLION IP GET calls per hour! > > > (20,000 per user for 100,000 users). > > > > > It sounds wrong to me. > > > > > Dewald > > > > > On Aug 6, 1:37 pm, Chad Etzel <[email protected]> wrote: > > >> Hi Inspector Gadget, er... Bob, > > > > >> Yes, the current whitelisted IP rate-limit allows 20k calls per hour > > >> *per user* on Basic Auth or OAuth or a combination thereof. > > > > >> Go, go gadget data! > > > > >> -Chad > > >> Twitter Platform Support > > > > >> On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishel<[email protected]> > wrote: > > > > >> > Well it seems as though Twitter is saying that 20k calls per user is > > >> > the intended functionality. Chad or someone else can you confirm > this? > > > > >> > Also if the correct functionality is 20k per ip per hour will you > then > > >> > fail over to 150 per user per hour or is it cut off? > > > > >> > Thanks > > > > >> > -Bob > > > > >> > On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretorius<[email protected]> > wrote: > > > > >> >> Bob, > > > > >> >> Don't base your app on the assumption that it is 20,000 calls per > hour > > >> >> per user. > > > > >> >> You get 20,000 GET calls per whitelisted IP address, period. It > does > > >> >> not matter if you use those calls for one Twitter account or 10,000 > > >> >> Twitter accounts. > > > > >> >> If the API is currently behaving differently, then it is a bug. > > > > >> >> I have had discussions with Twitter engineers about this, and the > > >> >> intended behavior is an aggregate 20,000 calls per whitelisted IP > > >> >> address as I mentioned above. > > > > >> >> Dewald > > > > >> >> On Aug 6, 4:09 am, Robert Fishel <[email protected]> wrote: > > >> >>> Wowzers (bonus points for getting the reference) > > > > >> >>> It appears as if each user does get 20k (according to the linked > > >> >>> threads) this is I think what they intended and makes apps a LOT > > >> >>> easier to develop as you can now do rate limiting (ie caching and > > >> >>> sleeping etc...) based on each user and not on an entire server > pool, > > >> >>> makes sessions much cleaner. > > > > >> >>> I am whitelisted and I'll test this tomorrow evening to make > double > > >> >>> sure but this sounds great!. > > > > >> >>> Thanks > > > > >> >>> -Bob > > > > >> >>> On Thu, Aug 6, 2009 at 2:53 AM, srikanth > > > > >> >>> reddy<[email protected]> wrote: > > >> >>> > With a whitelisted IP you can make 20k auth calls per hour for > each user. > > >> >>> > Once you reach this limit for a user you cannot make any auth > calls from > > >> >>> > that IP in that duration. But the user can still use his 150 > limit from > > >> >>> > other apps. > > > > >> >>> > > http://groups.google.com/group/twitter-development-talk/browse_thread... > > > > >> >>> > On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel < > [email protected]> wrote: > > > > >> >>> >> From the Rate Limiting documentation: > > > > >> >>> >> "IP whitelisting takes precedence to account rate limits. GET > requests > > >> >>> >> from a whitelisted IP address made on a user's behalf will be > deducted > > >> >>> >> from the whitelisted IP's limit, not the users. Therefore, > IP-based > > >> >>> >> whitelisting is a best practice for applications that request > many > > >> >>> >> users' data." > > > > >> >>> >> Say for example I wanted to simply replicate the twitter > website. One > > >> >>> >> page per user that just monitors for new statuses with > authenticated > > >> >>> >> (to catch protected users) calls to > > >> >>> >>http://twitter.com/statuses/friends_timeline.json > > > > >> >>> >> Say I was very popular and had 20k people on the site. Would > this > > >> >>> >> limit me to 1 call per minute per user or would it fall over to > the > > >> >>> >> user limit of 150 an hour once I hit my 20k? If so how can I > tell it > > >> >>> >> has fallen over besides for simply keeping track of the number > of > > >> >>> >> calls per hour my server has made. > > > > >> >>> >> Thanks > > > > >> >>> >> -Bob >
