[twitter-dev] Re: rate limiting due to invalid oauth credentials
Read the docs...thought the call was authenticated, but discovered that the library I was using had a bug for situations where the call included certain characters. The authentication failed, but the query continued as an unauthenticated call. No error message. Bitch of a time figuring out what was happening; had to reverse engineer the library and step through it. On Jul 4, 2:09 pm, Mo'b Mo'b mobingapapi...@gmail.com wrote: Authenticated Rate Limit --- 350 Calls per hour. Unauthenticated Rate Limit --- 150 Calls per hour. Please read the docs. -- Have you visited the Developer Discussions feature on https://dev.twitter.com/discussions yet? Twitter developer links: Documentation and resources: https://dev.twitter.com/docs API updates via Twitter: https://twitter.com/twitterapi Unsubscribe or change your group membership settings: http://groups.google.com/group/twitter-development-talk/subscribe
Re: [twitter-dev] Re: rate limiting due to invalid oauth credentials
Hey Mike, Sorry to hear you were having problems there. If the credentials were being rejected we would have returned an X-Warning header in the response letting you know. There is more information about this in our rate limiting documentation: https://dev.twitter.com/docs/rate-limiting#rest Hope that helps, @themattharris https://twitter.com/intent/follow?screen_name=themattharris Twitter, Inc. On Wed, Aug 10, 2011 at 8:50 AM, mike3s mike.3.sulli...@gmail.com wrote: Read the docs...thought the call was authenticated, but discovered that the library I was using had a bug for situations where the call included certain characters. The authentication failed, but the query continued as an unauthenticated call. No error message. Bitch of a time figuring out what was happening; had to reverse engineer the library and step through it. On Jul 4, 2:09 pm, Mo'b Mo'b mobingapapi...@gmail.com wrote: Authenticated Rate Limit --- 350 Calls per hour. Unauthenticated Rate Limit --- 150 Calls per hour. Please read the docs. -- Have you visited the Developer Discussions feature on https://dev.twitter.com/discussions yet? Twitter developer links: Documentation and resources: https://dev.twitter.com/docs API updates via Twitter: https://twitter.com/twitterapi Unsubscribe or change your group membership settings: http://groups.google.com/group/twitter-development-talk/subscribe -- Have you visited the Developer Discussions feature on https://dev.twitter.com/discussions yet? Twitter developer links: Documentation and resources: https://dev.twitter.com/docs API updates via Twitter: https://twitter.com/twitterapi Unsubscribe or change your group membership settings: http://groups.google.com/group/twitter-development-talk/subscribe
[twitter-dev] Re: Rate limiting questions
Hi there,
My application gives Read, Write and Direct Message permissions.
Additionally, I was able to retrieve direct messages using the
appropriate access tokens earlier in the day, now none of it works.
Is there some specific thing I need to do in order to have the
permission level of these tokens set beyond the normal OAuth process?
Thanks for the help.
On Jul 5, 7:46 pm, Taylor Singletary taylorsinglet...@twitter.com
wrote:
Hi there,
This is actually a different error than your library may be leading you to
believe -- the library is suggesting that the 403 may be due to rate
limiting, but in this case it's actually due to a recent permission model
change.
The permission model gas change whereas requesting a user's direct messages
now requires a re-authorization at the appropriate access request level
(RWD). You can read more about the new permission model and what to do next
here:http://dev.twitter.com/pages/application-permission-model..
@episod http://twitter.com/intent/user?screen_name=episod - Taylor
Singletary
On Tue, Jul 5, 2011 at 1:35 PM, YupiqDZ mrclea...@gmail.com wrote:
Hi,
I'm currently in the process of writing an application which
periodically gets directed messages from a user (once every 120
seconds checks if the user has new messages).
The API I am using is the Twitter4j library in Java. Here's the error
I get:
403:The request is understood, but it has been refused. An
accompanying error message will explain why.
TwitterException{exceptionCode=[ece01d6a-01eb72d6], statusCode=403,
retryAfter=0,
rateLimitStatus=RateLimitStatusJSONImpl{remainingHits=326,
hourlyLimit=350, resetTimeInSeconds=1309899, secondsUntilReset=1629,
resetTime=Tue Jul 05 13:58:10 PDT 2011}, version=2.1.10}
I am confused as to how my remainingHIts is still 326 yet I am rate
limited? Additionally, even if I wait until the reset time indicated
before hitting the service again, I receive the rate limit exception
again with the reset time pushed back by an hour.
I am fairly confident that this service does not poll Twitter more
than 350 times a second, is there something else that could be causing
me to be rate limited?
Thanks
--
Twitter developer documentation and resources:https://dev.twitter.com/doc
API updates via Twitter:https://twitter.com/twitterapi
Issues/Enhancements Tracker:
https://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
https://groups.google.com/forum/#!forum/twitter-development-talk
--
Twitter developer documentation and resources: https://dev.twitter.com/doc
API updates via Twitter: https://twitter.com/twitterapi
Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Re: Rate limiting questions
I figured out what was happening.
It looks like I was using the AuthenticationURL instead of the
AuthorizationURL and this was causing me to not obtain the proper
permissions.
Thank for your help!
On Jul 6, 9:40 am, YupiqDZ mrclea...@gmail.com wrote:
Hi there,
My application gives Read, Write and Direct Message permissions.
Additionally, I was able to retrieve direct messages using the
appropriate access tokens earlier in the day, now none of it works.
Is there some specific thing I need to do in order to have the
permission level of these tokens set beyond the normal OAuth process?
Thanks for the help.
On Jul 5, 7:46 pm, Taylor Singletary taylorsinglet...@twitter.com
wrote:
Hi there,
This is actually a different error than your library may be leading you to
believe -- the library is suggesting that the 403 may be due to rate
limiting, but in this case it's actually due to a recent permission model
change.
The permission model gas change whereas requesting a user's direct messages
now requires a re-authorization at the appropriate access request level
(RWD). You can read more about the new permission model and what to do next
here:http://dev.twitter.com/pages/application-permission-model..
@episod http://twitter.com/intent/user?screen_name=episod - Taylor
Singletary
On Tue, Jul 5, 2011 at 1:35 PM, YupiqDZ mrclea...@gmail.com wrote:
Hi,
I'm currently in the process of writing an application which
periodically gets directed messages from a user (once every 120
seconds checks if the user has new messages).
The API I am using is the Twitter4j library in Java. Here's the error
I get:
403:The request is understood, but it has been refused. An
accompanying error message will explain why.
TwitterException{exceptionCode=[ece01d6a-01eb72d6], statusCode=403,
retryAfter=0,
rateLimitStatus=RateLimitStatusJSONImpl{remainingHits=326,
hourlyLimit=350, resetTimeInSeconds=1309899, secondsUntilReset=1629,
resetTime=Tue Jul 05 13:58:10 PDT 2011}, version=2.1.10}
I am confused as to how my remainingHIts is still 326 yet I am rate
limited? Additionally, even if I wait until the reset time indicated
before hitting the service again, I receive the rate limit exception
again with the reset time pushed back by an hour.
I am fairly confident that this service does not poll Twitter more
than 350 times a second, is there something else that could be causing
me to be rate limited?
Thanks
--
Twitter developer documentation and resources:https://dev.twitter.com/doc
API updates via Twitter:https://twitter.com/twitterapi
Issues/Enhancements Tracker:
https://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
https://groups.google.com/forum/#!forum/twitter-development-talk
--
Twitter developer documentation and resources: https://dev.twitter.com/doc
API updates via Twitter: https://twitter.com/twitterapi
Issues/Enhancements Tracker: https://code.google.com/p/twitter-api/issues/list
Change your membership to this group:
https://groups.google.com/forum/#!forum/twitter-development-talk
[twitter-dev] Re: Rate Limiting
And again I'm locked out. 400 is saying Rate limit exceeded. Clients may not make more than 150 requests per hour., however rate_limit_status is saying You have 145 api calls left until 12:20 when it will be reset to 150. (local time 11:39) On Jul 6, 10:59 pm, Rich rhyl...@gmail.com wrote: Yep I'm now locked out of my own account with this issue On Jul 6, 10:38 pm, Matt Harris thematthar...@twitter.com wrote: We are aware rate limits are being reported incorrectly. We are tracking the issue on the API tracker:http://code.google.com/p/twitter-api/issues/detail?id=1728 Matt On Tue, Jul 6, 2010 at 10:10 AM, Johnson johnson...@gmail.com wrote: I want to know what's the metrics of this dynamic logout. I have several twitter account/twitter development accounts. It started around 1am PST, one of my account keep getting 400. I woke up this morning, that only account is still getting 400. I tried a different pair comsumer key/token using twurl, it works. I tried on mobile, Echolfon, failed, but twitbird goes thru. Obviously the web works! What other experiments do u want me to run? Are twitter giving favor on specific accounts? On Jul 6, 9:57 am, Pascal Jürgens lists.pascal.juerg...@googlemail.com wrote: Just a sidenote: This can be coincidental. Unless you try several dozen times with each client, no valid inference can be drawn from the tests. Pascal On Jul 6, 2010, at 18:46 , Johnson wrote: I notice that the rate limit is application specific. I've tried a few clients, some of them goes thru, some don't. -- Matt Harris Developer Advocate, Twitterhttp://twitter.com/themattharris
[twitter-dev] Re: Rate Limiting
For my whitelisted client everything is fine again, limit is up at 20k again :) On Jul 7, 12:40 pm, artesea ryancul...@gmail.com wrote: And again I'm locked out. 400 is saying Rate limit exceeded. Clients may not make more than 150 requests per hour., however rate_limit_status is saying You have 145 api calls left until 12:20 when it will be reset to 150. (local time 11:39) On Jul 6, 10:59 pm, Rich rhyl...@gmail.com wrote: Yep I'm now locked out of my own account with this issue On Jul 6, 10:38 pm, Matt Harris thematthar...@twitter.com wrote: We are aware rate limits are being reported incorrectly. We are tracking the issue on the API tracker:http://code.google.com/p/twitter-api/issues/detail?id=1728 Matt On Tue, Jul 6, 2010 at 10:10 AM, Johnson johnson...@gmail.com wrote: I want to know what's the metrics of this dynamic logout. I have several twitter account/twitter development accounts. It started around 1am PST, one of my account keep getting 400. I woke up this morning, that only account is still getting 400. I tried a different pair comsumer key/token using twurl, it works. I tried on mobile, Echolfon, failed, but twitbird goes thru. Obviously the web works! What other experiments do u want me to run? Are twitter giving favor on specific accounts? On Jul 6, 9:57 am, Pascal Jürgens lists.pascal.juerg...@googlemail.com wrote: Just a sidenote: This can be coincidental. Unless you try several dozen times with each client, no valid inference can be drawn from the tests. Pascal On Jul 6, 2010, at 18:46 , Johnson wrote: I notice that the rate limit is application specific. I've tried a few clients, some of them goes thru, some don't. -- Matt Harris Developer Advocate, Twitterhttp://twitter.com/themattharris
[twitter-dev] Re: Rate Limiting
I've got the same problem, according to the api response the reset time is in the past. On 6 Jul., 14:20, artesea ryancul...@gmail.com wrote: I'm being locked out on my account using the API and I'm seeing reports from others. At the moment making a request tohttp://api.twitter.com/version/account/rate_limit_status.jsoncomes back saying I have 8 calls left and it will be reset at 07:54, but the time is currently 13:18! The time for the reset has been the same since at least 8am.
[twitter-dev] Re: Rate Limiting
Finally have access again, with the time to reset now at 15:27 (around an hour). Ryan
Re: [twitter-dev] Re: Rate Limiting
Quoting artesea ryancul...@gmail.com: Finally have access again, with the time to reset now at 15:27 (around an hour). Ryan Ouch - sounds like my IP address got blocked then.
[twitter-dev] Re: Rate Limiting
On Jul 6, 9:23 am, M. Edward (Ed) Borasky zn...@borasky- research.net wrote: Quoting artesea ryancul...@gmail.com: Finally have access again, with the time to reset now at 15:27 (around an hour). Ryan Ouch - sounds like my IP address got blocked then. I appear to have the same issue. The status keeps reporting 0 remaining, yet I'm well past the reset time. When I try on another IP address, using the same authentication, I get 75 remaining. I don't understand how my IP address could have gotten blocked. My application does not attempt any further access to the API once the rate limit has been reached.
Re: [twitter-dev] Re: Rate Limiting
We're looking into this rate limiting issue. -John Kalucki http://twitter.com/jkalucki Infrastructure, Twitter Inc. On Tue, Jul 6, 2010 at 7:45 AM, Jeff Randall jeffr...@swbell.net wrote: On Jul 6, 9:23 am, M. Edward (Ed) Borasky zn...@borasky- research.net wrote: Quoting artesea ryancul...@gmail.com: Finally have access again, with the time to reset now at 15:27 (around an hour). Ryan Ouch - sounds like my IP address got blocked then. I appear to have the same issue. The status keeps reporting 0 remaining, yet I'm well past the reset time. When I try on another IP address, using the same authentication, I get 75 remaining. I don't understand how my IP address could have gotten blocked. My application does not attempt any further access to the API once the rate limit has been reached.
[twitter-dev] Re: Rate Limiting
I notice that the rate limit is application specific. I've tried a few clients, some of them goes thru, some don't. On Jul 6, 5:20 am, artesea ryancul...@gmail.com wrote: I'm being locked out on my account using the API and I'm seeing reports from others. At the moment making a request tohttp://api.twitter.com/version/account/rate_limit_status.jsoncomes back saying I have 8 calls left and it will be reset at 07:54, but the time is currently 13:18! The time for the reset has been the same since at least 8am.
Re: [twitter-dev] Re: Rate Limiting
Just a sidenote: This can be coincidental. Unless you try several dozen times with each client, no valid inference can be drawn from the tests. Pascal On Jul 6, 2010, at 18:46 , Johnson wrote: I notice that the rate limit is application specific. I've tried a few clients, some of them goes thru, some don't.
[twitter-dev] Re: Rate Limiting
I want to know what's the metrics of this dynamic logout. I have several twitter account/twitter development accounts. It started around 1am PST, one of my account keep getting 400. I woke up this morning, that only account is still getting 400. I tried a different pair comsumer key/token using twurl, it works. I tried on mobile, Echolfon, failed, but twitbird goes thru. Obviously the web works! What other experiments do u want me to run? Are twitter giving favor on specific accounts? On Jul 6, 9:57 am, Pascal Jürgens lists.pascal.juerg...@googlemail.com wrote: Just a sidenote: This can be coincidental. Unless you try several dozen times with each client, no valid inference can be drawn from the tests. Pascal On Jul 6, 2010, at 18:46 , Johnson wrote: I notice that the rate limit is application specific. I've tried a few clients, some of them goes thru, some don't.
Re: [twitter-dev] Re: Rate Limiting
We are aware rate limits are being reported incorrectly. We are tracking the issue on the API tracker: http://code.google.com/p/twitter-api/issues/detail?id=1728 Matt On Tue, Jul 6, 2010 at 10:10 AM, Johnson johnson...@gmail.com wrote: I want to know what's the metrics of this dynamic logout. I have several twitter account/twitter development accounts. It started around 1am PST, one of my account keep getting 400. I woke up this morning, that only account is still getting 400. I tried a different pair comsumer key/token using twurl, it works. I tried on mobile, Echolfon, failed, but twitbird goes thru. Obviously the web works! What other experiments do u want me to run? Are twitter giving favor on specific accounts? On Jul 6, 9:57 am, Pascal Jürgens lists.pascal.juerg...@googlemail.com wrote: Just a sidenote: This can be coincidental. Unless you try several dozen times with each client, no valid inference can be drawn from the tests. Pascal On Jul 6, 2010, at 18:46 , Johnson wrote: I notice that the rate limit is application specific. I've tried a few clients, some of them goes thru, some don't. -- Matt Harris Developer Advocate, Twitter http://twitter.com/themattharris
[twitter-dev] Re: Rate Limiting
Yep I'm now locked out of my own account with this issue On Jul 6, 10:38 pm, Matt Harris thematthar...@twitter.com wrote: We are aware rate limits are being reported incorrectly. We are tracking the issue on the API tracker:http://code.google.com/p/twitter-api/issues/detail?id=1728 Matt On Tue, Jul 6, 2010 at 10:10 AM, Johnson johnson...@gmail.com wrote: I want to know what's the metrics of this dynamic logout. I have several twitter account/twitter development accounts. It started around 1am PST, one of my account keep getting 400. I woke up this morning, that only account is still getting 400. I tried a different pair comsumer key/token using twurl, it works. I tried on mobile, Echolfon, failed, but twitbird goes thru. Obviously the web works! What other experiments do u want me to run? Are twitter giving favor on specific accounts? On Jul 6, 9:57 am, Pascal Jürgens lists.pascal.juerg...@googlemail.com wrote: Just a sidenote: This can be coincidental. Unless you try several dozen times with each client, no valid inference can be drawn from the tests. Pascal On Jul 6, 2010, at 18:46 , Johnson wrote: I notice that the rate limit is application specific. I've tried a few clients, some of them goes thru, some don't. -- Matt Harris Developer Advocate, Twitterhttp://twitter.com/themattharris
[twitter-dev] Re: Rate Limiting with desktop applications using C# HttpRequest
Alright, I figured it out...
The problem is I am using basic access authentication (I know.. my
bad... OAuth it is... I swear I will switch to that...). But C#
HttpWebRequest doesn't pack up the basic access Authentication
credential into the headers unless you program it to. Therefore the
GET calls from my program goes unauthenticated and falls into the
quota of my IP.
here are three links helped me:
http://forums.silverlight.net/forums/t/18631.aspx
http://en.wikipedia.org/wiki/Basic_access_authentication
http://geekswithblogs.net/dtotzke/articles/24571.aspx
the trick in the last link works. MAKE SURE to attach Basic in front
of your Base64 string of username/password pair in your header
value I came up with the same workaround but missed that piece,
and I almost thought it didn't work until I found my problem by
reading the last link
Now all my calls is counted against my account quota. Nice.
-Jian
On Jun 21, 10:52 am, Jian Lu tristan@gmail.com wrote:
I have whitelisted my account (but no ip), and am sending requests
through my desktop application by wrapping my credentials with C#
HttpRequest.
Very frequently, using the method above, I see my rates drop back to
150/hour and it drains out even I am not making any calls. Here is a
sample reponse I got by checking my rate limit:
{
remaining_hits = 0,
hourly_limit = 150,
reset_time_in_seconds = 1277141653,
reset_time = Mon Jun 21 17:34:13 + 2010
}
But strangely if I submit my rate-checking request on the SAME MACHINE
at the SAME TIME through curl, I got the correct limit, and it doesn't
automatically drain out if I don't make api request:
{reset_time_in_seconds:1277145186,remaining_hits:
2,reset_time:Mon Jun 21 18:33:06 + 2010,hourly_limit:
2}
The pattern I am seeing is:
1a. Right after limit RESET, if I query my rate limit through my C#
HttpRequest code, I get my whitelisted rates. BUT it drains out
automatically and quickly even I am not using it at all...
1b. But if I use curl to do a query to check my rate limit, even
though the checking in #1a shows something lower than my real usage,
the result returned from curl is always correct and equal to my real
usage.
3. When that limit drains out, I started to get the 150/hour limit
response but the remaining hits is 0 by issuing C# HttpRequests.
What I suspected is it has something to do with my network setting. I
am behind my company domain and maybe different machines within my
company network have the same external IP?
Is there any way to avoid that?
-Jian
Re: [twitter-dev] Re: Rate Limiting Twitter
Speaking of rate limiting, I was trying to learn the Twitter API and Net::Twitter by making six degrees of twitter that would figure out how many hops it was from Account A to Account B when I ran up against limiting. The way I had thought of doing it was basically the list of whom a person follows and if they follow back its a valid Nth degree hop. Then get whom that hop is following, see if they are following back, and just cycle outwards until you find the destination account. I'm sure you can see how quickly that eats up a rate limit. Is the only way around that to apply for whitelisted status? Or are there some API tricks that I, as a noob, haven't stumbled across yet?
[twitter-dev] Re: Rate Limiting Twitter
Thank you for your quick reply. I found out I was indeed rate limited, allthough i cant imagine having done 150 requests testing my app only once. Thanks anyway. On 15 feb, 23:41, Abraham Williams 4bra...@gmail.com wrote: You can check if you are getting rate limited with this method:http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-account%C2%A0ra... If you are using OAuth onhttp://api.twitter.comthen you should be getting 350 (last I heard) hits per hour. Otherwise you will be limited to 150/h. Abraham On Mon, Feb 15, 2010 at 07:32, b...@mindbus.nl b...@mindbus.nl wrote: Hi, I was creating a simple application which requests statuses of a certain list within my Twitter account using the following command: http://api.twitter.com/1/id/lists/list-id/statuses.xml. For some reason, after doing some requests, i get a HTTP 400 Bad Request. I have read that a client may only do 500 requests per hour, but i dont believe that i have done that many requests by simply testing my application. After some time all is well and i receive the expected result. Has someone else encountered this problem? I have already sent a request to whitelist my IP, but i want to make sure that not else is causing this problem. Thanks in advance. -- Abraham Williams | Community Advocate |http://abrah.am Project | Out Loud |http://outloud.labs.poseurtech.com This email is: [ ] shareable [x] ask first [ ] private. Sent from Seattle, WA, United States- Tekst uit oorspronkelijk bericht niet weergeven - - Tekst uit oorspronkelijk bericht weergeven -
[twitter-dev] Re: Rate Limiting question
Hi all, thanks for your responses. John, I did take a look at the stream api but was put off by the big disclaimer saying it could change very frequently and be down for extended periods. Also, I was kinda trying to avoid the issue I was seeing in search where certain tweets were not being indexed - now perhaps this is because my test tweets unwittingly looked like duplicates (and not worthy of indexing) but it made me think ok, is there a way to avoid this problem - scanning the users timelines seemed the way to go. The streaming API looks interesting though and I get the idea of having a single connection - which in my case would be a shell based PHP script dumping the results to (say) a flat file for subsequent import/processing. I'll need to find some best practice PHP scripts - phirehose looked interesting. Can I check something with rate limits and streaming API? If I have 20K requests per hour this basically means I can use the streaming API 20K times per hour? Or is that too simplistic. Joel
Re: [twitter-dev] Re: Rate Limiting question
You probably wouldn't use the streaming API 20k times/hr. You would open one connection and consume data from it during that hour. ---Mark http://twitter.com/mccv On Sun, Jan 3, 2010 at 12:15 AM, Joel Hughes joelhug...@gmail.com wrote: Hi all, thanks for your responses. John, I did take a look at the stream api but was put off by the big disclaimer saying it could change very frequently and be down for extended periods. Also, I was kinda trying to avoid the issue I was seeing in search where certain tweets were not being indexed - now perhaps this is because my test tweets unwittingly looked like duplicates (and not worthy of indexing) but it made me think ok, is there a way to avoid this problem - scanning the users timelines seemed the way to go. The streaming API looks interesting though and I get the idea of having a single connection - which in my case would be a shell based PHP script dumping the results to (say) a flat file for subsequent import/processing. I'll need to find some best practice PHP scripts - phirehose looked interesting. Can I check something with rate limits and streaming API? If I have 20K requests per hour this basically means I can use the streaming API 20K times per hour? Or is that too simplistic. Joel
[twitter-dev] Re: Rate Limiting question
Hey Kyle, Is that true re the rate limits? I've always assumed that ANY request on an IP is xounted against that IP's rate limit - so comes of the 20,000; and each user has their own rate limit... normally 150 per hour. So an authenticated request comes off both the user's and and IP's rate limit? In the very least, the user's limit of 150 per hour is affected first, and then if that is maxed out, then the IP limit will come into play? Could someone from Twitter clarify this, and Kyle's suggestion, as the API docs are very vague in this area Thanks Simon On Jan 1, 5:21 pm, Kyle Mulka repalvigla...@yahoo.com wrote: My experience with rate limiting shows that each authenticated request is counted against that user's limit on your IP. So, you get 20,000 requests per IP, per user, per hour assuming all your requests are authenticated. Any unauthenticated requests go towards the 20,000 request limit per IP, per hour. In my case, all Twitter API calls are authenticated and cached for an hour. The way my app is set up, under normal usage, no user will use more than 20,000 Twitter API requests. -- Kyle Mulka Founder, Congo Labshttp://twilk.com On Jan 1, 5:43 pm, jojet j...@jojet.com wrote: Hi all, I was feeling a little clever after working on some Twitter API stuff but then thought oh! I'd better think about Twitters rate limiting...and then that's where my brain started to melt! A few bits of info: my web app needs people to authenticate (OAUTH) and, from then on, the app analyses their tweets and occasionally updates registered user's statuses. I've applied for the webserver IP to be white listed which I believe gives the app 20,000 requests per hour. I've not found the search API to be great when looking for a hashtag (sometimes tweets just don't seem to get indexed) so I've gone a stage further and am analysing the individual timelines of all my registered users via a cron job (the cron job sucks in all of a persons tweets greater than the last analysed tweet of the user). This call is made via OAUTH/authenticated so I believe such a call depletes the user's rate limit quota rather than the IP/authenticated account of the webserver quota? Is that correct? Thanks for any thoughts here Joel
[twitter-dev] Re: Rate Limiting question
My experience with rate limiting shows that each authenticated request is counted against that user's limit on your IP. So, you get 20,000 requests per IP, per user, per hour assuming all your requests are authenticated. Any unauthenticated requests go towards the 20,000 request limit per IP, per hour. In my case, all Twitter API calls are authenticated and cached for an hour. The way my app is set up, under normal usage, no user will use more than 20,000 Twitter API requests. -- Kyle Mulka Founder, Congo Labs http://twilk.com On Jan 1, 5:43 pm, jojet j...@jojet.com wrote: Hi all, I was feeling a little clever after working on some Twitter API stuff but then thought oh! I'd better think about Twitters rate limiting...and then that's where my brain started to melt! A few bits of info: my web app needs people to authenticate (OAUTH) and, from then on, the app analyses their tweets and occasionally updates registered user's statuses. I've applied for the webserver IP to be white listed which I believe gives the app 20,000 requests per hour. I've not found the search API to be great when looking for a hashtag (sometimes tweets just don't seem to get indexed) so I've gone a stage further and am analysing the individual timelines of all my registered users via a cron job (the cron job sucks in all of a persons tweets greater than the last analysed tweet of the user). This call is made via OAUTH/authenticated so I believe such a call depletes the user's rate limit quota rather than the IP/authenticated account of the webserver quota? Is that correct? Thanks for any thoughts here Joel
[twitter-dev] Re: Rate Limiting Question
Craig, I just ran a test, and I can also confirm what you have found. Unauthenticated calls decrease per IP 20,000 Authenticated calls decrease per-IP per-user 20,000 Dewald On Aug 13, 4:27 pm, CaMason stasisme...@googlemail.com wrote: The behaviour at the moment is definitely as-described above: Unauthenticated calls decrease IP 20,000 Authenticated calls decrease per-user 20,000 My app only uses authenticated calls during normal use, and the IP- based limit isn't decreasing at-all 20,000 per-user is pretty silly - With 1000 users, I would be allowed to make 5,555 calls per second. A max of say 500 authenticated calls per-user would be more sensible, and would allow apps with many users to scale :) -Craig
[twitter-dev] Re: Rate Limiting Question
Hi There, What you all have been confirming is correct. The intended behavior is 20k per IP unauthenticated, and 20k per IP *per user* authenticated. This is not a bug. -Chad On Thu, Aug 13, 2009 at 4:43 PM, Abraham Williams4bra...@gmail.com wrote: I've been reading I have confirmed emails from 5 different threads for the last 2 weeks. Can we hold off until Chad gets back to us with an official answer. :) Thanks Abraham 2009/8/13 Dewald Pretorius dpr...@gmail.com Craig, I just ran a test, and I can also confirm what you have found. Unauthenticated calls decrease per IP 20,000 Authenticated calls decrease per-IP per-user 20,000 Dewald On Aug 13, 4:27 pm, CaMason stasisme...@googlemail.com wrote: The behaviour at the moment is definitely as-described above: Unauthenticated calls decrease IP 20,000 Authenticated calls decrease per-user 20,000 My app only uses authenticated calls during normal use, and the IP- based limit isn't decreasing at-all 20,000 per-user is pretty silly - With 1000 users, I would be allowed to make 5,555 calls per second. A max of say 500 authenticated calls per-user would be more sensible, and would allow apps with many users to scale :) -Craig -- Abraham Williams | Community Evangelist | http://web608.org Hacker | http://abrah.am | http://twitter.com/abraham Project | http://fireeagle.labs.poseurtech.com This email is: [ ] blogable [x] ask first [ ] private.
[twitter-dev] Re: Rate Limiting Question
Holy Thanks, Chad. :) On Aug 13, 4:58 pm, Chad Etzel c...@twitter.com wrote: Hi There, What you all have been confirming is correct. The intended behavior is 20k per IP unauthenticated, and 20k per IP *per user* authenticated. This is not a bug. -Chad On Thu, Aug 13, 2009 at 4:43 PM, Abraham Williams4bra...@gmail.com wrote: I've been reading I have confirmed emails from 5 different threads for the last 2 weeks. Can we hold off until Chad gets back to us with an official answer. :) Thanks Abraham 2009/8/13 Dewald Pretorius dpr...@gmail.com Craig, I just ran a test, and I can also confirm what you have found. Unauthenticated calls decrease per IP 20,000 Authenticated calls decrease per-IP per-user 20,000 Dewald On Aug 13, 4:27 pm, CaMason stasisme...@googlemail.com wrote: The behaviour at the moment is definitely as-described above: Unauthenticated calls decrease IP 20,000 Authenticated calls decrease per-user 20,000 My app only uses authenticated calls during normal use, and the IP- based limit isn't decreasing at-all 20,000 per-user is pretty silly - With 1000 users, I would be allowed to make 5,555 calls per second. A max of say 500 authenticated calls per-user would be more sensible, and would allow apps with many users to scale :) -Craig -- Abraham Williams | Community Evangelist |http://web608.org Hacker |http://abrah.am|http://twitter.com/abraham Project |http://fireeagle.labs.poseurtech.com This email is: [ ] blogable [x] ask first [ ] private.
[twitter-dev] Re: Rate Limiting Question
Just to make things crystal clear, it should be stated that the 20k rate limits apply only to GET requests to the so-called REST-API. Other request types (I.e., POST) and / or other APIs (I.e., search, streaming) have other rate limits. Jim Renkel On Aug 13, 3:58 pm, Chad Etzel c...@twitter.com wrote: Hi There, What you all have been confirming is correct. The intended behavior is 20k per IP unauthenticated, and 20k per IP *per user* authenticated. This is not a bug. -Chad On Thu, Aug 13, 2009 at 4:43 PM, Abraham Williams4bra...@gmail.com wrote: I've been reading I have confirmed emails from 5 different threads for the last 2 weeks. Can we hold off until Chad gets back to us with an official answer. :) Thanks Abraham 2009/8/13 Dewald Pretorius dpr...@gmail.com Craig, I just ran a test, and I can also confirm what you have found. Unauthenticated calls decrease per IP 20,000 Authenticated calls decrease per-IP per-user 20,000 Dewald On Aug 13, 4:27 pm, CaMason stasisme...@googlemail.com wrote: The behaviour at the moment is definitely as-described above: Unauthenticated calls decrease IP 20,000 Authenticated calls decrease per-user 20,000 My app only uses authenticated calls during normal use, and the IP- based limit isn't decreasing at-all 20,000 per-user is pretty silly - With 1000 users, I would be allowed to make 5,555 calls per second. A max of say 500 authenticated calls per-user would be more sensible, and would allow apps with many users to scale :) -Craig -- Abraham Williams | Community Evangelist |http://web608.org Hacker |http://abrah.am|http://twitter.com/abraham Project |http://fireeagle.labs.poseurtech.com This email is: [ ] blogable [x] ask first [ ] private.
[twitter-dev] Re: Rate Limiting Question
YabadabaFrigginDoo!! I have no idea what kind of application would need to continuously make 5 authenticated calls per second on a particular Twitter account, but hey, if you can think of one, you know you won't be rate limited. Dewald On Aug 13, 5:58 pm, Chad Etzel c...@twitter.com wrote: Hi There, What you all have been confirming is correct. The intended behavior is 20k per IP unauthenticated, and 20k per IP *per user* authenticated. This is not a bug. -Chad
[twitter-dev] Re: Rate Limiting Question
On Thu, Aug 13, 2009 at 5:44 PM, Dewald Pretoriusdpr...@gmail.com wrote: YabadabaFrigginDoo!! I have no idea what kind of application would need to continuously make 5 authenticated calls per second on a particular Twitter account, but hey, if you can think of one, you know you won't be rate limited. Dewald Twitterbation? ∞ Andy Badera ∞ This email is: [ ] bloggable [x] ask first [ ] private ∞ Google me: http://www.google.com/search?q=(andrew+badera)+OR+(andy+badera)
[twitter-dev] Re: Rate Limiting Question
In fact, with an API response time of 0.3 seconds, you won't even run out of rate limit if your authenticated GET script goes into an endless loop. Dewald On Aug 13, 6:44 pm, Dewald Pretorius dpr...@gmail.com wrote: YabadabaFrigginDoo!! I have no idea what kind of application would need to continuously make 5 authenticated calls per second on a particular Twitter account, but hey, if you can think of one, you know you won't be rate limited. Dewald On Aug 13, 5:58 pm, Chad Etzel c...@twitter.com wrote: Hi There, What you all have been confirming is correct. The intended behavior is 20k per IP unauthenticated, and 20k per IP *per user* authenticated. This is not a bug. -Chad
[twitter-dev] Re: Rate Limiting Problem
Get the accounts themselves whitelisted. ∞ Andy Badera ∞ This email is: [ ] bloggable [x] ask first [ ] private ∞ Google me: http://www.google.com/search?q=(andrew+badera)+OR+(andy+badera) On Thu, Aug 13, 2009 at 5:13 PM, arjunsegmentationremo...@gmail.com wrote: We are a research group in Georgia Tech working on a Recommender System for Twitter. We have 10 accounts and 3 ips whitelisted. However, since the accounts use the same ips, the rate limit of the ips (20,000) is causing a bottleneck. We would like to get the ips off the white-list if thats the only solution or we would like to know if there is a better solution. Is there someone who could help us out with this?
[twitter-dev] Re: Rate Limiting Problem
w00t! Go Jackets! I'll contact you off list to figure this one out with you guys. -Chad On Thu, Aug 13, 2009 at 5:13 PM, arjunsegmentationremo...@gmail.com wrote: We are a research group in Georgia Tech working on a Recommender System for Twitter. We have 10 accounts and 3 ips whitelisted. However, since the accounts use the same ips, the rate limit of the ips (20,000) is causing a bottleneck. We would like to get the ips off the white-list if thats the only solution or we would like to know if there is a better solution. Is there someone who could help us out with this?
[twitter-dev] Re: Rate Limiting Problem
Don't need to anymore. Any users authenticating from whitelisted IPs now get 20k hits/hour each. So sayeth the Chad. Abraham 2009/8/13 Andrew Badera and...@badera.us Get the accounts themselves whitelisted. ∞ Andy Badera ∞ This email is: [ ] bloggable [x] ask first [ ] private ∞ Google me: http://www.google.com/search?q=(andrew+badera)+OR+(andy+badera)http://www.google.com/search?q=%28andrew+badera%29+OR+%28andy+badera%29 On Thu, Aug 13, 2009 at 5:13 PM, arjunsegmentationremo...@gmail.com wrote: We are a research group in Georgia Tech working on a Recommender System for Twitter. We have 10 accounts and 3 ips whitelisted. However, since the accounts use the same ips, the rate limit of the ips (20,000) is causing a bottleneck. We would like to get the ips off the white-list if thats the only solution or we would like to know if there is a better solution. Is there someone who could help us out with this? -- Abraham Williams | Community Evangelist | http://web608.org Hacker | http://abrah.am | http://twitter.com/abraham Project | http://fireeagle.labs.poseurtech.com This email is: [ ] blogable [x] ask first [ ] private. Sent from Fairbanks, Alaska, United States
[twitter-dev] Re: Rate Limiting Question
Hi Chad - Now that the DDoS attacks are (sort of) behind us, can we seek some closure on this? I'm dying to know the official, undisputed, written- in-stone, we-can-finally-stop-arguing-about-it answer to the following (which I think simplifies the question): If my IP is whitelisted and I have 20 simultaneous users logged in to my app for 1 hour and each user generates 1,000 requests, have I used up my rate limit for that hour or could each user send 19,000 more requests? If it's the latter, is that a bug that's going to be fixed, or does Twitter really love us that much? :) On Aug 6, 1:04 pm, Chad Etzel c...@twitter.com wrote: Hi Dewald, I asked The Powers That Be about it, and that was the response I got. However, I am double and triple checking because that does soundtoogoodto betrue:)
[twitter-dev] Re: Rate Limiting Question
With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread/thread/d1664c633972a7c1/9f49c1ad096e9139?lnk=gstq=API+rate+limit#9f49c1ad096e9139 On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread/thread/d1664c633972a7c1/9f49c1ad096e9139?lnk=gstq=API+rate+limit#9f49c1ad096e9139 On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
Hi Inspector Gadget, er... Bob, Yes, the current whitelisted IP rate-limit allows 20k calls per hour *per user* on Basic Auth or OAuth or a combination thereof. Go, go gadget data! -Chad Twitter Platform Support On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishelbobfis...@gmail.com wrote: Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
Good questions. I agree the phrasing surrounding this topic in the documentation is not extremely clear. I am digging for answers. -Chad On Thu, Aug 6, 2009 at 12:44 PM, Jesse Stayjesses...@gmail.com wrote: Chad, did that change recently? I was told by Alex and others there that it was 20,000 calls per hour, period, per IP. When did that change and why weren't we notified? This will save me a lot of money if it is indeed true. Jesse On Thu, Aug 6, 2009 at 12:37 PM, Chad Etzel c...@twitter.com wrote: Hi Inspector Gadget, er... Bob, Yes, the current whitelisted IP rate-limit allows 20k calls per hour *per user* on Basic Auth or OAuth or a combination thereof. Go, go gadget data! -Chad Twitter Platform Support On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishelbobfis...@gmail.com wrote: Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
Chad, Are you 100% sure of that? I mean, in terms of rate limiting that simply does not make sense. For my site, TweetLater.com, it would mean I have an effective hourly rate limit, per IP address, of 2 BILLION IP GET calls per hour! (20,000 per user for 100,000 users). It sounds wrong to me. Dewald On Aug 6, 1:37 pm, Chad Etzel c...@twitter.com wrote: Hi Inspector Gadget, er... Bob, Yes, the current whitelisted IP rate-limit allows 20k calls per hour *per user* on Basic Auth or OAuth or a combination thereof. Go, go gadget data! -Chad Twitter Platform Support On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishelbobfis...@gmail.com wrote: Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
Hi Dewald, I asked The Powers That Be about it, and that was the response I got. However, I am double and triple checking because that does sound too good to be true :) -Chad On Thu, Aug 6, 2009 at 1:01 PM, Dewald Pretoriusdpr...@gmail.com wrote: Chad, Are you 100% sure of that? I mean, in terms of rate limiting that simply does not make sense. For my site, TweetLater.com, it would mean I have an effective hourly rate limit, per IP address, of 2 BILLION IP GET calls per hour! (20,000 per user for 100,000 users). It sounds wrong to me. Dewald On Aug 6, 1:37 pm, Chad Etzel c...@twitter.com wrote: Hi Inspector Gadget, er... Bob, Yes, the current whitelisted IP rate-limit allows 20k calls per hour *per user* on Basic Auth or OAuth or a combination thereof. Go, go gadget data! -Chad Twitter Platform Support On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishelbobfis...@gmail.com wrote: Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
Chad, did that change recently? I was told by Alex and others there that it was 20,000 calls per hour, period, per IP. When did that change and why weren't we notified? This will save me a lot of money if it is indeed true. Jesse On Thu, Aug 6, 2009 at 12:37 PM, Chad Etzel c...@twitter.com wrote: Hi Inspector Gadget, er... Bob, Yes, the current whitelisted IP rate-limit allows 20k calls per hour *per user* on Basic Auth or OAuth or a combination thereof. Go, go gadget data! -Chad Twitter Platform Support On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishelbobfis...@gmail.com wrote: Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
That would be the same as having no rate limit at all, because really, which app would beed to make 20,000 GET calls per hour on one Twitter account? If that's how it is enforced currently, then that is the reason why the API often gets so overloaded and slow. Dewald On Aug 6, 2:04 pm, Chad Etzel c...@twitter.com wrote: Hi Dewald, I asked The Powers That Be about it, and that was the response I got. However, I am double and triple checking because that does sound too good to be true :) -Chad On Thu, Aug 6, 2009 at 1:01 PM, Dewald Pretoriusdpr...@gmail.com wrote: Chad, Are you 100% sure of that? I mean, in terms of rate limiting that simply does not make sense. For my site, TweetLater.com, it would mean I have an effective hourly rate limit, per IP address, of 2 BILLION IP GET calls per hour! (20,000 per user for 100,000 users). It sounds wrong to me. Dewald On Aug 6, 1:37 pm, Chad Etzel c...@twitter.com wrote: Hi Inspector Gadget, er... Bob, Yes, the current whitelisted IP rate-limit allows 20k calls per hour *per user* on Basic Auth or OAuth or a combination thereof. Go, go gadget data! -Chad Twitter Platform Support On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishelbobfis...@gmail.com wrote: Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
Just some background. I talked with Doug about this a few months ago, because I observed in the Rate Limit Header of get calls that the 20,000 number decremented by user, not by IP address in aggregate. Doug informed me that he was going to hand the issue over to Matt, who was on vacation at that point, to look into when he got back from vacation. Doug specifically said that the intended behavior was for the 20,000 rate limit to be by IP address only. So, the point I'm trying to make is, at one point the API did count the 20,000 rate limit per IP address per user, but that was a bug that should have been fixed. I have not checked whether it is actually fixed. But, it's easy to check. Just do a GET call from a whitelisted IP with one user's credentials, check the remaining rate limit number, and then do the same call with another user's credentials. If each call gives you 19,999 remaining, then you know the bug still exists, and consequently no IP rate limiting is currently being done. Dewald On Aug 6, 2:04 pm, Chad Etzel c...@twitter.com wrote: Hi Dewald, I asked The Powers That Be about it, and that was the response I got. However, I am double and triple checking because that does sound too good to be true :) -Chad On Thu, Aug 6, 2009 at 1:01 PM, Dewald Pretoriusdpr...@gmail.com wrote: Chad, Are you 100% sure of that? I mean, in terms of rate limiting that simply does not make sense. For my site, TweetLater.com, it would mean I have an effective hourly rate limit, per IP address, of 2 BILLION IP GET calls per hour! (20,000 per user for 100,000 users). It sounds wrong to me. Dewald On Aug 6, 1:37 pm, Chad Etzel c...@twitter.com wrote: Hi Inspector Gadget, er... Bob, Yes, the current whitelisted IP rate-limit allows 20k calls per hour *per user* on Basic Auth or OAuth or a combination thereof. Go, go gadget data! -Chad Twitter Platform Support On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishelbobfis...@gmail.com wrote: Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the site. Would this limit me to 1 call per minute per user or would it fall over to the user limit of 150 an hour once I hit my 20k? If so how can I tell it has fallen over besides for simply keeping track of the number of calls per hour my server has made. Thanks -Bob
[twitter-dev] Re: Rate Limiting Question
I got the same response from Alex awhile back (and I think confirmed by Doug). And I'm seeing the same results, as well. I'm pretty sure it's 20,000 per IP without regard to user. Jesse On Thu, Aug 6, 2009 at 1:24 PM, Dewald Pretorius dpr...@gmail.com wrote: Just some background. I talked with Doug about this a few months ago, because I observed in the Rate Limit Header of get calls that the 20,000 number decremented by user, not by IP address in aggregate. Doug informed me that he was going to hand the issue over to Matt, who was on vacation at that point, to look into when he got back from vacation. Doug specifically said that the intended behavior was for the 20,000 rate limit to be by IP address only. So, the point I'm trying to make is, at one point the API did count the 20,000 rate limit per IP address per user, but that was a bug that should have been fixed. I have not checked whether it is actually fixed. But, it's easy to check. Just do a GET call from a whitelisted IP with one user's credentials, check the remaining rate limit number, and then do the same call with another user's credentials. If each call gives you 19,999 remaining, then you know the bug still exists, and consequently no IP rate limiting is currently being done. Dewald On Aug 6, 2:04 pm, Chad Etzel c...@twitter.com wrote: Hi Dewald, I asked The Powers That Be about it, and that was the response I got. However, I am double and triple checking because that does sound too good to be true :) -Chad On Thu, Aug 6, 2009 at 1:01 PM, Dewald Pretoriusdpr...@gmail.com wrote: Chad, Are you 100% sure of that? I mean, in terms of rate limiting that simply does not make sense. For my site, TweetLater.com, it would mean I have an effective hourly rate limit, per IP address, of 2 BILLION IP GET calls per hour! (20,000 per user for 100,000 users). It sounds wrong to me. Dewald On Aug 6, 1:37 pm, Chad Etzel c...@twitter.com wrote: Hi Inspector Gadget, er... Bob, Yes, the current whitelisted IP rate-limit allows 20k calls per hour *per user* on Basic Auth or OAuth or a combination thereof. Go, go gadget data! -Chad Twitter Platform Support On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishelbobfis...@gmail.com wrote: Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob On Thu, Aug 6, 2009 at 2:53 AM, srikanth reddysrikanth.yara...@gmail.com wrote: With a whitelisted IP you can make 20k auth calls per hour for each user. Once you reach this limit for a user you cannot make any auth calls from that IP in that duration. But the user can still use his 150 limit from other apps. http://groups.google.com/group/twitter-development-talk/browse_thread... On Thu, Aug 6, 2009 at 7:50 AM, Bob Fishel b...@bobforthejob.com wrote: From the Rate Limiting documentation: IP whitelisting takes precedence to account rate limits. GET requests from a whitelisted IP address made on a user's behalf will be deducted from the whitelisted IP's limit, not the users. Therefore, IP-based whitelisting is a best practice for applications that request many users' data. Say for example I wanted to simply replicate the twitter website. One page per user that just monitors for new statuses with authenticated (to catch protected users) calls to http://twitter.com/statuses/friends_timeline.json Say I was very popular and had 20k people on the
[twitter-dev] Re: Rate Limiting Question
@Dewald Pretorius For my site, TweetLater.com, it would mean I have an effective hourly rate limit, per IP address, of 2 BILLION IP GET calls per hour! I believe 20k limit per user is the desirable behavior, but i don't think twitter will allow you to make infinite calls in which case they will black list you. I have not checked whether it is actually fixed. But, it's easy to check. Just do a GET call from a whitelisted IP with one user's credentials, check the remaining rate limit number, and then do the same call with another user's credentials. If each call gives you 19,999 remaining, then you know the bug still exists, and consequently no IP rate limiting is currently being done. You can verify this here http://twxlate.com This bug was closed very recently (about a month and a half ago) as working as intended http://code.google.com/p/twitter-api/issues/detail?id=617 That would be the same as having no rate limit at all, because really, which app would beed to make 20,000 GET calls per hour on one Twitter account? we dont know the rationale behind that number but if the limit is per IP then your app is easily susceptible to DOS attacks. I believe there are many apps (not whitelisted) out there which make more than 20k calls/hour (150 users /hour and 150 calls) The limit should always be per user(whether IP is whitelisted or not) On Thu, Aug 6, 2009 at 10:54 PM, Dewald Pretorius dpr...@gmail.com wrote: Just some background. I talked with Doug about this a few months ago, because I observed in the Rate Limit Header of get calls that the 20,000 number decremented by user, not by IP address in aggregate. Doug informed me that he was going to hand the issue over to Matt, who was on vacation at that point, to look into when he got back from vacation. Doug specifically said that the intended behavior was for the 20,000 rate limit to be by IP address only. So, the point I'm trying to make is, at one point the API did count the 20,000 rate limit per IP address per user, but that was a bug that should have been fixed. I have not checked whether it is actually fixed. But, it's easy to check. Just do a GET call from a whitelisted IP with one user's credentials, check the remaining rate limit number, and then do the same call with another user's credentials. If each call gives you 19,999 remaining, then you know the bug still exists, and consequently no IP rate limiting is currently being done. Dewald On Aug 6, 2:04 pm, Chad Etzel c...@twitter.com wrote: Hi Dewald, I asked The Powers That Be about it, and that was the response I got. However, I am double and triple checking because that does sound too good to be true :) -Chad On Thu, Aug 6, 2009 at 1:01 PM, Dewald Pretoriusdpr...@gmail.com wrote: Chad, Are you 100% sure of that? I mean, in terms of rate limiting that simply does not make sense. For my site, TweetLater.com, it would mean I have an effective hourly rate limit, per IP address, of 2 BILLION IP GET calls per hour! (20,000 per user for 100,000 users). It sounds wrong to me. Dewald On Aug 6, 1:37 pm, Chad Etzel c...@twitter.com wrote: Hi Inspector Gadget, er... Bob, Yes, the current whitelisted IP rate-limit allows 20k calls per hour *per user* on Basic Auth or OAuth or a combination thereof. Go, go gadget data! -Chad Twitter Platform Support On Thu, Aug 6, 2009 at 12:13 PM, Robert Fishelbobfis...@gmail.com wrote: Well it seems as though Twitter is saying that 20k calls per user is the intended functionality. Chad or someone else can you confirm this? Also if the correct functionality is 20k per ip per hour will you then fail over to 150 per user per hour or is it cut off? Thanks -Bob On Thu, Aug 6, 2009 at 7:54 AM, Dewald Pretoriusdpr...@gmail.com wrote: Bob, Don't base your app on the assumption that it is 20,000 calls per hour per user. You get 20,000 GET calls per whitelisted IP address, period. It does not matter if you use those calls for one Twitter account or 10,000 Twitter accounts. If the API is currently behaving differently, then it is a bug. I have had discussions with Twitter engineers about this, and the intended behavior is an aggregate 20,000 calls per whitelisted IP address as I mentioned above. Dewald On Aug 6, 4:09 am, Robert Fishel bobfis...@gmail.com wrote: Wowzers (bonus points for getting the reference) It appears as if each user does get 20k (according to the linked threads) this is I think what they intended and makes apps a LOT easier to develop as you can now do rate limiting (ie caching and sleeping etc...) based on each user and not on an entire server pool, makes sessions much cleaner. I am whitelisted and I'll test this tomorrow evening to make double sure but this sounds great!. Thanks -Bob
[twitter-dev] Re: Rate limiting message in search
Thanks Paul,
I'll log your message here as an issue and see what I can do. And as
far as WebException is concerned,
you can just cast its Response property to HttpWebResponse rather than
go digging in the header. That's exactly what I do to retrieve the
root's Response object. So that means you can just cast that to
HttpWebResponse and get the StatusCode.
catch(WebException ex)
{
if (ex.Response != null ex.Response is HttpWebResponse)
{
return ex.Response;
}
}
On Mar 3, 1:06 pm, Paul Kinlan paul.kin...@gmail.com wrote:
Thats pretty much where I am handling the 503, my client code intercepts the
exception and then inspects the header. The other thing I noticed, and it
is probably not best on this list is that you use WebRequest which raises a
WebException, and you can't get the 503 out of it easily (at least from what
I understand), where as HttpWebRequest raises HttpWebException which you can
directly check for a 503 error.
Anyway, I really enjoy using Tweet# and if any .Net devs out there need a
.Net Twitter library this is the one I recommend.
Paul
2009/3/3 Dimebrain daniel.cre...@gmail.com
Thanks for the feedback; right now you can get at the response in
instance.Root.Response (where instance is your FluentTwitter query),
which will give you the instance of the last response returned. I'll
look at this closer (unless you have a patch already of course).
Daniel
On Mar 3, 11:28 am, Paul Kinlan paul.kin...@gmail.com wrote:
Hi Daniel,
I am using tweet# a lot, and it would be good if you catch the 503 error
status on the rate limited requests (including the Retry-After header in
the
response), I have had to implement it in tweet# for our product.
Kind Regards,
Paul
2009/3/3 Dimebrain daniel.cre...@gmail.com
I have experienced sending search requests out which return a plain
string, rather than JSON representing a twitter error. It's this:
You have been rate limited. Enhance your calm.
a) What is the rate limiting based on, IP or client? What is the
limit? I develop a Twitter library (tweetsharp) and by default I send
the tweet# credentials along with the call. If this means that anyone
using my library will be rate limited because of that header
information, I need to know so I can force my users to provide their
own credentials so that the library isn't unusable in this area, and
b) Can we get his as XML, JSON and not a plain string?
[twitter-dev] Re: Rate limiting message in search
Hi Paul, What is tweet#? Can you let us know more about it? On Tue, Mar 3, 2009 at 2:28 PM, Paul Kinlan paul.kin...@gmail.com wrote: Hi Daniel, I am using tweet# a lot, and it would be good if you catch the 503 error status on the rate limited requests (including the Retry-After header in the response), I have had to implement it in tweet# for our product. Kind Regards, Paul 2009/3/3 Dimebrain daniel.cre...@gmail.com I have experienced sending search requests out which return a plain string, rather than JSON representing a twitter error. It's this: You have been rate limited. Enhance your calm. a) What is the rate limiting based on, IP or client? What is the limit? I develop a Twitter library (tweetsharp) and by default I send the tweet# credentials along with the call. If this means that anyone using my library will be rate limited because of that header information, I need to know so I can force my users to provide their own credentials so that the library isn't unusable in this area, and b) Can we get his as XML, JSON and not a plain string? -- Sincerely, Burhan Tanweer www.explorewww.com expl...@explorewww.com
[twitter-dev] Re: Rate limiting message in search
Hi Burhan, Tweet# is a .Net twitter client API. It has been developed in a fluent interface style so you construct your twitter requests in a manner that you can read from left to right. For example I use it to search: var result = FluentTwitter.CreateRequest().Search().Query().Containing(\exeter city\).Since(last_id).Return(10).Request(); It Reads: Create a Request of type Search using a Query Containing exeter city Since the last id returning up to 10 results. It is on google code http://code.google.com/p/tweetsharp/ Kind Regards, Paul Kinlan 2009/3/3 Burhan TANWEER btanw...@gmail.com Hi Paul, What is tweet#? Can you let us know more about it? On Tue, Mar 3, 2009 at 2:28 PM, Paul Kinlan paul.kin...@gmail.com wrote: Hi Daniel, I am using tweet# a lot, and it would be good if you catch the 503 error status on the rate limited requests (including the Retry-After header in the response), I have had to implement it in tweet# for our product. Kind Regards, Paul 2009/3/3 Dimebrain daniel.cre...@gmail.com I have experienced sending search requests out which return a plain string, rather than JSON representing a twitter error. It's this: You have been rate limited. Enhance your calm. a) What is the rate limiting based on, IP or client? What is the limit? I develop a Twitter library (tweetsharp) and by default I send the tweet# credentials along with the call. If this means that anyone using my library will be rate limited because of that header information, I need to know so I can force my users to provide their own credentials so that the library isn't unusable in this area, and b) Can we get his as XML, JSON and not a plain string? -- Sincerely, Burhan Tanweer www.explorewww.com expl...@explorewww.com
[twitter-dev] Re: Rate limiting message in search
Thanks for the feedback; right now you can get at the response in instance.Root.Response (where instance is your FluentTwitter query), which will give you the instance of the last response returned. I'll look at this closer (unless you have a patch already of course). Daniel On Mar 3, 11:28 am, Paul Kinlan paul.kin...@gmail.com wrote: Hi Daniel, I am using tweet# a lot, and it would be good if you catch the 503 error status on the rate limited requests (including the Retry-After header in the response), I have had to implement it in tweet# for our product. Kind Regards, Paul 2009/3/3 Dimebrain daniel.cre...@gmail.com I have experienced sending search requests out which return a plain string, rather than JSON representing a twitter error. It's this: You have been rate limited. Enhance your calm. a) What is the rate limiting based on, IP or client? What is the limit? I develop a Twitter library (tweetsharp) and by default I send the tweet# credentials along with the call. If this means that anyone using my library will be rate limited because of that header information, I need to know so I can force my users to provide their own credentials so that the library isn't unusable in this area, and b) Can we get his as XML, JSON and not a plain string?
[twitter-dev] Re: Rate limiting message in search
Thats pretty much where I am handling the 503, my client code intercepts the exception and then inspects the header. The other thing I noticed, and it is probably not best on this list is that you use WebRequest which raises a WebException, and you can't get the 503 out of it easily (at least from what I understand), where as HttpWebRequest raises HttpWebException which you can directly check for a 503 error. Anyway, I really enjoy using Tweet# and if any .Net devs out there need a .Net Twitter library this is the one I recommend. Paul 2009/3/3 Dimebrain daniel.cre...@gmail.com Thanks for the feedback; right now you can get at the response in instance.Root.Response (where instance is your FluentTwitter query), which will give you the instance of the last response returned. I'll look at this closer (unless you have a patch already of course). Daniel On Mar 3, 11:28 am, Paul Kinlan paul.kin...@gmail.com wrote: Hi Daniel, I am using tweet# a lot, and it would be good if you catch the 503 error status on the rate limited requests (including the Retry-After header in the response), I have had to implement it in tweet# for our product. Kind Regards, Paul 2009/3/3 Dimebrain daniel.cre...@gmail.com I have experienced sending search requests out which return a plain string, rather than JSON representing a twitter error. It's this: You have been rate limited. Enhance your calm. a) What is the rate limiting based on, IP or client? What is the limit? I develop a Twitter library (tweetsharp) and by default I send the tweet# credentials along with the call. If this means that anyone using my library will be rate limited because of that header information, I need to know so I can force my users to provide their own credentials so that the library isn't unusable in this area, and b) Can we get his as XML, JSON and not a plain string?
