Re: [twitter-dev] Streaming API limits...
John, Thank you for getting back to me. The doc lists a "For example, reconnect no more than twice every four minutes, or three times per six minutes, or some similar metric." but doesn't give a "Don't reconnect more than a few times a minute unless you are retrying automatically due to failures, but remember to use exponential backoff to a reasonable upper limit depending on the type of failure." ;) (also, mistype earlier: "nor does it get connected by the server" should have read "nor does it get disconnected by the server") The funny thing is, I've actually discarded the client that I was using, and have written one from scratch using Python's asynchronous sockets. Every other time I've written async socket clients/servers, when a connection is closed, the socket will be seen as "readable" but when read will return zero bytes; this is a semantic that has worked for me every other time in the past, is the standard behavior for countless async (and sync libraries), and is the most reliable cross-platform way to detect socket disconnect. Worst-case scenario, the socket will come up as being in an error condition (I handle that too, but it hasn't come up when connecting to stream.twitter.com). Looking at a wireshark dump, I see some sort of disconnect-like packet, but the socket is never seen as readable or in an error condition according to select (Python wraps the standard system routines), and both lsof AND netstat see the sockets as being "established" (as opposed to disconnected, time_wait, etc.) I have worked around the seeming lack of connect, I was just reporting what I was experiencing. Regards, - Josiah On Sunday, February 27, 2011 9:56:04 PM UTC-8, John Kalucki wrote: > > This is documented in painful detail here: > http://dev.twitter.com/pages/streaming_api_concepts#updating-filter-predicates > . > > If you connect a second time, you should get a TCP Close or Reset on the > first connection. It sounds like your client library isn't detecting the > connection close. > > -John Kalucki > http://twitter.com/jkalucki > Twitter, Inc. > > > On Thu, Feb 24, 2011 at 1:08 PM, Josiah Carlson wrote: > >> Now that I've got OAuth with statuses/follow.json working, I've been >> working through building a small part of our app. >> >> Part of the streaming API docs state that only one connection is allowed >> (reasonable). Upon making a second connection, the first no longer receives >> any data (not even anti-timeout newlines), nor does it get connected by the >> server. On my end of things, I've written an async client which can detect >> such a condition (it watches a shared Redis key looking for a changed state >> when it doesn't receive any data for a while), and automatically >> disconnects. >> >> The streaming API docs also state that repeated reconnections, etc., are >> frowned upon and may result in banning. >> >> My question is simple: how often can I reconnect to follow different >> people/keywords? Obviously ten times a second is well beyond reasonable and >> would probably get us banned in seconds. But isat most once every 5 minutes >> okay? At most once every minute? At what level would we be safe? >> >> Thank you, >> - Josiah >> >> -- >> Twitter developer documentation and resources: http://dev.twitter.com/doc >> API updates via Twitter: http://twitter.com/twitterapi >> Issues/Enhancements Tracker: >> http://code.google.com/p/twitter-api/issues/list >> Change your membership to this group: >> http://groups.google.com/group/twitter-development-talk >> > > -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API limits...
This is documented in painful detail here: http://dev.twitter.com/pages/streaming_api_concepts#updating-filter-predicates . If you connect a second time, you should get a TCP Close or Reset on the first connection. It sounds like your client library isn't detecting the connection close. -John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Thu, Feb 24, 2011 at 1:08 PM, Josiah Carlson wrote: > Now that I've got OAuth with statuses/follow.json working, I've been > working through building a small part of our app. > > Part of the streaming API docs state that only one connection is allowed > (reasonable). Upon making a second connection, the first no longer receives > any data (not even anti-timeout newlines), nor does it get connected by the > server. On my end of things, I've written an async client which can detect > such a condition (it watches a shared Redis key looking for a changed state > when it doesn't receive any data for a while), and automatically > disconnects. > > The streaming API docs also state that repeated reconnections, etc., are > frowned upon and may result in banning. > > My question is simple: how often can I reconnect to follow different > people/keywords? Obviously ten times a second is well beyond reasonable and > would probably get us banned in seconds. But isat most once every 5 minutes > okay? At most once every minute? At what level would we be safe? > > Thank you, > - Josiah > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk > -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
[twitter-dev] Streaming API limits...
Now that I've got OAuth with statuses/follow.json working, I've been working through building a small part of our app. Part of the streaming API docs state that only one connection is allowed (reasonable). Upon making a second connection, the first no longer receives any data (not even anti-timeout newlines), nor does it get connected by the server. On my end of things, I've written an async client which can detect such a condition (it watches a shared Redis key looking for a changed state when it doesn't receive any data for a while), and automatically disconnects. The streaming API docs also state that repeated reconnections, etc., are frowned upon and may result in banning. My question is simple: how often can I reconnect to follow different people/keywords? Obviously ten times a second is well beyond reasonable and would probably get us banned in seconds. But isat most once every 5 minutes okay? At most once every minute? At what level would we be safe? Thank you, - Josiah -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
Re: [twitter-dev] Streaming API limits
Hi Thiago, We allow each account to have one standing connection to the streaming API. Multiple connections receive the same stream of Tweets so connecting multiple times would not give access to any more Tweets. If you need more to have more track keywords you can contact us explaining your use case so the team can make a decision and if necessary grant you higher access. You can read more about access and rate limiting on the streaming API in our developer documentation: http://dev.twitter.com/pages/streaming_api_concepts#access-rate-limiting Hope that helps, Matt On Mon, Aug 16, 2010 at 10:05 AM, Tom van der Woerdt wrote: > On 8/16/10 4:38 PM, Thiago Souza wrote: > > Hi, > > > > I know that it's possible to track up to 200 keywords per account, > > but how many accounts per IP is allowed? > > > > Regards, > > Thiago Souza > > Hi Thiago, > > I don't think that Twitter will give you these numbers. Just don't > create more connections than necessary, and if you need more track > keywords, you should consider asking Twitter for an upgrade of your > account. > > Tom > -- Matt Harris Developer Advocate, Twitter http://twitter.com/themattharris
Re: [twitter-dev] Streaming API limits
On 8/16/10 4:38 PM, Thiago Souza wrote: > Hi, > > I know that it's possible to track up to 200 keywords per account, > but how many accounts per IP is allowed? > > Regards, > Thiago Souza Hi Thiago, I don't think that Twitter will give you these numbers. Just don't create more connections than necessary, and if you need more track keywords, you should consider asking Twitter for an upgrade of your account. Tom
[twitter-dev] Streaming API limits
Hi, I know that it's possible to track up to 200 keywords per account, but how many accounts per IP is allowed? Regards, Thiago Souza
