There is no way to receive every status. Private statuses, for example, are not distributed.
The only sanctioned way to receive any substantial proportion of public statuses in real time is via the Streaming API. You can also receive a non-representative non-random, real-time sample, which may make your results worthless, by hitting the Search API. All other approaches will be rate limited or considered scraping. Scraping is frowned upon and you'll be quickly detected and blocked. I'd suggest recasting your project to the sample available in the gardenhose. -John Kalucki twitter.com/jkalucki Services, Twitter Inc. On Jul 17, 4:57 am, CreativeEye <creativv...@gmail.com> wrote: > Myself and my friend are doing a research based on twitter. We need to > analyse each and every tweet real time. Can you guide how to approach > this. > > There could be 2 ways of doing this (without Firehose): > > 1) Get Twitter Public timeline repeatedly. > > Thankfully Twitter's caching has not been problem to me, they seem to > fetch me new data every request. But there are a lot of limitation for > this: > > According to TweeSpeed.com: > - Rate of New tweets in the Twitter Server is right now (Wed Jul 17 > 11:47:02 - GMT) at 9233 tweets/minute. > - Ranges between 7K to 20K on an average Weekday. > - On June 26 (MJ's death) - reached 25K tweets/minute. > > Let us now consider the limitation of API requests per hour. > - Currently @ 20K per hour. > - 1 Req = 20 Tweets > - Need 1K Req per minute = 60K req per hour. > > To Use 1K Requests per minute, we should be using around 17 requests > per second. But my server is able to process only 28-33 requests/ > minute. > > Is this the right way to proceed, or am I fundamentally wrong on the > approach. > > 2) Get follower network - user profiles and get their statuses. > Frequency of request their new status updates could be set against > their general update frequency. But this is Google-like old way of > indexing things, which does not quite stand today in the REAL TIME > twitter. > > I do know Firehose is an option, but that would again be something > like Approach 1. right? > > Please guide me how to proceed.
