Why are you writing your own PHP Streaming API client? It's somewhat complicated to get all the details right. The Phirehose library seems to be well received, and it's incorporated lots of fixes for corner cases.
-John Kalucki http://twitter.com/jkalucki Twitter, Inc. On Tue, Sep 21, 2010 at 6:12 AM, Nicolas Grasset <nicolas.gras...@gmail.com>wrote: > Thanks for the quick reply! > > I went back to testing with command line with a curl call, it worked > so I realized the PHP code was buffering and I did not realize it on > large amount of results... > > The problem was with an internal buffer between cURL and the PHP > Stream. I went another way instead and posting a PHP base class if it > may help anyone else. > > I typically extend it to override handleTweet($tweet). > > > <?php > > class snow_twitter_bot { > > protected $buffer; > > > /** > * Read a stream looking for track keywords on behalf of a user > * > * > * @param string $track Comma separated > values keywords > * @param string $twitter_user Twitter username > * @param string $twitter_pass Twitter password > */ > public function readStreamFilter( $track, $twitter_user, > $twitter_pass ) > { > > $wait = 10; > while( true ) > { > > echo "\n" . date("c") . " - Connecting to Twitter > Stream\n"; > > // Configuration of curl > $ch = curl_init(); > curl_setopt($ch, CURLOPT_URL, > "http://stream.twitter.com/1/ > statuses/filter.json"); > curl_setopt($ch, CURLOPT_HEADER, > 0); > curl_setopt($ch, CURLOPT_RETURNTRANSFER, > "default"); > curl_setopt($ch, CURLOPT_POST, > true); > curl_setopt($ch, CURLOPT_POSTFIELDS, > "track=$track"); > curl_setopt($ch, CURLOPT_HTTPAUTH, > CURLAUTH_BASIC ) ; > curl_setopt($ch, CURLOPT_USERPWD, > "$twitter_user: > $twitter_pass"); > > curl_setopt($ch, CURLOPT_WRITEFUNCTION, > array($this, > "receiveResponse")); > > curl_exec($ch); > > $info = curl_getinfo($ch); > > if( $info['http_code'] == 200 ) > $wait = 10; > else > { > echo "\n" . date("c") . " - ERROR > ($info[http_code]) - Waiting > $wait sec \n"; > sleep ( $wait ); > $wait *= ($wait >= 240 ) ? 1 : 2; > } > } > > > curl_close($ch); > } > > > /** > * Call back function for cURL > * > * > * @param curl handler $curlHandle > * @param string $data > */ > private function receiveResponse( $curlHandle, $data ) > { > $lines = explode("\n", $data); > > // The buffer contains the end of the last line from previous > time > // => Is goes at the beginning of the first line we are > getting this time > $lines[0] = $this->buffer . $lines[0]; > > // And the last line os only partial > // => save it for next time, and remove it from the list this > time > $nb_lines = count($lines); > $this->buffer = $lines[$nb_lines-1]; > unset($lines[$nb_lines-1]); > > // Here, do your work with the lines you have in the buffer > foreach( $lines as $one ) > { > $tweet = @json_decode($one,1); > if(isset($tweet['user']['screen_name'])) > { > $this->handleTweet( $tweet ); > } > } > > return strlen( $data ); > } > > > > /** > * Called for each tweet. > * > * Override me with you logic /twss > * > * > * @param array $tweet A single tweet parsed from json, see online > doc. > */ > public function handleTweet( $tweet ) > { > echo '@'.$tweet['user']['screen_name']." at " . > $tweet['created_at']. " says:\n"; > echo $tweet['text']."\n"; > echo "\n------ Memory: " . round(memory_get_usage(true)/ > 1024) ."kb\n"; > } > > > } > > > ?> > > > On Sep 21, 12:57 am, John Kalucki <j...@twitter.com> wrote: > > You say you are backing off on HTTP connections. Are you connecting > > constantly, or just once, and staying connected for very long periods? > Are > > you tracking the number of limit messages you are receiving? Perhaps your > > track search was rate limited? > > > > -John Kaluckihttp://twitter.com/jkalucki > > Twitter, Inc. > > > > On Mon, Sep 20, 2010 at 2:46 PM, Nicolas Grasset > > <nicolas.gras...@gmail.com>wrote: > > > > > > > > > Hi there, > > > > > I have been reading many comments here, and while it seems at least > > > another person had the same issue, I could not find an answer. > > > > > I have a test account (@hemtexdream) using the statuses/filter stream > > > api which works on words like twitter or trending hashtags like > > > #ireallythink. > > > > > However, I would like it to work with track=#dreamproject and it gets > > > none of the tweets from my main account (@fellowshipofone). Tried over > > > long period of time and with a few settings. I am fairly sure I am not > > > rate limited since I implement the logic to report http error > > > responses and back off exponentially. > > > > > Also, if of any help, both accounts can find at least 2 tweets in the > > > website search page. > > > > > Is it an index question? Is it a "sample" question? > > > > > Thanks for your help! > > > > > -- > > > Twitter developer documentation and resources: > http://dev.twitter.com/doc > > > API updates via Twitter:http://twitter.com/twitterapi > > > Issues/Enhancements Tracker: > > >http://code.google.com/p/twitter-api/issues/list > > > Change your membership to this group: > > >http://groups.google.com/group/twitter-development-talk?hl=en > > -- > Twitter developer documentation and resources: http://dev.twitter.com/doc > API updates via Twitter: http://twitter.com/twitterapi > Issues/Enhancements Tracker: > http://code.google.com/p/twitter-api/issues/list > Change your membership to this group: > http://groups.google.com/group/twitter-development-talk?hl=en > -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk?hl=en
