[twitter-dev] Re: Streaming API Rate Limiting

2011-04-01 Thread Colin Surprenant
for the whole US seems small. Colin On Apr 1, 2:40 pm, Augusto Santos wrote: > Sorry Colin, but where did you get this information? Doesn't match with the > reality. Not at all. > > On Fri, Apr 1, 2011 at 12:35 PM, Colin Surprenant < > > > > > > > > > >

[twitter-dev] Re: Streaming API Rate Limiting

2011-04-01 Thread Colin Surprenant
As a side note, currently only 3-4% of the total tweets (firehose) are geo-tagged and are eligible to be selected in a stream location bounding box. If the current firehose rate is about 140M tweets/day, that makes ~5M eligible tweets/day. I do not know what the proportion of tweets from the US is

[twitter-dev] Re: Search API rate limit change?

2011-03-21 Thread Colin Surprenant
By adjusting the rate limits to reduce the stress on your search api without notice you have significantly increased the stress level on our end :P Seriously, advanced notice of the situation would have been welcome. In particular what created lots of confusion on our end is that even after pausin

[twitter-dev] Re: should search and streaming apis return similar tweets for equivalent geolocation areas

2011-02-24 Thread Colin Surprenant
my context but it is not possible to get this using the streaming API. - Can we expect both methods to be supported in the future and can we expect to get a streaming version of what the search API does today? Thanks, Colin On Feb 15, 2:19 pm, Colin Surprenant wrote: > So basically today we h

[twitter-dev] Re: should search and streaming apis return similar tweets for equivalent geolocation areas

2011-02-15 Thread Colin Surprenant
I did ran tests on keyword search and found only very marginal differences between polling the search api and using the streaming api. I also followed up in your thread. Colin On Feb 15, 2:39 pm, Karussell wrote: > Hmmh, would you mind to test this without the geo location filter? > And report y

[twitter-dev] Re: Streaming API vs. Search API: no API returns >95% of intented tweets

2011-02-15 Thread Colin Surprenant
First your test set is a bit small. Did you take into account the extra data you will get in your first search api poll? Typically your first poll will return 100 items then subsequent polls will return only "new" data if using since_id and/or dedupping. Make sure both your poller and stream reade

[twitter-dev] Re: should search and streaming apis return similar tweets for equivalent geolocation areas

2011-02-15 Thread Colin Surprenant
So basically today we have two options for geo search: - use the search api and get results that will include some incorrectly geolocalized tweets when falling back on the user location field. - use the streaming api and retrieve significantly far less tweets but with a higher degree of confidence

[twitter-dev] Re: should search and streaming apis return similar tweets for equivalent geolocation areas

2011-02-12 Thread Colin Surprenant
,-121.992630991336,38.0961869903135 which correspond to the bounding box around 37.736784, -122.44709, 40km. Why is there such huge difference and can we expect the streaming API to eventually match what the search API produces for geolocalized searches? Thanks, Colin On Feb 10, 5:32 pm, Colin Surprenant wrote: > Hi, &g

[twitter-dev] should search and streaming apis return similar tweets for equivalent geolocation areas

2011-02-10 Thread Colin Surprenant
Hi, I have been running some tests to gather tweets from users within a geo area using both the search API (with the geocode parameter) and the streaming API (with the statuses/filter method & locations parameter). I have noticed that the streaming API returns far less tweets for an equivalent ar

[twitter-dev] Re: language and geocode problem

2010-11-30 Thread Colin Surprenant
Currently it looks like any geo based search queries are returning zero result. ex, on New York: http://search.twitter.com/search.json?geocode=40.739454,-73.883743,75km The question is: what is the expected timeframe for a fix on this? hours, days, weeks? They only say "Engineering working on a f

[twitter-dev] Re: statuses/filter streaming api vs Gnip announcement

2010-11-22 Thread Colin Surprenant
bump - anyone? On Nov 17, 4:16 pm, Colin Surprenant wrote: > Hi, > > - does the usage of the statuses/filter method on the streaming api > impacted by the Gnip announcement? > > - do we know the maximum rate (or approximation) allowed through the > statuses/filter method? (i

[twitter-dev] statuses/filter streaming api vs Gnip announcement

2010-11-17 Thread Colin Surprenant
Hi, - does the usage of the statuses/filter method on the streaming api impacted by the Gnip announcement? - do we know the maximum rate (or approximation) allowed through the statuses/filter method? (incidentally, at which point in terms of rate we have to consider firehosing, therefore, Gnip op