[twitter-dev] Re: Questions about opening the firehose
BTW, I see that applications that authenticate with oAuth are going to get a 10X increase in the number of API calls they can make per hour. When does that go into effect? On Dec 20, 3:55 pm, M. Edward (Ed) Borasky zzn...@gmail.com wrote: On Dec 18, 9:23 am, Raffi Krikorian ra...@twitter.com wrote: Will the geo-hose streaming API return only those tweets which are geo- tagged, or will it also return tweets for users whose location (in profile) falls within the lat, lon specified the geo-hose will only return tweets that are geotagged. while the volume may be low (comparably) now, its definitely growing. Given the realities of social networking, media, privacy, word-of- mouth, attention spans, user laziness, etc., those people who want users to geo-tag will need to offer those users incentive to do so, for example, discounts on goods and services. Or you could do what Facebook did - make geo-tagging the default and then send out a complicated set of instructions on how to make your location private again. ;-) -- Raffi Krikorian Twitter Platform Teamhttp://twitter.com/raffi
[twitter-dev] Re: Questions about opening the firehose
On Dec 18, 9:23 am, Raffi Krikorian ra...@twitter.com wrote: Will the geo-hose streaming API return only those tweets which are geo- tagged, or will it also return tweets for users whose location (in profile) falls within the lat, lon specified the geo-hose will only return tweets that are geotagged. while the volume may be low (comparably) now, its definitely growing. Given the realities of social networking, media, privacy, word-of- mouth, attention spans, user laziness, etc., those people who want users to geo-tag will need to offer those users incentive to do so, for example, discounts on goods and services. Or you could do what Facebook did - make geo-tagging the default and then send out a complicated set of instructions on how to make your location private again. ;-) -- Raffi Krikorian Twitter Platform Teamhttp://twitter.com/raffi
[twitter-dev] Re: Questions about opening the firehose
Hi Raffi, Will the geo-hose streaming API return only those tweets which are geo- tagged, or will it also return tweets for users whose location (in profile) falls within the lat, lon specified. If it returns only tweets that are geo-tagged, I guess the volume of returned tweets will be pretty low for now. /Amitab Twaller.com (@mytwaller) On Dec 17, 9:19 pm, Raffi Krikorian ra...@twitter.com wrote: For one thing, I do a lot of location-based processing. I'm quite interested in what's happening in Portland, Oregon, and not so much about the rest of the world. As far as I can tell, there's no geocode parameter for filter. In addition, I can do a search back in time with Twitter search - with filter, if I don't know what I'm looking for, it's going to go right by me. ;-) the geo-hose will be eventually available to help specifically with this feature -http://www.slideshare.net/raffikrikorian/whats-happening-here. -- Raffi Krikorian Twitter Platform Teamhttp://twitter.com/raffi
Re: [twitter-dev] Re: Questions about opening the firehose
Will the geo-hose streaming API return only those tweets which are geo- tagged, or will it also return tweets for users whose location (in profile) falls within the lat, lon specified the geo-hose will only return tweets that are geotagged. while the volume may be low (comparably) now, its definitely growing. -- Raffi Krikorian Twitter Platform Team http://twitter.com/raffi
[twitter-dev] Re: Questions about opening the firehose
On Dec 15, 9:58 am, John Kalucki j...@twitter.com wrote: Bandwidth is likely to only be a small fraction of your total cost when consuming the firehose. If you want to focus on this small part and ignore all the other dominating costs, the prudent systems engineer would provision 2x to 3x daily peak to account for traffic spikes, growth, backlog retrieval, and to keep latency to a minimum. Not all have such requirements, though. So, somewhere between 5 and 15 mbit, very very roughly. Your requirements will certainly vary. The filtered and sampled streams are where virtually everyone will wind up. -John Kaluckihttp://twitter.com/jkalucki Services, Twitter Inc. I'm using the sampled stream at the moment and it's doing most of what I need. It's certainly more than enough for development and testing the algorithms. The filter stream, on the other hand, seems next to useless to me when compared with the stream coming out of Twitter search. For one thing, I do a lot of location-based processing. I'm quite interested in what's happening in Portland, Oregon, and not so much about the rest of the world. As far as I can tell, there's no geocode parameter for filter. In addition, I can do a search back in time with Twitter search - with filter, if I don't know what I'm looking for, it's going to go right by me. ;-) But really, I'm much more concerned about legal issues with the firehose than I am with technical issues. There are resellers of firehose data now. They have an advantage over random developers like myself, because they have a business relationship with Twitter and I don't. I can't make a credible business plan without knowing what I will and will not be able to legally do with firehose data, or how much it will cost me for access. -- M. Edward (Ed) Borasky http://borasky-research.net I've always regarded nature as the clothing of God. ~Alan Hovhaness
Re: [twitter-dev] Re: Questions about opening the firehose
For one thing, I do a lot of location-based processing. I'm quite interested in what's happening in Portland, Oregon, and not so much about the rest of the world. As far as I can tell, there's no geocode parameter for filter. In addition, I can do a search back in time with Twitter search - with filter, if I don't know what I'm looking for, it's going to go right by me. ;-) the geo-hose will be eventually available to help specifically with this feature - http://www.slideshare.net/raffikrikorian/whats-happening-here. -- Raffi Krikorian Twitter Platform Team http://twitter.com/raffi
[twitter-dev] Re: Questions about opening the firehose
Excellent! That's exactly what I need! If something gets past the filter, I can always backsearch for it. On Dec 17, 9:19 pm, Raffi Krikorian ra...@twitter.com wrote: For one thing, I do a lot of location-based processing. I'm quite interested in what's happening in Portland, Oregon, and not so much about the rest of the world. As far as I can tell, there's no geocode parameter for filter. In addition, I can do a search back in time with Twitter search - with filter, if I don't know what I'm looking for, it's going to go right by me. ;-) the geo-hose will be eventually available to help specifically with this feature -http://www.slideshare.net/raffikrikorian/whats-happening-here. -- Raffi Krikorian Twitter Platform Teamhttp://twitter.com/raffi
Re: [twitter-dev] Re: Questions about opening the firehose
The other levels of Streaming access are not only considerably more cost effective for all parties, they are also (nearly) sufficient for the vast majority of applications. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Mon, Dec 14, 2009 at 10:16 PM, M. Edward (Ed) Borasky zzn...@gmail.comwrote: Thanks!! At this point, I'm not sure I'll be using the firehose even if it is available -I don't think I can afford the pipe width to consume it. ;-) On Dec 14, 9:59 pm, John Kalucki j...@twitter.com wrote: There will be further announcements about Streaming API access early next year. -John Kaluckihttp://twitter.com/jkalucki Services, Twitter Inc. On Mon, Dec 14, 2009 at 9:09 PM, M. Edward (Ed) Borasky zzn...@gmail.comwrote: Last week at Le Web, Twitter's Platform Director, Ryan Sarver, announced that Twitter will be opening the firehose to all developers. As I recall, there were a number of reasons why Twitter kept the firehose restricted. Some of these were legal reasons. I'm starting to put together an action plan for 2010, and I'm really curious - what has changed legally since then that would allow Twitter to open the firehose to all developers? What legal agreements / licenses / contracts must a developer commit to in order to gain access to the firehose? -- M. Edward (Ed) Borasky http://borasky-research.net I've always regarded nature as the clothing of God. ~Alan Hovhaness
Re: [twitter-dev] Re: Questions about opening the firehose
Just so we all can guestimate if we're equiped for and financially able to consider consumption of the firehose, in average, what's the daily data throughput on a firehose stream? Thanks, Michael. On Dec 15, 2009, at 9:38 AM, John Kalucki j...@twitter.com wrote: The other levels of Streaming access are not only considerably more cost effective for all parties, they are also (nearly) sufficient for the vast majority of applications. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Mon, Dec 14, 2009 at 10:16 PM, M. Edward (Ed) Borasky zzn...@gmail.com wrote: Thanks!! At this point, I'm not sure I'll be using the firehose even if it is available -I don't think I can afford the pipe width to consume it. ;-) On Dec 14, 9:59 pm, John Kalucki j...@twitter.com wrote: There will be further announcements about Streaming API access early next year. -John Kaluckihttp://twitter.com/jkalucki Services, Twitter Inc. On Mon, Dec 14, 2009 at 9:09 PM, M. Edward (Ed) Borasky zzn...@gmail.comwrote: Last week at Le Web, Twitter's Platform Director, Ryan Sarver, announced that Twitter will be opening the firehose to all developers. As I recall, there were a number of reasons why Twitter kept the firehose restricted. Some of these were legal reasons. I'm starting to put together an action plan for 2010, and I'm really curious - what has changed legally since then that would allow Twitter to open the firehose to all developers? What legal agreements / licenses / contracts must a developer commit to in order to gain access to the firehose? -- M. Edward (Ed) Borasky http://borasky-research.net I've always regarded nature as the clothing of God. ~Alan Hovhaness
Re: [twitter-dev] Re: Questions about opening the firehose
Bandwidth is likely to only be a small fraction of your total cost when consuming the firehose. If you want to focus on this small part and ignore all the other dominating costs, the prudent systems engineer would provision 2x to 3x daily peak to account for traffic spikes, growth, backlog retrieval, and to keep latency to a minimum. Not all have such requirements, though. So, somewhere between 5 and 15 mbit, very very roughly. Your requirements will certainly vary. The filtered and sampled streams are where virtually everyone will wind up. -John Kalucki http://twitter.com/jkalucki Services, Twitter Inc. On Tue, Dec 15, 2009 at 9:45 AM, Michael Steuer mste...@gmail.com wrote: Just so we all can guestimate if we're equiped for and financially able to consider consumption of the firehose, in average, what's the daily data throughput on a firehose stream? Thanks, Michael. On Dec 15, 2009, at 9:38 AM, John Kalucki j...@twitter.com wrote: The other levels of Streaming access are not only considerably more cost effective for all parties, they are also (nearly) sufficient for the vast majority of applications. -John Kalucki http://twitter.com/jkaluckihttp://twitter.com/jkalucki Services, Twitter Inc. On Mon, Dec 14, 2009 at 10:16 PM, M. Edward (Ed) Borasky zzn...@gmail.com zzn...@gmail.com wrote: Thanks!! At this point, I'm not sure I'll be using the firehose even if it is available -I don't think I can afford the pipe width to consume it. ;-) On Dec 14, 9:59 pm, John Kalucki j...@twitter.com wrote: There will be further announcements about Streaming API access early next year. -John Kaluckihttp:// http://twitter.com/jkaluckitwitter.com/jkalucki Services, Twitter Inc. On Mon, Dec 14, 2009 at 9:09 PM, M. Edward (Ed) Borasky zzn...@gmail.comwrote: Last week at Le Web, Twitter's Platform Director, Ryan Sarver, announced that Twitter will be opening the firehose to all developers. As I recall, there were a number of reasons why Twitter kept the firehose restricted. Some of these were legal reasons. I'm starting to put together an action plan for 2010, and I'm really curious - what has changed legally since then that would allow Twitter to open the firehose to all developers? What legal agreements / licenses / contracts must a developer commit to in order to gain access to the firehose? -- M. Edward (Ed) Borasky http://borasky-research.nethttp://borasky-research.net I've always regarded nature as the clothing of God. ~Alan Hovhaness
[twitter-dev] Re: Questions about opening the firehose
Thanks!! At this point, I'm not sure I'll be using the firehose even if it is available -I don't think I can afford the pipe width to consume it. ;-) On Dec 14, 9:59 pm, John Kalucki j...@twitter.com wrote: There will be further announcements about Streaming API access early next year. -John Kaluckihttp://twitter.com/jkalucki Services, Twitter Inc. On Mon, Dec 14, 2009 at 9:09 PM, M. Edward (Ed) Borasky zzn...@gmail.comwrote: Last week at Le Web, Twitter's Platform Director, Ryan Sarver, announced that Twitter will be opening the firehose to all developers. As I recall, there were a number of reasons why Twitter kept the firehose restricted. Some of these were legal reasons. I'm starting to put together an action plan for 2010, and I'm really curious - what has changed legally since then that would allow Twitter to open the firehose to all developers? What legal agreements / licenses / contracts must a developer commit to in order to gain access to the firehose? -- M. Edward (Ed) Borasky http://borasky-research.net I've always regarded nature as the clothing of God. ~Alan Hovhaness
