Kafka at ApacheCon this week

2013-02-25 Thread Jun Rao
Hi, I will be giving a Kafka replication talk at ApacheCon Tue afternoon. http://na.apachecon.com/schedule/presentation/115/ We also plan to have a Kafka meetup Tue at 8pm. The meetup doesn't require conference registration. http://wiki.apache.org/apachecon/ApacheMeetupsNA13 Hope to see some of

Re: Kafka at ApacheCon this week

2013-02-25 Thread Vaibhav Puranik
Hi Jun, This meetup is in Portland if I understand it correctly. Is there a plan to have a meet up in the bay area soon? We are a LA based company. I would like to attend if there is any meetup coming up soon in San Francisco bay area. Replication/Upgrading the existing cluster to 0.8 is what I a

Re: Kafka at ApacheCon this week

2013-02-25 Thread Neha Narkhede
Vaibhav, We do plan to do one in the SF area once 0.8 is released. It could be held in April or so. Thanks, Neha On Mon, Feb 25, 2013 at 11:24 AM, Vaibhav Puranik wrote: > Hi Jun, > > This meetup is in Portland if I understand it correctly. Is there a plan to > have a meet up in the bay area s

Re: 'simple' partitioning in 0.8

2013-02-25 Thread Neha Narkhede
Chris, Partitioning is independent of serialization. The key data is serialized using the serializer specified through the "key.serializer.class" property in the Producer. It defaults to a no-op encoder if you don't specify one. Here, since you want to use an integer key, you'd have to plug in an

Question about function SimpleConsumer.getOffsetsBefore

2013-02-25 Thread Sining Ma
I have a question about function getOffsetsBefore() in class SimpleConsumer. I am not sure about the input value of maxNumOffsets. If I set this value to a small number, I cannot get enough valid Offset values. If I set to a big number like 10,000, I think this function will consume a lot of time

Re: Kafka at ApacheCon this week

2013-02-25 Thread Vaibhav Puranik
Thanks Neha. I will definitely attend it. Regards, Vaibhav On Mon, Feb 25, 2013 at 11:27 AM, Neha Narkhede wrote: > Vaibhav, > > We do plan to do one in the SF area once 0.8 is released. It could be held > in April or so. > > Thanks, > Neha > > > On Mon, Feb 25, 2013 at 11:24 AM, Vaibhav Puranik

Kafka talk tomorrow night in Madison WI

2013-02-25 Thread Matthew Rathbone
Hey all, If any of you are in the midwest I'll get giving a talk on Kafka at the BigDataMadison meetup [1] in Madison WI. Stop by if you're around, there will be pizza :-). It's at 6pm tomorrow (feb 26th). [1]: http://www.meetup.com/BigDataMadison/ -- Matthew Rathbone Foursquare | Software Eng

Re: Question about function SimpleConsumer.getOffsetsBefore

2013-02-25 Thread Neha Narkhede
The value of maxNumOffsets depends on what you are trying to accomplish. In most cases, this value is 1. For a value > 1, it gives you the first offset of the segments since the segment that matched your offset query. Even if you set a very high value, it will not have those many segments and hence

Re: Question about function SimpleConsumer.getOffsetsBefore

2013-02-25 Thread Sining Ma
If right now this function can return many offsets, will it take long time? Based on your experience, what is the best value for maxNumOffsets? Thanks, Sining Ma On Mon, Feb 25, 2013 at 5:24 PM, Neha Narkhede wrote: > The value of maxNumOffsets depends on what you are trying to accomplish. In