Re: Kafka wiki Documentation conventions - looking for feedback

2013-05-01 Thread Chris Curtin
I've tested my examples with the new (4/30) release and they work, so I've updated the documentation. Thanks, Chris On Mon, Apr 29, 2013 at 6:18 PM, Jun Rao jun...@gmail.com wrote: Thanks. I also updated your producer example to reflect a recent config change (broker.list =

Re: Kafka wiki Documentation conventions - looking for feedback

2013-05-01 Thread Jun Rao
Chris, Thanks. This is very helpful. I linked your wiki pages to our website. A few more comments: 1. Producer: The details of the meaning of request.required.acks are described in http://kafka.apache.org/08/configuration.html. It would be great if you can add a link to the description in your

Re: Kafka wiki Documentation conventions - looking for feedback

2013-05-01 Thread Chris Curtin
Hi Jun I've added #1 and #2. I'll need to think about where to put #3, maybe even adding a 'tips and tricks' section? I've not had to do any encoder/decoders. Can anyone else offer some example code I can incorporate into an example? Thanks, Chris On Wed, May 1, 2013 at 11:45 AM, Jun Rao

Re: Kafka wiki Documentation conventions - looking for feedback

2013-05-01 Thread Jun Rao
The following are sample encoder/decoder in java. class StringEncode implements EncoderString { private String encoding = null; public StringEncoder(VerifiableProperties props) { if(props == null) encoding = UTF8; else encoding = props.getString(serializer.encoding,

Re: Kafka wiki Documentation conventions - looking for feedback

2013-04-29 Thread Jun Rao
Chris, Thanks for the writeup. Looks great overall. A couple of comments. 1. At the beginning, it sounds like that one can't run multiple processes of consumers in the same group. This is actually not true. We can create multiple instances of consumers for the same group in the same JVM or

Re: Kafka wiki Documentation conventions - looking for feedback

2013-04-29 Thread Chris Curtin
Jun, can you explain this a little better? I thought when using Consumer Groups that on startup Kafka connects to ZooKeeper and finds the last read offset for every partition in the topic being requested for the group. That is then the starting point for the consumer threads. If a second process

Re: Kafka wiki Documentation conventions - looking for feedback

2013-04-29 Thread Chris Curtin
Thanks, I missed that the addition of consumers can cause a re-balance. Thought it was only on Leader changes. I've updated the wording in the example. I'll pull down the beta and test my application then change the names on the properties. Thanks, Chris On Mon, Apr 29, 2013 at 11:58 AM, Jun

Re: Kafka wiki Documentation conventions - looking for feedback

2013-04-29 Thread Jun Rao
Thanks. I also updated your producer example to reflect a recent config change (broker.list = metadata.broker.list). Jun On Mon, Apr 29, 2013 at 11:19 AM, Chris Curtin curtin.ch...@gmail.comwrote: Thanks, I missed that the addition of consumers can cause a re-balance. Thought it was only on

Re: Kafka wiki Documentation conventions - looking for feedback

2013-04-28 Thread Jun Rao
Chris, Any update of the high level consumer example? Also, in the Producer example, it would be useful to describe how to write a customized encoder. One subtle thing is that the encoder needs a constructor that takes a a single VerifiableProperties argument (

Re: Kafka wiki Documentation conventions - looking for feedback

2013-04-22 Thread Jun Rao
Chris, Thanks for the wiki. We are getting close to releasing 0.8.0 beta and your writeup is very helpful. The following are some comments for the 0.8 Producer wiki. 1. The following sentence is inaccurate. The producer will do random assignment as long as the key in KeyedMessage is null. If a

Re: Kafka wiki Documentation conventions - looking for feedback

2013-04-22 Thread Chris Curtin
Hi Jun, #1 and #2 are done, thanks for the code-review! I'll work on getting a High Level consumer example this week. I don't have one readily usable (we quickly found the lack of control over offsets didn't meet our needs) but I can get something this week. Congratulations on getting closer to

Re: Kafka wiki Documentation conventions - looking for feedback

2013-03-29 Thread Chris Curtin
Hi, I've added an example program for using a SimpleConsumer for 0.8.0. Turns out to be a little more complicated once you add Broker failover. I'm not 100% thrilled with how I detect and recover, so if someone has a better way of doing this please let me (and this list) know.

Kafka wiki Documentation conventions - looking for feedback

2013-03-25 Thread Chris Curtin
Hi David, Thanks for the feedback. I've seen the example before and after in different books/articles and it doesn't matter to me. Anyone else want to help define a style guide or is there one I didn't see already? Thanks, Chris On Thu, Mar 21, 2013 at 7:46 PM, David Arthur mum...@gmail.com