Re: How to commit offset every time, automatically

2013-10-27 Thread Cheolwoo Choi
Thank you for your reply. The 2nd one is what I want. I has tested with it, and it works fine. Thanks. I hope it supports 'consumerConnector.gracefulShutdown()' to finalize a connector gracefully (with some user properties). :-) On Mon, Oct 28, 2013 at 10:16 AM, Philip O'Toole wrote: > You

Re: How to commit offset every time, automatically

2013-10-27 Thread Philip O'Toole
You have two choices. -- Do what you say, and write your own consumer, based on the SimpleConsumer. Handle all commits, ZK accesses, and balancing yourself. -- Use a ConsumerConnector for every partition, and call commitOffsets() explicitly when you have processed a message. This does a commit fo

How to commit offset every time, automatically

2013-10-27 Thread Cheolwoo Choi
Hi, all I'm using kafka-0.7.2 I'd like to commit offset every time reading a message block. "autocommit.interval.ms" property is not enough. To do that, should I use SimpleConsumer, and write a commit logic manually? (i.e. zkUtils.commitOffset(groupId, messageAndOffset.offset() ) Is there no way