Hi Neha, Is moving to zookeeper 3.4.x is a big change ?.
Can you please explain parts it affects consumer API for example ?. Thanks, Balaji -----Original Message----- From: Neha Narkhede [mailto:neha.narkh...@gmail.com] Sent: Friday, May 17, 2013 7:35 AM To: users@kafka.apache.org Subject: RE: are commitOffsets botched to zookeeper? Upgrading to a new zookeeper version is not an easy change. Also zookeeper 3.3.4 is much more stable compared to 3.4.x. We think it is better not to club 2 big changes together. So most likely this will be a post 08 item for stability purposes. Thanks, Neha On May 17, 2013 6:31 AM, "Withers, Robert" <robert.with...@dish.com> wrote: > Awesome! Thanks for the clarification. I would like to offer my strong > vote that this get tackled before a beta, to get it firmly into 0.8. > Stabilize everything else to the existing use, but make offset updates > batched. > > thanks, > rob > ________________________________________ > From: Neha Narkhede [neha.narkh...@gmail.com] > Sent: Friday, May 17, 2013 7:17 AM > To: users@kafka.apache.org > Subject: RE: are commitOffsets botched to zookeeper? > > Sorry I wasn't clear. Zookeeper 3.4.x has this feature. As soon as 08 is > stable and released it will be worth looking into when we can use zookeeper > 3.4.x. > > Thanks, > Neha > On May 16, 2013 10:32 PM, "Rob Withers" <reefed...@gmail.com> wrote: > > > Can a request be made to zookeeper for this feature? > > > > Thanks, > > rob > > > > > -----Original Message----- > > > From: Neha Narkhede [mailto:neha.narkh...@gmail.com] > > > Sent: Thursday, May 16, 2013 9:53 PM > > > To: users@kafka.apache.org > > > Subject: Re: are commitOffsets botched to zookeeper? > > > > > > Currently Kafka depends on zookeeper 3.3.4 that doesn't have a batch > > write > > > api. So if you commit after every message at a high rate, it will be > slow > > and > > > inefficient. Besides it will cause zookeeper performance to degrade. > > > > > > Thanks, > > > Neha > > > On May 16, 2013 6:54 PM, "Rob Withers" <reefed...@gmail.com> wrote: > > > > > > > We are calling commitOffsets after every message consumption. It > > > > looks to be ~60% slower, with 29 partitions. If a single KafkaStream > > > > thread is from a connector, and there are 29 partitions, then > > > > commitOffsets sends 29 offset updates, correct? Are these offset > > > > updates batched in one send to zookeeper? > > > > > > > > thanks, > > > > rob > > > > >
