Re: OutOfMemoryError in mirror maker

2015-06-27 Thread Guozhang Wang
Tao, I have added you to the contributor list of Kafka so you can assign tickets to yourself now. I will review the patch soon. Guozhang On Thu, Jun 25, 2015 at 2:54 AM, tao xiao wrote: > Patch updated. please review > > On Mon, 22 Jun 2015 at 12:24 tao xiao wrote: > > > Yes, you are right. W

Re: zookeeper bottleneck in kafka operations

2015-06-27 Thread Jason Gustafson
Hi Shushant, Write throughput on zoookeeper can be a problem depending on your commit policy. Typically you can handle this by just committing less frequently (with the obvious tradeoff). The consumer also supports storing offsets in Kafka itself through the "offsets.storage" option (see http://ka

zookeeper bottleneck in kafka operations

2015-06-27 Thread Shushant Arora
Hi Is zookeeper not fit for high writes? since write goes via leader node then how kafka maintains offsets using zookeeper update operation very efficiently. If there are a lot of subscribers you may feel a squeeze though. zookeeper is also not strictly consistent then how its guranteed cosumer