Re: kafka simpleconsumer question
Hi Guozhang, Just a follow up question. Is one simpleconsumer instance can be shared by multiple threads and fetch data from multiple topics and commit offsets ? It seems all the implementation are synchronized which means it can be shared between multiple threads ? Is my understanding correct ? Thanks, Weide On Tue, Jul 1, 2014 at 5:36 PM, Guozhang Wang wangg...@gmail.com wrote: Hi Weide, 1. The old consumer is still depending on the ZK no matter if you use ZK for offsets or not, since it depends on ZK for group manage anyways. In the new consumer (0.9) we have completely the ZK dependance. 2. Yes, it queries the broker for leader info. 3. You can use a single consumer for multiple topics, but a single consumer can only talk to a single broker; if your topic spans across multiple brokers you would also need multiple simple consumers. Guozhang On Tue, Jul 1, 2014 at 5:14 PM, Weide Zhang weo...@gmail.com wrote: Hi , Just want to ask some basic question about kafka simple consumer. 1. if I'm using simple consumer and doesn't really depend on zookeeper to manage partition offset. (application manage offset themselves). Will that remove the zookeeper dependency for consumer ? 2. if zookeeper dies, will simple consumer still able to get partition leadership information from broker itself ? Or it indirectly uses zookeeper to find out the partition leadership for particular topics ? 3. does simpleconsumer api have topicfilter logic similar to high level consumer provides ? Or I have to create multiple simpleconsumer for multiple topics themselves explicitly ? Thanks a lot, Weide -- -- Guozhang
Re: kafka simpleconsumer question
Yes it can be shared. Guozhang On Thu, Jul 10, 2014 at 11:12 AM, Weide Zhang weo...@gmail.com wrote: Hi Guozhang, Just a follow up question. Is one simpleconsumer instance can be shared by multiple threads and fetch data from multiple topics and commit offsets ? It seems all the implementation are synchronized which means it can be shared between multiple threads ? Is my understanding correct ? Thanks, Weide On Tue, Jul 1, 2014 at 5:36 PM, Guozhang Wang wangg...@gmail.com wrote: Hi Weide, 1. The old consumer is still depending on the ZK no matter if you use ZK for offsets or not, since it depends on ZK for group manage anyways. In the new consumer (0.9) we have completely the ZK dependance. 2. Yes, it queries the broker for leader info. 3. You can use a single consumer for multiple topics, but a single consumer can only talk to a single broker; if your topic spans across multiple brokers you would also need multiple simple consumers. Guozhang On Tue, Jul 1, 2014 at 5:14 PM, Weide Zhang weo...@gmail.com wrote: Hi , Just want to ask some basic question about kafka simple consumer. 1. if I'm using simple consumer and doesn't really depend on zookeeper to manage partition offset. (application manage offset themselves). Will that remove the zookeeper dependency for consumer ? 2. if zookeeper dies, will simple consumer still able to get partition leadership information from broker itself ? Or it indirectly uses zookeeper to find out the partition leadership for particular topics ? 3. does simpleconsumer api have topicfilter logic similar to high level consumer provides ? Or I have to create multiple simpleconsumer for multiple topics themselves explicitly ? Thanks a lot, Weide -- -- Guozhang -- -- Guozhang
Re: kafka simpleconsumer question
Hi Weide, 1. The old consumer is still depending on the ZK no matter if you use ZK for offsets or not, since it depends on ZK for group manage anyways. In the new consumer (0.9) we have completely the ZK dependance. 2. Yes, it queries the broker for leader info. 3. You can use a single consumer for multiple topics, but a single consumer can only talk to a single broker; if your topic spans across multiple brokers you would also need multiple simple consumers. Guozhang On Tue, Jul 1, 2014 at 5:14 PM, Weide Zhang weo...@gmail.com wrote: Hi , Just want to ask some basic question about kafka simple consumer. 1. if I'm using simple consumer and doesn't really depend on zookeeper to manage partition offset. (application manage offset themselves). Will that remove the zookeeper dependency for consumer ? 2. if zookeeper dies, will simple consumer still able to get partition leadership information from broker itself ? Or it indirectly uses zookeeper to find out the partition leadership for particular topics ? 3. does simpleconsumer api have topicfilter logic similar to high level consumer provides ? Or I have to create multiple simpleconsumer for multiple topics themselves explicitly ? Thanks a lot, Weide -- -- Guozhang
