Re: multiple queue consumers on one partition

2013-08-14 Thread Wang Guozhang
Hello Jiang, In Kafka parallelism is achieve through partitioning, meaning you can specify n partitions for a topic and m (<=n) consumers each consuming from one or more partitions. Guozhang On Wed, Aug 14, 2013 at 12:59 PM, Wu, Jiang2 wrote: > Hi, > > In our experiments, we find that if multi

Re: multiple queue consumers on one partition

2013-08-14 Thread Eric Sites
Correction: So if you have n consumers in the same group you need n partitions per topic. On 8/15/13 12:14 AM, "Eric Sites" wrote: >Yes, that is the expected. > > >You need more partitions per topic, at lest 1 partition per consumer in >the group. > >So if you have n consumers you need n topics.

Re: multiple queue consumers on one partition

2013-08-14 Thread Eric Sites
Yes, that is the expected. You need more partitions per topic, at lest 1 partition per consumer in the group. So if you have n consumers you need n topics. All messages sent to the topic will be divided amongst the partitions. Cheers, Eric Sites On 8/14/13 3:59 PM, "Wu, Jiang2 " wrote: >Hi,

multiple queue consumers on one partition

2013-08-14 Thread Wu, Jiang2
Hi, In our experiments, we find that if multiple consumers in the same group listen to the same partition, then one consumer will receive all messages on this partition, and others get none. Is this inherent to Kafka design, or it can be changed by some configuration? Basically we expect ems qu