If you have 2 consumer groups, each group will read from all partitions automaticcally if you are using HighLevel consumer ( In your case it would be each consumer gets 2 partitons). You don't have to specify the partitions it should read from.
Thanks, Mayuresh On Mon, Mar 9, 2015 at 9:59 AM, Jiangjie Qin <j...@linkedin.com.invalid> wrote: > HI Phill, > > Do you mean you are using 6 consumers with the same group id? Or you have > 3 consumers using one group id, and another 3 using another different > group id? > For the example you mentioned, what you can do is to run several consumers > on different physical machine with the same group id, they will balance > the partition among themselves if you are using high level consumer. > > Jiangjie (Becket) Qin > > On 3/9/15, 1:27 AM, "Phill Tomlinson" <philltomlin...@fico.com> wrote: > > >Hi, > > > >I have a topic with 6 partitions. I have two consumer groups with 3 > >consumers each, both with the same group.id. However only one group > >appears to consume from the topic. > > > >Is this expected behaviour? I would expect to be able to concurrently use > >two consumer groups on the same topic to provide better throughput across > >multiple nodes (if I wanted 100 partitions for a topic for example I > >don't want 100 threads running on a single 4 core processor for example). > >It would be ideal if each consumer group decided which partitions they > >would read from and auto-balance between them. Currently it just waits in > >the background and will only consume if the other group fails. > > > >Thanks, > >Phill > > > >This email and any files transmitted with it are confidential, > >proprietary and intended solely for the individual or entity to whom they > >are addressed. If you have received this email in error please delete it > >immediately. > > -- -Regards, Mayuresh R. Gharat (862) 250-7125
