Hi, I have two separate consumer groups on different JVM processes, but both have the same "group.id". They are high level consumer groups with each group containing 3 consumers.
Only one group consumes at a given time - and I would like both groups, with the same id to share the load and common offsets. Jiangjie - are you saying I need to run both consumer groups on completely separate JVMs on different hosts? -----Original Message----- From: Mayuresh Gharat [mailto:gharatmayures...@gmail.com] Sent: 09 March 2015 17:07 To: users@kafka.apache.org Subject: Re: Multiple consumer groups with same group id on a single topic If you have 2 consumer groups, each group will read from all partitions automaticcally if you are using HighLevel consumer ( In your case it would be each consumer gets 2 partitons). You don't have to specify the partitions it should read from. Thanks, Mayuresh On Mon, Mar 9, 2015 at 9:59 AM, Jiangjie Qin <j...@linkedin.com.invalid> wrote: > HI Phill, > > Do you mean you are using 6 consumers with the same group id? Or you > have > 3 consumers using one group id, and another 3 using another different > group id? > For the example you mentioned, what you can do is to run several > consumers on different physical machine with the same group id, they > will balance the partition among themselves if you are using high level > consumer. > > Jiangjie (Becket) Qin > > On 3/9/15, 1:27 AM, "Phill Tomlinson" <philltomlin...@fico.com> wrote: > > >Hi, > > > >I have a topic with 6 partitions. I have two consumer groups with 3 > >consumers each, both with the same group.id. However only one group > >appears to consume from the topic. > > > >Is this expected behaviour? I would expect to be able to concurrently > >use two consumer groups on the same topic to provide better > >throughput across multiple nodes (if I wanted 100 partitions for a > >topic for example I don't want 100 threads running on a single 4 core > >processor for example). > >It would be ideal if each consumer group decided which partitions > >they would read from and auto-balance between them. Currently it just > >waits in the background and will only consume if the other group fails. > > > >Thanks, > >Phill > > > >This email and any files transmitted with it are confidential, > >proprietary and intended solely for the individual or entity to whom > >they are addressed. If you have received this email in error please > >delete it immediately. > > -- -Regards, Mayuresh R. Gharat (862) 250-7125 This email and any files transmitted with it are confidential, proprietary and intended solely for the individual or entity to whom they are addressed. If you have received this email in error please delete it immediately.
