Hi Onur, It was in my previous email. But here it is again.
============================================================ 1. Better rebalance timing. We will try to rebalance only when all the consumers in a group have joined. The challenge would be someone has to define what does ALL consumers mean, it could either be a time or number of consumers, etc. 2. Avoid frequent rebalance. For example, if there are 100 consumers in a group, today, in the worst case, we may end up with 100 rebalances even if all the consumers joined the group in a reasonably small amount of time. Frequent rebalance is also a bad thing for brokers. Having a client side configuration may solve problem 1 better because each consumer group can potentially configure their own timing. However, it does not really prevent frequent rebalance in general because some of the consumers can be misconfigured. (This may have something to do with KIP-124 as well. But if quota is applied on the JoinGroup/SyncGroup request it may cause some unwanted cascading effects.) Having a broker side configuration may result in less flexibility for each consumer group, but it can prevent frequent rebalance better. I think with some reasonable design, the rebalance timing issue can be resolved on the broker side as well. Matthias had a good point on extending the delay when a new consumer joins a group (we actually did something similar to batch ISR change propagation). For example, let's say on the broker side, we will always delay 2 seconds each time we see a new consumer joining a consumer group. This would probably work for most of the consumer groups and will also limit the rebalance frequency to protect the brokers. I am not sure about the streams use case here, but if something like 2 seconds of delay is acceptable for streams, I would prefer adding the configuration to the broker so that we can address both problems. On Thu, 6 Apr 2017 at 17:11 Onur Karaman <onurkaraman.apa...@gmail.com> wrote: > Hi Damian. > > Can you copy the point Becket made earlier that you say isn't addressed? > > On Thu, Apr 6, 2017 at 2:51 AM, Damian Guy <damian....@gmail.com> wrote: > > > Thanks all, the Vote is now closed and the KIP has been accepted with 9 > +1s > > > > 3 binding:: > > Guozhang, > > Jason, > > Ismael > > > > 6 non-binding: > > Bill, > > Eno, > > Mathieu, > > Matthias, > > Dong, > > Mickael > > > > Thanks, > > Damian > > > > On Thu, 6 Apr 2017 at 09:26 Ismael Juma <ism...@juma.me.uk> wrote: > > > > > Thanks for the KIP, +1 (binding). > > > > > > Ismael > > > > > > On Thu, Mar 30, 2017 at 8:55 PM, Jason Gustafson <ja...@confluent.io> > > > wrote: > > > > > > > +1 Thanks for the KIP! > > > > > > > > On Thu, Mar 30, 2017 at 12:51 PM, Guozhang Wang <wangg...@gmail.com> > > > > wrote: > > > > > > > > > +1 > > > > > > > > > > Sorry about the previous email, Gmail seems be collapsing them > into a > > > > > single thread on my inbox. > > > > > > > > > > Guozhang > > > > > > > > > > On Thu, Mar 30, 2017 at 11:34 AM, Guozhang Wang < > wangg...@gmail.com> > > > > > wrote: > > > > > > > > > > > Damian, could you create a new thread for the voting process? > > > > > > > > > > > > Thanks! > > > > > > > > > > > > Guozhang > > > > > > > > > > > > On Thu, Mar 30, 2017 at 10:33 AM, Bill Bejeck <bbej...@gmail.com > > > > > > wrote: > > > > > > > > > > > >> +1(non-binding) > > > > > >> > > > > > >> On Thu, Mar 30, 2017 at 1:30 PM, Eno Thereska < > > > eno.there...@gmail.com > > > > > > > > > > >> wrote: > > > > > >> > > > > > >> > +1 (non binding) > > > > > >> > > > > > > >> > Thanks > > > > > >> > Eno > > > > > >> > > On 30 Mar 2017, at 18:01, Matthias J. Sax < > > > matth...@confluent.io> > > > > > >> wrote: > > > > > >> > > > > > > > >> > > +1 > > > > > >> > > > > > > > >> > > On 3/30/17 3:46 AM, Damian Guy wrote: > > > > > >> > >> Hi All, > > > > > >> > >> > > > > > >> > >> I'd like to start the voting thread on KIP-134: > > > > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > > >> > 134%3A+Delay+initial+consumer+group+rebalance > > > > > >> > >> > > > > > >> > >> Thanks, > > > > > >> > >> Damian > > > > > >> > >> > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > -- Guozhang > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > -- Guozhang > > > > > > > > > > > > > > >
