Thanks Jason for the comment! I answered it on the discuss thread. Folks, could we continue the vote for this KIP? This is a very critical improvement for our streaming system stability and we need to get things rolling right at the start of 2019.
Thank you for your time! Boyang ________________________________ From: Jason Gustafson <ja...@confluent.io> Sent: Tuesday, December 18, 2018 7:40 AM To: dev Subject: Re: [VOTE] KIP-345: Introduce static membership protocol to reduce consumer rebalances Hi Boyang, Thanks, the KIP looks good. Just one comment. The new schema for the LeaveGroup request is slightly odd since it is handling both the single consumer use case and the administrative use case. I wonder we could make it consistent from a batching perspective. In other words, instead of this: LeaveGroupRequest => GroupId MemberId [GroupInstanceId] Maybe we could do this: LeaveGroupRequest => GroupId [GroupInstanceId MemberId] For dynamic members, GroupInstanceId could be empty, which is consistent with JoinGroup. What do you think? Also, just for clarification, what is the expected behavior if the current memberId of a static member is passed to LeaveGroup? Will the static member be removed? I know the consumer will not do this, but we'll still have to handle the case on the broker. Best, Jason On Mon, Dec 10, 2018 at 11:54 PM Boyang Chen <bche...@outlook.com> wrote: > Thanks Stanislav！ > > Get Outlook for iOS<https://aka.ms/o0ukef> > > ________________________________ > From: Stanislav Kozlovski <stanis...@confluent.io> > Sent: Monday, December 10, 2018 11:28 PM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-345: Introduce static membership protocol to > reduce consumer rebalances > > This is great work, Boyang. Thank you very much. > > +1 (non-binding) > > On Mon, Dec 10, 2018 at 6:09 PM Boyang Chen <bche...@outlook.com> wrote: > > > Hey there, could I get more votes on this thread? > > > > Thanks for the vote from Mayuresh and Mike :) > > > > Best, > > Boyang > > ________________________________ > > From: Mayuresh Gharat <gharatmayures...@gmail.com> > > Sent: Thursday, December 6, 2018 10:53 AM > > To: dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-345: Introduce static membership protocol to > > reduce consumer rebalances > > > > +1 (non-binding) > > > > Thanks, > > > > Mayuresh > > > > On Tue, Dec 4, 2018 at 6:58 AM Mike Freyberger < > mike.freyber...@xandr.com> > > wrote: > > > > > +1 (non binding) > > > > > > On 12/4/18, 9:43 AM, "Patrick Williams" < > patrick.willi...@storageos.com > > > > > > wrote: > > > > > > Pls take me off this VOTE list > > > > > > Best, > > > > > > Patrick Williams > > > > > > Sales Manager, UK & Ireland, Nordics & Israel > > > StorageOS > > > +44 (0)7549 676279 > > > patrick.willi...@storageos.com > > > > > > 20 Midtown > > > 20 Proctor Street > > > Holborn > > > London WC1V 6NX > > > > > > Twitter: @patch37 > > > LinkedIn: linkedin.com/in/patrickwilliams4 < > > > > > > https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Flinkedin.com%2Fin%2Fpatrickwilliams4&data=02%7C01%7C%7C9b12ec4ce9ae4454db8a08d65f3a4862%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636801101252994092&sdata=ipDTX%2FGARrFkwZfRuOY0M5m3iJ%2Bnkxovv6u9bBDaTyc%3D&reserved=0 > > > > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fslack.storageos.com%2F&data=02%7C01%7C%7C9b12ec4ce9ae4454db8a08d65f3a4862%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636801101252994092&sdata=hxuKU6aZdQU%2FpxpqaaThR6IjpEmwIP5%2F3NhYzMYijkw%3D&reserved=0 > > > > > > > > > > > > On 03/12/2018, 17:34, "Guozhang Wang" <wangg...@gmail.com> wrote: > > > > > > Hello Boyang, > > > > > > I've browsed through the new wiki and there are still a couple of > > > minor > > > things to notice: > > > > > > 1. RemoveMemberFromGroupOptions seems not defined anywhere. > > > > > > 2. LeaveGroupRequest added a list of group instance id, but still > > > keep the > > > member id as a singleton; is that intentional? I think to make > > the > > > protocol > > > consistent both member id and instance ids could be plural. > > > > > > 3. About the *kafka-remove-member-from-group.sh *tool, I'm > > > wondering if we > > > can defer adding this while just add the corresponding calls of > > the > > > LeaveGroupRequest inside Streams until we have used it in > > > production and > > > hence have a better understanding on how flexible or extensible > > if > > > we want > > > to add any cmd tools. The rationale is that if we do not > > > necessarily need > > > it now, we can always add it later with a more think-through API > > > design, > > > but if we add the tool in a rush, we may need to extend or modify > > > it soon > > > after we realize its limits in operations. > > > > > > Otherwise, I'm +1 on the proposal. > > > > > > Guozhang > > > > > > > > > On Mon, Dec 3, 2018 at 9:14 AM Boyang Chen <bche...@outlook.com> > > > wrote: > > > > > > > Hey community friends, > > > > > > > > after another month of polishing, KIP-345< > > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-345%253A%2BIntroduce%2Bstatic%2Bmembership%2Bprotocol%2Bto%2Breduce%2Bconsumer%2Brebalances&data=02%7C01%7C%7C9b12ec4ce9ae4454db8a08d65f3a4862%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636801101252994092&sdata=T4i7L1i0nIeHrrjTeLOOgYKsfzfNEMGDhTazvBEZbXw%3D&reserved=0 > > > > > > > > design is ready for vote. Feel free to add your comment on the > > > discussion > > > > thread or here. > > > > > > > > Thanks for your time! > > > > > > > > Boyang > > > > ________________________________ > > > > From: Boyang Chen <bche...@outlook.com> > > > > Sent: Friday, November 9, 2018 6:35 AM > > > > To: dev@kafka.apache.org > > > > Subject: [VOTE] KIP-345: Introduce static membership protocol > > to > > > reduce > > > > consumer rebalances > > > > > > > > Hey all, > > > > > > > > > > > > thanks so much for all the inputs on KIP-345 so far. The > > > original proposal > > > > has enhanced a lot with your help. To make sure the > > > implementation go > > > > smoothly without back and forth, I would like to start a vote > > on > > > the final > > > > design agreement now: > > > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-&data=02%7C01%7C%7C9b12ec4ce9ae4454db8a08d65f3a4862%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636801101252994092&sdata=g4%2BMXKpkiQLZXg5HJWfJhw1kc1PbDNwyiX9zkREVqGE%3D&reserved=0 > > < > > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-345%253A%2BIntroduce%2Bstatic%2Bmembership%2Bprotocol%2Bto%2Breduce%2Bconsumer%2Brebalances&data=02%7C01%7C%7C9b12ec4ce9ae4454db8a08d65f3a4862%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636801101252994092&sdata=T4i7L1i0nIeHrrjTeLOOgYKsfzfNEMGDhTazvBEZbXw%3D&reserved=0 > > > > > > > > > > > > > > > > > > > 345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances< > > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-345%253A%2BIntroduce%2Bstatic%2Bmembership%2Bprotocol%2Bto%2Breduce%2Bconsumer%2Brebalances&data=02%7C01%7C%7C9b12ec4ce9ae4454db8a08d65f3a4862%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636801101252994092&sdata=T4i7L1i0nIeHrrjTeLOOgYKsfzfNEMGDhTazvBEZbXw%3D&reserved=0 > > > > > > > > > > > > > KIP-345: Introduce static membership protocol to reduce ...< > > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-345%253A%2BIntroduce%2Bstatic%2Bmembership%2Bprotocol%2Bto%2Breduce%2Bconsumer%2Brebalances&data=02%7C01%7C%7C9b12ec4ce9ae4454db8a08d65f3a4862%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636801101252994092&sdata=T4i7L1i0nIeHrrjTeLOOgYKsfzfNEMGDhTazvBEZbXw%3D&reserved=0 > > > > > > > > > cwiki.apache.org > > > > For stateful applications, one of the biggest performance > > > bottleneck is > > > > the state shuffling. In Kafka consumer, there is a concept > > called > > > > "rebalance" which means that for given M partitions and N > > > consumers in one > > > > consumer group, Kafka will try to balance the load between > > > consumers and > > > > ideally have ... > > > > > > > > > > > > Let me know if you have any questions. > > > > > > > > > > > > Best, > > > > > > > > Boyang > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > > > > > > > > > > > > > > -- > > -Regards, > > Mayuresh R. Gharat > > (862) 250-7125 > > > > > -- > Best, > Stanislav >
