回复:回复:回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove members in StreamsResetter

2020-05-29 Thread feyman2009
Hi, team
I updated the KIP-571 since we took a slightly different implementation 
in the PR https://github.com/apache/kafka/pull/8589, basically:
In RemoveMembersFromConsumerGroupOptions, leveraging empty members 
rather than introducing a new field to imply the removeAll scenario.
   Please let me know if you have any concerns, thanks a lot!

Feyman


--
发件人:feyman2009 
发送时间:2020年4月13日(星期一) 08:47
收件人:dev 
主 题:回复:回复:回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove members in 
StreamsResetter

Thanks , John and Guochang!
--
发件人:Guozhang Wang 
发送时间:2020年4月11日(星期六) 03:07
收件人:dev 
主 题:Re: 回复:回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove members in 
StreamsResetter

Thanks Feyman,

I've looked at the update that you incorporated from Matthias and that LGTM
too. I'm still +1 :)

Guozhang

On Fri, Apr 10, 2020 at 11:18 AM John Roesler  wrote:

> Hey Feyman,
>
> Just to remove any ambiguity, I've been casually following the discussion,
> I've just looked at the KIP document again, and I'm still +1 (binding).
>
> Thanks,
> -John
>
> On Fri, Apr 10, 2020, at 01:44, feyman2009 wrote:
> > Hi, all
> > KIP-571 has already collected 4 bind +1 (John, Guochang, Bill,
> > Matthias) and 3 non-binding +1(Boyang, Sophie), I will mark it as
> > approved and create a PR shortly.
> > Thanks!
> >
> > Feyman
> > --
> > 发件人:feyman2009 
> > 发送时间:2020年4月8日(星期三) 14:21
> > 收件人:dev ; Boyang Chen 
> > 主 题:回复:回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove
> > members in StreamsResetter
> >
> > Hi Boyang,
> > Thanks for reminding me of that!
> > I'm not sure about the convention, I thought it would need to
> > re-collect votes if the KIP has changed~
> > Let's leave the vote thread here for 2 days, if no objection, I
> > will take it as approved and update the PR accordingly.
> >
> > Thanks!
> > Feyman
> >
> >
> >
> > ------------------
> > 发件人:Boyang Chen 
> > 发送时间:2020年4月8日(星期三) 12:42
> > 收件人:dev ; feyman2009 
> > 主 题:Re: 回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove
> > members in StreamsResetter
> >
> > You should already get enough votes if I'm counting correctly
> > (Guozhang, John, Matthias)
> > On Tue, Apr 7, 2020 at 6:59 PM feyman2009
> >  wrote:
> > Hi, Boyang
> >  I think Matthias's proposal makes sense, but we can use the admin
> > tool for this scenario as Boyang mentioned or follow up later, so I
> > prefer to keep this KIP unchanged to minimize the scope.
> >  Calling for vote ~
> >
> >  Thanks!
> >  Feyman
> >
> >  --
> >  发件人:Boyang Chen 
> >  发送时间:2020年4月8日(星期三) 02:15
> >  收件人:dev 
> >  主 题:Re: 回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove
> > members in StreamsResetter
> >
> >  Hey Feyman,
> >
> >  I think Matthias' suggestion is optional, and we could just use admin
> tool
> >  to remove single static members as well.
> >
> >  Boyang
> >
> >  On Tue, Apr 7, 2020 at 11:00 AM Matthias J. Sax 
> wrote:
> >
> >  > > Would you mind to elaborate why we still need that
> >  >
> >  > Sure.
> >  >
> >  > For static memership, the session timeout it usually set quite high.
> >  > This make scaling in an application tricky: if you shut down one
> >  > instance, no rebalance would happen until `session.timeout.ms` hits.
> >  > This is specific to Kafka Streams, because when a Kafka Stream
> > client is
> >  > closed, it does _not_ send a `LeaveGroupRequest`. Hence, the
> >  > corresponding partitions would not be processed for a long time and
> >  > thus, fall back.
> >  >
> >  > Given that each instance will have a unique `instance.id` provided by
> >  > the user, we could allow users to remove the instance they want to
> >  > decommission from the consumer group without the need to wait for
> >  > `session.timeout.ms`.
> >  >
> >  > Hence, it's not an application reset scenario for which one wants to
> >  > remove all members, but a scaling-in scenario. For dynamic
> > membership,
> >  > this issue usually does not occur because the `session.timeout.ms` is
> >  > set to a fairly low value and a rebalance would happen quick

回复:回复:回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove members in StreamsResetter

2020-04-12 Thread feyman2009
Thanks , John and Guochang!
--
发件人:Guozhang Wang 
发送时间:2020年4月11日(星期六) 03:07
收件人:dev 
主 题:Re: 回复:回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove members in 
StreamsResetter

Thanks Feyman,

I've looked at the update that you incorporated from Matthias and that LGTM
too. I'm still +1 :)

Guozhang

On Fri, Apr 10, 2020 at 11:18 AM John Roesler  wrote:

> Hey Feyman,
>
> Just to remove any ambiguity, I've been casually following the discussion,
> I've just looked at the KIP document again, and I'm still +1 (binding).
>
> Thanks,
> -John
>
> On Fri, Apr 10, 2020, at 01:44, feyman2009 wrote:
> > Hi, all
> > KIP-571 has already collected 4 bind +1 (John, Guochang, Bill,
> > Matthias) and 3 non-binding +1(Boyang, Sophie), I will mark it as
> > approved and create a PR shortly.
> > Thanks!
> >
> > Feyman
> > --
> > 发件人:feyman2009 
> > 发送时间:2020年4月8日(星期三) 14:21
> > 收件人:dev ; Boyang Chen 
> > 主 题:回复:回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove
> > members in StreamsResetter
> >
> > Hi Boyang,
> > Thanks for reminding me of that!
> > I'm not sure about the convention, I thought it would need to
> > re-collect votes if the KIP has changed~
> > Let's leave the vote thread here for 2 days, if no objection, I
> > will take it as approved and update the PR accordingly.
> >
> > Thanks!
> > Feyman
> >
> >
> >
> > ------------------
> > 发件人:Boyang Chen 
> > 发送时间:2020年4月8日(星期三) 12:42
> > 收件人:dev ; feyman2009 
> > 主 题:Re: 回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove
> > members in StreamsResetter
> >
> > You should already get enough votes if I'm counting correctly
> > (Guozhang, John, Matthias)
> > On Tue, Apr 7, 2020 at 6:59 PM feyman2009
> >  wrote:
> > Hi, Boyang
> >  I think Matthias's proposal makes sense, but we can use the admin
> > tool for this scenario as Boyang mentioned or follow up later, so I
> > prefer to keep this KIP unchanged to minimize the scope.
> >  Calling for vote ~
> >
> >  Thanks!
> >  Feyman
> >
> >  --
> >  发件人:Boyang Chen 
> >  发送时间:2020年4月8日(星期三) 02:15
> >  收件人:dev 
> >  主 题:Re: 回复:回复:回复:回复:回复:[Vote] KIP-571: Add option to force remove
> > members in StreamsResetter
> >
> >  Hey Feyman,
> >
> >  I think Matthias' suggestion is optional, and we could just use admin
> tool
> >  to remove single static members as well.
> >
> >  Boyang
> >
> >  On Tue, Apr 7, 2020 at 11:00 AM Matthias J. Sax 
> wrote:
> >
> >  > > Would you mind to elaborate why we still need that
> >  >
> >  > Sure.
> >  >
> >  > For static memership, the session timeout it usually set quite high.
> >  > This make scaling in an application tricky: if you shut down one
> >  > instance, no rebalance would happen until `session.timeout.ms` hits.
> >  > This is specific to Kafka Streams, because when a Kafka Stream
> > client is
> >  > closed, it does _not_ send a `LeaveGroupRequest`. Hence, the
> >  > corresponding partitions would not be processed for a long time and
> >  > thus, fall back.
> >  >
> >  > Given that each instance will have a unique `instance.id` provided by
> >  > the user, we could allow users to remove the instance they want to
> >  > decommission from the consumer group without the need to wait for
> >  > `session.timeout.ms`.
> >  >
> >  > Hence, it's not an application reset scenario for which one wants to
> >  > remove all members, but a scaling-in scenario. For dynamic
> > membership,
> >  > this issue usually does not occur because the `session.timeout.ms` is
> >  > set to a fairly low value and a rebalance would happen quickly after
> > an
> >  > instance is decommissioned.
> >  >
> >  > Does this make sense?
> >  >
> >  > As said before, we may or may not include this in this KIP. It's up
> > to
> >  > you if you want to address it or not.
> >  >
> >  >
> >  > -Matthias
> >  >
> >  >
> >  >
> >  > On 4/7/20 7:12 AM, feyman2009 wrote:
> >  > > Hi, Matthias
> >  > > Thanks a lot!
> >  > > So you do not plan so support removing a _single stat