Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

2019-06-14 Thread Mike Freyberger
Yes that's the plan! ____ From: Mike Freyberger Sent: Saturday, March 9, 2019 10:04 AM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id Hi Boyang, Is th

Re: [DISCUSS] KIP-315: Stream Join Sticky Assignor

2019-05-06 Thread Mike Freyberger
How to you handle reassignment and state movement? Curious to see if we could improve Kafka Streams :) -Matthias On 4/30/19 3:09 AM, Mike Freyberger wrote: > In light of KIP-429, I think there will be an increased demand for sticky assignors. So, I'd like to r

Re: [DISCUSS] KIP-315: Stream Join Sticky Assignor

2019-04-29 Thread Mike Freyberger
on the overall idea and the proposed implementation. Thanks, Mike On 6/20/18, 5:47 PM, "Mike Freyberger" wrote: Matthias, Thanks for the feedback. For our use case, we have some complexities that make using the existing Streams API more complicated than using the Kafka

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

2019-03-08 Thread Mike Freyberger
Hi Boyang, Is this work targeted for Kafka 2.3? I am eager to use this new feature. Thanks, Mike Freyberger On 12/21/18, 1:21 PM, "Mayuresh Gharat" wrote: Hi Boyang, Regarding "However, we shall still attempt to remove the member static info if the given `me

Re: [VOTE] KIP-345: Introduce static membership protocol to reduce consumer rebalances

2018-12-04 Thread Mike Freyberger
+1 (non binding) On 12/4/18, 9:43 AM, "Patrick Williams" wrote: Pls take me off this VOTE list Best, Patrick Williams Sales Manager, UK & Ireland, Nordics & Israel StorageOS +44 (0)7549 676279 patrick.willi...@storageos.com 20 Midtown

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

2018-11-05 Thread Mike Freyberger
ure and provision a new instance. Maybe on the order of 10 minutes is more reasonable. In any case, it's probably a good idea to have an administrative way to force deregistration. One option is to extend the DeleteGroups API with a list of members names. -Jason

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

2018-08-24 Thread Mike Freyberger
Jason, Regarding step 4 in your proposal which suggests beginning a long timer (30 minutes) when a static member leaves the group, would there also be the ability for an admin to force a static membership expiration? I’m thinking that during particular types of outages or upgrades users would

Re: [VOTE] KIP-341: Update Sticky Assignor's User Data Protocol

2018-08-03 Thread Mike Freyberger
+1 On 8/3/18, 2:13 PM, "Ted Yu" wrote: +1 On Fri, Aug 3, 2018 at 10:40 AM Jason Gustafson wrote: > +1 Thanks Vahid. > > On Thu, Aug 2, 2018 at 1:27 PM, Vahid S Hashemian < > vahidhashem...@us.ibm.com > > wrote: > > > Hi everyone, > > > > I

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

2018-07-30 Thread Mike Freyberger
c member changes are not expected often. Hence we would not react to those membership-changing events immediately but wait for longer specified time expecting some other systems like k8s will resume the group members. WDYT? Guozhang On Mon, Jul 30, 2018 at 3:05

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

2018-07-30 Thread Mike Freyberger
Jason, I really appreciate the broader conversation that you are bringing up here. I've been working on an application that does streaming joins for a while now, and we face a similar issue with group membership being dynamic. We are currently using our own StickyAssignor and take special care

Re: [DISCUSS] KIP-341: Update Sticky Assignor's User Data Protocol

2018-07-13 Thread Mike Freyberger
on KIP-315, which is another Sticky Assignor, which also requires some kind of epoch/generation marker to be protected against zombies. So, I’d be in favor of a generic solution here that other assignors can leverage. Best, Mike Freyberger > On Jul 13, 2018, at 6:15 PM, Vahid S Hashem

Re: [DISCUSS] KIP-315: Stream Join Sticky Assignor

2018-06-20 Thread Mike Freyberger
ot for the KIP. I am wondering, why Streams API cannot be used for perform the join? Would be good to understand the advantage of adding a `StickyStreamJoinAssignor` compared to using Streams API? Atm, it seems to be a redundant feature to me. -Matthias On 6/20/18 1:07

[DISCUSS] KIP-315: Stream Join Sticky Assignor

2018-06-20 Thread Mike Freyberger
Hi everybody, I’ve created a proposal document for KIP-315 which outlines the motivation of adding a new partition assignment strategy that can used for streaming join use cases. It’d be great to get feedback on the overall idea and the proposed implementation. KIP Link:

Perms to create KIP

2018-06-13 Thread Mike Freyberger
Can I please have permission to create a KIP? WikiID: mfreyberger Email: mfreyber...@appnexus.com