Hi Chia-Ping, Thanks for pointing out this issue.
I’m thinking that maybe we might need to define a new Exception to handle this scenario. Once the client receives this exception, it should consider the exception as a serious error and stop the process, and the user will need to assign a new memberId. What do you think? Best Regards, TengYao Chia-Ping Tsai <chia7...@apache.org> 於 2024年9月20日 週五 下午7:00寫道: > > This part is not clear either. It basically says that if a member joins > with an existing member id but a different epoch, it will be fenced. Then > it must rejoin with the same member id and epoch zero. This is already the > current behavior and it does not help with detecting duplicates, right? > > The duplicates issue is interesting. The member id is generated on > server-side by UUID before, so it seems to me the fenced member epoch > happens only if the client miss the response with bumped epoch, and this > case is recoverable. > > however, v1 brings a different fenced epoch scenario. Users, now, have > responsibility to generate unique member id, hence clients may encounter > infinite fenced error if there are >1 clients having same member id due to > config error (or other engineer error) > > Maybe we can highlight this scenario if we start to requires clients to > generate unique member id. > > Best, > Chia-Ping > > On 2024/09/19 18:36:48 David Jacot wrote: > > Hi, > > > > Thanks for the update. I have a few nits: > > > > > If the member ID is null or empty, the server will reject the request > > with an InvalidRequestException. > > We should clarify that this should only apply to version >= 1. > > > > > The consumer instance must generate a member ID, and this ID should > > remain consistent for the duration of the consumer's session. Here, a > > "session" is defined as the period from the consumer's first heartbeat > > until it leaves the group, either through a graceful shutdown, a > heartbeat > > timeout, or the process stopping or dying. The consumer instance should > > reuse the same member ID for all heartbeats and rejoin attempts to > maintain > > continuity within the group. > > > > This part is not clear to me. When the member leaves the group, it should > > not reset the member id. I would rather say that the member must generate > > its member id when it starts and it must keep it until the process stops. > > It is basically an incarnation of the process. > > > > > If a conflict arises where the member ID generated by the client is > > detected to be a duplicate within the same group (for example, the same > > member ID is associated with another active member in the group), the > > server will handle this by comparing the memberEpoch values of the > > conflicting members. The member with the lower memberEpoch is considered > > outdated and will be fenced off by the server. When this occurs, the > server > > responds with a FENCED_MEMBER_EPOCH error to the client, signaling it to > > rejoin the group with the same member ID while resetting the memberEpoch > to > > zero. This ensures that the client properly resynchronizes and maintains > > the continuity and consistency of the group membership. > > > > This part is not clear either. It basically says that if a member joins > > with an existing member id but a different epoch, it will be fenced. Then > > it must rejoin with the same member id and epoch zero. This is already > the > > current behavior and it does not help with detecting duplicates, right? > > Should we just remove the paragraph? > > > > > A member ID mismatch occurs within a session: If the server detects a > > mismatch between the provided member ID and the expected member ID for an > > ongoing session, it should return a UNKNOWN_MEMBER_ID error. > > > > How could we detect a mismatch between the provided and the expected > member > > id? My understanding is that we can only know whether the provided member > > id exists or not. This is already implemented. > > > > Thanks, > > David > > > > On Sat, Sep 14, 2024 at 9:31 AM TengYao Chi <kiting...@gmail.com> wrote: > > > > > Hello everyone, > > > > > > Since this KIP has been fully discussed, I will initiate a vote for it > next > > > Monday. > > > Thank you and have a nice weekend. > > > > > > Best regards, > > > TengYao > > > > > > TengYao Chi <kiting...@gmail.com> 於 2024年9月5日 週四 下午2:19寫道: > > > > > > > Hello everyone, > > > > > > > > KT2: It looks like everyone who has expressed an opinion supports the > > > > second option: “Document a recommendation for clients to use UUIDs as > > > > member IDs, without strictly enforcing it.” > > > > I have updated the KIP accordingly. > > > > Please take a look, and let me know if you have any thoughts or > feedback. > > > > > > > > Thank you! > > > > > > > > Best regards, > > > > TengYao > > > > > > > > Chia-Ping Tsai <chia7...@gmail.com> 於 2024年8月30日 週五 下午9:56寫道: > > > > > > > >> hi TengYao > > > >> > > > >> KT2: +1 to second approach > > > >> > > > >> Best, > > > >> Chia-Ping > > > >> > > > >> > > > >> David Jacot <dja...@confluent.io.invalid> 於 2024年8月30日 週五 下午9:15寫道: > > > >> > > > >> > Hi TengYao, > > > >> > > > > >> > KT2: I don't think that we can realistically validate the uuid on > the > > > >> > server. It is basically a string of chars. So I lean towards > having a > > > >> good > > > >> > recommendation in the KIP and in the document of the field in the > > > RPC's > > > >> > definition. > > > >> > > > > >> > Best, > > > >> > David > > > >> > > > > >> > On Fri, Aug 30, 2024 at 3:02 PM TengYao Chi <kiting...@gmail.com> > > > >> wrote: > > > >> > > > > >> > > Hello Kirk ! > > > >> > > > > > >> > > Thank you for your comments ! > > > >> > > > > > >> > > KT1: Yes, you are correct. The issue is not unique to the > initial > > > >> > > heartbeat; there can always be cases where the broker might lose > > > >> > connection > > > >> > > with a member. > > > >> > > > > > >> > > KT2: Currently, if the client doesn't have a member ID and the > > > >> > memberEpoch > > > >> > > equals 0, the coordinator will generate a UUID as the member ID > for > > > >> the > > > >> > > client. However, at the RPC level, the member ID is sent as a > > > literal > > > >> > > string, meaning there are no restrictions on the format at this > > > level. > > > >> > > This also reminds me that we haven't reached a final conclusion > on > > > >> how to > > > >> > > enforce the use of UUIDs. > > > >> > > From our previous discussions, I recall two possible approaches: > > > >> > > The first is to validate the UUID on the server side, and if > it's > > > not > > > >> > > valid, throw an exception to the client. > > > >> > > The second is to document a recommendation for clients to use > UUIDs > > > as > > > >> > > member IDs, without strictly enforcing it. > > > >> > > I think it's time to decide on the approach we want to take. > > > >> > > > > > >> > > KT3: Yes, "session" can be considered synonymous with > "membership" > > > in > > > >> > this > > > >> > > context. > > > >> > > > > > >> > > KT4: Thank you for pointing that out. I will update the wording > to > > > >> > > specifically say this behavior is for consumers. > > > >> > > > > > >> > > Thanks again for your comments. > > > >> > > > > > >> > > Best regards, > > > >> > > TengYao > > > >> > > > > > >> > > Kirk True <k...@kirktrue.pro> 於 2024年8月30日 週五 上午12:39寫道: > > > >> > > > > > >> > > > Hi TengYao! > > > >> > > > > > > >> > > > Sorry for being late to the discussion... > > > >> > > > > > > >> > > > After reading the thread and then the KIP, I had a few > > > >> > > questions/comments: > > > >> > > > > > > >> > > > KT1: In Motivation, it states: "This scenario can result in > the > > > >> broker > > > >> > > > registering a new member for which it will never receive a > proper > > > >> leave > > > >> > > > request.” Just to be clear, the broker will always have cases > > > where > > > >> it > > > >> > > > might lose connection with a member. That’s not unique to the > > > >> initial > > > >> > > > heartbeat, right? > > > >> > > > > > > >> > > > KT2: There was a bit of back and forth about format of the > member > > > >> ID. > > > >> > > From > > > >> > > > what I gathered in the thread, the member ID is still defined > in > > > the > > > >> > RPC > > > >> > > as > > > >> > > > a string and not a UUID, right? The KIP states that the > “client > > > must > > > >> > > > generate a UUID as the member ID” and that the “server will > > > validate > > > >> > > that a > > > >> > > > valid UUID is provided.” Is that a change for the server, or > is it > > > >> > > already > > > >> > > > enforced as a UUID? > > > >> > > > > > > >> > > > KT3: Lianet mentioned some confusion over the use of the word > > > >> > “session.” > > > >> > > > Isn’t “session” synonymous with “membership?” > > > >> > > > > > > >> > > > KT4: Under “Member ID Lifecycle,” it states: "The client > should > > > >> reuse > > > >> > the > > > >> > > > same UUID as the member ID for all heartbeats and rejoin > attempts > > > to > > > >> > > > maintain continuity within the group.” Could we change the > first > > > >> part > > > >> > of > > > >> > > > that to “The Consumer instance should…” We do have lifetimes > that > > > >> > extend > > > >> > > > past the lifetime of a client instance (such as the > transaction > > > ID). > > > >> > > > > > > >> > > > Thanks, > > > >> > > > Kirk > > > >> > > > > > > >> > > > > On Aug 29, 2024, at 1:28 AM, TengYao Chi < > kiting...@gmail.com> > > > >> > wrote: > > > >> > > > > > > > >> > > > > Hi David, > > > >> > > > > > > > >> > > > > Thank you for pointing that out. > > > >> > > > > I have updated the content of the KIP based on Lianet's and > your > > > >> > > > feedback. > > > >> > > > > Please take a look and let me know your thoughts. > > > >> > > > > > > > >> > > > > Best regards, > > > >> > > > > TengYao > > > >> > > > > > > > >> > > > > David Jacot <dja...@confluent.io.invalid> 於 2024年8月29日 週四 > > > >> 下午3:20寫道: > > > >> > > > > > > > >> > > > >> Hi TengYao, > > > >> > > > >> > > > >> > > > >> Thanks for the update. I haven't fully read it yet but I > will > > > >> soon. > > > >> > > > >> > > > >> > > > >> LM4: This is incorrect. The consumer must keep its member > id > > > >> during > > > >> > > its > > > >> > > > >> entire lifetime (until the process stops or dies). The > protocol > > > >> > > > stipulates > > > >> > > > >> that a member must rejoin with the same member id and the > > > member > > > >> > epoch > > > >> > > > set > > > >> > > > >> to zero when an FENCED_MEMBER_EPOCH occurs. This allows the > > > >> member > > > >> > to > > > >> > > > >> resynchronize itself. We should not change this behavior. I > > > think > > > >> > that > > > >> > > > we > > > >> > > > >> should see the client side generation id as an incarnation > id > > > of > > > >> the > > > >> > > > >> application. It is generated once and kept until it stops > or > > > >> dies. > > > >> > > > >> > > > >> > > > >> Best, > > > >> > > > >> David > > > >> > > > >> > > > >> > > > >> On Thu, Aug 29, 2024 at 6:21 AM TengYao Chi < > > > kiting...@gmail.com > > > >> > > > > >> > > > wrote: > > > >> > > > >> > > > >> > > > >>> Hello Lianet ! > > > >> > > > >>> > > > >> > > > >>> Thanks for the reviews and suggestions! > > > >> > > > >>> > > > >> > > > >>> LM1: Indeed, we plan to enforce client-side ID generation > in > > > the > > > >> > > > future, > > > >> > > > >>> and it is not an alternative. I will change the title > > > >> accordingly. > > > >> > > > >>> > > > >> > > > >>> LM2: Yes, that's the expectation. I will add that > statement to > > > >> the > > > >> > > > public > > > >> > > > >>> interface section. > > > >> > > > >>> > > > >> > > > >>> LM3: Thank you for the high-level perspective review. I > think > > > >> > you're > > > >> > > > >> right; > > > >> > > > >>> our intention isn't very clear since it was placed at the > end > > > of > > > >> > the > > > >> > > > >>> section. I will try to rephrase that section to make it > more > > > >> > obvious. > > > >> > > > >>> > > > >> > > > >>> LM4: Regarding the definition of "session" in this KIP, I > > > >> believe > > > >> > it > > > >> > > > >> refers > > > >> > > > >>> to the period between the *first-time heartbeat* and when > the > > > >> > > *consumer > > > >> > > > >>> leaves the group* (whether through a graceful shutdown or > a > > > >> > heartbeat > > > >> > > > >>> timeout). The consumer should reuse its UUID if it has > been > > > >> > generated > > > >> > > > >>> before. The only situation in which it will regenerate the > > > UUID > > > >> is > > > >> > if > > > >> > > > the > > > >> > > > >>> coordinator finds that there is already a consumer with > the > > > same > > > >> > > UUID. > > > >> > > > >>> IIRC, the coordinator should compare the member epochs, > and > > > the > > > >> > > > >>> later-joined consumer should be fenced off by the > coordinator > > > >> due > > > >> > to > > > >> > > > >> having > > > >> > > > >>> a lower member epoch. Once the consumer receives a > > > >> > > > `FENCED_MEMBER_EPOCH` > > > >> > > > >>> error, it will generate a new UUID and attempt to rejoin. > I > > > will > > > >> > > > clarify > > > >> > > > >>> this in the KIP. > > > >> > > > >>> > > > >> > > > >>> Thanks again for your reviews, I really appreciate it. > > > >> > > > >>> > > > >> > > > >>> Best regards, > > > >> > > > >>> TengYao > > > >> > > > >>> > > > >> > > > >>> Lianet M. <liane...@gmail.com> 於 2024年8月28日 週三 下午7:12寫道: > > > >> > > > >>> > > > >> > > > >>>> Hello TengYao! Thanks for taking on this issue, we've > been > > > >> going > > > >> > > > around > > > >> > > > >>> it > > > >> > > > >>>> for a while. > > > >> > > > >>>> > > > >> > > > >>>> LM1: About the title of the KIP: "Enable ID Generation > for > > > >> Clients > > > >> > > > over > > > >> > > > >>> the > > > >> > > > >>>> ConsumerGroupHeartbeat RPC". I find it confusing because > it > > > >> hints > > > >> > > that > > > >> > > > >>>> we're adding it as an alternative (which was discussed > and > > > >> > > discarded, > > > >> > > > >> in > > > >> > > > >>>> favour of really enforcing it). It's also missing the > core > > > >> change > > > >> > > imo, > > > >> > > > >>>> which is "where" the generation happens. So, maybe more > to > > > the > > > >> > point > > > >> > > > >> with > > > >> > > > >>>> something along the lines of "Client-side generated ID > for > > > >> clients > > > >> > > > over > > > >> > > > >>>> ConsumerGroupHeartbeat RPC"? > > > >> > > > >>>> > > > >> > > > >>>> LM2: On the public interfaces section, the KIP states > that > > > "the > > > >> > > server > > > >> > > > >>> will > > > >> > > > >>>> reject the request", but we should agree on the specific > > > error > > > >> > > type. I > > > >> > > > >>>> expect it should fail with an InvalidRequestException, is > > > that > > > >> the > > > >> > > > >>>> intention? (This was also suggested in the discussion > thread > > > >> > before > > > >> > > > but > > > >> > > > >>> is > > > >> > > > >>>> not in the KIP). > > > >> > > > >>>> > > > >> > > > >>>> LM3. Related to my previous point, I find that to be the > true > > > >> > > > >>> public-facing > > > >> > > > >>>> change (member ID mandatory at the protocol level), but > it's > > > >> only > > > >> > at > > > >> > > > >> the > > > >> > > > >>>> end of the Public interfaces changes, kind of lost among > > > >> details > > > >> > of > > > >> > > > how > > > >> > > > >>>> we're going to do it. Should we rephrase that section > with > > > the > > > >> > > actual > > > >> > > > >>>> change first, and the hows after (ex. Bumping the > version is > > > >> not > > > >> > the > > > >> > > > >>>> public-facing change in this case, it's just the > mechanism to > > > >> > > properly > > > >> > > > >>>> introduce our change) > > > >> > > > >>>> > > > >> > > > >>>> LM4. Regarding the lifetime of the UUID: the KIP states > we > > > will > > > >> > > > "Verify > > > >> > > > >>>> that the UUID remains consistent across all subsequent > > > >> heartbeats > > > >> > > > >> during > > > >> > > > >>>> the session". What is this "session" referring to here? I > > > would > > > >> > > expect > > > >> > > > >>> that > > > >> > > > >>>> the UUID is associated to a consumer instance (generated > for > > > >> the > > > >> > > > >> consumer > > > >> > > > >>>> the first time it needs to send a HB if it doesn't have > the > > > >> UUID > > > >> > > yet. > > > >> > > > >>> From > > > >> > > > >>>> there on, every time it needs to send a "first HB" > again, it > > > >> will > > > >> > > > reuse > > > >> > > > >>> its > > > >> > > > >>>> UUID, is that the intention? Note that we should consider > > > that > > > >> the > > > >> > > > same > > > >> > > > >>>> consumer instance may have many "first heartbeats", > meaning > > > >> > > heartbeats > > > >> > > > >> to > > > >> > > > >>>> join the group when it's not part of it (ex. consumer > > > >> unsubscribe > > > >> > + > > > >> > > > >>>> subscribe, fenced, stale). Is this the intention or are > you > > > >> > > > considering > > > >> > > > >>> the > > > >> > > > >>>> lifetime differently? We should clarify it in the KIP. > > > >> > > > >>>> > > > >> > > > >>>> Thanks! > > > >> > > > >>>> > > > >> > > > >>>> Lianet > > > >> > > > >>>> > > > >> > > > >>>> On Tue, Aug 27, 2024 at 2:27 AM TengYao Chi < > > > >> kiting...@gmail.com> > > > >> > > > >> wrote: > > > >> > > > >>>> > > > >> > > > >>>>> Hi everyone, > > > >> > > > >>>>> > > > >> > > > >>>>> I have revised this KIP multiple times based on the > feedback > > > >> from > > > >> > > our > > > >> > > > >>>>> discussions. > > > >> > > > >>>>> I would greatly appreciate it if you could review it > when > > > you > > > >> > have > > > >> > > > >> the > > > >> > > > >>>>> time. > > > >> > > > >>>>> If there are no further comments or suggestions, I plan > to > > > >> > proceed > > > >> > > > >> with > > > >> > > > >>>>> initiating a vote soon. > > > >> > > > >>>>> > > > >> > > > >>>>> Best regards, > > > >> > > > >>>>> TengYao > > > >> > > > >>>>> > > > >> > > > >>>>> TengYao Chi <kiting...@gmail.com> 於 2024年8月23日 週五 > 下午2:43寫道: > > > >> > > > >>>>> > > > >> > > > >>>>>> Hi Andrew, > > > >> > > > >>>>>> Thank you for your previous feedback and insights. > > > >> > > > >>>>>> Your contributions have added valuable perspectives to > the > > > >> > > > >>> discussions. > > > >> > > > >>>>>> And we also benefit from the comparison of different > > > >> solutions. > > > >> > > > >>>>>> I’m also looking forward to seeing an initial version > in > > > >> > KIP-932, > > > >> > > > >> as > > > >> > > > >>> it > > > >> > > > >>>>>> will provide a good reference for future > implementations. > > > >> > > > >>>>>> > > > >> > > > >>>>>> Regarding your comment on AS2, I wanted to clarify > that my > > > >> > > > >>>> specification > > > >> > > > >>>>>> references org.apache.kafka.common.Uuid. > > > >> > > > >>>>>> I believe we’re referring to the same class, and it > might > > > >> just > > > >> > be > > > >> > > a > > > >> > > > >>>> small > > > >> > > > >>>>>> oversight due to the busy schedule. > > > >> > > > >>>>>> > > > >> > > > >>>>>> I want to express my gratitude once again for your many > > > >> > insightful > > > >> > > > >>>>>> comments, which have helped the discussion progress > > > smoothly. > > > >> > > > >>>>>> > > > >> > > > >>>>>> Best regards, > > > >> > > > >>>>>> TengYao > > > >> > > > >>>>>> > > > >> > > > >>>>>> > > > >> > > > >>>>>> Andrew Schofield <andrew_schofi...@live.com> 於 > 2024年8月22日 > > > 週四 > > > >> > > > >>>> 下午11:28寫道: > > > >> > > > >>>>>> > > > >> > > > >>>>>>> Hi TengYao, > > > >> > > > >>>>>>> I’ve been reading through the comments and I’m happy > that > > > >> the > > > >> > > > >> lobby > > > >> > > > >>>>>>> approach has not gained support. > > > >> > > > >>>>>>> > > > >> > > > >>>>>>> Assuming that this KIP is voted, I will be happy to > change > > > >> > > KIP-932 > > > >> > > > >>> so > > > >> > > > >>>>>>> that it only supports client-side member ID > generation. > > > >> Because > > > >> > > > >> that > > > >> > > > >>>> KIP > > > >> > > > >>>>>>> is still > > > >> > > > >>>>>>> under development, I can do this in the first version > of > > > >> > > > >>>>>>> ShareGroupHeartbeat. > > > >> > > > >>>>>>> > > > >> > > > >>>>>>> AS2: For the encoding section, I suppose the specific > > > >> encoding > > > >> > > > >> which > > > >> > > > >>>>>>> is used is what org.apache.kafka.utils.Uuid uses. > > > >> > > > >>>>>>> > > > >> > > > >>>>>>> Thanks, > > > >> > > > >>>>>>> Andrew > > > >> > > > >>>>>>> > > > >> > > > >>>>>>>> On 14 Aug 2024, at 17:00, TengYao Chi < > > > kiting...@gmail.com > > > >> > > > > >> > > > >>> wrote: > > > >> > > > >>>>>>>> > > > >> > > > >>>>>>>> Hello Apoorv, > > > >> > > > >>>>>>>> Thank you for your feedback. > > > >> > > > >>>>>>>> Regarding the questions you raised, unfortunately, > this > > > KIP > > > >> > > > >> cannot > > > >> > > > >>>>>>>> guarantee the order of heartbeats. As with many > classic > > > >> > > > >>> distributed > > > >> > > > >>>>>>> system > > > >> > > > >>>>>>>> challenges, what we can do is make our best effort to > > > >> ensure > > > >> > > > >> that > > > >> > > > >>>>> there > > > >> > > > >>>>>>> are > > > >> > > > >>>>>>>> no idle members or stale assignments under normal > > > >> > circumstances. > > > >> > > > >>>>>>>> > > > >> > > > >>>>>>>> As for the lobby approach, I’m not a fan of it > because it > > > >> > > > >> requires > > > >> > > > >>>>>>> adding a > > > >> > > > >>>>>>>> mechanism to maintain client state within the > > > >> ConsumerGroup, > > > >> > > > >>> which, > > > >> > > > >>>> in > > > >> > > > >>>>>>> my > > > >> > > > >>>>>>>> view, resembles something like a two-phase commit. > This > > > >> would > > > >> > > > >>>>> introduce > > > >> > > > >>>>>>>> more complexity than the proposal in this KIP, which > is > > > >> > > > >> something > > > >> > > > >>> we > > > >> > > > >>>>>>> want > > > >> > > > >>>>>>>> to avoid. KIP-848 aims to simplify the existing > protocol, > > > >> and > > > >> > > > >>> while > > > >> > > > >>>>> the > > > >> > > > >>>>>>>> lobby approach is a good one, I believe it is not the > > > right > > > >> > fit > > > >> > > > >>> for > > > >> > > > >>>>> this > > > >> > > > >>>>>>>> particular situation. > > > >> > > > >>>>>>>> > > > >> > > > >>>>>>>> Best regards, > > > >> > > > >>>>>>>> TengYao > > > >> > > > >>>>>>>> > > > >> > > > >>>>>>>> TengYao Chi <kiting...@gmail.com> 於 2024年8月14日 週三 > > > >> 下午11:45寫道: > > > >> > > > >>>>>>>> > > > >> > > > >>>>>>>>> Hi David, > > > >> > > > >>>>>>>>> > > > >> > > > >>>>>>>>> I really appreciate your review and suggestions. As > I am > > > >> > still > > > >> > > > >>>>> gaining > > > >> > > > >>>>>>>>> experience in writing KIPs, your input has been > > > incredibly > > > >> > > > >>>> helpful. I > > > >> > > > >>>>>>> am > > > >> > > > >>>>>>>>> currently applying your suggestions to the KIP and > will > > > >> > > > >> complete > > > >> > > > >>> it > > > >> > > > >>>>> as > > > >> > > > >>>>>>> soon > > > >> > > > >>>>>>>>> as possible. > > > >> > > > >>>>>>>>> Regarding the UUID part, I think we haven’t reached > a > > > >> > > > >> conclusion > > > >> > > > >>>>>>> yet.(So > > > >> > > > >>>>>>>>> far according to this thread) > > > >> > > > >>>>>>>>> However, I will review the current implementation > in the > > > >> > Kafka > > > >> > > > >>>> `Uuid` > > > >> > > > >>>>>>>>> class and include a brief specification in the KIP. > > > >> > > > >>>>>>>>> > > > >> > > > >>>>>>>>> Once again, thank you so much for your help. > > > >> > > > >>>>>>>>> > > > >> > > > >>>>>>>>> Best regards, > > > >> > > > >>>>>>>>> TengYao > > > >> > > > >>>>>>>>> > > > >> > > > >>>>>>>>> Chia-Ping Tsai <chia7...@gmail.com> 於 2024年8月14日 週三 > > > >> > 下午11:14寫道: > > > >> > > > >>>>>>>>> > > > >> > > > >>>>>>>>>> hi Apoorv > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>>>>>> As the memberId is now known to the client, and > client > > > >> > might > > > >> > > > >>> send > > > >> > > > >>>>> the > > > >> > > > >>>>>>>>>> leave > > > >> > > > >>>>>>>>>> group heartbeat on shutdown prior to receiving the > > > >> initial > > > >> > > > >>>> heartbeat > > > >> > > > >>>>>>>>>> response. If that's true then how do we guarantee > that > > > >> the 2 > > > >> > > > >>>>> requests > > > >> > > > >>>>>>> to > > > >> > > > >>>>>>>>>> join and leave will be processed in order, which > could > > > >> still > > > >> > > > >>> leave > > > >> > > > >>>>>>> stale > > > >> > > > >>>>>>>>>> members or throw unknown member id exceptions? > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>>>>> This is definitely a good question. the short > answer: > > > no > > > >> > > > >>> guarantee > > > >> > > > >>>>> but > > > >> > > > >>>>>>>>>> best > > > >> > > > >>>>>>>>>> efforts > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>>>>> Please notice the root cause is "we have no enough > time > > > >> to > > > >> > > > >> wait > > > >> > > > >>>>>>> member id > > > >> > > > >>>>>>>>>> (response) when closing consumer". Sadly, we can' > > > >> guarantee > > > >> > > > >> the > > > >> > > > >>>>>>> request > > > >> > > > >>>>>>>>>> order due to the same reason. > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>>>>> However, in contrast to previous behavior, there > is one > > > >> big > > > >> > > > >>>> benefit > > > >> > > > >>>>>>> of new > > > >> > > > >>>>>>>>>> approach - we can try STONITH because we know the > > > member > > > >> id > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>>>>> Best, > > > >> > > > >>>>>>>>>> Chia-Ping > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>>>>> Apoorv Mittal <apoorvmitta...@gmail.com> 於 > 2024年8月14日 > > > 週三 > > > >> > > > >>>> 下午8:55寫道: > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>>>>>> Hi TengYao, > > > >> > > > >>>>>>>>>>> Thanks for the KIP. Continuing on the point which > > > Andrew > > > >> > > > >>>> mentioned > > > >> > > > >>>>> as > > > >> > > > >>>>>>>>>> AS1. > > > >> > > > >>>>>>>>>>> > > > >> > > > >>>>>>>>>>> As the memberId is now known to the client, and > client > > > >> > might > > > >> > > > >>> send > > > >> > > > >>>>> the > > > >> > > > >>>>>>>>>> leave > > > >> > > > >>>>>>>>>>> group heartbeat on shutdown prior to receiving the > > > >> initial > > > >> > > > >>>>> heartbeat > > > >> > > > >>>>>>>>>>> response. If that's true then how do we guarantee > that > > > >> the > > > >> > 2 > > > >> > > > >>>>>>> requests to > > > >> > > > >>>>>>>>>>> join and leave will be processed in order, which > could > > > >> > still > > > >> > > > >>>> leave > > > >> > > > >>>>>>> stale > > > >> > > > >>>>>>>>>>> members or throw unknown member id exceptions? > > > >> > > > >>>>>>>>>>> > > > >> > > > >>>>>>>>>>> Though the client side member id generation is > helpful > > > >> > which > > > >> > > > >>> will > > > >> > > > >>>>>>>>>> represent > > > >> > > > >>>>>>>>>>> the same group perspective as from client and > broker's > > > >> end. > > > >> > > > >>> But I > > > >> > > > >>>>>>> think > > > >> > > > >>>>>>>>>> the > > > >> > > > >>>>>>>>>>> major concern we want to solve here is Stale > Partition > > > >> > > > >>>> Assignments > > > >> > > > >>>>>>> which > > > >> > > > >>>>>>>>>>> might still exist with the new approach. I am > leaning > > > >> > towards > > > >> > > > >>> the > > > >> > > > >>>>>>>>>>> suggestion mentioned by Andrew where partition > > > >> assignment > > > >> > > > >>>> triggers > > > >> > > > >>>>> on > > > >> > > > >>>>>>>>>>> subsequent heartbeat when client acknowledges the > > > >> initial > > > >> > > > >>>>> heartbeat, > > > >> > > > >>>>>>>>>>> delayed partition assignment. > > > >> > > > >>>>>>>>>>> > > > >> > > > >>>>>>>>>>> Though on a separate note, I have a different > > > question. > > > >> > What > > > >> > > > >>>>> happens > > > >> > > > >>>>>>>>>> when > > > >> > > > >>>>>>>>>>> there is an issue with the client which sends the > > > >> initial > > > >> > > > >>>> heartbeat > > > >> > > > >>>>>>>>>> without > > > >> > > > >>>>>>>>>>> memberId, then crashes and restarts? I think we > must > > > be > > > >> > > > >>>>> re-triggering > > > >> > > > >>>>>>>>>>> assignments and expiring members only after the > > > >> heartbeat > > > >> > > > >>> session > > > >> > > > >>>>>>>>>> timeout? > > > >> > > > >>>>>>>>>>> If that's true then shall delayed partition > assignment > > > >> can > > > >> > > > >> help > > > >> > > > >>>>>>> benefit > > > >> > > > >>>>>>>>>> us > > > >> > > > >>>>>>>>>>> from this situation as well? > > > >> > > > >>>>>>>>>>> > > > >> > > > >>>>>>>>>>> Regards, > > > >> > > > >>>>>>>>>>> Apoorv Mittal > > > >> > > > >>>>>>>>>>> > > > >> > > > >>>>>>>>>>> > > > >> > > > >>>>>>>>>>> On Wed, Aug 14, 2024 at 12:51 PM David Jacot > > > >> > > > >>>>>>>>>> <dja...@confluent.io.invalid> > > > >> > > > >>>>>>>>>>> wrote: > > > >> > > > >>>>>>>>>>> > > > >> > > > >>>>>>>>>>>> Hi Andrew, > > > >> > > > >>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>> Personally, I don't like the lobby approach. It > makes > > > >> > things > > > >> > > > >>>> more > > > >> > > > >>>>>>>>>>>> complicated and it would require changing the > records > > > >> on > > > >> > the > > > >> > > > >>>>> server > > > >> > > > >>>>>>>>>> too. > > > >> > > > >>>>>>>>>>>> This is why I initially suggested the rejected > > > >> alternative > > > >> > > > >> #2 > > > >> > > > >>>>> which > > > >> > > > >>>>>>> is > > > >> > > > >>>>>>>>>>>> pretty close but also not perfect. > > > >> > > > >>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>> I'd like to clarify one thing. The > > > >> ConsumerGroupHeartbeat > > > >> > > > >> API > > > >> > > > >>>>>>> already > > > >> > > > >>>>>>>>>>>> supports generating the member id on the client > so we > > > >> > don't > > > >> > > > >>> need > > > >> > > > >>>>> any > > > >> > > > >>>>>>>>>>>> conditional logic on the client side. This is > > > actually > > > >> > what > > > >> > > > >> we > > > >> > > > >>>>>>> wanted > > > >> > > > >>>>>>>>>> to > > > >> > > > >>>>>>>>>>> do > > > >> > > > >>>>>>>>>>>> in the first place but the idea got pushed back > by > > > >> Magnus > > > >> > > > >> back > > > >> > > > >>>>> then > > > >> > > > >>>>>>>>>>> because > > > >> > > > >>>>>>>>>>>> generating uuid from librdkafka required a new > > > >> dependency. > > > >> > > > >> It > > > >> > > > >>>>> turns > > > >> > > > >>>>>>>>>> out > > > >> > > > >>>>>>>>>>>> that librdkafka has that dependency today. In > > > >> retrospect, > > > >> > we > > > >> > > > >>>>> should > > > >> > > > >>>>>>>>>> have > > > >> > > > >>>>>>>>>>>> pushed back on this. Long story short, we can > just do > > > >> it. > > > >> > > > >> The > > > >> > > > >>>>>>>>>> proposal in > > > >> > > > >>>>>>>>>>>> this KIP is to make the member id required in > future > > > >> > > > >> versions. > > > >> > > > >>>> We > > > >> > > > >>>>>>>>>> could > > > >> > > > >>>>>>>>>>>> also decide not to do it and to keep supporting > both > > > >> > > > >>>> approaches. I > > > >> > > > >>>>>>>>>> would > > > >> > > > >>>>>>>>>>>> also be fine with this. > > > >> > > > >>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>> Best, > > > >> > > > >>>>>>>>>>>> David > > > >> > > > >>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>> On Wed, Aug 14, 2024 at 12:30 PM Andrew > Schofield < > > > >> > > > >>>>>>>>>>>> andrew_schofi...@live.com> > > > >> > > > >>>>>>>>>>>> wrote: > > > >> > > > >>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>> Hi TengYao, > > > >> > > > >>>>>>>>>>>>> Thanks for your response. I’ll have just one > more > > > try > > > >> to > > > >> > > > >>>>> persuade. > > > >> > > > >>>>>>>>>>>>> I feel that I will need to follow the approach > with > > > >> > KIP-932 > > > >> > > > >>>> when > > > >> > > > >>>>>>>>>> we’ve > > > >> > > > >>>>>>>>>>>>> made a decision, so I do have more than a > passing > > > >> > interest > > > >> > > > >> in > > > >> > > > >>>>> this. > > > >> > > > >>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>> A group member in the lobby is in the group, > but it > > > >> does > > > >> > > > >> not > > > >> > > > >>>> have > > > >> > > > >>>>>>>>>> any > > > >> > > > >>>>>>>>>>>>> assignments. A member of a consumer group can > have > > > no > > > >> > > > >>> assigned > > > >> > > > >>>>>>>>>>>>> partitions (such as 5 CG members subscribed to a > > > topic > > > >> > > > >> with 4 > > > >> > > > >>>>>>>>>>>> partitions), > > > >> > > > >>>>>>>>>>>>> so it’s a situation that consumer group members > > > >> already > > > >> > > > >>> expect. > > > >> > > > >>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>> One of Kafka’s strengths is the way that we > handle > > > API > > > >> > > > >>>>> versioning. > > > >> > > > >>>>>>>>>>>>> But, there is a cost - the behaviour is > different > > > >> > depending > > > >> > > > >>> on > > > >> > > > >>>>> the > > > >> > > > >>>>>>>>>> RPC > > > >> > > > >>>>>>>>>>>>> version. KIP-848 is on the cusp of completion, > but > > > >> we’re > > > >> > > > >>>> already > > > >> > > > >>>>>>>>>> adding > > > >> > > > >>>>>>>>>>>>> conditional logic for v0/v1 for > > > >> ConsumerGroupHeartbeat. > > > >> > > > >>> That’s > > > >> > > > >>>> a > > > >> > > > >>>>>>>>>> pity. > > > >> > > > >>>>>>>>>>>>> Only a minor issue, but it’s unfortunate. > > > >> > > > >>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>> Thanks, > > > >> > > > >>>>>>>>>>>>> Andrew > > > >> > > > >>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>> On 14 Aug 2024, at 08:47, TengYao Chi < > > > >> > > > >> kiting...@gmail.com> > > > >> > > > >>>>>>>>>> wrote: > > > >> > > > >>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>> Hello Andrew > > > >> > > > >>>>>>>>>>>>>> Thank you for your thoughtful suggestions and > > > getting > > > >> > the > > > >> > > > >>>>>>>>>> discussion > > > >> > > > >>>>>>>>>>>>> going. > > > >> > > > >>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>> To AS1: > > > >> > > > >>>>>>>>>>>>>> In the current scenario where the server > generates > > > >> the > > > >> > > > >> UUID, > > > >> > > > >>>> if > > > >> > > > >>>>>>>>>> the > > > >> > > > >>>>>>>>>>>>> client > > > >> > > > >>>>>>>>>>>>>> shuts down before receiving the memberId > generated > > > by > > > >> > the > > > >> > > > >> GC > > > >> > > > >>>>>>>>>>>> (regardless > > > >> > > > >>>>>>>>>>>>> of > > > >> > > > >>>>>>>>>>>>>> whether it’s a graceful shutdown or not), the > GC > > > will > > > >> > > > >> still > > > >> > > > >>>> have > > > >> > > > >>>>>>>>>> to > > > >> > > > >>>>>>>>>>>> wait > > > >> > > > >>>>>>>>>>>>>> for the heartbeat timeout because the client > > > doesn’t > > > >> > know > > > >> > > > >>> its > > > >> > > > >>>>>>>>>>> memberId. > > > >> > > > >>>>>>>>>>>>>> This KIP indeed cannot completely resolve the > > > >> > idempotency > > > >> > > > >>>> issue, > > > >> > > > >>>>>>>>>> but > > > >> > > > >>>>>>>>>>> it > > > >> > > > >>>>>>>>>>>>> can > > > >> > > > >>>>>>>>>>>>>> better handle shutdown scenarios under normal > > > >> > > > >> circumstances > > > >> > > > >>>>>>>>>> because > > > >> > > > >>>>>>>>>>> the > > > >> > > > >>>>>>>>>>>>>> client always knows its memberId. Even if the > > > client > > > >> > shuts > > > >> > > > >>>> down > > > >> > > > >>>>>>>>>>>>> immediately > > > >> > > > >>>>>>>>>>>>>> after the initial heartbeat, as long as it > > > performs a > > > >> > > > >>> graceful > > > >> > > > >>>>>>>>>>> shutdown > > > >> > > > >>>>>>>>>>>>> and > > > >> > > > >>>>>>>>>>>>>> sends a leave heartbeat, the GC can manage the > > > >> situation > > > >> > > > >> and > > > >> > > > >>>>>>>>>> remove > > > >> > > > >>>>>>>>>>> the > > > >> > > > >>>>>>>>>>>>>> member. Therefore, the goal of this KIP is to > > > address > > > >> > the > > > >> > > > >>>> issue > > > >> > > > >>>>>>>>>> where > > > >> > > > >>>>>>>>>>>> the > > > >> > > > >>>>>>>>>>>>>> GC has to wait for the heartbeat timeout due > to the > > > >> > client > > > >> > > > >>>>> leaving > > > >> > > > >>>>>>>>>>>>> without > > > >> > > > >>>>>>>>>>>>>> knowing its memberId, which leads to reduced > > > >> throughput > > > >> > > > >> and > > > >> > > > >>>>>>>>>> limited > > > >> > > > >>>>>>>>>>>>>> scalability. > > > >> > > > >>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>> The solution you suggest has also been > proposed by > > > >> > David. > > > >> > > > >>> The > > > >> > > > >>>>>>>>>> concern > > > >> > > > >>>>>>>>>>>>> with > > > >> > > > >>>>>>>>>>>>>> this approach is that it introduces additional > > > >> > complexity > > > >> > > > >>> for > > > >> > > > >>>>>>>>>>>>>> compatibility, as the new server would not > > > >> immediately > > > >> > add > > > >> > > > >>> the > > > >> > > > >>>>>>>>>> member > > > >> > > > >>>>>>>>>>>> to > > > >> > > > >>>>>>>>>>>>>> the group, while the old server would. This > > > requires > > > >> > > > >> clients > > > >> > > > >>>> to > > > >> > > > >>>>>>>>>>>>>> differentiate whether their memberId has been > added > > > >> to > > > >> > the > > > >> > > > >>>> group > > > >> > > > >>>>>>>>>> or > > > >> > > > >>>>>>>>>>>> not, > > > >> > > > >>>>>>>>>>>>>> which could result in unexpected logs. > > > >> > > > >>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>> Best Regards, > > > >> > > > >>>>>>>>>>>>>> TengYao > > > >> > > > >>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>> Andrew Schofield <andrew_schofi...@live.com> 於 > > > >> > 2024年8月14日 > > > >> > > > >>> 週三 > > > >> > > > >>>>>>>>>>>> 上午12:29寫道: > > > >> > > > >>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>> Hi TengYao, > > > >> > > > >>>>>>>>>>>>>>> Thanks for the KIP. I wonder if there’s a > > > different > > > >> way > > > >> > > > >> to > > > >> > > > >>>>> close > > > >> > > > >>>>>>>>>>> what > > > >> > > > >>>>>>>>>>>>>>> is quite a small window. > > > >> > > > >>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>> AS1: It is true that the initial heartbeat is > not > > > >> > > > >>> idempotent, > > > >> > > > >>>>> but > > > >> > > > >>>>>>>>>>> this > > > >> > > > >>>>>>>>>>>>>>> remains > > > >> > > > >>>>>>>>>>>>>>> true with this KIP. It’s just differently not > > > >> > idempotent. > > > >> > > > >>> If > > > >> > > > >>>>> the > > > >> > > > >>>>>>>>>>>> client > > > >> > > > >>>>>>>>>>>>>>> makes its > > > >> > > > >>>>>>>>>>>>>>> own member ID, sends a request and dies, the > GC > > > will > > > >> > > > >> still > > > >> > > > >>>> have > > > >> > > > >>>>>>>>>>> added > > > >> > > > >>>>>>>>>>>>>>> the member to the group and it will hang > around > > > >> until > > > >> > the > > > >> > > > >>>>> session > > > >> > > > >>>>>>>>>>>>> expires. > > > >> > > > >>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>> I wonder if the GC could still generate the > member > > > >> ID > > > >> > in > > > >> > > > >>>>>>>>>> response to > > > >> > > > >>>>>>>>>>>> the > > > >> > > > >>>>>>>>>>>>>>> first > > > >> > > > >>>>>>>>>>>>>>> heartbeat, and put the member in a special > PENDING > > > >> > state > > > >> > > > >>> with > > > >> > > > >>>>> no > > > >> > > > >>>>>>>>>>>>>>> assignments until the client sends the next > > > >> heartbeat, > > > >> > > > >> thus > > > >> > > > >>>>>>>>>>> confirming > > > >> > > > >>>>>>>>>>>>> it > > > >> > > > >>>>>>>>>>>>>>> has received the member ID. This would not be > a > > > >> > protocol > > > >> > > > >>>> change > > > >> > > > >>>>>>>>>> at > > > >> > > > >>>>>>>>>>>> all, > > > >> > > > >>>>>>>>>>>>>>> just > > > >> > > > >>>>>>>>>>>>>>> a change to the GC to keep a new member in the > > > lobby > > > >> > > > >> until > > > >> > > > >>>> it’s > > > >> > > > >>>>>>>>>>>>> comfirmed > > > >> > > > >>>>>>>>>>>>>>> it knows its member ID. > > > >> > > > >>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>> Thanks, > > > >> > > > >>>>>>>>>>>>>>> Andrew > > > >> > > > >>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>> On 13 Aug 2024, at 15:59, TengYao Chi < > > > >> > > > >>> kiting...@gmail.com> > > > >> > > > >>>>>>>>>> wrote: > > > >> > > > >>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>> Hi Chia-Ping, > > > >> > > > >>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>> Thanks for review and suggestions. > > > >> > > > >>>>>>>>>>>>>>>> I have updated the content of KIP > accordingly. > > > >> > > > >>>>>>>>>>>>>>>> Please take a look. > > > >> > > > >>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>> Best regards, > > > >> > > > >>>>>>>>>>>>>>>> TengYao > > > >> > > > >>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>> Chia-Ping Tsai <chia7...@apache.org> 於 > > > 2024年8月13日 > > > >> 週二 > > > >> > > > >>>>> 下午9:45寫道: > > > >> > > > >>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>> hi TengYao > > > >> > > > >>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>> thanks for this KIP. > > > >> > > > >>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>> 1) could you please describe the > before/after > > > >> > behavior > > > >> > > > >> in > > > >> > > > >>>> the > > > >> > > > >>>>>>>>>>>>> "Proposed > > > >> > > > >>>>>>>>>>>>>>>>> Changes" section? IIRC, current RPC allows > HB > > > >> having > > > >> > > > >>> member > > > >> > > > >>>>> id > > > >> > > > >>>>>>>>>>>>>>> generated by > > > >> > > > >>>>>>>>>>>>>>>>> client, right? If HB has no member ID, > server > > > will > > > >> > > > >>> generate > > > >> > > > >>>>> one > > > >> > > > >>>>>>>>>>> and > > > >> > > > >>>>>>>>>>>>> then > > > >> > > > >>>>>>>>>>>>>>>>> return. The new behavior will enforce HB > "must" > > > >> have > > > >> > > > >>> member > > > >> > > > >>>>> ID. > > > >> > > > >>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>> 2) could you please write the version number > > > >> > explicitly > > > >> > > > >>> in > > > >> > > > >>>>> the > > > >> > > > >>>>>>>>>> KIP > > > >> > > > >>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>> 3) how new client code handle the old HB? > Does > > > it > > > >> > > > >> always > > > >> > > > >>>>>>>>>> generate > > > >> > > > >>>>>>>>>>>>> member > > > >> > > > >>>>>>>>>>>>>>>>> ID on client-side even though that is not > > > >> restricted? > > > >> > > > >>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>> Best, > > > >> > > > >>>>>>>>>>>>>>>>> Chia-Ping > > > >> > > > >>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>> On 2024/08/13 06:20:42 TengYao Chi wrote: > > > >> > > > >>>>>>>>>>>>>>>>>> Hello everyone, > > > >> > > > >>>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>>> I would like to start a discussion thread > on > > > >> > KIP-1082, > > > >> > > > >>>> which > > > >> > > > >>>>>>>>>>>> proposes > > > >> > > > >>>>>>>>>>>>>>>>>> enabling id generation for clients over the > > > >> > > > >>>>>>>>>>> ConsumerGroupHeartbeat > > > >> > > > >>>>>>>>>>>>> RPC. > > > >> > > > >>>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>>> Here is the KIP Link: KIP-1082 > > > >> > > > >>>>>>>>>>>>>>>>>> < > > > >> > > > >>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>> > > > >> > > > >>>>>>>>>>> > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>> > > > >> > > > >>>>> > > > >> > > > >>>> > > > >> > > > >>> > > > >> > > > >> > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1082%3A+Enable+ID+Generation+for+Clients+over+the+ConsumerGroupHeartbeat+RPC > > > >> > > > >>>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>>> Please take a look and let me know what you > > > >> think, > > > >> > > > >> and I > > > >> > > > >>>>> would > > > >> > > > >>>>>>>>>>>>>>> appreciate > > > >> > > > >>>>>>>>>>>>>>>>>> any suggestions and feedback. > > > >> > > > >>>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>>> Best regards, > > > >> > > > >>>>>>>>>>>>>>>>>> TengYao > > > >> > > > >>>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>>> > > > >> > > > >>>>>>>>>>>> > > > >> > > > >>>>>>>>>>> > > > >> > > > >>>>>>>>>> > > > >> > > > >>>>>>>>> > > > >> > > > >>>>>>> > > > >> > > > >>>>>>> > > > >> > > > >>>>> > > > >> > > > >>>> > > > >> > > > >>> > > > >> > > > >> > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > >