Hi, Andrew, Thanks for the reply.
JR34. Transaction coordinator uses ApiVersion to negotiate the PRC versions. Therefore, if there is no functional change, there is no need to bump the TV version. I am not sure about EligibleLeaderReplicasVersion and KRaftVersion. It would be useful to have the experts chime in. JR35. Envelope/forwarding interaction: When a broker forwards a client request to the controller via EnvelopeUtils (parseForwardedRequestHeader), the inner header may have its clientId optimized to null. Some quota enforcement (e.g. controller mutation) may depend on the non-null clientId. The per-connection caching occurs on the original client↔broker connection, not the broker↔controller one. Should the forwarded request preserve the non-null resolved (cached) client.id? JR36. "This KIP does not need to introduce any feature version bumps." This is not very accurate since MV is a feature version and is bumped. Jun On Fri, Jun 26, 2026 at 12:42 AM Andrew Schofield <[email protected]> wrote: > Hi Jun, > I've updated the KIP. > > JS34: I don't believe that we need to introduce any new feature versions > for this KIP. For example, we do not need a new ShareVersion since that > enabled share groups (SV_1) and controls the versioning of the records on > the internal topics (SV_2). I think the same applies for the others. I > understand GroupVersion and StreamsVersion, and I think they follow the > same reasoning as ShareVersion. I don't know TransactionVersion or > EligibleLeaderReplicasVersion, so let me know whether those are different > in this regard. > > Thanks, > Andrew > > On 2026/06/25 21:32:27 Andrew Schofield wrote: > > Hi Jun, > > Thanks for the reply. > > > > JR27: Yes, makes sense. So request header v3 is binary-compatible with > request header v2. > > > > JS34: Yes, I think so. > > > > I'll update the KIP tomorrow morning. > > > > Thanks, > > Andrew > > > > On 2026/06/25 20:23:57 Jun Rao via dev wrote: > > > Hi, Andrew, > > > > > > Thanks for the reply. > > > > > > JR27. I was thinking of the following, which is slightly different from > > > what you described. > > > * Bump RequestHeader to v3, and introduce ClientInstanceId as a tagged > > > field for versions 3+ and change the semantic of the existing field > > > clientId (only need to be populated by the first request). > > > * Bump the request versions for all RPCs including ApiVersion and map > to > > > the new version to use v3 of RequestHeader. > > > * On the server side, when a new request version is seen, we enforce > the > > > presence of ClientInstanceId and reject the request if it is missing. > > > Essentially, treat it as a required field. We also cache the clientId > from > > > the first request and ignore it from subsequent ones. > > > * We would introduce OffsetDelete v1 as the first flexible version so > that > > > it uses the v3 request header. > > > * We would update PushTelemetry and GetTelemetrySubscriptions as > described > > > in the KIP and the ClientInstanceId would move to the request header. > > > > > > In the future, if we want to add a new truly optional field to > > > RequestHeader, we can continue adding it as a tagged field without > bumping > > > up the request version. To add a required field, we will add it as a > tagged > > > field but force a version bump to all requests. > > > > > > The benefit of this approach is that (1) it's compatible with the > format of > > > the old request header; (2) it avoids duplicating new fields; and (3) > there > > > is no constraint on deprecating the old request header. It's slightly > weird > > > to bump up the version when adding a tagged field. But it still seems > > > better than the alternatives. > > > > > > JR34. Since we now have different feature versions gating different > RPCs, > > > should we bump up GV, TV, etc., too? > > > > > > Jun > > > > > > On Thu, Jun 25, 2026 at 10:37 AM Andrew Schofield < > [email protected]> > > > wrote: > > > > > > > Hi Jun, > > > > Thanks for the reply. > > > > > > > > JR27.1: What if we want to add non-optional fields to the request > header > > > > in the future? I think that such fields would be by definition future > > > > fields which would not be understood by existing brokers. For > serialization > > > > reasons, I think we would use version 2+ tagged fields and the > requirement > > > > for non-optionality would be enforced by code, rather than by the > protocol > > > > schema. This avoids the field duplication. > > > > > > > > JR27.2: Yes, I don't like it, but when we inevitably do another > KIP-896 to > > > > remove support for ancient parts of the protocol, it's going to be > another > > > > KIP. That's all I meant. > > > > > > > > If I understand your proposal correctly compared with the existing > KIP > > > > text, it would look like this: > > > > * Remove RequestHeader v3, and introduce ClientInstanceId as a tagged > > > > field for versions 2+. > > > > * Bump the request versions for all RPCs. We'd update the request > JSON > > > > schemas with something like "ClientInstanceIdVersions": "X+". This > would > > > > mean that the request would only be properly formed if it had the > > > > ClientInstanceId present, and would also mean that ClientId could be > null > > > > for non-initial requests on each connection. > > > > * We would create ApiVersions v6, simply as a matter of the bumping > of > > > > request versions for all RPCs, but its schema would be unchanged > from v5. > > > > * We would introduce OffsetDelete v1 as the first flexible version > so that > > > > it uses the v2 request header. > > > > * We would update PushTelemetry and GetTelemetrySubscriptions as > described > > > > in the KIP and the ClientInstanceId would move to the request header. > > > > > > > > I think that works roughly the same as what we have today. I don't > really > > > > like the mandatory nature of the tagged field, but then I don't > really like > > > > the way that ApiVersions is pegged on a historical request header > version. > > > > > > > > Do I have this correct? If so, I can update the KIP accordingly. > > > > > > > > Thanks, > > > > Andrew > > > > > > > > On 2026/06/24 23:40:06 Jun Rao via dev wrote: > > > > > Hi, Andrew, > > > > > > > > > > Thanks for the reply. > > > > > > > > > > JR27.1 What if we want to add non-optional fields to the request > header > > > > in > > > > > the future? The choice I see is duplicating the field in the new > request > > > > > header and the new ApiVersion request. But it's not very clean. > > > > > JR27.2 Requiring a major version API compatibility KIP is also a > bit > > > > > awkward. > > > > > > > > > > Here is yet another proposal. We add the ClientInstanceId to the > request > > > > > header as a tagged field and also bump up all request versions. > This > > > > avoids > > > > > the header compatibility issue in SocketServer. The bumped-up > version > > > > > signals that the new field is actually required. On the server > side, when > > > > > receiving the new version of a request, we verify that the > > > > ClientInstanceId > > > > > field exists. If it's not present, we reject the request. This > will be > > > > the > > > > > convention for adding future required fields to the header. > Compared to > > > > the > > > > > current proposal, this avoids duplicating the field between request > > > > header > > > > > and ApiVersionRequest, and it provides a path for deprecating the > older > > > > > version of the request header. What do you think? > > > > > > > > > > Jun > > > > > > > > > > On Wed, Jun 24, 2026 at 2:15 AM Andrew Schofield < > [email protected]> > > > > > wrote: > > > > > > > > > > > Hi Jun, > > > > > > Thanks for the reply. > > > > > > > > > > > > JR27: Yes, I enhanced that section of the KIP. > > > > > > JR27.1: This KIP was unusual because we wanted to add a concept > which > > > > > > existed in the KIP-714 RPCs to the request header, complicating > the > > > > > > behavior of those RPCs, AND we wanted to change the handling of > client > > > > ID > > > > > > without breaking older brokers. If we simply wanted to add a > brand new > > > > > > field to the request header, we can do it as a tagged field in > request > > > > > > header versions 2+. > > > > > > JR27.2: Covered this too. We'd need a major version API > compatibility > > > > KIP. > > > > > > > > > > > > JR31: Yes. KIP updated. > > > > > > > > > > > > JR32: Yes. This is a minor inconvenience I feel. KIP updated. > > > > > > > > > > > > JR33: I don't see the value of a string comparison here. > Personally, > > > > I'm > > > > > > happy with "assume". > > > > > > > > > > > > Thanks, > > > > > > Andrew > > > > > > > > > > > > On 2026/06/23 22:52:15 Jun Rao via dev wrote: > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > Thanks for the reply. > > > > > > > > > > > > > > JR27. The code that changed the request header parsing was > > > > introduced in > > > > > > > 2019 ( > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/apache/kafka/pull/7372__;!!Ayb5sqE7!scbWbTEEm2o-Y3MBGi829H2cpMWPRyrF9g2CW3lmRHyRmgBU2MyoRif2mFJpnfmvQ2x7fF4LrAJXdwH86ERR$ > > > > > > ). I agree that fixing it > > > > > > > now is too late. We should document the path forward for > ApiVersion. > > > > > > Could > > > > > > > we answer at least the following questions in the > Compatibility, > > > > > > > Deprecation, and Migration Plan section? > > > > > > > JR27.1 What happens to ApiVersion if we change the request > header in > > > > the > > > > > > > future? > > > > > > > JR27.2 What happens if we ever want to deprecate v2 of the > request > > > > > > header? > > > > > > > > > > > > > > JR28. The new MV needs to be covered in the Compatibility, > > > > Deprecation, > > > > > > and > > > > > > > Migration Plan section. > > > > > > > > > > > > > > JR31. "If a connection uses the v3 request header in its first > > > > request, > > > > > > it > > > > > > > must specify a non-zero client instance ID and it must specify > the > > > > same > > > > > > > client instance ID for all subsequent requests." > > > > > > > Do we do the same if the first request is v6 of ApiVersion? > > > > > > > > > > > > > > JR32. "For subsequent requests using an earlier version of the > > > > request > > > > > > > header, the client sends the client ID". This means that > subsequent > > > > > > > ApiVersion V6 requests will carry the non-null clientID > > > > unnecessarily. > > > > > > > > > > > > > > JR33. "After this KIP, the broker will assume that the client > ID > > > > from the > > > > > > > initial request applies to all subsequent requests on a > connection, > > > > and > > > > > > it > > > > > > > will ignore the client ID specified on any subsequent > requests." > > > > > > > Should the broker verify that the clientId from subsequent > request > > > > match > > > > > > > the first cached one, if not null? > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > On Tue, Jun 23, 2026 at 6:05 AM Andrew Schofield < > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Jun, > > > > > > > > Thanks for your reply. > > > > > > > > > > > > > > > > JR27: Unfortunately, I don't think this is acceptable. We > would be > > > > > > saying > > > > > > > > that AK 4.5 clients can only connect to AK 4.5 or later > brokers, > > > > and > > > > > > that > > > > > > > > we'd create patch release for AK 4.0.x to AK 4.4.x. so AK 4.5 > > > > clients > > > > > > could > > > > > > > > connect to those too. We'd be preventing connection to AK 3.x > > > > brokers > > > > > > by AK > > > > > > > > 4.5 clients. I don't see how we can break compatibility in a > minor > > > > > > release. > > > > > > > > Here's the existing compatibility table: > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://kafka.apache.org/41/getting-started/compatibility/__;!!Ayb5sqE7!ppWU_jI9f7EY9DYwcbiOESSBG2TmPn9CHfLnVM5TaIMwWqHUbUbIiGre1GGKMr-HmelQlyP6ibUSyORCAgte$ > > > > > > > > > > > > > > > > JR29: OK, makes sense to me. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Andrew > > > > > > > > > > > > > > > > On 2026/06/22 21:50:43 Jun Rao via dev wrote: > > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > > > > > Thanks for the reply. > > > > > > > > > > > > > > > > > > JR27. I am not sure I like the approach of pinning the > header to > > > > v2 > > > > > > for > > > > > > > > > ApiVersionsRequest. This means that in the future, each > change > > > > to the > > > > > > > > > request header must be duplicated in the request itself. > Here is > > > > an > > > > > > > > > alternative. Note that to detect an unsupported > > > > ApiVersionRequest, we > > > > > > > > don't > > > > > > > > > need to parse the full v2 request header. Parsing the > first two > > > > > > fields > > > > > > > > > ApiKey and ApiVersion is enough. So, we could patch all > the 4.x > > > > > > releases > > > > > > > > > with a change that short-circuits parsing the full request > header > > > > > > based > > > > > > > > on > > > > > > > > > the first two fields. Then, we recommend upgrading from the > > > > latest > > > > > > 4.x > > > > > > > > > releases to future releases. This way, we can still let > > > > > > > > ApiVersionsRequest > > > > > > > > > depend on the request header for current and future > changes. > > > > > > > > > > > > > > > > > > JR29. The simplest thing is to upgrade the KRaft RPCs too > to be > > > > > > > > consistent. > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > On Sat, Jun 20, 2026 at 3:38 AM Andrew Schofield < > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Jun, > > > > > > > > > > Thanks for these insightful comments. I did prototype the > > > > earlier > > > > > > v2 > > > > > > > > > > request header with tagged fields variant. I have now > > > > prototyped > > > > > > the v3 > > > > > > > > > > request header variant. > > > > > > > > > > > > > > > > > > > > JR27: You were of course entirely correct that a new > version of > > > > > > > > > > ApiVersions request cannot use a request header which is > not > > > > > > > > > > binary-compatible with what has gone before. As a > result, I > > > > have > > > > > > > > introduced > > > > > > > > > > a new version of ApiVersions request which includes > > > > > > ClientInstanceId > > > > > > > > in the > > > > > > > > > > request body but continues to use the v2 request header. > > > > > > > > > > > > > > > > > > > > JR28: Yes, we need a new metadata version, which is > > > > provisionally > > > > > > > > > > IBP_4_5_IV0. I'll update the KIP with the final value > when it > > > > is > > > > > > > > assigned. > > > > > > > > > > > > > > > > > > > > JR29: I don't have any strong feelings on this. I > suppose for > > > > each > > > > > > > > inter > > > > > > > > > > broker RPC version that is bumped, we need to introduce > an MV > > > > > > check. We > > > > > > > > > > cannot avoid this for RPCs which are also used by > clients such > > > > as > > > > > > > > Fetch. > > > > > > > > > > For purely inter-broker RPCs, we could just exempt them > from > > > > the > > > > > > > > version > > > > > > > > > > bump. What do you recommend here? > > > > > > > > > > > > > > > > > > > > JR30: Done. INVALID_REQUEST. I did consider permitting > zero > > > > client > > > > > > > > > > instance ID for inter-broker connections, but I decided > that I > > > > > > > > preferred > > > > > > > > > > not creating another rule. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > On 2026/06/17 22:45:07 Andrew Schofield wrote: > > > > > > > > > > > Hi Jun, > > > > > > > > > > > Thanks for your response. > > > > > > > > > > > > > > > > > > > > > > JR27: I am familiar with KIP-511 and the oddness of > > > > ApiVersions. > > > > > > You > > > > > > > > > > make an excellent point. I suspect that > ApiVersionsRequest will > > > > > > have > > > > > > > > to be > > > > > > > > > > limited to the v2 request header, just as it is > restricted to > > > > the > > > > > > v0 > > > > > > > > > > response header. > > > > > > > > > > > > > > > > > > > > > > I'll do some experimentation. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > > > On 2026/06/17 17:35:17 Jun Rao via dev wrote: > > > > > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the reply. A few more comments. > > > > > > > > > > > > > > > > > > > > > > > > JR27. It seems that we have a compatibility issue > with > > > > > > > > > > ApiVersionRequest v6 > > > > > > > > > > > > depending on RequestHeader v3. This happens when a > new > > > > client > > > > > > > > issues > > > > > > > > > > > > ApiVersionRequest v6 to an old broker (one that only > > > > supports > > > > > > > > > > > > ApiVersionRequest v5). The SocketServer in the > broker will > > > > > > first > > > > > > > > try to > > > > > > > > > > > > read the request header. It tries to find the header > > > > version > > > > > > from > > > > > > > > the > > > > > > > > > > > > request version. Since the broker doesn't know > > > > > > ApiVersionRequest > > > > > > > > v6, it > > > > > > > > > > > > uses v2 to parse the request header. > > > > > > > > > > > > > > > > > > > > > > > > The v2's wire format for the request header is: > ApiKey | > > > > > > > > ApiVersion | > > > > > > > > > > > > CorrelationId | ClientId | tagged-fields-varint > > > > > > > > > > > > The v3's wire format for the request header is: > ApiKey | > > > > > > > > ApiVersion | > > > > > > > > > > > > CorrelationId | ClientId | ClientInstanceId(16 > bytes) | > > > > > > > > tagged-fields > > > > > > > > > > varint > > > > > > > > > > > > The first 4 fields are identical, allowing the > broker to > > > > parse > > > > > > > > them. > > > > > > > > > > The > > > > > > > > > > > > broker then tries to read the tagged-fields-varint > using > > > > the > > > > > > > > serialized > > > > > > > > > > > > bytes intended for ClientInstanceId. This could lead > to a > > > > > > parsing > > > > > > > > > > error and > > > > > > > > > > > > raise an invalid request error. The broker will > close the > > > > > > > > connection > > > > > > > > > > > > instead of sending a v0 response for > ApiVersionRequest. > > > > Then > > > > > > the > > > > > > > > client > > > > > > > > > > > > will never be able to detect supported API versions > on the > > > > old > > > > > > > > broker. > > > > > > > > > > > > > > > > > > > > > > > > The same problem didn't exist when ApiVersionRequest > > > > changed > > > > > > from > > > > > > > > the > > > > > > > > > > v1 > > > > > > > > > > > > request header to v2 because the wire format for v1 > is a > > > > > > prefix of > > > > > > > > v2. > > > > > > > > > > This > > > > > > > > > > > > problem exists for this KIP because the wire format > of v2 > > > > is no > > > > > > > > longer > > > > > > > > > > a > > > > > > > > > > > > prefix of v3. > > > > > > > > > > > > > > > > > > > > > > > > JR28. Since we are bumping up the version of inter > broker > > > > > > > > requests, we > > > > > > > > > > need > > > > > > > > > > > > to gate them by introducing a new MV. > > > > > > > > > > > > > > > > > > > > > > > > JR29. Could you specify whether the version for > KRaft PRCs > > > > > > will be > > > > > > > > > > bumped > > > > > > > > > > > > too? > > > > > > > > > > > > > > > > > > > > > > > > JR30. "If a connection uses the v3 request header in > its > > > > first > > > > > > > > > > request, it > > > > > > > > > > > > must specify a non-zero client instance ID" > > > > > > > > > > > > Could we define the broker's behavior when a v3 > request > > > > header > > > > > > > > > > contains a 0 > > > > > > > > > > > > client instance ID? > > > > > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 16, 2026 at 1:25 AM Andrew Schofield < > > > > > > > > > > [email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi Jun, > > > > > > > > > > > > > Thanks for your response. > > > > > > > > > > > > > > > > > > > > > > > > > > JR26: Yes, I was considering open-ended version > ranges. > > > > I've > > > > > > > > updated > > > > > > > > > > the > > > > > > > > > > > > > KIP. > > > > > > > > > > > > > > > > > > > > > > > > > > Now that we have a new request header version, I > think > > > > it's > > > > > > safe > > > > > > > > to > > > > > > > > > > > > > reintroduce the client ID optimisation. I've added > that > > > > into > > > > > > the > > > > > > > > KIP > > > > > > > > > > also. > > > > > > > > > > > > > > > > > > > > > > > > > > Let me know what you think. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > > > > > > > On 2026/06/15 17:46:00 Jun Rao via dev wrote: > > > > > > > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the reply. Just a minor comment. > > > > > > > > > > > > > > > > > > > > > > > > > > > > JR26. "The keys of the map are either single > versions > > > > like > > > > > > "0" > > > > > > > > or > > > > > > > > > > closed > > > > > > > > > > > > > > version ranges like "1-2"." > > > > > > > > > > > > > > It will be useful to support open-ended > versions. That > > > > way, > > > > > > > > "3": > > > > > > > > > > "3" > > > > > > > > > > > > > could > > > > > > > > > > > > > > be "3+": "3". This part won't need to change in > the > > > > future > > > > > > > > when we > > > > > > > > > > bump > > > > > > > > > > > > > up > > > > > > > > > > > > > > the request version without changing the header. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 12, 2026 at 2:12 PM Andrew Schofield > < > > > > > > > > > > [email protected]> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Jun, > > > > > > > > > > > > > > > Thanks for your response. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > JR25: I agree. I've changed to a > headerVersions map > > > > in > > > > > > both > > > > > > > > the > > > > > > > > > > request > > > > > > > > > > > > > > > and response schemas. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Let me know what you think. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2026/06/12 03:39:39 Jun Rao via dev wrote: > > > > > > > > > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for the updated KIP. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > JR25. I have some concerns about adding the > > > > > > > > > > clientInstanceIdVersions > > > > > > > > > > > > > > > field > > > > > > > > > > > > > > > > to the schema. First of all, it only applies > to > > > > > > requests, > > > > > > > > but > > > > > > > > > > not > > > > > > > > > > > > > > > > responses. So, it's a bit weird to add it to > the > > > > > > general > > > > > > > > > > schema. > > > > > > > > > > > > > > > Secondly, > > > > > > > > > > > > > > > > it's very specific to the new field added to > the > > > > > > request > > > > > > > > > > header. If > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > change the request header again in the > future, we > > > > need > > > > > > > > another > > > > > > > > > > way > > > > > > > > > > > > > to map > > > > > > > > > > > > > > > > request versions to the new header version. > > > > Ideally, we > > > > > > > > should > > > > > > > > > > have a > > > > > > > > > > > > > > > > generic way to map request versions to all > future > > > > > > header > > > > > > > > > > versions. > > > > > > > > > > > > > One > > > > > > > > > > > > > > > > possibility is to introduce a headerVersion > field > > > > in > > > > > > the > > > > > > > > > > schema. > > > > > > > > > > > > > Then, > > > > > > > > > > > > > > > each > > > > > > > > > > > > > > > > request/response can use this field to map > its > > > > > > versions to > > > > > > > > the > > > > > > > > > > header > > > > > > > > > > > > > > > > versions. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 9, 2026 at 4:55 AM Andrew > Schofield < > > > > > > > > > > > > > [email protected]> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Matthias, > > > > > > > > > > > > > > > > > Thanks for your response. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have change the version property for > > > > introducing > > > > > > client > > > > > > > > > > instance > > > > > > > > > > > > > ID > > > > > > > > > > > > > > > into > > > > > > > > > > > > > > > > > the request headers to be > > > > `clientInstanceIdVersions` > > > > > > > > which > > > > > > > > > > is now > > > > > > > > > > > > > an > > > > > > > > > > > > > > > > > open-ended range, such as "3+", more > closely > > > > matching > > > > > > > > what > > > > > > > > > > we have > > > > > > > > > > > > > for > > > > > > > > > > > > > > > > > flexible versions. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I also added a change to the > > > > > > > > > > RebalanceConsumer.clientInstanceId > > > > > > > > > > > > > method > > > > > > > > > > > > > > > > > introduced in KIP-1306. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2026/06/03 18:31:06 "Matthias J. Sax" > wrote: > > > > > > > > > > > > > > > > > > Thanks Andrew. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From a KIP-714 perspective, this is a > great > > > > > > change, > > > > > > > > as it > > > > > > > > > > make > > > > > > > > > > > > > the > > > > > > > > > > > > > > > new > > > > > > > > > > > > > > > > > > RPCs and client/broker cross-version > > > > compatibility > > > > > > > > clean. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > One question about your `DeleteGroups` > RPC > > > > example. > > > > > > > > Should > > > > > > > > > > the > > > > > > > > > > > > > new > > > > > > > > > > > > > > > > > > version of the RPC say > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > "clientInstanceIdVersion": "3+", > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ie, 3+ instead of 3? Similar for other > request, > > > > > > which > > > > > > > > > > don't use > > > > > > > > > > > > > `+` > > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > > `clientInstanceIdVersion` field. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Matthias > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 6/2/26 7:42 AM, Andrew Schofield > wrote: > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > Following on from the discussion in the > > > > > > community, > > > > > > > > and > > > > > > > > > > the > > > > > > > > > > > > > > > difficulty > > > > > > > > > > > > > > > > > of trying to accommodate the client > instance ID > > > > in a > > > > > > > > tagged > > > > > > > > > > field > > > > > > > > > > > > > with > > > > > > > > > > > > > > > > > acceptable complexity, I have made a > significant > > > > > > change > > > > > > > > to > > > > > > > > > > the KIP. > > > > > > > > > > > > > > > Now, it > > > > > > > > > > > > > > > > > introduces v3 of the request header > containing > > > > the > > > > > > client > > > > > > > > > > instance > > > > > > > > > > > > > ID > > > > > > > > > > > > > > > as a > > > > > > > > > > > > > > > > > proper field. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1313*3A*Client*instance*ID*in*all*request*headers__;JSsrKysrKys!!Ayb5sqE7!pUZS25PuTKdhiqvTGcwDKbclqpG8dFOzw2Ax9WxVM6Sb09PVQ0gl1i0gjlr2_Krr66agbuVC2qLGgUcg7vrW$ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please review and let me know what you > think. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 2026/05/28 21:51:01 "Matthias J. > Sax" > > > > wrote: > > > > > > > > > > > > > > > > > > >> Thanks Andrew. I agree to the issue > that > > > > tagged > > > > > > > > fields > > > > > > > > > > are > > > > > > > > > > > > > > > optional. > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> Nevertheless, I find the proposed > > > > compatibility > > > > > > > > > > protocol too > > > > > > > > > > > > > > > complex > > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > >> unnecessary, and would still prefer a > > > > version > > > > > > bump > > > > > > > > for > > > > > > > > > > > > > > > > > > >> > `GetTelemetrySubscriptionsRequest/Reponse` > > > > to > > > > > > clean > > > > > > > > it > > > > > > > > > > all > > > > > > > > > > > > > up. -- > > > > > > > > > > > > > > > For > > > > > > > > > > > > > > > > > > >> `PushTelemetryRequest` the argument > from > > > > above > > > > > > > > applies, > > > > > > > > > > so > > > > > > > > > > > > > > > leaving it > > > > > > > > > > > > > > > > > > >> untouched is ok. > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> I don't see why > > > > > > `GetTelemetrySubscriptionsRequest` > > > > > > > > > > would need > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > encode > > > > > > > > > > > > > > > > > > >> the `clientInstandId` in it's request > body. > > > > > > Given > > > > > > > > that > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >> `clientInstanceId` is a random UUID, > the > > > > > > returned > > > > > > > > > > subscription > > > > > > > > > > > > > > > cannot > > > > > > > > > > > > > > > > > > >> really depend on it. So why would we > need to > > > > > > send > > > > > > > > it to > > > > > > > > > > the > > > > > > > > > > > > > > > broker? > > > > > > > > > > > > > > > > > > >> Right now, it's set to ZERO only for > the > > > > > > purpose to > > > > > > > > get > > > > > > > > > > the > > > > > > > > > > > > > > > broker to > > > > > > > > > > > > > > > > > > >> assign a new UUID. But it has nothing > to do > > > > > > with the > > > > > > > > > > actual > > > > > > > > > > > > > "what > > > > > > > > > > > > > > > > > > >> metrics should I send" question. > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> To be fair, KIP-714 lists > > > > `client_instance_id` > > > > > > as a > > > > > > > > > > field that > > > > > > > > > > > > > > > can be > > > > > > > > > > > > > > > > > > >> used to define a subscription. But it > seems > > > > > > rather > > > > > > > > > > useless in > > > > > > > > > > > > > > > practice > > > > > > > > > > > > > > > > > > >> to me? I a user defines a > subscription, they > > > > > > cannot > > > > > > > > > > know the > > > > > > > > > > > > > UUID. > > > > > > > > > > > > > > > > > Thus, > > > > > > > > > > > > > > > > > > >> I think we should actually drop > supporting > > > > > > > > > > > > > `client_instance_id` > > > > > > > > > > > > > > > as a > > > > > > > > > > > > > > > > > > >> "subscription matching parameter"? Of > > > > course, > > > > > > there > > > > > > > > is a > > > > > > > > > > > > > backward > > > > > > > > > > > > > > > > > > >> compatibility question, but I think > we can > > > > > > address > > > > > > > > > > this: if > > > > > > > > > > > > > > > brokers > > > > > > > > > > > > > > > > > are > > > > > > > > > > > > > > > > > > >> upgraded and they have an existing > > > > > > > > `client_instance_id` > > > > > > > > > > based > > > > > > > > > > > > > > > > > > >> subscription defined, they could > advertise > > > > to > > > > > > only > > > > > > > > > > support > > > > > > > > > > > > > v0, and > > > > > > > > > > > > > > > > > they > > > > > > > > > > > > > > > > > > >> should log a warning that this > feature is > > > > > > > > deprecated. > > > > > > > > > > New > > > > > > > > > > > > > brokers > > > > > > > > > > > > > > > > > would > > > > > > > > > > > > > > > > > > >> also not allow to use > `client_instance_id` > > > > any > > > > > > > > longer to > > > > > > > > > > > > > define a > > > > > > > > > > > > > > > > > > >> subscription. > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> Similarly, for > > > > > > `GetTelemetrySubscriptionsResponse`, > > > > > > > > we > > > > > > > > > > only > > > > > > > > > > > > > need > > > > > > > > > > > > > > > > > > >> `clientInstanceId` if the broker > computes > > > > the > > > > > > UUID. > > > > > > > > But > > > > > > > > > > if we > > > > > > > > > > > > > let > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >> client compute it, this field is not > needed > > > > any > > > > > > > > longer. > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> -Matthias > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> On 5/28/26 1:21 AM, Andrew Schofield > wrote: > > > > > > > > > > > > > > > > > > >>> Hi Jun, > > > > > > > > > > > > > > > > > > >>> Thanks for your response. > > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > >>> JR24: I'll update the KIP shortly. > Here's a > > > > > > > > summary. > > > > > > > > > > The > > > > > > > > > > > > > various > > > > > > > > > > > > > > > > > modern group protocol heartbeat requests > encode > > > > the > > > > > > > > member > > > > > > > > > > ID as a > > > > > > > > > > > > > > > string > > > > > > > > > > > > > > > > > for historical reasons, even though it's in > > > > practice > > > > > > a > > > > > > > > UUID. > > > > > > > > > > The > > > > > > > > > > > > > Apache > > > > > > > > > > > > > > > > > Kafka Java client happens to use > > > > > > > > > > org.apache.kafka.common.Uuid to > > > > > > > > > > > > > > > create the > > > > > > > > > > > > > > > > > member ID and convert it into a string. > This > > > > means > > > > > > that > > > > > > > > when > > > > > > > > > > the > > > > > > > > > > > > > > > > > clientInstanceId is converted into a > string, its > > > > > > encoding > > > > > > > > > > matches > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > member ID and it all lines up. However, > there are > > > > > > other > > > > > > > > ways > > > > > > > > > > to > > > > > > > > > > > > > encode > > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > > UUID into a string and I know for a fact > that the > > > > > > > > librdkafka > > > > > > > > > > client > > > > > > > > > > > > > > > does it > > > > > > > > > > > > > > > > > differently. > > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > >>> Thanks, > > > > > > > > > > > > > > > > > > >>> Andrew > > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > >>> On 2026/05/27 17:11:25 Jun Rao via > dev > > > > wrote: > > > > > > > > > > > > > > > > > > >>>> Hi, Andrew, > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > >>>> Thanks for the reply. One more > comment. > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > >>>> JR24. "The alignment of other > identifiers > > > > is > > > > > > by > > > > > > > > > > convention > > > > > > > > > > > > > (and > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > Java > > > > > > > > > > > > > > > > > > >>>> client will follow the convention) > rather > > > > than > > > > > > > > > > mandate." > > > > > > > > > > > > > Could > > > > > > > > > > > > > > > you > > > > > > > > > > > > > > > > > describe > > > > > > > > > > > > > > > > > > >>>> the convention to convert a > > > > clientInstanceId > > > > > > > > (UUID) > > > > > > > > > > to a > > > > > > > > > > > > > > > memberId > > > > > > > > > > > > > > > > > (String)? > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > >>>> Jun > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > >>>> On Tue, May 19, 2026 at 2:36 AM > Andrew > > > > > > Schofield < > > > > > > > > > > > > > > > > > [email protected]> > > > > > > > > > > > > > > > > > > >>>> wrote: > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > >>>>> Hi Jun, > > > > > > > > > > > > > > > > > > >>>>> Thanks for your response. > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > >>>>> JR23: You are absolutely correct. > It > > > > seems > > > > > > to me > > > > > > > > > > that not > > > > > > > > > > > > > > > sending a > > > > > > > > > > > > > > > > > > >>>>> clientInstanceId in the header and > > > > explicitly > > > > > > > > > > sending a > > > > > > > > > > > > > zero > > > > > > > > > > > > > > > UUID > > > > > > > > > > > > > > > > > as the > > > > > > > > > > > > > > > > > > >>>>> clientInstanceId in the header can > be > > > > > > treated as > > > > > > > > > > > > > semantically > > > > > > > > > > > > > > > > > equivalent. > > > > > > > > > > > > > > > > > > >>>>> I've tweaked the words slightly. > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > >>>>> Thanks, > > > > > > > > > > > > > > > > > > >>>>> Andrew > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > >>>>> On 2026/05/19 03:42:16 Jun Rao via > dev > > > > wrote: > > > > > > > > > > > > > > > > > > >>>>>> Hi, Andrew, > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > >>>>>> Thanks for the reply. > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > >>>>>> JR23. Our message protocol doc > says "Any > > > > > > fields > > > > > > > > in > > > > > > > > > > the > > > > > > > > > > > > > message > > > > > > > > > > > > > > > > > object > > > > > > > > > > > > > > > > > > >>>>> that > > > > > > > > > > > > > > > > > > >>>>>> are not present in the version > that you > > > > are > > > > > > > > > > deserializing > > > > > > > > > > > > > > > will be > > > > > > > > > > > > > > > > > reset > > > > > > > > > > > > > > > > > > >>>>> to > > > > > > > > > > > > > > > > > > >>>>>> default values. Unless a custom > > > > default has > > > > > > > > been > > > > > > > > > > set:". > > > > > > > > > > > > > Uuid > > > > > > > > > > > > > > > > > fields > > > > > > > > > > > > > > > > > > >>>>>> default to zero uuid. > > > > > > > > > > > > > > > > > > >>>>>> So if the server gets > > > > > > header.clientInstanceId=0 > > > > > > > > in > > > > > > > > > > the > > > > > > > > > > > > > > > > > deserialized > > > > > > > > > > > > > > > > > > >>>>> header, > > > > > > > > > > > > > > > > > > >>>>>> could it distinguish between the > ID not > > > > > > being > > > > > > > > > > present > > > > > > > > > > > > > (since > > > > > > > > > > > > > > > > > client is > > > > > > > > > > > > > > > > > > >>>>> old) > > > > > > > > > > > > > > > > > > >>>>>> and the ID being explicitly set > to 0 by > > > > the > > > > > > > > client? > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > >>>>>> Jun > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > >>>>>> On Mon, May 18, 2026 at 7:45 PM > Andrew > > > > > > > > Schofield < > > > > > > > > > > > > > > > > > [email protected]> > > > > > > > > > > > > > > > > > > >>>>>> wrote: > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > >>>>>>> Hi Jun, > > > > > > > > > > > > > > > > > > >>>>>>> Thanks for your reply. It's > tricky > > > > > > squaring a > > > > > > > > > > circle. > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>> JR23: For > GetTelemetrySubscriptions, I > > > > have > > > > > > > > > > changed it so > > > > > > > > > > > > > > > that a > > > > > > > > > > > > > > > > > client > > > > > > > > > > > > > > > > > > >>>>>>> which omits the ClientInstanceId > from > > > > the > > > > > > > > request > > > > > > > > > > header > > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > permitted > > > > > > > > > > > > > > > > > > >>>>> to > > > > > > > > > > > > > > > > > > >>>>>>> specify a zero ClientInstanceId > in the > > > > > > request > > > > > > > > > > body, > > > > > > > > > > > > > > > following > > > > > > > > > > > > > > > > > original > > > > > > > > > > > > > > > > > > >>>>>>> KIP-714 precedent. However, a > client > > > > which > > > > > > > > > > specifies a > > > > > > > > > > > > > > > > > > >>>>> ClientInstanceId in > > > > > > > > > > > > > > > > > > >>>>>>> the request header MUST specify > the > > > > same > > > > > > > > > > > > > ClientInstanceId in > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >>>>> request > > > > > > > > > > > > > > > > > > >>>>>>> body. This ensures that the > header and > > > > > > > > telemetry > > > > > > > > > > UUIDs > > > > > > > > > > > > > are > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > same. > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>> Thanks, > > > > > > > > > > > > > > > > > > >>>>>>> Andrew > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>> On 2026/05/12 17:48:23 Andrew > Schofield > > > > > > wrote: > > > > > > > > > > > > > > > > > > >>>>>>>> Hi Jun, > > > > > > > > > > > > > > > > > > >>>>>>>> Thanks for the reply and > digging into > > > > the > > > > > > > > details. > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>> JR23: Correct. The client > telemetry > > > > > > component > > > > > > > > > > will use > > > > > > > > > > > > > > > UUID-B > > > > > > > > > > > > > > > > > as the > > > > > > > > > > > > > > > > > > >>>>>>> client instance ID. > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>> JR23.1: Yes, I agree. It's not > ideal. > > > > > > When I > > > > > > > > was > > > > > > > > > > > > > drawing up > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >>>>> tables, > > > > > > > > > > > > > > > > > > >>>>>>> I was thinking that this might > be a > > > > > > > > possibility, > > > > > > > > > > but I'm > > > > > > > > > > > > > less > > > > > > > > > > > > > > > > > convinced > > > > > > > > > > > > > > > > > > >>>>>>> now. I think that I should > mandate > > > > that if > > > > > > a > > > > > > > > client > > > > > > > > > > > > > specifies > > > > > > > > > > > > > > > > > > >>>>>>> header.ClientInstanceId on > > > > > > > > > > GetTelemetrySubscriptions > > > > > > > > > > > > > request, > > > > > > > > > > > > > > > > > then > > > > > > > > > > > > > > > > > > >>>>>>> request.ClientInstanceId must > either be > > > > > > zero or > > > > > > > > > > equal to > > > > > > > > > > > > > > > > > > >>>>>>> header.ClientInstanceId. > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>> JR23.2: This is perhaps the > > > > interesting > > > > > > one. > > > > > > > > From > > > > > > > > > > its > > > > > > > > > > > > > > > original > > > > > > > > > > > > > > > > > > >>>>> intent, > > > > > > > > > > > > > > > > > > >>>>>>> it should be UUID-B (the > telemetry > > > > UUID), > > > > > > but > > > > > > > > then > > > > > > > > > > that > > > > > > > > > > > > > > > > > contradicts the > > > > > > > > > > > > > > > > > > >>>>>>> change in signature to remove the > > > > timeout. > > > > > > > > Unless > > > > > > > > > > I make > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > change > > > > > > > > > > > > > > > > > > >>>>> above, > > > > > > > > > > > > > > > > > > >>>>>>> in which case it will be UUID-H. > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>> Thanks, > > > > > > > > > > > > > > > > > > >>>>>>>> Andrew > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>> On 2026/05/12 17:23:58 Jun Rao > via dev > > > > > > wrote: > > > > > > > > > > > > > > > > > > >>>>>>>>> Hi, Andrew, > > > > > > > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>> Thanks for the reply. > > > > > > > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>> JR23. In the new client -> old > broker > > > > > > case, > > > > > > > > we > > > > > > > > > > have > > > > > > > > > > > > > > > > > > >>>>>>>>> header.ClientInstanceId=UUID-H > > > > > > > > > > > > > > > > > > >>>>>>>>> request.ClientInstanceId=UUID-B > > > > > > > > > > > > > > > > > > >>>>>>>>> response.ClientInstanceId=0 > > > > > > > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>> On the server side, I guess the > > > > telemetry > > > > > > > > > > component > > > > > > > > > > > > > will > > > > > > > > > > > > > > > use > > > > > > > > > > > > > > > > > > >>>>> UUID-B as > > > > > > > > > > > > > > > > > > >>>>>>> the > > > > > > > > > > > > > > > > > > >>>>>>>>> clientInstanceId? This has a > couple > > > > of > > > > > > > > > > implications. > > > > > > > > > > > > > > > > > > >>>>>>>>> JR23.1 On the server side, we > have > > > > two > > > > > > > > different > > > > > > > > > > > > > > > > > clientInstanceIds > > > > > > > > > > > > > > > > > > >>>>>>> used in > > > > > > > > > > > > > > > > > > >>>>>>>>> different places, UUID-H for > request > > > > > > logging > > > > > > > > and > > > > > > > > > > > > > UUID-B in > > > > > > > > > > > > > > > > > > >>>>> telemetry. > > > > > > > > > > > > > > > > > > >>>>>>> This > > > > > > > > > > > > > > > > > > >>>>>>>>> seems confusing since we can't > > > > uniquely > > > > > > > > identify > > > > > > > > > > a > > > > > > > > > > > > > client > > > > > > > > > > > > > > > on > > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >>>>> server > > > > > > > > > > > > > > > > > > >>>>>>>>> side. > > > > > > > > > > > > > > > > > > >>>>>>>>> JR23.2 On the client side. > what uuid > > > > does > > > > > > > > > > > > > > > > > clientInstanceId(Duration > > > > > > > > > > > > > > > > > > >>>>>>>>> timeout) return? If it returns > > > > UUID-H, it > > > > > > > > will be > > > > > > > > > > > > > confusing > > > > > > > > > > > > > > > > > since > > > > > > > > > > > > > > > > > > >>>>> it > > > > > > > > > > > > > > > > > > >>>>>>>>> doesn't match the ID used for > > > > telemetry > > > > > > on > > > > > > > > the > > > > > > > > > > server. > > > > > > > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>> Jun > > > > > > > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>> On Tue, May 12, 2026 at > 12:58 AM > > > > Andrew > > > > > > > > > > Schofield < > > > > > > > > > > > > > > > > > > >>>>>>> [email protected]> > > > > > > > > > > > > > > > > > > >>>>>>>>> wrote: > > > > > > > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>> Hi Jun, > > > > > > > > > > > > > > > > > > >>>>>>>>>> Thanks for your response. > > > > > > > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>> JR20: I have improved (I > hope) the > > > > > > wording. > > > > > > > > The > > > > > > > > > > client > > > > > > > > > > > > > > > sends > > > > > > > > > > > > > > > > > > >>>>>>>>>> request.clientInstanceId = 0 > and > > > > > > > > > > > > > header.clientInstanceId = > > > > > > > > > > > > > > > > > > >>>>> UUID-H, > > > > > > > > > > > > > > > > > > >>>>>>> and the > > > > > > > > > > > > > > > > > > >>>>>>>>>> broker responds > > > > > > > > > > response.clientInstanceId=UUID-H. In > > > > > > > > > > > > > this > > > > > > > > > > > > > > > way, > > > > > > > > > > > > > > > > > > >>>>> the > > > > > > > > > > > > > > > > > > >>>>>>> broker > > > > > > > > > > > > > > > > > > >>>>>>>>>> will have taken the UUID-H > from the > > > > > > header, > > > > > > > > and > > > > > > > > > > told > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > client > > > > > > > > > > > > > > > > > > >>>>> to > > > > > > > > > > > > > > > > > > >>>>>>> use it > > > > > > > > > > > > > > > > > > >>>>>>>>>> for client telemetry also. > > > > > > > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>> JR21: Done. Look for > "henceforth". > > > > > > > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>> JR22: Summary table added. > > > > > > > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>> Thanks, > > > > > > > > > > > > > > > > > > >>>>>>>>>> Andrew > > > > > > > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>> On 2026/05/11 19:18:24 Jun > Rao via > > > > dev > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > >>>>>>>>>>> Hi, Andrew, > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> Thanks for the reply. > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> JR20. "If the client > requests a new > > > > > > client > > > > > > > > > > instance > > > > > > > > > > > > > ID > > > > > > > > > > > > > > > on its > > > > > > > > > > > > > > > > > > >>>>>>> initial > > > > > > > > > > > > > > > > > > >>>>>>>>>>> GetTelemetrySubscriptions > request > > > > and > > > > > > it > > > > > > > > > > sends a > > > > > > > > > > > > > client > > > > > > > > > > > > > > > > > > >>>>> instance > > > > > > > > > > > > > > > > > > >>>>>>> ID in > > > > > > > > > > > > > > > > > > >>>>>>>>>> the > > > > > > > > > > > > > > > > > > >>>>>>>>>>> request header, the broker > will > > > > send > > > > > > back > > > > > > > > that > > > > > > > > > > client > > > > > > > > > > > > > > > > > instance > > > > > > > > > > > > > > > > > > >>>>> ID > > > > > > > > > > > > > > > > > > >>>>>>> rather > > > > > > > > > > > > > > > > > > >>>>>>>>>>> than generating a new UUID. > This > > > > will > > > > > > > > > > automatically > > > > > > > > > > > > > > > align the > > > > > > > > > > > > > > > > > > >>>>> UUID > > > > > > > > > > > > > > > > > > >>>>>>> in the > > > > > > > > > > > > > > > > > > >>>>>>>>>>> request headers and client > > > > telemetry." > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> This seems inconsistent with > > > > what's in > > > > > > the > > > > > > > > > > table. In > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >>>>> table, for > > > > > > > > > > > > > > > > > > >>>>>>>>>>> example, if the client has > the > > > > > > following: > > > > > > > > > > > > > > > > > > >>>>>>>>>>> GetTelemetrySubscriptions v0 > > > > > > > > > > > > > > > > > > >>>>>>>>>>> header.ClientInstanceId = > UUID-H > > > > > > > > > > > > > > > > > > >>>>>>>>>>> request.ClientInstanceId = > UUID-H > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> or > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> GetTelemetrySubscriptions v0 > > > > > > > > > > > > > > > > > > >>>>>>>>>>> header.ClientInstanceId = > UUID-H > > > > > > > > > > > > > > > > > > >>>>>>>>>>> request.ClientInstanceId = > UUID-R > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> the broker returns > > > > > > > > > > > > > > > > > > >>>>>>>>>>> response.ClientInstanceId = > 0. > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> JR21. It will be useful to > document > > > > > > what > > > > > > > > the > > > > > > > > > > new > > > > > > > > > > > > > client > > > > > > > > > > > > > > > does > > > > > > > > > > > > > > > > > > >>>>> with > > > > > > > > > > > > > > > > > > >>>>>>> the > > > > > > > > > > > > > > > > > > >>>>>>>>>>> returned > response.ClientInstanceId. > > > > > > Note > > > > > > > > that > > > > > > > > > > return > > > > > > > > > > > > > > > value > > > > > > > > > > > > > > > > > may > > > > > > > > > > > > > > > > > > >>>>> or > > > > > > > > > > > > > > > > > > >>>>>>> may not > > > > > > > > > > > > > > > > > > >>>>>>>>>>> be 0. > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> JR22. It's probably clearer > if we > > > > could > > > > > > > > > > populate the > > > > > > > > > > > > > > > table > > > > > > > > > > > > > > > > > > >>>>> with 4 > > > > > > > > > > > > > > > > > > >>>>>>>>>>> combinations: old/new > clients with > > > > > > old/new > > > > > > > > > > brokers. > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> Jun > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> On Fri, May 8, 2026 at > 2:49 AM > > > > Andrew > > > > > > > > > > Schofield < > > > > > > > > > > > > > > > > > > >>>>>>> [email protected]> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> wrote: > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> Hi Jun and Chia-Ping, > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> I've overhauled part of the > KIP > > > > to do > > > > > > with > > > > > > > > > > > > > alignment of > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >>>>>>> request > > > > > > > > > > > > > > > > > > >>>>>>>>>> header > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> client instance ID, client > > > > telemetry > > > > > > > > client > > > > > > > > > > > > > instance ID > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > >>>>> group > > > > > > > > > > > > > > > > > > >>>>>>>>>> protocol > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> member IDs. The alignment > is by > > > > > > > > convention, > > > > > > > > > > not > > > > > > > > > > > > > mandate > > > > > > > > > > > > > > > > > > >>>>> (SHOULD > > > > > > > > > > > > > > > > > > >>>>>>> not > > > > > > > > > > > > > > > > > > >>>>>>>>>> MUST). > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> It would be possible to go > around > > > > the > > > > > > > > > > existing RPCs > > > > > > > > > > > > > > > such as > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> ConsumerGroupHeartbeat and > > > > > > > > > > > > > GetTelemetrySubscriptions, > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > > >>>>> remove > > > > > > > > > > > > > > > > > > >>>>>>> the > > > > > > > > > > > > > > > > > > >>>>>>>>>> fields > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> containing the existing > > > > identifiers > > > > > > which > > > > > > > > are > > > > > > > > > > > > > intended > > > > > > > > > > > > > > > to be > > > > > > > > > > > > > > > > > > >>>>>>> aligned. > > > > > > > > > > > > > > > > > > >>>>>>>>>> Doing > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> so would be a bad idea > though, > > > > > > because we > > > > > > > > > > would then > > > > > > > > > > > > > > > have > > > > > > > > > > > > > > > > > RPC > > > > > > > > > > > > > > > > > > >>>>>>> versions > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> which essentially depend > upon the > > > > > > > > presence of > > > > > > > > > > a > > > > > > > > > > > > > tagged > > > > > > > > > > > > > > > field > > > > > > > > > > > > > > > > > > >>>>> in > > > > > > > > > > > > > > > > > > >>>>>>> the > > > > > > > > > > > > > > > > > > >>>>>>>>>> request > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> header. This is a > > > > > > protocol-compatibility > > > > > > > > > > nightmare. > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> I have removed the new > versions of > > > > > > > > > > > > > > > GetTelemetrySubscriptions > > > > > > > > > > > > > > > > > > >>>>> and > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> PushTelemetry. I have also > > > > explained > > > > > > the > > > > > > > > > > behavior of > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> GetTelemetrySubscriptions > in the > > > > > > presence > > > > > > > > and > > > > > > > > > > > > > absence > > > > > > > > > > > > > > > of a > > > > > > > > > > > > > > > > > > >>>>> client > > > > > > > > > > > > > > > > > > >>>>>>>>>> instance > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> ID in the request header. > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> Let me know what you think. > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> Thanks, > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> Andrew > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> On 2026/05/07 15:09:31 > Andrew > > > > > > Schofield > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> Hi Jun and Chia-Ping, > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> I've been thinking and > > > > discussing the > > > > > > > > > > changes to > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >>>>> KIP-714 > > > > > > > > > > > > > > > > > > >>>>>>> RPCs. > > > > > > > > > > > > > > > > > > >>>>>>>>>> There > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> are too many combinations > for my > > > > > > liking > > > > > > > > at the > > > > > > > > > > > > > moment. I > > > > > > > > > > > > > > > > > > >>>>> want to > > > > > > > > > > > > > > > > > > >>>>>>> take > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> another pass at this area > and will > > > > > > make an > > > > > > > > > > update > > > > > > > > > > > > > in a > > > > > > > > > > > > > > > few > > > > > > > > > > > > > > > > > > >>>>> days. > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> I intend to start a new > vote > > > > once we > > > > > > have > > > > > > > > > > consensus > > > > > > > > > > > > > > > because > > > > > > > > > > > > > > > > > > >>>>>>> the spec > > > > > > > > > > > > > > > > > > >>>>>>>>>> has > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> changed somewhat since the > > > > earliest > > > > > > votes. > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> Thanks, > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> Andrew > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> On 2026/05/06 17:28:27 > Chia-Ping > > > > Tsai > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>> hi Andrew > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>> chia_0: If the consensus > is to > > > > > > remove > > > > > > > > the > > > > > > > > > > > > > "duplicate" > > > > > > > > > > > > > > > > > > >>>>> field > > > > > > > > > > > > > > > > > > >>>>>>> from > > > > > > > > > > > > > > > > > > >>>>>>>>>> the > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> RPC payloads, the tagged > field in > > > > the > > > > > > > > header > > > > > > > > > > will > > > > > > > > > > > > > > > > > essentially > > > > > > > > > > > > > > > > > > >>>>>>> become a > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> required field. This means > the > > > > broker > > > > > > > > needs to > > > > > > > > > > > > > handle > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >>>>> edge > > > > > > > > > > > > > > > > > > >>>>>>> case > > > > > > > > > > > > > > > > > > >>>>>>>>>> where > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> both the header and the > request > > > > body > > > > > > have > > > > > > > > no > > > > > > > > > > > > > > > > > > >>>>> ClientInstanceId, > > > > > > > > > > > > > > > > > > >>>>>>> right? > > > > > > > > > > > > > > > > > > >>>>>>>>>> If > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> so, would you mind > clarifying the > > > > > > expected > > > > > > > > > > broker > > > > > > > > > > > > > > > behavior > > > > > > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > > >>>>>>> the KIP? > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>> Best, > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>> Chia-Ping > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>> On 2026/04/03 16:17:37 > Andrew > > > > > > Schofield > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>>> Hi, > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>>> I would like to start the > > > > > > discussion on > > > > > > > > > > KIP-1313. > > > > > > > > > > > > > > > This > > > > > > > > > > > > > > > > > > >>>>>>> adds a > > > > > > > > > > > > > > > > > > >>>>>>>>>> unique > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> client instance ID to the > request > > > > > > header > > > > > > > > of > > > > > > > > > > all > > > > > > > > > > > > > Kafka > > > > > > > > > > > > > > > > > > >>>>> protocol > > > > > > > > > > > > > > > > > > >>>>>>>>>> requests to > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> give a unique identifier > which > > > > can be > > > > > > > > used to > > > > > > > > > > > > > correlate > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > >>>>>>> requests > > > > > > > > > > > > > > > > > > >>>>>>>>>> from > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> each client for the > purposes of > > > > > > problem > > > > > > > > > > > > > determination. > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1313*3A*Client*instance*ID*in*all*request*headers__;JSsrKysrKys!!Ayb5sqE7!uqWf0-b_X82WmpmCYImD2W2rht_s_q5vHcqB9ToMV4IaeQbZF42eMJyS5XC5b5qE_qJJUj3KTCXcqEvYbwYS$ > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>>> Thanks, > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>>> Andrew > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > > > > > > > > >>>>>>> > > > > > > > > > > > > > > > > > > >>>>>> > > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
