Hi, Andrew, Thanks for the reply.
JR14. So the description for GetTelemetrySubscriptions "Unique id for this client instance, must be set to 0 on the first request (v0 only)." is not accurate. It would be useful to clarify that in the KIP. Jun On Tue, May 5, 2026 at 11:34 PM Andrew Schofield <[email protected]> wrote: > Hi Jun, > Thanks for the reply. > > JR14: With this new KIP, the client uses the client instance ID that it > itself generated, even in the GetTelemetrySubscriptions v0 request. A > broker supporting only GetTelemetrySubscriptions v0 will already be able to > handle this. > > In the original behavior of KIP-714, the client would arbitrarily choose a > broker for its first GetTelemetrySubscriptions request and send a zero > clientInstanceId to get the broker to generate an id and return it in the > response. Once received, the client would then use this clientInstanceId in > all future KIP-714 requests. The client was already permitted to send > GetTelemetrySubscriptions/PushTelemetry to any broker with this > clientInstanceId, even to brokers which did not generate the id. There was > already the idea that the KIP-714 request received by a broker could > contain a non-zero clientInstanceId that it did not generate. With > KIP-1313, this is always the case. The client will always generate its own > clientInstanceId, and it will always send a non-zero value. > > Thanks, > Andrew > > On 2026/05/05 20:35:20 Jun Rao via dev wrote: > > Hi, Andrew, > > > > Thanks for the reply. > > > > JR14. Consider the case that a new client connects to an old broker. The > > new client generates a new clientInstanceId (uuidA) and includes it in > the > > initial request. From the ApiVersionResponse, the client realizes that > the > > broker only supports GetTelemetrySubscriptions V0. It issues that request > > and gets a different clientInstanceId (uuidB) in the > > GetTelemetrySubscriptions V0 response. What clientInstanceId should the > > client use in subsequent requests? If the client uses uuidA, that ID > won't > > match the broker's registration for this client. Using uuidB violates the > > assumption that clientInstanceId is immutable and may also impact > memberId. > > > > Jun > > > > > > On Tue, May 5, 2026 at 12:22 PM Andrew Schofield <[email protected]> > > wrote: > > > > > Hi Jun, > > > Thanks for the reply. > > > > > > JR7: I'm guessing at a detail of the implementation but I think it > would > > > be reasonable to call a method on ChannelMetadataRegistry on every > request > > > with a V2 header that sets or checks the clientInstanceId. > > > > > > JR13: Thanks. Updated. > > > > > > Thanks, > > > Andrew > > > > > > On 2026/05/05 19:06:52 Jun Rao via dev wrote: > > > > Hi, Andrew, > > > > > > > > Thanks for the reply. > > > > > > > > JR7. ChannelMetadataRegistry.registerClientInformation() is currently > > > only > > > > called on ApiVersionRequest. When registering clientInstanceId in > > > > ChannelMetadataRegistry, do we plan to do that on every request with > V2 > > > > header? > > > > > > > > JR13. GetTelemetrySubscriptions has the following, which seems > outdated. > > > > "After this KIP, the Apache Kafka Java client will send the > > > > ClientInstanceId in the request header and also in the request body" > > > > > > > > Jun > > > > > > > > On Tue, May 5, 2026 at 10:33 AM Andrew Schofield < > [email protected]> > > > > wrote: > > > > > > > > > Hi Jun, > > > > > Thanks for your comments. > > > > > > > > > > JR5: This is a very interesting one. I did have some nervousness > about > > > > > changing client ID without RPC version bumps, and you've brought > up an > > > > > excellent example. > > > > > > > > > > I have removed the change to the client ID behavior from this KIP. > > > While I > > > > > still think it would be beneficial to stop sending the client ID > > > strings > > > > > repeatedly, it needs to be done properly. That was not the main > > > purpose of > > > > > this KIP, so I think it's best to pull back on this part. > > > > > > > > > > JR6: Done. > > > > > > > > > > JR7: I wonder why the authentication needs to have completed > before the > > > > > client instance ID would be cached. The client instance ID is > > > generated by > > > > > the client and the client should use the same value for all of its > > > > > requests, and the broker will check that it has. > > > > > > > > > > JR8: As mentioned in my response to JR5, I've removed the client ID > > > > > changes. > > > > > > > > > > JR9: Done. > > > > > > > > > > JR10: I've clarified that the checking only applies to requests > that > > > use > > > > > the v2 header. > > > > > > > > > > JR11: I've added v1 for GetTelemetrySubscriptions and > PushTelemetry in > > > > > this KIP. > > > > > > > > > > JR12: Done. > > > > > > > > > > Thanks, > > > > > Andrew > > > > > > > > > > On 2026/05/05 00:04:13 Jun Rao via dev wrote: > > > > > > Hi, Andrew, > > > > > > > > > > > > Thanks for the updated KP. A few more comments. > > > > > > > > > > > > JR5. It seems that this design could break the clientId based > quota > > > if a > > > > > > new client communicates with an old broker. The KIP proposes > sending > > > > > > ClientId only on the first request, then null for all subsequent > > > > > requests, > > > > > > without bumping up the request version. So, the old server will > > > receive > > > > > the > > > > > > first request and ignore ClientId. The client doesn't know the > server > > > > > > ignores the clientId and sends a null clientId to an old broker > for > > > > > > subsequent requests. The broker will enforce quotas against a > null > > > > > > clientId, which is incorrect. > > > > > > > > > > > > JR6. Motivation: It would be useful to motivate the benefit of > > > including > > > > > > clientInstanceID on every request. For example, one immediate > > > benefit is > > > > > > that it can be included in request logging for troubleshooting. > > > > > > > > > > > > JR7. "The initial client instance ID for each connection will be > > > cached > > > > > by > > > > > > the broker for checking" > > > > > > It would be useful to clarify that the caching occurs only after > > > > > > authentication completes. Some requests can be issued before > > > > > authentication > > > > > > completes. > > > > > > > > > > > > JR8. "This KIP also proposes a second change to the Kafka > protocol. > > > It > > > > > > proposes sending the client ID only on the initial request on > each > > > > > > connection, and then sending a null client ID for all subsequent > > > > > requests." > > > > > > Similar to the above, it would be useful to clarify that the > > > clientId is > > > > > > included on the first request after authentication completes. > > > > > > > > > > > > JR9. GetTelemetrySubscriptionsRequest > > > > > > "about": "Unique id for this client instance, must be set > to 0 > > > on > > > > > the > > > > > > first request." > > > > > > Could we change the description above since clientInstanceId is > no > > > > > longer 0 > > > > > > on first request? > > > > > > > > > > > > JR10. "Once a client has specified a client instance ID in the > > > request > > > > > > header of its first request, any subsequent requests which are > > > missing > > > > > the > > > > > > client instance ID or which specify a different value for the > client > > > > > > instance ID will be rejected with error code INVALID_REQUEST." > > > > > > Hmm, does this work for a new AdminClient sending an > > > OffsetDeleteRequest > > > > > > that still uses a v1 request header? > > > > > > > > > > > > JR11. Since ClientInstanceId exists in the header, should we > remove > > > > > > clientInstanceId from both GetTelemetrySubscriptions and > > > PushTelemetry? > > > > > > > > > > > > JR12. Could we add the description in RequestHeader to explain > the > > > > > > difference between ClientId and ClientInstanceId? Historically, > > > clientId > > > > > is > > > > > > meant for identifying an application, not an instance of an > > > application. > > > > > > > > > > > > Jun > > > > > > > > > > > > On Thu, Apr 30, 2026 at 3:00 AM Andrew Schofield < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi Apoorv, > > > > > > > Thanks for your suggestion. > > > > > > > > > > > > > > AM1: That's a great idea. Done. > > > > > > > > > > > > > > Thanks, > > > > > > > Andrew > > > > > > > > > > > > > > On 2026/04/27 12:24:12 Apoorv Mittal wrote: > > > > > > > > Hi Andrew, > > > > > > > > Thanks for the KIP. I have one question: > > > > > > > > > > > > > > > > AM1: With KIP-1082, the consumer now generates its own > member ID > > > at > > > > > > > > construction time. With KIP-1313, the client will generate > its > > > own > > > > > client > > > > > > > > instance ID as a Uuid at construction time. Both are > > > client-generated > > > > > > > > UUIDs, immediately available at startup, and remain constant > for > > > the > > > > > > > > process's lifetime. For a consumer (or share consumer), these > > > two IDs > > > > > > > serve > > > > > > > > overlapping identification purposes at the same scope: > > > identifying > > > > > this > > > > > > > > particular client incarnation. The ClientInstanceIds > interface in > > > > > Streams > > > > > > > > already maintains a separate client instance ID per consumer > per > > > > > thread, > > > > > > > so > > > > > > > > there is no cardinality mismatch even in multi-threaded > Streams > > > > > > > > applications. Has unifying these been considered? The > consumer > > > could > > > > > > > > generate one UUID at startup and use it both as its member > ID in > > > > > > > heartbeat > > > > > > > > requests and as the client instance ID in request headers and > > > > > telemetry. > > > > > > > > > > > > > > > > Regards, > > > > > > > > Apoorv Mittal > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Apr 24, 2026 at 5:48 PM Jun Rao via dev < > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > > > > > Thanks for the reply. > > > > > > > > > > > > > > > > > > JR1. Caching the clientID from the first request in > > > > > > > ChannelMetadataRegistry > > > > > > > > > sounds good. We can then use it to fill the request header > in > > > > > > > subsequent > > > > > > > > > requests. > > > > > > > > > > > > > > > > > > JR2.1 Ok. It's fine to include clientInstanceID in the > request > > > > > header. > > > > > > > It > > > > > > > > > would be useful to document that it's only set on the first > > > > > request and > > > > > > > > > cached in ChannelMetadataRegistry. > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > On Fri, Apr 24, 2026 at 6:28 AM Andrew Schofield < > > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Jun, > > > > > > > > > > Thanks for your comments. > > > > > > > > > > > > > > > > > > > > JR1: I was thinking of attaching it to some channel-based > > > > > context, > > > > > > > such > > > > > > > > > as > > > > > > > > > > the ChannelMetadataRegistry. I've not dug into the code > in > > > > > detail at > > > > > > > this > > > > > > > > > > point. However, as mentioned in the KIP, removing the > client > > > ID > > > > > from > > > > > > > all > > > > > > > > > > except the initial request is perhaps overreach in this > KIP. > > > I > > > > > wonder > > > > > > > > > > whether actually this would be better handled in a major > > > release. > > > > > > > Let me > > > > > > > > > > know your thoughts here. > > > > > > > > > > > > > > > > > > > > JR2.1: I did consider put it in the ApiVersionsRequest, > but > > > I do > > > > > > > want it > > > > > > > > > > in every request to help with problem diagnosis. Also, > > > > > ApiVersions is > > > > > > > > > not a > > > > > > > > > > mandatory part of the protocol, so we can't rely on it > being > > > the > > > > > > > first > > > > > > > > > > request. > > > > > > > > > > > > > > > > > > > > JR2.2: Similar to JR1, I was thinking of hanging it off > some > > > > > > > > > channel-based > > > > > > > > > > context. > > > > > > > > > > > > > > > > > > > > JR3: Indeed. I've added share consumer. > > > > > > > > > > > > > > > > > > > > JR4: Thinking about migration more, it seems to me that > it is > > > > > best > > > > > > > not to > > > > > > > > > > introduce new versions of the PushTelemetry and > > > > > > > GetTelemetrySubscriptions > > > > > > > > > > RPCs. For an AK 4.4 Java client, we know that the client > will > > > > > > > generate a > > > > > > > > > > client instance ID before its initial connection. For any > > > other > > > > > > > client, > > > > > > > > > we > > > > > > > > > > do not know that. Consequently, even in the broker > supports a > > > > > newer > > > > > > > > > version > > > > > > > > > > of PushTelemetry which omits ClientInstanceId from the > > > request > > > > > > > schema, we > > > > > > > > > > would still need to support the broker-side > ClientInstanceId > > > > > > > > > initialization > > > > > > > > > > in which the response contains the ID created by the > broker. > > > I > > > > > think > > > > > > > that > > > > > > > > > > leaving the slightly redundant ClientInstanceId in the > > > KIP-714 > > > > > RPCs > > > > > > > is > > > > > > > > > > best. I've changed the text accordingly. Let me know > what you > > > > > think. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > On 2026/04/23 19:04:37 Jun Rao via dev wrote: > > > > > > > > > > > Hi, Andrew, > > > > > > > > > > > > > > > > > > > > > > Thanks for the KIP. A few comments. > > > > > > > > > > > > > > > > > > > > > > JR1. Currently, you can configure a client quota based > on > > > > > > > > > > > clientId. Currently, the broker extracts the clientId > from > > > the > > > > > > > request > > > > > > > > > > > header. If clientId is not included in every request, > can > > > we > > > > > > > document > > > > > > > > > > where > > > > > > > > > > > the server will obtain it? > > > > > > > > > > > > > > > > > > > > > > JR2. client instance ID: > > > > > > > > > > > JR2.1 Is it only sent on the first request in a > > > connection? If > > > > > so, > > > > > > > have > > > > > > > > > > we > > > > > > > > > > > considered making it part of the ApiVersionRequest > instead > > > of > > > > > the > > > > > > > > > request > > > > > > > > > > > header? > > > > > > > > > > > JR2.2 It would be useful to include the client > instance ID > > > in > > > > > > > request > > > > > > > > > > > logging on the server side. Could we document how to > make > > > that > > > > > > > > > available > > > > > > > > > > > for each request the server processes? > > > > > > > > > > > > > > > > > > > > > > JR3. "The client instance ID will be calculated during > the > > > > > > > constructor > > > > > > > > > of > > > > > > > > > > > the Producer , Consumer and Admin". > > > > > > > > > > > What about ShareConsumer? > > > > > > > > > > > > > > > > > > > > > > JR4. GetTelemetrySubscriptions: Could we document the > > > changes > > > > > in > > > > > > > the > > > > > > > > > > > response too? > > > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Apr 3, 2026 at 9:18 AM Andrew Schofield < > > > > > > > > > > [email protected]> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > I would like to start the discussion on KIP-1313. > This > > > adds a > > > > > > > unique > > > > > > > > > > > > client instance ID to the request header of all Kafka > > > > > protocol > > > > > > > > > > requests to > > > > > > > > > > > > give a unique identifier which can be used to > correlate > > > the > > > > > > > requests > > > > > > > > > > from > > > > > > > > > > > > each client for the purposes of problem > determination. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/KAFKA/KIP-1313*3A*Client*instance*ID*in*all*request*headers__;JSsrKysrKys!!Ayb5sqE7!txpES9zmXlCh8usguNsiJLRnSXvMtgWuM6R1rx8VxYYyi-sThQTmaOrq3ZfhArG25CjhHKuaLaJ63z_dzny8$ > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
