Two quick mistakes to clarify: When I say ClusterMetadata, I mean request 60, DescribeCluster.
Also, the email subject of this entire thread should be "[DISCUSS] KIP-885: Expose Broker's Name and Version to Clients”. I must have either accidentally pasted the “Skip to end of metadata”, or did not delete something. Cheers, -Travis On 2022/11/11 16:45:12 Travis Bischel wrote: > Thanks for the replies David and Magnus > > David: > > 02: From a client implementation perspective, it is easier to gate features > based on the max version numbers returned per request, rather than any > textual representation of a string. I’m not really envisioning a client > implementation trying to match on an undefined string, especially if it’s > documented as just metadata information. > > 03: Interesting, I may be one of the few that does query the version > directly. Perhaps this can be some new information that is instead added to > request 60, ClusterMetadata? The con with ClusterMetadata is that I’m > interested in this information on a per-broker basis. We could add these > fields per each broker in the Brokers field, though. > > Magnus: > > As far as I can see, only my franz-go client offers this ability to “guess” > the version of a broker — and it’s historically done so through ApiVersions, > which was the only way to do this. This feature was used in three projects by > two people: my kcl project, and the formerly-known-as Kowl and Kminion > projects. > > After reading through most of the discussion thread on KIP-35, it seems that > the conversation about using a broker version string / cluster aggregate > version was specifically related to having the client choose how to behave > (i.e., choose what versions of requests to use). The discussion was not > around having the broker version as a piece of information that a client can > use in log lines or for end-user presentation purposes. > > It seems a bit of an misdirected worry that a client implementor may > accidentally use an unstructured string field for versioning purposes, > especially when another field (ApiKeys) exists for versioning purposes and is > widely known. Implementing a Kafka client is quite complex and there are so > many other areas an implementor can go wrong, I’m not sure that we should be > worried about an unstructured and documented metadata field. > > "the existing ApiVersionsReq … this information is richer than a single > version string" > > Agree, this true for clients. However, it’s completely useless visually for > end users. > > The reason Kminion used the version guess was two fold: to emit log a log on > startup that the process was talking to Kafka v#.#, and to emit a const gauge > metric for Prometheus where people could monitor external to Kafka what > version each broker was running. > > Kowl uses the version guess to display the Kafka version the process is > talking to immediately when you go to the Brokers panel. I envision that this > same UI display can be added to Conduktor, even Confluent, etc. > > kcl uses the version guess as an extremely quick debugging utility: software > engineers (not cluster administrators) might not always know what version of > Kafka they are talking to, but they are trying to use a client. I often > receive questions about “why isn’t this xyz thing working”, I ask for their > cluster version with kcl, and then we can jump into diagnosing the problem > much quicker. > > I think, if we focus on the persona of a cluster administrator, it’s not > obvious what the need for this KIP is. For me, focusing on the perspective of > an end-user of a client makes the need a bit clearer. It is not the > responsibility of an end-user to manage the cluster version, but it is the > responsibility of an end-user to know which version of a cluster they are > talking to so that they know which fields / requests / behaviors are > supported in a client > > All that said: I think this information is worth it and unlikely to be > misused. IMO, ApiVersions is the main place to include this information, but > another alternative is ClusterMetadata. Adding these fields to > ClusterMetadata might be a bit awkward and may return stale information > sometimes during a rolling upgrade. > > Curious to know your thoughts, and again thank you for the consideration and > replies, > - Travis > > On 2022/11/11 13:07:37 Magnus Edenhill wrote: > > Hi Travis and thanks for the KIP, two comments below: > > > > > > Den fre 11 nov. 2022 kl 13:37 skrev David Jacot <da...@gmail.com>: > > > > > 02: I am a bit concerned by clients that could misuse these information. > > > For instance, one may be tempted to rely on the version to decide whether > > > a > > > feature is enabled or not. The api versions should remain the source of > > > truth but we cannot enforce it with the proposed approach. That may be > > > acceptable though. > > > > > > > We proposed including this in the original ApiVersionRequest KIP-35 (waaay > > back), but it was rejected > > for exactly this reason; that it may(will) be misused by clients. > > > > > > > > I would also like to question the actual end-user use of this information, > > the existing ApiVersionsReq > > already provides - on a detailed level - what features and functionality > > the broker supports - > > this information is richer than a single version string. > > > > The KIP says "End users can quickly check from a client if the cluster is > > up to date" and > > "Clients can also use the broker version in log lines so that end users can > > quickly see > > if a version looks about right or if something is seriously broken.": > > > > In my mind that's not typically the end-users role or responsibility, but > > that of the Kafka cluster operator, > > who'll already know the version being deployed. > > > > > > Regards, > > Magnus > > > > > > > > > Le jeu. 10 nov. 2022 à 19:10, Travis Bischel <tr...@gmail.com> a > > > écrit : > > > > > > > Hi all, > > > > > > > > I've written a KIP to expose the BrokerSoftwareName and > > > > BrokerSoftwareVersion to clients: > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-885%3A+Expose+Broker%27s+Name+and+Version+to+Clients > > > > > > > > If we agree this is useful, it would be great to have this in by 3.4. > > > > > > > > Thank you, > > > > - Travis > > > > > > > > >
