Hello, do we have a metric showing the uptime? We could tag that metric with version information as well.
I like the idea of adding the version as a tag as well. However, I am not inclined to tag each metric with a KafkaVersion information. We could discuss which metrics could be tagged but let's keep that out of scope from this discussion. On Wed, 11 Oct 2023 at 07:37, Sophie Blee-Goldman <ableegold...@gmail.com> wrote: > Just to chime in here since I recently went through a similar thing, I > support adding the version > as a tag instead of introducing an entirely new metric for this. In fact I > just implemented exactly this > in a project that uses Kafka, for these reasons: > > 1. Adding the version as a tag means that all metrics which are already > collected will benefit, and lets you easily tell > at a glance which version a specific client metric corresponds to. This is > incredibly useful when looking at a dashboard > covering multiple instances from different sources. For example, imagine a > graph that plots the performance (eg bytes > consumed rate) of many individual consumers and which shows several of them > maxing out much lower than the rest. > If the metric is tagged with the version already, you can easily check if > the slow consumers are all using a specific version > and may be displaying a performance regression. If the version info has to > be plotted separately as its own metric, this is > much more of a hassle to check. > 2. Additional metrics can be expensive, but additional tags are almost > always free (at least, that is my understanding) > 3. As you guys already discussed, many systems (like Prometheus) require > numeric values, and it's pretty much impossible > to come up with a readable scheme for all the relevant versioning info -- > even if we removed the dots we're left with a rather > unreadable representation of the version and of course will need to solve > the "-SNAPSHOT" issue somehow. But beyond that, > in addition to the raw version we also wanted to emit the specific commit > id, which really needs to be a string. > > I'm pretty sure Kafka client metrics also include the commit id in addition > to the version. If we add the version to the tags, > we should consider adding the commit id as well. This is incredibly useful > for intermediate/SNAPSHOT versions, which > don't uniquely identify the specific code that is running. > > I would personally love to see a KIP start tagging the existing metrics > with the version info, and it sounds like this would also > solve your problem in a very natural way > > On Tue, Oct 10, 2023 at 5:42 AM Mickael Maison <mickael.mai...@gmail.com> > wrote: > > > Hi Hudeqi, > > > > Rather than creating a gauge with a dummy value, could we add the > > version (and commitId) as tags to an existing metric. > > For example, the alongside the existing Version and CommitId metrics > > we have StartTimeMs. Maybe we can have a StartTimeMs metrics with the > > version and commitId) as tags on it? The existing metric already has > > the brokerid (id) as tag. WDYT? > > > > Thanks, > > Mickael > > > > On Thu, Aug 31, 2023 at 4:59 AM hudeqi <16120...@bjtu.edu.cn> wrote: > > > > > > Thank you for your answer, Mickael. > > > If set the value of gauge to a constant value of 1, adding that tag key > > is "version" and value is the version value of the obtained string type, > > does this solve the problem? We can get the version by tag in prometheus. > > > > > > best, > > > hudeqi > > > > > > "Mickael Maison" <mickael.mai...@gmail.com>写道： > > > > Hi, > > > > > > > > Prometheus only support numeric values for metrics. This means it's > > > > not able to handle the kafka.server:type=app-info metric since Kafka > > > > versions are not valid numbers (3.5.0). > > > > As a workaround we could create a metric with the version without the > > > > dots, for example with value 350 for Kafka 3.5.0. > > > > > > > > Also in between releases Kafka uses the -SNAPSHOT suffix (for example > > > > trunk is currently 3.7.0-SNAPSHOT) so we should also consider a way > to > > > > handle those. > > > > > > > > Thanks, > > > > Mickael > > > > > > > > On Wed, Aug 30, 2023 at 2:51 PM hudeqi <16120...@bjtu.edu.cn> wrote: > > > > > > > > > > Hi, Kamal, thanks your reminding, but I have a question: It seems > > that I can't get this metric through "jmx_prometheus"? Although I > observed > > this metric through other tools. > > > > > > > > > > best, > > > > > hudeqi > > > > > > > > > > "Kamal Chandraprakash" & > lt;kamal.chandraprak...@gmail.com > > >写道： > > > > > > Hi Hudeqi, > > > > > > > > > > > > Kafka already emits the version metric. Can you check whether the > > below > > > > > > metric satisfies your requirement? > > > > > > > > > > > > kafka.server:type=app-info,id=0 > > > > > > > > > > > > -- > > > > > > Kamal > > > > > > > > > > > > On Mon, Aug 28, 2023 at 2:29 PM hudeqi <16120...@bjtu.edu.cn> > > wrote: > > > > > > > > > > > > > Hi, all, I want to submit a minor kip to add a metric, which > > supports to > > > > > > > get the running kafka server verison, the wiki url is here > > > > > > > > > > > > > > Motivation > > > > > > > > > > > > > > At present, it is impossible to perceive the Kafka version that > > the broker > > > > > > > is running from the perspective of metrics. If multiple Kafka > > versions are > > > > > > > deployed in a cluster due to various reasons, it is difficult > > for us to > > > > > > > intuitively understand the version distribution. > > > > > > > > > > > > > > So, I want to add a kafka version metric indicating the version > > of the > > > > > > > current running kafka server, it can help us to perceive the > > mixed > > > > > > > distribution of multiple versions, and to perceive the progress > > of version > > > > > > > upgrade in the cluster in real time. > > > > > > > > > > > > > > Proposed Changes > > > > > > > > > > > > > > When instantiating kafkaServer/BrokerServer, register > > `KafkaVersion` gauge > > > > > > > metric, whose value is obtained by `VersionInfo.getVersion`. > And > > remove all > > > > > > > related metrics when kafkaServer/BrokerServer shutdown. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > best, > > > > > > > > > > > > > > hudeqi > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
