[
https://issues.apache.org/jira/browse/KAFKA-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Charles-Eddy updated KAFKA-19753:
---------------------------------
Description:
Hello,
Since Kafka 4.1.0, we're experiencing an issue with Kafka Streams metrics for
topics containing dots in their names. (I think the problem is also for simple
usage of consumers not only kstream)
In FetchMetricsManager, methods like {{recordRecordsFetched()}} now create
duplicate sensors for the same topic if they contain dot in their name:
{code:java}
void recordRecordsFetched(String topic, int records) {
String name = topicRecordsFetchedMetricName(topic);
maybeRecordDeprecatedRecordsFetched(name, topic, records); <-- here we create
another sensor if the topic name contains dot
Sensor recordsFetched = new SensorBuilder(metrics, name, () -> Map.of("topic",
topic))
.withAvg(metricsRegistry.topicRecordsPerRequestAvg)
.withMeter(metricsRegistry.topicRecordsConsumedRate,
metricsRegistry.topicRecordsConsumedTotal)
.build();
recordsFetched.record(records);
} {code}
It currently record two sensors, one with my original topic name, one time with
a topic name with dots replaced by underscore.
-While we can work around this by reversing the transformation in our case
(replacing underscores back to dots in Micrometer filters) or by removing this
specific list of metrics, this does not feel like a long-term solution for us.-
Currently using spring-boot and micrometer, it's really hard to make one of
those duplicated metrics to be removed.
Could a configuration option be added to disable one of the two sensor to avoid
having duplicated metrics?
Thanks in advance
was:
Hello,
Since Kafka 4.1.0, we're experiencing an issue with Kafka Streams metrics for
topics containing dots in their names. (I think the problem is also for simple
usage of consumers not only kstream)
In FetchMetricsManager, methods like {{recordRecordsFetched()}} now create
duplicate sensors for the same topic if they contain dot in their name:
{code:java}
void recordRecordsFetched(String topic, int records) {
String name = topicRecordsFetchedMetricName(topic);
maybeRecordDeprecatedRecordsFetched(name, topic, records); <-- here we create
another sensor if the topic name contains dot
Sensor recordsFetched = new SensorBuilder(metrics, name, () -> Map.of("topic",
topic))
.withAvg(metricsRegistry.topicRecordsPerRequestAvg)
.withMeter(metricsRegistry.topicRecordsConsumedRate,
metricsRegistry.topicRecordsConsumedTotal)
.build();
recordsFetched.record(records);
} {code}
It currently record two sensors, one with my original topic name, one time with
a topic name with dots replaced by underscore.
While we can work around this by reversing the transformation in our case
(replacing underscores back to dots in Micrometer filters) or by removing this
specific list of metrics, this does not feel like a long-term solution for us.
Could a configuration option be added to disable the deprecated
underscore-replaced metrics?
Thanks in advance
> Metrics from FetchMetricsManager containing a topic tag are duplicated
> ----------------------------------------------------------------------
>
> Key: KAFKA-19753
> URL: https://issues.apache.org/jira/browse/KAFKA-19753
> Project: Kafka
> Issue Type: Bug
> Components: consumer, metrics
> Affects Versions: 4.1.0
> Reporter: Charles-Eddy
> Priority: Minor
> Attachments: Screenshot 2025-10-03 at 09.44.11.png, Screenshot
> 2025-10-03 at 09.46.42.png
>
>
> Hello,
> Since Kafka 4.1.0, we're experiencing an issue with Kafka Streams metrics for
> topics containing dots in their names. (I think the problem is also for
> simple usage of consumers not only kstream)
> In FetchMetricsManager, methods like {{recordRecordsFetched()}} now create
> duplicate sensors for the same topic if they contain dot in their name:
> {code:java}
> void recordRecordsFetched(String topic, int records) {
> String name = topicRecordsFetchedMetricName(topic);
> maybeRecordDeprecatedRecordsFetched(name, topic, records); <-- here we
> create another sensor if the topic name contains dot
> Sensor recordsFetched = new SensorBuilder(metrics, name, () ->
> Map.of("topic", topic))
> .withAvg(metricsRegistry.topicRecordsPerRequestAvg)
> .withMeter(metricsRegistry.topicRecordsConsumedRate,
> metricsRegistry.topicRecordsConsumedTotal)
> .build();
> recordsFetched.record(records);
> } {code}
> It currently record two sensors, one with my original topic name, one time
> with a topic name with dots replaced by underscore.
> -While we can work around this by reversing the transformation in our case
> (replacing underscores back to dots in Micrometer filters) or by removing
> this specific list of metrics, this does not feel like a long-term solution
> for us.-
> Currently using spring-boot and micrometer, it's really hard to make one of
> those duplicated metrics to be removed.
> Could a configuration option be added to disable one of the two sensor to
> avoid having duplicated metrics?
> Thanks in advance
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
