Guilherme created KAFKA-14272:
---------------------------------
Summary: Consumer metrics being double emitted
Key: KAFKA-14272
URL: https://issues.apache.org/jira/browse/KAFKA-14272
Project: Kafka
Issue Type: Bug
Components: consumer
Reporter: Guilherme
Attachments: graph.png
Consumer metrics are emitted at consumer level, and also with topic and
partition as tags. for example `records-conumed-rate` is emitted per
[topic|https://github.com/apache/kafka/blob/1daa149730e3c56b7b6fe8f14369178273e8efc4/clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetcherMetricsRegistry.java#L123]
as well as
[overall|https://github.com/apache/kafka/blob/1daa149730e3c56b7b6fe8f14369178273e8efc4/clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetcherMetricsRegistry.java#L84].
Using the same metric names lead to metric systems rolling up both values,
imposing challenges on users creating dashboards / alerts for higher level
aggregations such as "Records consumed rate" across all instances of a
particular app - which use:
{code:java}
sum(records-consumed-rate){code}
which would end up summing the rate per topic and overall leading to double
counting the rate.
Attached a sample chart which shows both individual time series (with topic tag
and without), which was discovered while checking consumer-lag vs consumed-rate
based alerts.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
