[
https://issues.apache.org/jira/browse/KAFKA-9175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mickael Maison updated KAFKA-9175:
----------------------------------
Description:
While looking at MirrorMaker 2 metrics with [~ecomar], we noticed the topic
partition metrics were invalid.
!image-2019-11-12-17-42-45-773.png|width=822,height=615!
There is no traffic on the topic spp.hello but its metrics are constantly
updating.
The issue is in {{MirrorMetrics.PartitionMetrics}}. In the constructor, Sensors
are built using {{metrics.sensor()}} with a name that does not include the
topic partition. The method {{metrics.sensor()}} does not always create a new
Sensor but can return an existing Sensor if one exists for the specified name.
So in practice, if a Task is handling many topic partitions, they all share the
same Sensors!
This renders the topic partition metrics unusable and really prevents running
MirrorMaker 2 in a production environment.
was:
While looking at MirrorMaker 2 metrics with [~ecomar], we noticed the topic
partition metrics were invalid.
!image-2019-11-12-17-42-45-773.png|width=822,height=615!
There is no traffic on the topic spp.hello but its metrics are constantly
updating.
The issue is in MirrorMetrics.PartitionMetrics. In the constructor, Sensors are
built using metrics.sensor() with a name that does not include the topic
partition. metrics.sensor() does not always create a new Sensor but can return
an existing Sensor if one exists for the specified name. So in practice, if a
Task is handling many topic partitions, they all share the same Sensors!
This renders the topic partition metrics unusable and really prevents running
MirrorMaker 2 in a production environment.
> MirrorMaker 2 emits invalid topic partition metrics
> ---------------------------------------------------
>
> Key: KAFKA-9175
> URL: https://issues.apache.org/jira/browse/KAFKA-9175
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 2.4.0
> Reporter: Mickael Maison
> Assignee: Mickael Maison
> Priority: Blocker
> Attachments: image-2019-11-12-17-42-45-773.png
>
>
> While looking at MirrorMaker 2 metrics with [~ecomar], we noticed the topic
> partition metrics were invalid.
> !image-2019-11-12-17-42-45-773.png|width=822,height=615!
> There is no traffic on the topic spp.hello but its metrics are constantly
> updating.
> The issue is in {{MirrorMetrics.PartitionMetrics}}. In the constructor,
> Sensors are built using {{metrics.sensor()}} with a name that does not
> include the topic partition. The method {{metrics.sensor()}} does not always
> create a new Sensor but can return an existing Sensor if one exists for the
> specified name. So in practice, if a Task is handling many topic partitions,
> they all share the same Sensors!
> This renders the topic partition metrics unusable and really prevents running
> MirrorMaker 2 in a production environment.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
