[jira] [Commented] (KAFKA-7219) Add topic/partition level metrics.
[ https://issues.apache.org/jira/browse/KAFKA-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563199#comment-16563199 ] Ismael Juma commented on KAFKA-7219: Thanks for the JIRA. Note that adding metrics requires a KIP (https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals). The main drawback is the additional memory usage of all these new metrics. > Add topic/partition level metrics. > -- > > Key: KAFKA-7219 > URL: https://issues.apache.org/jira/browse/KAFKA-7219 > Project: Kafka > Issue Type: Improvement > Components: metrics >Reporter: Satish Duggana >Assignee: Satish Duggana >Priority: Major > Labels: needs-kip > > Currently, Kafka generates different metrics for topics on a broker. > - MessagesInPerSec > - BytesInPerSec > - BytesOutPerSec > - BytesRejectedPerSec > - ReplicationBytesInPerSec > - ReplicationBytesOutPerSec > - FailedProduceRequestsPerSec > - FailedFetchRequestsPerSec > - TotalProduceRequestsPerSec > - TotalFetchRequestsPerSec > - FetchMessageConversionsPerSec > - ProduceMessageConversionsPerSec > Add metrics for individual partitions instead of having only at topic level. > Some of these partition level metrics are useful for monitoring applications > to monitor individual topic/partitions. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7219) Add topic/partition level metrics.
[ https://issues.apache.org/jira/browse/KAFKA-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563562#comment-16563562 ] Satish Duggana commented on KAFKA-7219: --- I will work on having KIP with the details and update this issue with the location when it is ready. > Add topic/partition level metrics. > -- > > Key: KAFKA-7219 > URL: https://issues.apache.org/jira/browse/KAFKA-7219 > Project: Kafka > Issue Type: Improvement > Components: metrics >Reporter: Satish Duggana >Assignee: Satish Duggana >Priority: Major > Labels: needs-kip > > Currently, Kafka generates different metrics for topics on a broker. > - MessagesInPerSec > - BytesInPerSec > - BytesOutPerSec > - BytesRejectedPerSec > - ReplicationBytesInPerSec > - ReplicationBytesOutPerSec > - FailedProduceRequestsPerSec > - FailedFetchRequestsPerSec > - TotalProduceRequestsPerSec > - TotalFetchRequestsPerSec > - FetchMessageConversionsPerSec > - ProduceMessageConversionsPerSec > Add metrics for individual partitions instead of having only at topic level. > Some of these partition level metrics are useful for monitoring applications > to monitor individual topic/partitions. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7219) Add topic/partition level metrics.
[ https://issues.apache.org/jira/browse/KAFKA-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565641#comment-16565641 ] Sriharsha Chintalapani commented on KAFKA-7219: --- [~satish.duggana] one more suggestion to add a Highwatermark metric to each topic/partition. This will make it easier to calculate the consumer lag from metrics instead of making a broker request to get the highwatermark for a topic partition. > Add topic/partition level metrics. > -- > > Key: KAFKA-7219 > URL: https://issues.apache.org/jira/browse/KAFKA-7219 > Project: Kafka > Issue Type: Improvement > Components: metrics >Reporter: Satish Duggana >Assignee: Satish Duggana >Priority: Major > Labels: needs-kip > > Currently, Kafka generates different metrics for topics on a broker. > - MessagesInPerSec > - BytesInPerSec > - BytesOutPerSec > - BytesRejectedPerSec > - ReplicationBytesInPerSec > - ReplicationBytesOutPerSec > - FailedProduceRequestsPerSec > - FailedFetchRequestsPerSec > - TotalProduceRequestsPerSec > - TotalFetchRequestsPerSec > - FetchMessageConversionsPerSec > - ProduceMessageConversionsPerSec > Add metrics for individual partitions instead of having only at topic level. > Some of these partition level metrics are useful for monitoring applications > to monitor individual topic/partitions. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7219) Add topic/partition level metrics.
[ https://issues.apache.org/jira/browse/KAFKA-7219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565692#comment-16565692 ] George Teo commented on KAFKA-7219: --- Another slightly bigger ask would be to also emit Lowwatermark metric for each topic-partition. This makes it easier to compute retention safety (consumer offset - lowwatermark) metrics. This could potentially be put into the log cleaner. > Add topic/partition level metrics. > -- > > Key: KAFKA-7219 > URL: https://issues.apache.org/jira/browse/KAFKA-7219 > Project: Kafka > Issue Type: Improvement > Components: metrics >Reporter: Satish Duggana >Assignee: Satish Duggana >Priority: Major > Labels: needs-kip > > Currently, Kafka generates different metrics for topics on a broker. > - MessagesInPerSec > - BytesInPerSec > - BytesOutPerSec > - BytesRejectedPerSec > - ReplicationBytesInPerSec > - ReplicationBytesOutPerSec > - FailedProduceRequestsPerSec > - FailedFetchRequestsPerSec > - TotalProduceRequestsPerSec > - TotalFetchRequestsPerSec > - FetchMessageConversionsPerSec > - ProduceMessageConversionsPerSec > Add metrics for individual partitions instead of having only at topic level. > Some of these partition level metrics are useful for monitoring applications > to monitor individual topic/partitions. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
