[jira] [Commented] (KAFKA-6504) Connect: Some per-task-metrics not working
[ https://issues.apache.org/jira/browse/KAFKA-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355785#comment-16355785 ] Robert Yokota commented on KAFKA-6504: -- [~steff1193], it's not just the metric name change. The PR also changes calls to metricGroup.metrics().sensor() with metricGroup.sensor(). This ensures that sensors names will be prepended with the metric group ID to ensure they are not shared across metric groups. > Connect: Some per-task-metrics not working > -- > > Key: KAFKA-6504 > URL: https://issues.apache.org/jira/browse/KAFKA-6504 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 1.0.0 >Reporter: Per Steffensen >Priority: Minor > > Some Kafka-Connect-metrics seems to be wrong with respect to per-task - at > least it seems like MBean > "kafka.connect:type=source-task-metrics,connector=,task=x" > attribute "source-record-active-count" reports the same number for all x > tasks running in the same Kafka-Connect instance/JVM. E.g. if I have a > source-connector "my-connector" with 2 tasks that both run in the same > Kafka-Connect instance, but I know that only one of them actually produces > anything (and therefore can have "active source-records") both > "kafka.connect:type=source-task-metrics,connector=my-connector,task=0" and > "kafka.connect:type=source-task-metrics,connector=my-connector,task=1" goes > up (following each other). It should only go up for the one task that > actually produces something. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-6504) Connect: Some per-task-metrics not working
[ https://issues.apache.org/jira/browse/KAFKA-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355562#comment-16355562 ] Randall Hauch commented on KAFKA-6504: -- [~rayokota], ping. Please see [~steff1193]'s comments. Also, [~hachikuji] this issue was not closed when the PR was merged, and there's no fix version. > Connect: Some per-task-metrics not working > -- > > Key: KAFKA-6504 > URL: https://issues.apache.org/jira/browse/KAFKA-6504 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 1.0.0 >Reporter: Per Steffensen >Priority: Minor > > Some Kafka-Connect-metrics seems to be wrong with respect to per-task - at > least it seems like MBean > "kafka.connect:type=source-task-metrics,connector=,task=x" > attribute "source-record-active-count" reports the same number for all x > tasks running in the same Kafka-Connect instance/JVM. E.g. if I have a > source-connector "my-connector" with 2 tasks that both run in the same > Kafka-Connect instance, but I know that only one of them actually produces > anything (and therefore can have "active source-records") both > "kafka.connect:type=source-task-metrics,connector=my-connector,task=0" and > "kafka.connect:type=source-task-metrics,connector=my-connector,task=1" goes > up (following each other). It should only go up for the one task that > actually produces something. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-6504) Connect: Some per-task-metrics not working
[
https://issues.apache.org/jira/browse/KAFKA-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354172#comment-16354172
]
ASF GitHub Bot commented on KAFKA-6504:
---
hachikuji closed pull request #4514: KAFKA-6504: Fix creation of a sensor to be
specific to a metric group so it is not shared
URL: https://github.com/apache/kafka/pull/4514
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java
b/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java
index 69614948147..4ee8ad6a808 100644
---
a/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java
+++
b/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java
@@ -677,34 +677,34 @@ public SinkTaskMetricsGroup(ConnectorTaskId id,
ConnectMetrics connectMetrics) {
// prevent collisions by removing any previously created metrics
in this group.
metricGroup.close();
-sinkRecordRead = metricGroup.metrics().sensor("sink-record-read");
+sinkRecordRead = metricGroup.sensor("sink-record-read");
sinkRecordRead.add(metricGroup.metricName(registry.sinkRecordReadRate), new
Rate());
sinkRecordRead.add(metricGroup.metricName(registry.sinkRecordReadTotal), new
Total());
-sinkRecordSend = metricGroup.metrics().sensor("sink-record-send");
+sinkRecordSend = metricGroup.sensor("sink-record-send");
sinkRecordSend.add(metricGroup.metricName(registry.sinkRecordSendRate), new
Rate());
sinkRecordSend.add(metricGroup.metricName(registry.sinkRecordSendTotal), new
Total());
-sinkRecordActiveCount =
metricGroup.metrics().sensor("sink-record-active-count");
+sinkRecordActiveCount =
metricGroup.sensor("sink-record-active-count");
sinkRecordActiveCount.add(metricGroup.metricName(registry.sinkRecordActiveCount),
new Value());
sinkRecordActiveCount.add(metricGroup.metricName(registry.sinkRecordActiveCountMax),
new Max());
sinkRecordActiveCount.add(metricGroup.metricName(registry.sinkRecordActiveCountAvg),
new Avg());
-partitionCount = metricGroup.metrics().sensor("partition-count");
+partitionCount = metricGroup.sensor("partition-count");
partitionCount.add(metricGroup.metricName(registry.sinkRecordPartitionCount),
new Value());
-offsetSeqNum = metricGroup.metrics().sensor("offset-seq-number");
+offsetSeqNum = metricGroup.sensor("offset-seq-number");
offsetSeqNum.add(metricGroup.metricName(registry.sinkRecordOffsetCommitSeqNum),
new Value());
-offsetCompletion =
metricGroup.metrics().sensor("offset-commit-completion");
+offsetCompletion = metricGroup.sensor("offset-commit-completion");
offsetCompletion.add(metricGroup.metricName(registry.sinkRecordOffsetCommitCompletionRate),
new Rate());
offsetCompletion.add(metricGroup.metricName(registry.sinkRecordOffsetCommitCompletionTotal),
new Total());
-offsetCompletionSkip =
metricGroup.metrics().sensor("offset-commit-completion-skip");
+offsetCompletionSkip =
metricGroup.sensor("offset-commit-completion-skip");
offsetCompletionSkip.add(metricGroup.metricName(registry.sinkRecordOffsetCommitSkipRate),
new Rate());
offsetCompletionSkip.add(metricGroup.metricName(registry.sinkRecordOffsetCommitSkipTotal),
new Total());
-putBatchTime = metricGroup.metrics().sensor("put-batch-time");
+putBatchTime = metricGroup.sensor("put-batch-time");
putBatchTime.add(metricGroup.metricName(registry.sinkRecordPutBatchTimeMax),
new Max());
putBatchTime.add(metricGroup.metricName(registry.sinkRecordPutBatchTimeAvg),
new Avg());
}
diff --git
a/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java
b/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java
index a172cdb45f0..473e2359578 100644
---
a/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java
+++
b/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java
@@ -509,7 +509,7 @@ public SourceTaskMetricsGroup(ConnectorTaskId id,
ConnectMetrics connectMetrics)
pollTime.add(metricGroup.metricName(registry.sourceRecordPollBatchTimeMax), new
Max());
pollTime.add(metricGroup.metricName(registry.sourceRecordPollBatchTimeAvg), new
Avg());
-
[jira] [Commented] (KAFKA-6504) Connect: Some per-task-metrics not working
[ https://issues.apache.org/jira/browse/KAFKA-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349545#comment-16349545 ] ASF GitHub Bot commented on KAFKA-6504: --- rayokota opened a new pull request #4514: KAFKA-6504: Fix source task metric caused by copy-paste error URL: https://github.com/apache/kafka/pull/4514 This is a simple change to correct "sink-active-record-count" with "source-active-record-count" for the This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Connect: Some per-task-metrics not working > -- > > Key: KAFKA-6504 > URL: https://issues.apache.org/jira/browse/KAFKA-6504 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 1.0.0 >Reporter: Per Steffensen >Priority: Minor > > Some Kafka-Connect-metrics seems to be wrong with respect to per-task - at > least it seems like MBean > "kafka.connect:type=source-task-metrics,connector=,task=x" > attribute "source-record-active-count" reports the same number for all x > tasks running in the same Kafka-Connect instance/JVM. E.g. if I have a > source-connector "my-connector" with 2 tasks that both run in the same > Kafka-Connect instance, but I know that only one of them actually produces > anything (and therefore can have "active source-records") both > "kafka.connect:type=source-task-metrics,connector=my-connector,task=0" and > "kafka.connect:type=source-task-metrics,connector=my-connector,task=1" goes > up (following each other). It should only go up for the one task that > actually produces something. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
