[GitHub] [kafka] ableegoldman commented on a change in pull request #8697: [WIP] KAFKA-9983: KIP-613, add task-level e2e latency metrics (min and max)

[GitBox]

ableegoldman commented on a change in pull request #8697:
URL: https://github.com/apache/kafka/pull/8697#discussion_r429396524



##########
File path: 
streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java
##########
@@ -137,6 +138,7 @@ public StreamTask(final TaskId id,
         }
         processRatioSensor = TaskMetrics.activeProcessRatioSensor(threadId, 
taskId, streamsMetrics);
         processLatencySensor = TaskMetrics.processLatencySensor(threadId, 
taskId, streamsMetrics);
+        recordE2ELatencySensor = TaskMetrics.recordE2ELatencySensor(threadId, 
taskId, streamsMetrics);

Review comment:
       I modified the proposal slightly to make these all processor-node level 
(will push the changes in a minute) but this question is still relevant, so 
here's the answer:
   We can't record the e2e latency in the sink node because not all topologies 
_have_ a sink node. For that reason we also can't record at the record 
collector. We need to figure out the terminal nodes when processing the 
topology, then record this metric after `child.process` in 
`ProcessorContext#forward`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org