alanlau28 commented on PR #22452:
URL: https://github.com/apache/kafka/pull/22452#issuecomment-4926672980
Reworked to address the per-record `System.currentTimeMillis()` overhead
concern.
Terminal e2e latency is now recorded once per task batch instead of at the
terminal node. It reuses the wall-clock time `TaskExecutor.processTask` already
reads at batch end for `recordProcessBatchTime`, so there are **no extra clock
reads** vs. the original cached-time behavior. During the batch each terminal
node accumulates only `count/min/max/offset-sum` of record timestamps (constant
size, independent of fan-out); at flush the sensor's avg/min/max are
reconstructed exactly from those aggregates and the single end time — `max =
end − minTs`, `min = end − maxTs`, and the interior samples filled so the
count-weighted avg is exact. Same number of samples as recording per record, so
weighting is unchanged.
Trade-off: every record in a batch is attributed the batch-end completion
time (a conservative upper bound) — the same per-batch model `process-latency`
uses.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
