[
https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331590#comment-17331590
]
Ming Liu edited comment on KAFKA-12713 at 4/27/21, 3:25 AM:
------------------------------------------------------------
The idea is:
# Add waitTimeMs in FetchResponse
# In processResponseCallback() of handleFetchRequest, set the waitTimeMs as
the time spent in purgatory.
# In FetcherStats, we will add a new meter to track the fetch latency, by
deduct the waitTimeMs from the latency.
Also, in FetchLatency, we should also report a time called TotalEffectiveTime =
TotalTime-RemoteTime.
Created KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-736%3A+Track+the+real+fetch+latency
was (Author: mingaliu):
The idea is:
# Add waitTimeMs in FetchResponse
# In processResponseCallback() of handleFetchRequest, set the waitTimeMs as
the time spent in purgatory.
# In FetcherStats, we will add a new meter to track the fetch latency, by
deduct the waitTimeMs from the latency.
Also, in FetchLatency, we should also report a time called TotalEffectiveTime =
TotalTime-RemoteTime.
Let me know for any suggestion/feedback. I like to propose a KIP on that
change.
> Report "REAL" follower/consumer fetch latency
> ---------------------------------------------
>
> Key: KAFKA-12713
> URL: https://issues.apache.org/jira/browse/KAFKA-12713
> Project: Kafka
> Issue Type: Bug
> Reporter: Ming Liu
> Priority: Major
>
> The fetch latency is an important metrics to monitor for the cluster
> performance. With ACK=ALL, the produce latency is affected primarily by
> broker fetch latency.
> However, currently the reported fetch latency didn't reflect the true fetch
> latency because it sometimes need to stay in purgatory and wait for
> replica.fetch.wait.max.ms when data is not available. This greatly affect the
> real P50, P99 etc.
> I like to propose a KIP to be able track the real fetch latency for both
> broker follower and consumer.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
