[
https://issues.apache.org/jira/browse/KAFKA-18369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lucy Liu reassigned KAFKA-18369:
--------------------------------
Assignee: Lucy Liu
> State updater's *-ratio metrics are incorrect
> ---------------------------------------------
>
> Key: KAFKA-18369
> URL: https://issues.apache.org/jira/browse/KAFKA-18369
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 3.8.1
> Reporter: Rafał Sumisławski
> Assignee: Lucy Liu
> Priority: Major
>
> h2. Background
> {{DefaultStateUpdater}} defines {{{}idle-ratio{}}},
> {{{}active-restore-ratio{}}}, {{{}standby-update-ratio{}}},
> {{checkpoint-ratio}} metrics here:
> [https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/DefaultStateUpdater.java#L1101-L1115]
> These metrics are averages, that are supposed to indicate "The fraction of
> time the thread spent on \{action}". But the metrics don't actually do that.
> h2. Issue
> Let me explain this with an example:
> For simplicity's sake, let's consider the following example involving just
> {{{}standby-update-ratio{}}}, {{checkpoint-ratio}} and ignoring the existence
> of the other two metrics.
> Let's say the thread did:
> * {{999}} iterations with {{standby-update}} taking {{1ms}} in each
> iteration and no {{checkpoint}} happening ({{{}0ms{}}}).
> * {{1}} iteration with {{standby-update}} taking {{1ms}} and {{checkpoint}}
> taking {{9000ms}}
> The thread spent {{10s}} working, of which it spent {{1s}} on
> {{standby-updates}} and {{9s}} on checkpoint, so the fraction of time it
> spent on checkpoint (checkpoint-ratio) is {{{}~0.001 (0.1%){}}}. Or at least
> that is what the metrics will say. I would instead argue that it spent
> {{9s/10s == 0.9 == 90%}} on checkpoint. If you agree with my logic, then you
> agree that this metrics is incorrect.
> The problem is that the code computes a ratio for each iteration, and then
> averages those ratios out, producing a number devoid of statistical meaning
> or practical application. It ignores the fact that the one iteration that
> took {{{}9s{}}}, should have a much higher weight than those quick 1ms
> iterations.
> {{(999*(0ms/1ms) + 1*(9000ms/9001ms))/1000 ~= 0.001}}
> h2. Solution
> What we would like to see instead is either a ratio of average/total times,
> not an average of ratios. I don't think this can be easily realised within
> the existing metrics system. So instead, what I propose as a solution is to
> report {{duration-total}} and/or {{duration-rate}} (with the unit of seconds
> per second) metric for each of {{{}idle{}}}, {{{}active-restore{}}},
> {{{}standby-restore{}}}, {{{}checkpoint{}}}. The observers of these metrics,
> when needed, could then derive the actual ratio of time spent on each
> operation for example as {{{}checkpoint-ratio = checkpoint-duration-rate /
> (idle-duration-rate + active-restore-duration-rate +
> standby-restore-duration-rate + checkpoint-duration-rate){}}}. Or by
> performing an analogical calculation on deltas of the {{total}} metrics.
> I can submit a PR once there's an agreement on the correct way to fix this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
