[
https://issues.apache.org/jira/browse/SAMZA-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shadi A. Noghabi updated SAMZA-912:
-----------------------------------
Attachment: (was: add-processNS-CDF.patch)
> Getting a cumulative distribution function of the processes latency as a
> metric
> --------------------------------------------------------------------------------
>
> Key: SAMZA-912
> URL: https://issues.apache.org/jira/browse/SAMZA-912
> Project: Samza
> Issue Type: New Feature
> Reporter: Shadi A. Noghabi
> Priority: Minor
> Attachments: add-processNS-CDF-v2.patch
>
>
> The processNs metrics reports the average of the processNS values over a
> length of time. However, there are some outliers in the data that heavily
> skew the data. The tail and 99th percentile is very large and affect the
> average. For example in one of my test, the processNs was reporting 300 us
> however, when I looked at the data unto the 95th percentile (i.e., remove all
> the dat points above 95th percentile), the average would be 7 us.
> I suggest reporting the cumulative distribution function (CDF) of the process
> ns , for cases that would dig more into the detail distribution of the
> latencies.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
