[ https://issues.apache.org/jira/browse/KAFKA-10177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17143599#comment-17143599 ]
Tom Bentley commented on KAFKA-10177: ------------------------------------- Have you considered using HrdHistogram, or reimplementing some of the ideas from it if you don't want to add it as a dependency? > Replace/improve Percentiles metrics > ----------------------------------- > > Key: KAFKA-10177 > URL: https://issues.apache.org/jira/browse/KAFKA-10177 > Project: Kafka > Issue Type: Improvement > Components: metrics > Reporter: Sophie Blee-Goldman > Priority: Major > > There's an existing – but seemingly unused – implementation of percentile > metrics that we attempted to use for end-to-end latency metrics in Streams. > Unfortunately a number of limitations became apparent, and we ultimately > pulled the metrics from the 2.6 release pending further > investigation/improvement. > The problems we encountered were > # Need to set a static upper/lower limit for the values > # Not well suited to a distribution with a long tail, ie setting the max > value too high caused the accuracy to plummet > # Required a lot of memory per metric for reasonable accuracy and caused us > to hit OOM (unclear if there was actually a memory leak, or it was just > gobbling up unnecessarily large amounts in general) > Since the Percentiles class is part of the public API, we may need to create > a new class altogether and possibly deprecate/remove the old one. > Alternatively we can consider just re-implementing the existing class from > scratch, and just deprecating the current constructors and associated > implementation (eg the constructor accepts a max) -- This message was sent by Atlassian Jira (v8.3.4#803005)