Nicholas Telford created KAFKA-15541: ----------------------------------------
Summary: RocksDB Iterator Metrics Key: KAFKA-15541 URL: https://issues.apache.org/jira/browse/KAFKA-15541 Project: Kafka Issue Type: Improvement Components: streams Reporter: Nicholas Telford Assignee: Nicholas Telford [KIP-989: RocksDB Iterator Metrics|https://cwiki.apache.org/confluence/display/KAFKA/KIP-989%3A+RocksDB+Iterator+Metrics] RocksDB {{Iterators}} must be closed after use, to prevent memory leaks due to [blocks being "pinned" in-memory|https://github.com/facebook/rocksdb/wiki/Memory-usage-in-RocksDB#blocks-pinned-by-iterators]. Pinned blocks can currently be tracked via the per-store {{block-cache-pinned-usage}} metric. However, it's common [(and even recommended)|https://docs.confluent.io/platform/current/streams/developer-guide/memory-mgmt.html#rocksdb] to share the Block Cache among all stores in an application, to enable users to globally bound native memory used by RocksDB. This results in the {{block-cache-pinned-usage}} reporting the same memory usage for every store in the application, irrespective of which store is actually pinning blocks in the block cache. To aid users in finding leaked Iterators, as well as identifying the cause of a high number of pinned blocks, we introduce two new metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)