Lucas Brutschy created KAFKA-14432:
--------------------------------------
Summary: RocksDBStore relies on finalizers to not leak memory
Key: KAFKA-14432
URL: https://issues.apache.org/jira/browse/KAFKA-14432
Project: Kafka
Issue Type: Bug
Reporter: Lucas Brutschy
Assignee: Lucas Brutschy
Relying on finalizers in RocksDB has been deprecated for a long time, and
starting with rocksdb 7, finalizers are removed completely (see
[https://github.com/facebook/rocksdb/pull/9523]).
Kafka Streams currently relies on finalizers in parts to not leak memory. This
needs to be resolved before we can upgrade to RocksDB 7.
See [https://github.com/apache/kafka/pull/12809] .
This is a native heap profile after running Kafka Streams without finalizers
for a few hours:
Total: 13547.5 MB
12936.3 95.5% 95.5% 12936.3 95.5% rocksdb::port::cacheline_aligned_alloc
438.5 3.2% 98.7% 438.5 3.2% rocksdb::BlockFetcher::ReadBlockContents
84.0 0.6% 99.3% 84.2 0.6% rocksdb::Arena::AllocateNewBlock
45.9 0.3% 99.7% 45.9 0.3% prof_backtrace_impl
8.1 0.1% 99.7% 14.6 0.1%
rocksdb::BlockBasedTable::PutDataBlockToCache
6.4 0.0% 99.8% 12941.4 95.5%
Java_org_rocksdb_Statistics_newStatistics___3BJ
6.1 0.0% 99.8% 6.9 0.1% rocksdb::LRUCacheShard::Insert@2d8b20
5.1 0.0% 99.9% 6.5 0.0%
rocksdb::VersionSet::ProcessManifestWrites
3.9 0.0% 99.9% 3.9 0.0%
rocksdb::WritableFileWriter::WritableFileWriter
3.2 0.0% 99.9% 3.2 0.0% std::string::_Rep::_S_create
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
