Re: [PR] KAFKA-16089: Fix memory leak in RocksDBStore [kafka]

2024-01-11 Thread via GitHub


lucasbru merged PR #15174:
URL: https://github.com/apache/kafka/pull/15174


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [PR] KAFKA-16089: Fix memory leak in RocksDBStore [kafka]

2024-01-11 Thread via GitHub


nicktelford commented on PR #15174:
URL: https://github.com/apache/kafka/pull/15174#issuecomment-1886931733

   @lucasbru I'm just writing up a more detailed explanation, including graphs, 
in the ticket. I'm also going to raise a bug with RocksDB, because I believe 
this to be a bug in RocksJNI.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[PR] KAFKA-16089: Fix memory leak in RocksDBStore [kafka]

2024-01-11 Thread via GitHub


nicktelford opened a new pull request, #15174:
URL: https://github.com/apache/kafka/pull/15174

   `ColumnFamilyDescriptor` is _not_ a `RocksObject`, which in theory means 
it's not backed by any native memory allocated by RocksDB.
   
   However, in practice, `ColumnFamilyHandle#getDescriptor()`, which returns a 
`ColumnFamilyDescriptor`, allocates an internal 
`rocksdb::db::ColumnFamilyDescriptor`, copying the name and options of the 
column family into it.
   
   Since the Java `ColumnFamilyDescriptor` is not a `RocksObject`, it's not 
possible to track this allocation and free it from Java.
   
   Fortunately, in our case, we can simply avoid calling 
`ColumnFamilyHandle#getDescriptor()`, since we're only interested in the column 
family name, which is already available on
   `ColumnFamilyHandle#getName()`, which does not leak memory.
   
   We can also optimize away the temporary `Options`, which was previously a 
source of memory leaks, because `userSpecifiedOptions` is an instance of 
`Options`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org