Hello Alexis, RocksDB resides off-heap and outside of JVM. The small subset of data ends up on the off-heap in the memory.
For more details, check the following link: https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/memory/mem_setup_tm/#managed-memory I hope this addresses your inquiry. On Thu, Feb 15, 2024 at 12:52 AM Alexis Sarda-Espinosa < sarda.espin...@gmail.com> wrote: > Hello, > > Most info regarding RocksDB memory for Flink focuses on what's needed > independently of the JVM (although the Flink process configures its limits > and so on). I'm wondering if there are additional special considerations > with regards to the JVM heap in the following scenario. > > Assuming a key used to partition a Flink stream and its state has a high > cardinality, but that the state of each key is small, when Flink prepares > the state to expose to a user function during a call (with a given > partition key), I guess it loads only the required subset from RocksDB, but > does this small subset end (temporarily) up on the JVM heap? And if it > does, does it stay "cached" in the JVM for some time or is it immediately > discarded after the user function completes? > > Maybe this isn't even under Flink's control, but I'm curious. > > Regards, > Alexis. >
