Re: TTL issue with large RocksDB keyed state

2024-06-03 Thread Yanfei Lei
Hi, > 1. After multiple full checkpoints and a NATIVE savepoint the size was > unchanged. I'm wondering if RocksDb compaction is because we never update > key values? The state is nearly fully composed of keys' space. Do keys not > get freed using RocksDb compaction filter for TTL? Regarding

TTL issue with large RocksDB keyed state

2024-06-02 Thread Cliff Resnick
Hi everyone, We have a Flink application that has a very large and perhaps unusual state. The basic shape of it is a very large and somewhat random keyed-stream partition space, each with a continuously growing map-state keyed by microsecond time Long values. There are never any overwrites in