Re:Re: Checkpoints and windows size

2024-06-19 Thread Feifan Wang
Hi banu: > Not all old sst files are present. Few are removed (i think it is because of > compaction). You are right, rocksdb implement delete a key by insert a entry with null value, the space will be release after compaction. > Now how can I maintain check points size under control??.

Re: Checkpoints and windows size

2024-06-19 Thread banu priya
Hi Wang, Thanks a lot for your reply. Currently I have 2s window and check point interval as 10s. Minimum pass between check point is 5s. What happens is my check points size is growing gradually. I checked the content inside my rocks db local dir and also the shared checkpoints directory.

Checkpoints and windows size

2024-06-18 Thread banu priya
Hi All, I have a flink job with key by, tumbling window(2sec window time processing time)and aggregator. How often should I run the check point??I don't need the data to be retained after 2s. I want to use incremental check point with rocksdb. Thanks Banupriya