Hi, Patricia. In my opinion, This parameter balances the trade-off between the read/write performance and storage space utilization (of course, smaller state also means better performance for the future).
I think the right value of longtimeNumberOfQueries depends on several factors, such as the size of your state, the rate of updates to your data. So just a personal suggestion: 1. If your state size is relatively small and does not update frequently, you may be able to use a larger value. 2. otherwise, you could use a smaller value for longTimeNumberOfQueries() to ensure that the compaction filter is executed more frequently. 3. Of course, You could also start with the default value and increase or decrease it gradually to see whether the performance and storage space works well for your job. On Fri, Jun 16, 2023 at 8:29 AM patricia lee <plee3...@gmail.com> wrote: > Hi, > > I am currently migrating our flink project from 1.8 to 1.17. > > The cleanUpInRocksDbCompactFilter() now accepts longtimeNumberOfQueries() > as parameter. The question is how would we know the right value. We set to > 1000 temporarily, is there a default value to set. > > > Regards, > Patricia > -- Best, Hangxiang.
