Thank you Yun for your reply. Yes, I did the same thing and posted my
solution here:
https://stackoverflow.com/questions/76308686/how-to-figure-out-whats-the-size-of-liststate?noredirect=1#comment134577594_76308686
Best,
Anir
On Wed, May 24, 2023 at 5:13 AM Yun Tang wrote:
> Hi Amir,
>
> For the current Flink, you have to iterator the returned Iterable of
> ListState#get().
>
> Why Flink lacks an API to get the size of listState directly? This is
> because Flink leverages RocksDB's merge operator [1] for high-performance
> listState#add [2], that is to say, we would not even read the original data
> when appending the listState. I think you can record the size in another
> valueState when implementing your logic.
>
>
> [1] https://github.com/facebook/rocksdb/wiki/Merge-Operator
> [2]
> https://github.com/apache/flink/blob/d8c64a808484cab78c8bd7b74a287edf7d1f3b01/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBListState.java#L131
>
> Best
> Yun Tang
>
> From: Amir Hossein Sharifzadeh
> Sent: Tuesday, May 23, 2023 23:51
> To: dev
> Subject: How to figure out what's the size of ListState?
>
> Dear Flink Dev team,
>
> It's about a while since I am dealing with an issue that can't figure out
> myself. I spent quite a lot of time trying to solve the problem myself, but
> I feel stuck.
>
> I explained the problem statement and the issue here:
>
> https://stackoverflow.com/questions/76308686/how-to-figure-out-whats-the-size-of-liststate
>
> I really appreciate any suggestion.
>
> Best,
> Amir
>
