Hi >From your description: "output the state every y seconds and remove old >elements", I think TTL [1] is the proper solution for your scenario. And you >could define the ttl of your state as y seconds so that processfunction could >only print elements in the last y seconds.
[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html#state-time-to-live-ttl Best Yun Tang ________________________________ From: Annemarie Burger <annemarie.bur...@campus.tu-berlin.de> Sent: Wednesday, May 13, 2020 2:46 To: user@flink.apache.org <user@flink.apache.org> Subject: Incremental state with purging Hi, I'm trying to implement the most efficient way to incrementally put incoming DataStream elements in my (map)state, while removing old elements (older that x) from that same state. I then want to output the state every y seconds. I've looked into using the ProcessFunction with onTimer, or building my own Trigger for a window function, but I struggle with putting all this together in a logical and efficient way. Since the state is very big I don't want to duplicate it over multiple (sliding)windows. Does anybody know the best way to achieve this? Some pseudo code would be very helpful. Thanks! -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
