Hi >From your description, you want to do two things: 1 update state and remote the state older than x 2 output the state every y second
>From my side, the first can be done by using TTL state as Yun said, the second can be done by using KeyedProcessFunction[1] If you want to have complex logic to remove the older state in step 1, maybe you can also use the KeyedProcessFunction and timer() [1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/stream/operators/process_function.html#the-keyedprocessfunction Best, Congxian Yun Tang <[email protected]> 于2020年5月13日周三 下午7:42写道： > Hi > > From your description: "output the state every y seconds and remove old > elements", I think TTL [1] is the proper solution for your scenario. And > you could define the ttl of your state as y seconds so that processfunction > could only print elements in the last y seconds. > > > [1] > https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html#state-time-to-live-ttl > > Best > Yun Tang > ------------------------------ > *From:* Annemarie Burger <[email protected]> > *Sent:* Wednesday, May 13, 2020 2:46 > *To:* [email protected] <[email protected]> > *Subject:* Incremental state with purging > > Hi, > > I'm trying to implement the most efficient way to incrementally put > incoming > DataStream elements in my (map)state, while removing old elements (older > that x) from that same state. I then want to output the state every y > seconds. I've looked into using the ProcessFunction with onTimer, or > building my own Trigger for a window function, but I struggle with putting > all this together in a logical and efficient way. Since the state is very > big I don't want to duplicate it over multiple (sliding)windows. Does > anybody know the best way to achieve this? Some pseudo code would be very > helpful. > > Thanks! > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >
