Hi
>From your description,  you want to do two things:
1 update state and remote the state older than x
2 output the state every y second

>From my side, the first can be done by using TTL state as Yun said,
the second can be done by using KeyedProcessFunction[1]

If you want to have complex logic to remove the older state in step 1,
maybe you can also use the KeyedProcessFunction and timer()

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/stream/operators/process_function.html#the-keyedprocessfunction
Best,
Congxian


Yun Tang <myas...@live.com> 于2020年5月13日周三 下午7:42写道:

> Hi
>
> From your description: "output the state every y seconds and remove old
> elements", I think TTL [1] is the proper solution for your scenario. And
> you could define the ttl of your state as y seconds so that processfunction
> could only print elements in the last y seconds.
>
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/state.html#state-time-to-live-ttl
>
> Best
> Yun Tang
> ------------------------------
> *From:* Annemarie Burger <annemarie.bur...@campus.tu-berlin.de>
> *Sent:* Wednesday, May 13, 2020 2:46
> *To:* user@flink.apache.org <user@flink.apache.org>
> *Subject:* Incremental state with purging
>
> Hi,
>
> I'm trying to implement the most efficient way to incrementally put
> incoming
> DataStream elements in my (map)state, while removing old elements (older
> that x) from that same state. I then want to output the state every y
> seconds. I've looked into using the ProcessFunction with onTimer, or
> building my own Trigger for a window function, but I struggle with putting
> all this together in a logical and efficient way. Since the state is very
> big I don't want to duplicate it over multiple (sliding)windows. Does
> anybody know the best way to achieve this? Some pseudo code would be very
> helpful.
>
> Thanks!
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>

Reply via email to