Re: Appropriate State to use to buffer events in ProcessFunction

Hi Xiaogang, Indeed, the MapState is what I was looking for in order to have efficient sorted state, as it would faciliate many use cases like this one, or joining streams, etc. I searched a bit and found your contribution of MapState for the next 1.3 re

Re: Appropriate State to use to buffer events in ProcessFunction

Hi Yassine, If I understand correctly, you are needing sorted states which unfortunately are not supported in Flink now. We have some ideas to provide such sorted states to facilitate the development of user applications. But it is still under discussion due to the concerns on back compatibility.

Re: Appropriate State to use to buffer events in ProcessFunction

Hi Timo, I thought about the ListState but quickly discarded It as it keeps the insersion order and not events order. After a second thought I think I will reconsider it since my events are occaionally out-of-order. Didn't know that Flink CEP operators 'next' and 'within', can handle event time, s

Re: Appropriate State to use to buffer events in ProcessFunction

Hi Yassine, have you thought about using a ListState? As far as I know, it keeps at least the insertion order. You could sort it once your trigger event has arrived. If you use a RocksDB as state backend, 100+ GB of state should not be a problem. Have you thought about using Flink's CEP librar

Appropriate State to use to buffer events in ProcessFunction

Hi all, I want to label events in a stream based on a condition on some future events. For example my stream contains events of type A and B and and I would like to assign a label 1 to an event E of type A if an event of type B happens within a duration x of E. I am using event time and my events