So it sounds like the timestamp of the event is not important here? If that is correct then order is not important and you can do something like this ( not tried it out so I might have missed detail)..
Use a CombiningState to keep a count of elements that are being passed through. On a new element check the value, if its < x then output the element if > x then set an EventTimer for time now+something and add the element to a bagstate In the timercode clear the bag state and reset the combinestate On Thu, 26 Jan 2023 at 12:08, Vignesh Kumar Kathiresan < vkath...@yahooinc.com> wrote: > Thanks Reza. Does this mean having a global window and triggering for > every new event? The use case I was mentioning was a throttle kind of > application using the key based state. Events flow though and the keyed > state is increased by 1 but once a particular key reaches a certain max > count, the newer events are skipped until the state is cleared after the > throttle period. I am looking at something similar to a stateful keyed > parDo so that all events of same key go to the same worker (assuming state > is local to worker as in flink) > > On Thu, Jan 26, 2023 at 8:13 AM Reza Ardeshir Rokni <raro...@gmail.com> > wrote: > >> Hi, >> >> For these types of use cases, folks will generally make use of the Global >> Window which is -/+ inf and Timers. Some key considerations when using the >> Global Window: >> >> 1- GC is not done by the system as the window will never close. >> 2- There are no order guarantees, so you will often need to make use of >> looping >> timer >> <https://urldefense.com/v3/__https://beam.apache.org/blog/looping-timers/__;!!Op6eflyXZCqGR5I!H02HPpgqudbKt18Xo0IC7a0LSrWD3znlzIBATW1EMItF4iCSrhnLM-ziCgsZd14hFgKlQEKBi4GQULrH$> >> patterns. >> >> Cheers >> >> Reza >> >> On Thu, 26 Jan 2023 at 00:09, Vignesh Kumar Kathiresan via user < >> user@beam.apache.org> wrote: >> >>> Hi Community, >>> >>> I am new to beam coming from flink. In flink state can be scoped to only >>> key. A datasteam(similar to Pcollections in beam) can be converted to a >>> keyed data stream. And a process function on this keyed stream can access >>> state scoped to only key. It also has state scoped to key+window. In beam >>> though I see only state scoped to key+window combination. Is my >>> understanding correct? How are use cases using state scoped to key without >>> windows solved in beam. >>> >>> thanks >>> Vignesh >>> >>
