Till, Thank you for your answer. That's true that there is the case window operator have not received all data for the key. I will go with the second idea.
Thanks! Hironori 2016-04-26 17:46 GMT+09:00 Till Rohrmann <trohrm...@apache.org>: > Hi Hironori, > > I would go with the second approach, because it is not guaranteed that all > events of a given key have been received by the window operator if the data > source says that all events for this key have been read. The events might > still be in flight. Furthermore, it integrates more nicely with Flink's > streaming model. > > Cheers, > Till > > On Tue, Apr 26, 2016 at 10:16 AM, Hironori Ogibayashi <ogibaya...@gmail.com> > wrote: >> >> Hello, >> >> I am using GlobalWindow and my custom trigger (similar to >> ContinuousProcessingTimeTrigger). >> In my trigger I want to control the TriggerResult based on external >> datasource. >> That datasource has flags for each key which describes if stream for that >> key has been finished (so, can be purged). >> >> I am thinking of two approaches, so could you give me some advise about >> which is better, or are there any other better solutions? >> >> 1. Check datasource in onProcessingTime() >> >> Query datasource (i.e. Redis) in onProcessingTime() and return FIRE or >> FIRE_AND_PURGE based on the result. >> Maybe I will create Jedis or JedisPool instance in the trigger's >> constructor? >> >> 2. External program periodically query datasource and send special >> event for keys of finished stream. >> >> The schema of the event will be the same as normal events in the >> stream, but has special value in a field. So, the trigger will be able >> to handle the event in onElement(). I need to filter that event >> afterward so that it does not affect the computation result. >> >> Thanks, >> Hironori Ogibayashi > >
