Re: Event time window eviction

2019-07-29 Thread taher koitawala
Hi Navneeth, There are 3 ways we can work with data now flowing and windows and not being fired because watermark flow stopped. 1. Write a custom trigger which fires the window if no elements arrive. 2. Your watermark assigner function can also house the logic that if no more watermarks are

Re: Event time window eviction

2019-07-29 Thread Navneeth Krishnan
Thanks Taher. Are there any examples for this? In my scenario I would have data coming in and it might stop for sometime but I need the window to end after the duration. Also, I believe in version 1.3 the event time will progress only if all partitions in a kafka topic pass the event time. Is that

Re: Event time window eviction

2019-07-29 Thread taher koitawala
I believe the approach to this is wrong... For fixing windows we can write our custom triggers to fire them... However what I'm not convinced with is switching between event and processing time. Write a custom triggers and fire the event time window if you don't see any activity. That's th

Re: Event time window eviction

2019-07-29 Thread Navneeth Krishnan
Hi All, Any suggestions? Thanks On Thu, Jul 25, 2019 at 11:45 PM Navneeth Krishnan wrote: > Hi All, > > I'm working on a very short tumbling window for 1 second per key. What I > want to achieve is if the event time per key doesn't progress after a > second I want to evict the window, basicall

Event time window eviction

2019-07-25 Thread Navneeth Krishnan
Hi All, I'm working on a very short tumbling window for 1 second per key. What I want to achieve is if the event time per key doesn't progress after a second I want to evict the window, basically a combination of event time and processing time. I'm currently achieving it by registering a processin