Re: Firing windows multiple times

[aj.h]

Hi, I'm interested in helping out on this project. I also want to implement a
continuous time-boxed sliding window, my current use case is a 60-second
sliding window that moves whenever a newer event arrives, discarding any
late events that arrive outside the current window, but *also* re-triggering
window processing for any late events within the current window. I
considered using sliding windows with a 1-second granularity, but I'd be
discarding a lot of windows on sparse data, and rebuilding pontetially very
large windows for relatively small 1-second updates.

I'm a fellow in the Insight Data Engineering program. We just got underway,
and I have 3 weeks in which to complete a project. I'd love to tackle this
one, and I'm trying to assess the practicality and feasibility of it.

I noticed that FLIP-2 and FLIP-4 are still under discussion; is it premature
to try to implement these enhancements? And would you be at all
willing/available to help me get up to speed?

Thank you much!



--
View this message in context: 
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Firing-windows-multiple-times-tp8424p8975.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at 
Nabble.com.

Re: Firing windows multiple times

[aj.h]

In the way that FLIP-2 would solve this problem, secondAggregate would ignore
the early firing updates from firstAggregate to prevent double-counting,
correct? If that's the case, I am trying to understand why we'd want to
trigger early-fires every 30 seconds for the secondAggregate if it's only
accepting new results at a daily rate, after firstAggregate's primary firing
at the end of the window. If we filter out results from early-fires,
wouldn't every 30-second result from secondAggregate remain unchanged within
the same 1-day window?

Similarly (compounded) for a 365-day window aggregating over a 30 day
window: if it filters out early fires, wouldn't it only produce new/unique
results every 30 days?

I very well may have misunderstood this solution.



--
View this message in context: 
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Firing-windows-multiple-times-tp8424p8994.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at 
Nabble.com.