On Tue, Mar 28, 2017 at 1:32 PM, Robert Bradshaw < [email protected]> wrote:
> Another alternative is to be able to set special timers, e.g. end of window > and expiration of window. That at least addresses (2). > Potentially a tangent, but that would perhaps fit in with the idea of removing TimeDomain from user APIs ( https://issues.apache.org/jira/browse/BEAM-1308) and instead having TimerSpecs.eventTimeTimer(), TimerSpecs.processingTimeTimer(), TimerSpecs.windowExpirationTimer() that each yield distinct sorts of parameters in @ProcessElement. A bit more heavyweight, syntactically. Kenn > > On Tue, Mar 28, 2017 at 1:27 PM, Kenneth Knowles <[email protected]> > wrote: > > > Hi all, > > > > I have a little extension to the stateful DoFn annotations to circulate > for > > feedback: Allow a method to be annotated with @OnWindowExpiration to > > automatically get a callback at some point after the window has expired, > > but before the state for the window has been cleared. > > > > Today, a user can pretty easily get the same effect by setting a timer > for > > the end of the window + allowed lateness in their @ProcessElement calls. > > But having just one annotation for it has a couple nice benefits: > > > > 1. Some users assume a naive implementation so they are concerned that > > setting a timer repeatedly is costly. This eliminates the cause for user > > alarm and allows a runner to do a better job in case it didn't already do > > it efficiently. > > > > 2. Getting the allowed lateness to be available to your @ProcessElement > is > > a little crufty. > > > > 3. Often, if you don't have @OnWindowExpiration, you are leaving behind > > state that might contain data that is otherwise lost. So I would even > > consider making it mandatory (with some way of indicating state you don't > > care about dropping) though that could be annoying. > > > > Another interesting moment in a window's lifecycle is @EndOfWindow. This > is > > not critical for correctness, though. > > > > Thoughts? > > > > Kenn > > >
