Yes. Using calendar day-based windows and watermark is completely caught up to today ... calendar window ends several days ago. I got EARLY panes for each element but never ON_TIME pane.
On Mon, Jan 13, 2020 at 4:16 PM Luke Cwik <lc...@google.com> wrote: > Is the watermark advancing past the end of the window? > > On Mon, Jan 13, 2020 at 2:02 PM Aaron Dixon <atdi...@gmail.com> wrote: > >> The window is not empty fwiw; it has elements; I get an early firing pane >> for the window but well after the watermark passes there is no ON_TIME >> pane. Would this be a bug in Dataflow? Seems fundamental, so I'm concerned >> perhaps the Beam spec doesn't obligate ON_TIME firings? >> >> >> >> >> >> On Mon, Jan 13, 2020 at 3:58 PM Luke Cwik <lc...@google.com> wrote: >> >>> I would have expected an empty on time pane since the default on time >>> behavior is FIRE_ALWAYS. >>> >>> On Mon, Jan 13, 2020 at 1:54 PM Aaron Dixon <atdi...@gmail.com> wrote: >>> >>>> Can anyone confirm? >>>> >>>> This is intermittent. Some (it seems, sparse) windows don't get an >>>> ON_TIME firing after watermark. Is this a bug or is there a reason to not >>>> expect ON_TIME firings for every window? >>>> >>>> On Mon, Jan 13, 2020 at 3:47 PM Rui Wang <ruw...@google.com> wrote: >>>> >>>>> If it indeed happened as you have described, I will be very interested >>>>> in the expected behaviour. >>>>> >>>>> Something I remembered before: the trigger condition meets just gives >>>>> the runner/engine "permission" to fire, but runner/engine may not fire >>>>> immediately. But I don't know if the engine/runner will guarantee to fire. >>>>> >>>>> >>>>> >>>>> -Rui >>>>> >>>>> On Mon, Jan 13, 2020 at 1:43 PM Aaron Dixon <atdi...@gmail.com> wrote: >>>>> >>>>>> I have the following trigger: >>>>>> >>>>>> .apply(Window >>>>>> .configure() >>>>>> .triggering(AfterWatermark >>>>>> .pastEndOfWindow() >>>>>> .withEarlyFirings(AfterPane >>>>>> .elementCountAtLeast(1))) >>>>>> .accumulatingFiredPanes() >>>>>> .withAllowedLateness(Duration.ZERO) >>>>>> >>>>>> But in Dataflow I notice that I never get an ON_TIME firing for my >>>>>> window -- I only see early firing for elements, and then nothing. >>>>>> >>>>>> My assumption is that AfterWatermark should give me a last, on-time >>>>>> pane under this configuration when the watermark surpasses the window's >>>>>> end. >>>>>> >>>>>> Is my expectation correct? >>>>>> >>>>>
