Generally you shouldn't need to hold the watermark. The fact that the input watermark of the DoFn has advanced to the final watermark (i.e. positive infinity) means that all of the windows expire. At this point, any window with buffered data that is not finished should have its remaining elements output. The remaining processing time timers should have nothing to do at that point, and can be safely dropped.
On Tue, May 2, 2017 at 9:57 PM, Thomas Weise <t...@apache.org> wrote: > Hi, > > While working on SDF support in the Apex runner, I see the scenario where > processing time timers are set in the future. These never trigger, the > topology exits with the final watermark before. > > What is the correct way to handle this? Should the final watermark be held > until all processing time timers are cleared? > > Thanks, > Thomas >
