Future processing time timers and final watermark

2017-05-02 Thread Thomas Weise
Hi, While working on SDF support in the Apex runner, I see the scenario where processing time timers are set in the future. These never trigger, the topology exits with the final watermark before. What is the correct way to handle this? Should the final watermark be held until all processing time

Re: Future processing time timers and final watermark

2017-05-04 Thread Thomas Groh
Generally you shouldn't need to hold the watermark. The fact that the input watermark of the DoFn has advanced to the final watermark (i.e. positive infinity) means that all of the windows expire. At this point, any window with buffered data that is not finished should have its remaining elements o