Re: Watermarking without aggregation with Structured Streaming

2018-10-25 Thread sanjay_awat
Hello peay-2, Were you able to get a solution to your problem ? Were you able to get watermark timestamp available through a function ? Regards, Sanjay peay-2 wrote > Thanks for the pointers. I guess right now the only workaround would be to > apply a "dummy" aggregation (e.g., group by the

Re: Watermarking without aggregation with Structured Streaming

2018-10-24 Thread sanjay_awat
Try this peay-2 wrote > For my purposes, an alternative solution to pushing it out to the source > would be to make the watermark timestamp available through a function so > that it can be used in a regular filter clause. Based on my experiments, > the timestamp is computed and updated even when