Hello peay-2, Were you able to get a solution to your problem ? Were you able to get watermark timestamp available through a function ?
Regards, Sanjay peay-2 wrote > Thanks for the pointers. I guess right now the only workaround would be to > apply a "dummy" aggregation (e.g., group by the timestamp itself) only to > have the stateful processing logic kick in and apply the filtering? > > For my purposes, an alternative solution to pushing it out to the source > would be to make the watermark timestamp available through a function so > that it can be used in a regular filter clause. Based on my experiments, > the timestamp is computed and updated even when no stateful computations > occur. I am not sure how easy that would be to contribute though, maybe > someone can suggest a starting point? -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
