GitHub user jose-torres opened a pull request: https://github.com/apache/spark/pull/20859
[SPARK-23702][SS] Forbid watermarks on both sides of stateful streaming operators. ## What changes were proposed in this pull request? Forbid watermarks on both sides of stateful streaming operators. Multiple sequential watermarks are in general not supported by the execution engine; support is only in parallel, e.g. on both sides of a join. We can normally resolve this by simply picking the topmost watermark operator and ignoring the rest, but this is not semantically valid when there's a stateful operator in between. ## How was this patch tested? new unit test You can merge this pull request into a Git repository by running: $ git pull https://github.com/jose-torres/spark triggerDisallow Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20859.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20859 ---- commit 051c85afe727f39ba9d505e00e162620f69c808f Author: Jose Torres <torres.joseph.f+github@...> Date: 2018-03-19T18:48:11Z disallow watermarks on both sides of stateful ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org