HeartSaVioR commented on issue #23576: [SPARK-26655] [SS] Support multiple aggregates in append mode URL: https://github.com/apache/spark/pull/23576#issuecomment-501180447 I just found the case I missed somehow - watermark for (N+1) level of stage shouldn't refer "input" watermark of N level of stage. It should refer "output" watermark of N level of stage. The option 2 doesn't address the input/output watermark hence it should be considered as well. (I just added comment on design doc.) Once input watermark and output watermark for each stateful operator are properly adopted, I think it would work.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
