When using structured streaming, we use "partitionBy" api to partition the output data, and use the watermark based on event-time to handle delay records, but how to tell downstream users that a partition is ready? For example, when to write an empty "hadoop.done" file in a paritition directory?
-- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
