Github user arunmahadevan commented on the issue:
https://github.com/apache/spark/pull/21721
For continuous queries, the progress could still be reported by posting
QueryProgressEvent to the listener for each epoch (instead of micro-batch). The
`StreamingQueryProgress` also could mostly be the same.
I am not clear on how the continuous query would get the metric updates. We
may need some mechanism to post metric updates while the query continues to
run. Right now the SQL metrics relies on accumulator and the accumulators might
not be updated unless the task completes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
