Github user arunmahadevan commented on the issue:

    https://github.com/apache/spark/pull/21721
  
    >It seems like its life cycle should be bound to an epoch, but 
unfortunately we don't have such an interface in continuous streaming to 
represent an epoch. Is it possible that we may end up with 2 sets of custom 
metrics APIs for micro-batch and continuous?
    
    @cloud-fan we could still report progress at the end of each epoch (e.g. 
[here](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala#L231)
 and via the EpochCordinator). There need not be separate interfaces for the 
progress or the custom metrics, just the mechanisms could be different.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to