Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21919 `numOutputRows` makes sense for all sinks, but I agree the counting should be done at the framework and not by individual sinks. For metrics that does not apply to all sinks, they could report it as some custom metrics if they want to. Heres a proposal to add collect and report custom metrics for sources and sinks - https://github.com/apache/spark/pull/21721
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org