Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/7770#issuecomment-127328436 @squito we won't remove `TaskMetrics` before 2.0 because it breaks compatibility, as you mentioned. Also, in my mind the semantics of accumulators during retries are fairly clear actually; if a task fails, its accumulator values are not counted, whereas with `TaskMetrics` you have to check the task end status in all the downstream listeners, thus requiring code duplication for correctness. I agree that accumulators are not perfect. The fact that there is a singleton `Accumulator` object per JVM is a little strange, for instance. For a clean redesign in 2.0 ideally we can clean up a lot of the remaining sources of confusion with accumulators and unify it with `TaskMetrics` somehow so we get the best of both worlds (e.g. provide type safety). Until then, I personally find accumulators easier to work with.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org