Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17596 @mridulm I actually have the same plan, I think it's an overkill to implement TaskMetrics with accumulators, we don't need to merge the accumulator updates at driver side for TaskMetrics accumulators. We should send back TaskMetrics directly with hearbeat, task failure and task finish, then we can just send a bunch of `long`s and compress. One thing I'm not 100% agree with you is about the bitmask. According to my experiment, most of the task metrics will not be 0, so the bitmask may not be very useful.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org