GitHub user advancedxy opened a pull request: https://github.com/apache/spark/pull/21165
Spark 20087: Attach accumulators / metrics to 'TaskKilled' end reason ## What changes were proposed in this pull request? The ultimate goal is for listeners to onTaskEnd to receive metrics when a task is killed intentionally, since the data is currently just thrown away. This is already done for ExceptionFailure, so this just copies the same approach. ## How was this patch tested? Updated existing tests. This is a rework of https://github.com/apache/spark/pull/17422, all credits should go to @noodle-fb You can merge this pull request into a Git repository by running: $ git pull https://github.com/advancedxy/spark SPARK-20087 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/21165.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #21165 ---- commit 6262c52f7eb4abcf06742e9afd9d1454f06cdf1f Author: Charles Lewis <noodle@...> Date: 2017-03-22T20:33:55Z report metrics for killed tasks commit 18308895ad07c9c757e96e08f5c40b5dcaaf3455 Author: Charles Lewis <noodle@...> Date: 2017-03-24T19:06:46Z add task killed to exception accum test commit cb276bc1f1583ca7dcf44b8e80a9fc2cd09953cf Author: Charles Lewis <noodle@...> Date: 2017-03-24T23:20:59Z extra fixes for task killed reason merge commit 30ae1457afd9899a5fa937c2fefac31f6a0752ed Author: Xianjin YE <advancedxy@...> Date: 2018-04-26T07:34:54Z Fix merge conflict and semantic difference commit 88b1cebd76d7414d4bbdb99e03fe10f74e25029a Author: Xianjin YE <advancedxy@...> Date: 2018-04-26T09:22:20Z Make accums in TaskKilled immutable and extract common logic in TaskRunner to reduce duplicate code ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org