[GitHub] spark pull request #17088: [SPARK-19753][CORE] All shuffle files on a host s...

2017-02-28 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/17088#discussion_r103552476 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1331,7 +1332,7 @@ class DAGScheduler( // TODO:

[GitHub] spark pull request #17088: [SPARK-19753][CORE] All shuffle files on a host s...

2017-02-28 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/17088#discussion_r103541395 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1331,7 +1332,7 @@ class DAGScheduler( //

[GitHub] spark pull request #17088: [SPARK-19753][CORE] All shuffle files on a host s...

2017-02-27 Thread sitalkedia
GitHub user sitalkedia opened a pull request: https://github.com/apache/spark/pull/17088 [SPARK-19753][CORE] All shuffle files on a host should be removed in … ## What changes were proposed in this pull request? Currently, when we detect fetch failure, we only remove the