Repository: spark
Updated Branches:
refs/heads/master faabdfa2b -> 6b781576a
[SPARK-11974][CORE] Not all the temp dirs had been deleted when the JVM exits
deleting the temp dir like that
```
scala> import scala.collection.mutable
import scala.collection.mutable
scala> val a = mutable.Set(1,2,3,4,7,0,8,98,9)
a: scala.collection.mutable.Set[Int] = Set(0, 9, 1, 2, 3, 7, 4, 8, 98)
scala> a.foreach(x => {a.remove(x) })
scala> a.foreach(println(_))
98
```
You may not modify a collection while traversing or iterating over it.This can
not delete all element of the collection
Author: Zhongshuai Pei <peizhongsh...@huawei.com>
Closes #9951 from DoingDone9/Bug_RemainDir.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6b781576
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/6b781576
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/6b781576
Branch: refs/heads/master
Commit: 6b781576a15d8d5c5fbed8bef1c5bda95b3d44ac
Parents: faabdfa
Author: Zhongshuai Pei <peizhongsh...@huawei.com>
Authored: Wed Nov 25 10:37:34 2015 -0800
Committer: Reynold Xin <r...@databricks.com>
Committed: Wed Nov 25 10:37:34 2015 -0800
----------------------------------------------------------------------
.../main/scala/org/apache/spark/util/ShutdownHookManager.scala | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/6b781576/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
----------------------------------------------------------------------
diff --git
a/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
b/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
index db4a8b3..4012dca 100644
--- a/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
+++ b/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
@@ -57,7 +57,9 @@ private[spark] object ShutdownHookManager extends Logging {
// Add a shutdown hook to delete the temp dirs when the JVM exits
addShutdownHook(TEMP_DIR_SHUTDOWN_PRIORITY) { () =>
logInfo("Shutdown hook called")
- shutdownDeletePaths.foreach { dirPath =>
+ // we need to materialize the paths to delete because deleteRecursively
removes items from
+ // shutdownDeletePaths as we are traversing through it.
+ shutdownDeletePaths.toArray.foreach { dirPath =>
try {
logInfo("Deleting directory " + dirPath)
Utils.deleteRecursively(new File(dirPath))
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
