[
https://issues.apache.org/jira/browse/SPARK-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-7917:
-----------------------------
Priority: Minor (was: Major)
> Spark doesn't clean up Application Directories (local dirs)
> ------------------------------------------------------------
>
> Key: SPARK-7917
> URL: https://issues.apache.org/jira/browse/SPARK-7917
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.3.0
> Reporter: Zach Fry
> Priority: Minor
>
> Similar to SPARK-4834.
> Spark does clean up the cache and lock files in the local dirs, however, it
> doesn't clean up the actual directories.
> We have to write custom scripts to go back through the local dirs and find
> directories that don't contain any files and clear those out.
> Its a pretty simple repro:
> Run a job that does some shuffling, wait for the shuffle files to get cleaned
> up, go and look on disk at spark.local.dir and notice that the directory(s)
> are still there, but there are no files in them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
