[
https://issues.apache.org/jira/browse/SPARK-7705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14559055#comment-14559055
]
Apache Spark commented on SPARK-7705:
-------------------------------------
User 'Sephiroth-Lin' has created a pull request for this issue:
https://github.com/apache/spark/pull/6409
> Cleanup of .sparkStaging directory fails if application is killed
> -----------------------------------------------------------------
>
> Key: SPARK-7705
> URL: https://issues.apache.org/jira/browse/SPARK-7705
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 1.3.0
> Reporter: Wilfred Spiegelenburg
> Priority: Minor
>
> When a streaming application is killed while running on YARN the
> .sparkStaging directory is not cleaned up. Setting
> spark.yarn.preserve.staging.files=false does not help and still leaves the
> files around.
> The changes in SPARK-7503 do not catch this case since there is no exception
> in the shutdown. When the application gets killed the AM gets told to
> shutdown and the shutdown hook is run but the clean up is not triggered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
