[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14153382#comment-14153382 ]
Matt Cheah commented on SPARK-1860: ----------------------------------- Cool, I see where you're coming from now. I'll whip up something. Thanks for the input! The cleanup is more for cosmetic sake than performance, I agree. > Standalone Worker cleanup should not clean up running executors > --------------------------------------------------------------- > > Key: SPARK-1860 > URL: https://issues.apache.org/jira/browse/SPARK-1860 > Project: Spark > Issue Type: Bug > Components: Deploy > Affects Versions: 1.0.0 > Reporter: Aaron Davidson > Priority: Blocker > > The default values of the standalone worker cleanup code cleanup all > application data every 7 days. This includes jars that were added to any > executors that happen to be running for longer than 7 days, hitting streaming > jobs especially hard. > Executor's log/data folders should not be cleaned up if they're still > running. Until then, this behavior should not be enabled by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org