Andrew Or created SPARK-6132: -------------------------------- Summary: Context cleaner thread lives across SparkContexts Key: SPARK-6132 URL: https://issues.apache.org/jira/browse/SPARK-6132 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.3.0 Reporter: Andrew Or Assignee: Andrew Or
The context cleaner thread is not stopped properly. If a SparkContext is started immediately after one stops, the context cleaner of the former can clean variables in the latter. This is because the cleaner.stop() just sets a flag and expects the thread to terminate asynchronously, but the code to clean broadcasts goes through `SparkEnv.get.blockManager`, which could belong to a different SparkContext. The right behavior is to wait until all currently running clean up tasks have finished. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org