Matt Whelan created SPARK-5841: ---------------------------------- Summary: Memory leak in DiskBlockManager Key: SPARK-5841 URL: https://issues.apache.org/jira/browse/SPARK-5841 Project: Spark Issue Type: Bug Components: Block Manager Affects Versions: 1.2.1 Reporter: Matt Whelan
DiskBlockManager registers a Runtime shutdown hook, which creates a hard reference to the entire Driver ActorSystem. If a long-running JVM repeatedly creates and destroys SparkContext instances, it leaks memory. I suggest we deregister the shutdown hook if DiskBlockManager.stop is called. It's redundant at that point. PR coming. See also http://mail-archives.apache.org/mod_mbox/spark-user/201501.mbox/%3CCA+kjH+w_DDTEBE9XB6NrPxLTUXD=nc_d-3ogxtumk_5v-e0...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org