[ https://issues.apache.org/jira/browse/SPARK-11276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-11276: ------------------------------------ Assignee: Apache Spark > SizeEstimator prevents class unloading > -------------------------------------- > > Key: SPARK-11276 > URL: https://issues.apache.org/jira/browse/SPARK-11276 > Project: Spark > Issue Type: Bug > Components: Block Manager, Spark Core > Affects Versions: 1.5.1 > Reporter: Sem Mulder > Assignee: Apache Spark > > The SizeEstimator keeps a cache of ClassInfos but this cache uses Class > objects as keys. > Which results in strong references to the Class objects. If these classes are > dynamically created > this prevents the corresponding ClassLoader from being GCed. Leading to > PermGen exhaustion. > An easy fix would be to use a WeakRef for the keys. A proposed fix can be > found here: > [https://github.com/Site2Mobile/spark/commit/21c572cbda5607d0c7c6643bfaf43e53c8aa6f8c] > We are currently running this in production and it seems to resolve the issue. > I will prepare a pull request ASAP. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org