[
https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629313#comment-15629313
]
Yin Huai commented on SPARK-17822:
----------------------------------
Basically, the problem that I have observed is a long running Spark driver runs
out of memory because JVMObjectTracker.objMap prevents objects that are not
used anymore from getting GCed. I am attaching a screenshot which shows the
objects inside the map.
> JVMObjectTracker.objMap may leak JVM objects
> --------------------------------------------
>
> Key: SPARK-17822
> URL: https://issues.apache.org/jira/browse/SPARK-17822
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Reporter: Yin Huai
> Attachments: screenshot-1.png
>
>
> JVMObjectTracker.objMap is used to track JVM objects for SparkR. However, we
> observed that JVM objects that are not used anymore are still trapped in this
> map, which prevents those object get GCed.
> Seems it makes sense to use weak reference (like persistentRdds in
> SparkContext).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
