[ https://issues.apache.org/jira/browse/SPARK-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-2255. ------------------------------ Resolution: Won't Fix I'm guessing this isn't going anywhere or is obsolete > create a self destructing iterator that releases records from hash maps > ----------------------------------------------------------------------- > > Key: SPARK-2255 > URL: https://issues.apache.org/jira/browse/SPARK-2255 > Project: Spark > Issue Type: Improvement > Components: SQL > Reporter: Reynold Xin > > This is a small thing to do that can help out with GC pressure. For > aggregations (and potentially joins), we don't really need to hold onto the > key value pairs as soon as we have iterate over them. We can create a self > destructing iterator for AppendOnlyMap / ExternalAppendOnlyMap that removes > references to the key value pair as the iterator goes through records so > those memory can be freed quickly. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org