[jira] [Updated] (SPARK-12402) Memory leak in pyspark
[ https://issues.apache.org/jira/browse/SPARK-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-12402: - Labels: bulk-closed (was: ) > Memory leak in pyspark > -- > > Key: SPARK-12402 > URL: https://issues.apache.org/jira/browse/SPARK-12402 > Project: Spark > Issue Type: Bug > Components: SQL >Reporter: Davies Liu >Priority: Major > Labels: bulk-closed > > After run some SQL query in PySpark, the DataFrame are still referenced by > py4j Gateway, they are freed after calll `gc.collect()` in Python. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-12402) Memory leak in pyspark
[ https://issues.apache.org/jira/browse/SPARK-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12402: --- Description: After run some SQL query in PySpark, the DataFrame are still referenced by py4j Gateway, they are freed after calll `gc.collect()` in Python. (was: The broadcasted HashRelation is not destroyed after query finished (also can't be reused).) > Memory leak in pyspark > -- > > Key: SPARK-12402 > URL: https://issues.apache.org/jira/browse/SPARK-12402 > Project: Spark > Issue Type: Bug > Components: SQL >Reporter: Davies Liu > > After run some SQL query in PySpark, the DataFrame are still referenced by > py4j Gateway, they are freed after calll `gc.collect()` in Python. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-12402) Memory leak in pyspark
[ https://issues.apache.org/jira/browse/SPARK-12402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12402: --- Summary: Memory leak in pyspark (was: Memory leak in broadcast hash join) > Memory leak in pyspark > -- > > Key: SPARK-12402 > URL: https://issues.apache.org/jira/browse/SPARK-12402 > Project: Spark > Issue Type: Bug > Components: SQL >Reporter: Davies Liu > > The broadcasted HashRelation is not destroyed after query finished (also > can't be reused). -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
