[ https://issues.apache.org/jira/browse/SPARK-27337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinoo Ganesh updated SPARK-27337: --------------------------------- Affects Version/s: (was: 2.4.0) > QueryExecutionListener never cleans up listeners from the bus after > SparkSession is closed > ------------------------------------------------------------------------------------------ > > Key: SPARK-27337 > URL: https://issues.apache.org/jira/browse/SPARK-27337 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 3.0.0 > Reporter: Vinoo Ganesh > Priority: Critical > Attachments: image001-1.png > > > As a result of > [https://github.com/apache/spark/commit/9690eba16efe6d25261934d8b73a221972b684f3], > it looks like there is a memory leak (specifically > [https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala#L131).] > > Because the Listener Bus on the context still has a reference to the listener > (even after the SparkSession is cleared), they are never cleaned up. This > means that if you close and remake spark sessions fairly frequently, you're > leaking every single time. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org