[ https://issues.apache.org/jira/browse/SPARK-27337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16823120#comment-16823120 ]
Vinoo Ganesh commented on SPARK-27337: -------------------------------------- Hey [~cltlfcjin] - the thread is called Closing a SparkSession stops the SparkContext. I'll put up a PR for this shortly. > QueryExecutionListener never cleans up listeners from the bus after > SparkSession is cleared > ------------------------------------------------------------------------------------------- > > Key: SPARK-27337 > URL: https://issues.apache.org/jira/browse/SPARK-27337 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.0 > Reporter: Vinoo Ganesh > Priority: Major > Attachments: image001-1.png > > > As a result of > [https://github.com/apache/spark/commit/9690eba16efe6d25261934d8b73a221972b684f3], > it looks like there is a memory leak (specifically > [https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala#L131).] > > Because the Listener Bus on the context still has a reference to the listener > (even after the SparkSession is cleared), they are never cleaned up. This > means that if you close and remake spark sessions fairly frequently, you're > leaking every single time. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org