Xi Lyu created SPARK-47819: ------------------------------ Summary: Use asynchronous callback for execution cleanup Key: SPARK-47819 URL: https://issues.apache.org/jira/browse/SPARK-47819 Project: Spark Issue Type: Improvement Components: Connect Affects Versions: 4.0.0 Reporter: Xi Lyu Fix For: 4.0.0
Expired sessions are regularly checked and cleaned up by a maintenance thread. However, currently, this process is synchronous. Therefore, in occasional cases, interrupting the execution thread of a query in a session can take hours, causing the entire maintenance process to stall, resulting in a large amount of memory not being cleared. We address this by introducing asynchronous callbacks for execution cleanup, avoiding synchronous joins of execution threads, and preventing the maintenance thread from stalling in the above occasional scenarios. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org