RamonZhou opened a new pull request, #58396: URL: https://github.com/apache/spark/pull/58396
### What changes were proposed in this pull request? This follow-up records when `SparkConnectClient.release_session()` completes successfully and makes the registered `atexit` handler a no-op for an already released session. It also adds regression coverage that verifies successful release updates the client state and that `_on_exit()` does not issue ML cache cleanup, release, or close RPCs afterward. This follows SPARK-55326 and the cleanup-after-close fix in #56140. ### Why are the changes needed? `_on_exit()` currently skips cleanup only when the client channel has been explicitly closed. Calling `release_session()` releases all server-side resources but leaves the client channel open. At process exit, the client therefore sends an unnecessary ML cache cleanup command to a session that was already released. Besides being redundant, that RPC can delay process exit when the server is unavailable. ### Does this PR introduce _any_ user-facing change? Yes. A process that exits after explicitly releasing its Spark Connect session no longer sends additional cleanup RPCs for that released session. ### How was this patch tested? Added unit coverage and ran: ```bash PYTHONPATH=python python -m unittest pyspark.sql.tests.connect.client.test_client ``` All 43 tests passed. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
