cloud-fan commented on PR #56140: URL: https://github.com/apache/spark/pull/56140#issuecomment-4557983896
Thanks @gaogaotiantian, you're absolutely right — I missed that. The [atexit docs](https://docs.python.org/3/library/atexit.html#atexit.register) explicitly forbid starting threads in registered functions (raises `RuntimeError` on 3.12+, risks race conditions on earlier versions), so the daemon-thread approach was broken on the very platform Spark targets. I've pushed a revision that drops the thread entirely and uses gRPC's native per-call deadline instead — `release_session` and `_cleanup_ml_cache` now accept an optional `timeout` that's passed through to the stub call, and `_on_exit` invokes them with `timeout=10.0`. If the deadline elapses, the RPC fails with `DEADLINE_EXCEEDED`, we swallow it, and the server's own session GC reclaims any leftover state — same end behavior, no thread involved. PR description updated accordingly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
