cloud-fan commented on PR #56140: URL: https://github.com/apache/spark/pull/56140#issuecomment-4563069285
Thanks for catching that, @gaogaotiantian — you're right that bypassing `execute_command` is a bigger change than I'd like to land without more discussion. I've rescoped this PR to only the uncontroversial fix: short-circuit `_on_exit` when `self._closed` is already `True`. That covers the case where the user has explicitly called `close()` but the `atexit` handler still re-issues a cleanup RPC afterward — which is wasted work and a hang risk if the server is unreachable. The broader case (user never called `close()`, server is unreachable) is left for a separate PR where we can decide whether bypassing the reattachable iterator for fire-and-forget cleanup is acceptable, or whether to thread a deadline through `execute_command` instead. cc @wbo4958 @HyukjinKwon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
