JoshRosen opened a new pull request, #45203: URL: https://github.com/apache/spark/pull/45203
### What changes were proposed in this pull request? This PR adds logic to avoid uncaught `RejectedExecutionException`s while `StandaloneSchedulerBackend` is shutting down. When the backend is shut down, its `stop()` method calls `executorDelayRemoveThread.shutdownNow()`. After this point, though, it's possible that its `StandaloneDriverEndpoint` might still process `onDisconnected` events and those might trigger calls to schedule new tasks on the `executorDelayRemoveThread`. This causes uncaught `java.util.concurrent.RejectedExecutionException`s to be thrown in RPC threads. This patch adds a `try-catch` to catch those exceptions and log a short warning if the exceptions occur while the scheduler is stopping. This approach is consistent with other similar code in Spark, including: - https://github.com/apache/spark/blob/9b53b803998001e4b706666e37e5f86f900a7430/core/src/main/scala/org/apache/spark/scheduler/TaskResultGetter.scala#L160-L163 - https://github.com/apache/spark/blob/9b53b803998001e4b706666e37e5f86f900a7430/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala#L754-L756 ### Why are the changes needed? Remove log and exception noise. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? No new tests: it is difficult to reliably reproduce the scenario that leads to the log noise. ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
