[GitHub] [spark] zsxwing commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver
zsxwing commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver URL: https://github.com/apache/spark/pull/24796#issuecomment-499298514 > If this is removed, I am not sure what goes wrong. If `onStop` happens before or after an `onReceive` during shutdown, that's fine. If it happens during `onReceive`, well, it has already been interrupted and won't complete normally. The potential issue is about thread-safety. Right now EventLoop ensures all `onXXX` calls run in sequence without any overlapping. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zsxwing commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver
zsxwing commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver URL: https://github.com/apache/spark/pull/24796#issuecomment-499764530 > `onReceive()` is interrupted before `onStop()` But there will be a race condition if removing `join`. We cannot guarantee that `onReceive` can return immediately when it receives the interrupt signal. By the way, is there any theory about how this deadlock can happen? As I mentioned here: https://github.com/apache/spark/pull/24796#discussion_r290908122 I could not reproduce it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] zsxwing commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver
zsxwing commented on issue #24796: [SPARK-27900][CORE] Add uncaught exception handler to the driver URL: https://github.com/apache/spark/pull/24796#issuecomment-501049193 Before making the decision, I would like to understand why this dead lock can happen. There are still some unknowns here, for example, why "eventThread.interrupt()" doesn't interrupt `Thread.join` in the `dag-scheduler-event-loop` thread. ? Is it just a special behavior in shutdown hooks, and it can happen in other cases as well? Without the answers, we don't know if any fix can really fix the root cause. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
