Github user zhuoliu commented on the pull request: https://github.com/apache/spark/pull/9946#issuecomment-175121155 Sure. A brief summary is that: (picked a few conclusive points from above): "At the point we call System.exit here all user code is done and we are terminating. If there is something you know of that the user should be allowed to do after this then we should create something like a shutdown hook so the user can properly clean it up. A timeout would probably work here but I'm not fond of it here, it is going to delay it for everyone and you don't give the resources back as quickly. You could also end up in the same situation where if you choose a timeout, say 3 seconds and if this theoretical user code didn't finish within that 3 seconds you would kill it anyway." "I also don't buy the existence of any valid user case for a user thread to block container exit," "I'm saying that it's the user's fault if his application depends on that non-predictable behavior. We're talking milliseconds here, that might be affected by everything from YARN's internal code to network delays to kernel scheduling. There's absolutely no argument for someone depending on that for the correct behavior of their application."' "I'm on board with this if it's true that YARN virtually immediately kills a JVM like this if it's not done by the time the NM thinks it's done. Then indeed regardless of what it does to an app that's tardy in cleaning up, it's not a materially different behavior. If it helps handle another bad app behavior, there's no downside to doing that."
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org