Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/9946#issuecomment-173298943 Just to point out here I may re-open this. I would still rather fix this, a known bug that I can reproduce then worry about a theoretical it might break something. Unless of course someone can give me a hard use case that this breaks. At the point we call System.exit here all user code is done and we are terminating. If there is something you know of that the user should be allowed to do after this then we should create something like a shutdown hook so the user can properly clean it up. A timeout would probably work here but I'm not fond of it here, it is going to delay it for everyone and you don't give the resources back as quickly. You could also end up in the same situation where if you choose a timeout, say 3 seconds and if this theoretical user code didn't finish within that 3 seconds you would kill it anyway. Actually thinking about this more, maybe this is a perfect time as we can put it in 2.0 so change in behavior is more reasonable. thoughts ?
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org