Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/16774 Thanks for getting back to this @MLnick. The only difference with daemon threads is they won't prevent the JVM from shutting down - which is what we would want in any case. The name is a bit misleading, but the JVM will not shutdown until all non-daemon threads have finished, so if the main thread is done and only daemon threads are still running, then the JVM will shutdown immediately. This is what we would want here because if the thread invoking CrossValidation is interrupted, then we do not want to wait for all the worker threads to finish, we want them to stop immediately. It doesn't matter if the work is incomplete because unless CrossValidation has run it's course the results are not really useful, so graceful shutdown is not really needed here. By cleanup, I mean the threadpool should be scoped to the `fit` method, not the class instance, so any resources wouldn't be hanging around after calling `fit` once the threadpool went out of scope - you don't really need to cleanup explicitly. We can still do this without using the factory. I guess the reason for me using the factory pattern was attempting to be as flexible as possible and still allow using the `numParallelEval` param with a custom executor. My thinking was thatcompletley maybe `numParallelEval` is a user level param and the `ExecutorService` would be configured on some higher level. Since none of this is in use today and it could be a completely different when it does, it would make sense to just make the API simple like you mentioned until it can be better hashed out. So I'll redo this with using `def setExecutorService(executorService: ExecutorService): Unit` - sorry for the long-winded response!
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org