Eric Snow <ericsnowcurren...@gmail.com> added the comment:
> This means that ThreadPoolExecutor's atexit runs before mine, > and since I never get a chance to cancel my tasks, it deadlocks. (assuming we want to support long-running tasks here) With all the above in mind, there are a few things that may help. The first that comes to mind is to have the atexit handler call ThreadPoolExecutor.shutdown() for each instance. So something like this: def _python_exit(): global _shutdown with _global_shutdown_lock: _shutdown = True for executor in list(_executors): executor.shutdown() That would require a little refactoring to make it work. However, the change is simpler if each executor has its own atexit handler: class ThreadPoolExecutor(_base.Executor): def __init__(self, ...): ... threading._register_atexit(self._atexit()) def _atexit(self): global _shutdown _shutdown = True self.shutdown() The value of either approach is that you can then subclass ThreadPoolExecutor to get what you want: class MyExecutor(ThreadPoolExecutor): def shutdown(self, *args, **kwargs): stop_my_tasks() super().shutdown(*args, **kwwargs) ---- One thing I thought about was supporting a per-task finalizer instead, since that aligns more closely with the way ThreadPoolExecutor works. It would only apply So something like one of the following: * ThreadPoolExecutor(finalize_task=<callable>) * ThreadPoolExecutor.submit(finalize=<callable) ---- Other things that could be helpful: * always cancel all the pending tasks during shutdown (and maybe let users opt out) * use a timeout during shutdown ---- FWIW, adding support for some sort of sub-atexit handler isn't so appealing. I'm talking about something like one of the following: * ThreadPoolExecutor(onshutdown=<callable>) * ThreadPoolExecutor.register_atexit(<callable>) * (classmethod) ThreadPoolExecutor.register_atexit(<callable>) * concurrent.futures.register_atexit(<callable>) (It would probably make sense to pass the list of currently running tasks to the callable.) ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue41962> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com