Ben Darnell <ben.darn...@gmail.com> added the comment:
> To be clear, by "cancel" you are not talking about Future.cancel(). Rather, > your handler causes all running tasks to finish (by sending a special message > on the socket corresponding to each running task). Is that right? Correct. My tasks here are calls to functions from the `select` module (one select call per executor task), and cancelling them means writing a byte to a pipe set up for this purpose. The select calls could be given a timeout so there is never an infinite task, but that's not ideal - a timeout that's too low has a performance cost as calls timeout and restart even when the system is "at rest", and a too-long timeout is still going to be perceived as a hanging application. > * it does not make sure the task associated with the socket finishes (no way > of knowing?) > * so if a task hangs while trying to stop then the running thread in the > ThreadPoolExecutor would block shutdown forever > * similarly, if a task is stuck handling a request then it will never receive > the special message on the socket, either blocking the send() in your handler > or causing ThreadPoolExecutor shutdown/atexit to wait forever Correct. If the task were buggy it could still cause a deadlock. In my case the task is simple enough (a single selector call) that this is not a risk. > * it vaguely implies a 1-to-1 relationship between sockets and *running* tasks > * likewise that pending (queued) tasks do not have an associated socket > (until started) Each task is associated with a selector object (managing a set of sockets), not a single socket. There is only ever one task at a time; a task is enqueued only after the previous one finishes. (This thread pool is not used for any other purpose) > * so once your handler finishes, any tasks pending in the ThreadPoolExecutor > queue will eventually get started but never get stopped by your handler; thus > you're back to the deadlock situation In my case this one-at-a-time rule means that the queue is always empty. But yes, in a more general solution you'd need some sort of interlock between cancelling existing tasks and starting new ones. > Alternately, perhaps ThreadPoolExecutor isn't the right fit here, as implied > by the route you ended up going. Yes, this is my conclusion as well. I filed this issue because I was frustrated that Python 3.9 broke previously-working code, but I'm willing to chalk this up to Hyrum's law and I'm not sure that this is something that ThreadPoolExecutor should be modified to support. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue41962> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
