[Python-ideas] Re: Non-blocking concurrent.futures ThreadPoolExecutor

2021-04-10 Thread Brunoais
(I know this has some time but I think it makes sense in context) That is much easier said than done in real life when the whole project is doing by a team and you are stuck with whatever you all do in the group. Even then, I prefer making my own programs with a mix of daemon and non-daemon

[Python-ideas] Re: Non-blocking concurrent.futures ThreadPoolExecutor

2020-03-12 Thread Remy NOEL
On Wed, Mar 11, 2020 at 3:20 AM Kyle Stanley wrote: > IIUC, I don't think there's a way that we could reasonably support > stability for provide any guarantees for spawning or interacting with the > Executors from a daemon thread. It might even be worth considering to > explicitly warn or

[Python-ideas] Re: Non-blocking concurrent.futures ThreadPoolExecutor

2020-03-10 Thread Kyle Stanley
Remy NOEL wrote: > This means that if a ThreadExecutor is called from both daemon and non-daemon threads (which is a very bad idea though), the kind of spawned threads might be unpredictable. IIUC, I don't think there's a way that we could reasonably support stability for provide any guarantees

[Python-ideas] Re: Non-blocking concurrent.futures ThreadPoolExecutor

2020-03-10 Thread Antoine Pitrou
On Tue, 10 Mar 2020 15:16:06 +0100 Remy NOEL wrote: > On Mon, Mar 9, 2020 at 11:18 PM Kyle Stanley wrote: > > > We're currently looking into adjusting the current Executor implementation > > (for both TPE and PPE) to not use daemon threads at all for subinterpreter > > compatibility, see

[Python-ideas] Re: Non-blocking concurrent.futures ThreadPoolExecutor

2020-03-10 Thread Remy NOEL
On Tue, Mar 10, 2020 at 1:04 PM Rhodri James wrote: > Well, yes. The non-ugly thing would be to make sure your threads are > well behaved ;-) > That is kinda hard to do for the general case, i prefer to have a failsafe. ___ Python-ideas mailing list

[Python-ideas] Re: Non-blocking concurrent.futures ThreadPoolExecutor

2020-03-10 Thread Remy NOEL
On Mon, Mar 9, 2020 at 11:18 PM Kyle Stanley wrote: > We're currently looking into adjusting the current Executor implementation > (for both TPE and PPE) to not use daemon threads at all for subinterpreter > compatibility, see https://bugs.python.org/issue39812. Instead of using > an atexit

[Python-ideas] Re: Non-blocking concurrent.futures ThreadPoolExecutor

2020-03-10 Thread Rhodri James
On 09/03/2020 17:50, Remy NOEL wrote: I ran into a problem which, according to a google search is not that uncommon: concurrent.futures ThreadPoolExecutor, installs an exit handler preventing program exit if any of the jobs it is running is blocked. (Which might be a surprising behaviour to

[Python-ideas] Re: Non-blocking concurrent.futures ThreadPoolExecutor

2020-03-09 Thread Kyle Stanley
> It would seems reasonable to have a separate _daemon_thread_queues dict (in contrast to _threads_queues) that would not get joined upon exit. > A "daemon" (or any better name) parameter of ThreadPoolExecutor initialisation would allow to let it add its workers in this dict rather than the