[issue39812] Avoid daemon threads in concurrent.futures

Change by Josh Rosenberg : -- Removed message: https://bugs.python.org/msg416876 ___ Python tracker ___ ___ Python-bugs-list

[issue39812] Avoid daemon threads in concurrent.futures

Josh Rosenberg added the comment: I think this is causing a regression for code that explicitly desires the ThreadPoolExecutor to go away abruptly when all other non-daemon threads complete (by choosing not to use a with statement, and if shutdown is called, calling it with wait=False, or

[issue39812] Avoid daemon threads in concurrent.futures

Jack DeVries added the comment: The regression that @janfrederik.konopka points out also has it's own open issue: bpo-43944. I'm trying to work on a fix for this regression. Slowly but surely. Now I've finally found these threads, this information will be a big help! Any tips are

[issue39812] Avoid daemon threads in concurrent.futures

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39812] Avoid daemon threads in concurrent.futures

Antoine Pitrou added the comment: @Jan, without taking a look, I'd answer that indeed you should avoid using the "fork" method if you're doing any kind of multithreading in the parent process. "forkserver" is a good choice nowadays on Linux and will result in more robust code. --

[issue39812] Avoid daemon threads in concurrent.futures

Jan Konopka added the comment: Hi all! While browsing StackOverflow I came across this question: https://stackoverflow.com/q/67273533/2111778 The user created a ThreadPoolExecutor which started a Process using multiprocessing. The Process produces an exitcode of 0 in Python 3.8 but an

[issue39812] Avoid daemon threads in concurrent.futures

Kyle Stanley added the comment: > Thanks Kyle for the test_asyncio fix! No problem! Thanks for the review. :-) -- ___ Python tracker ___

[issue39812] Avoid daemon threads in concurrent.futures

STINNER Victor added the comment: Kyle fixed bpo-40115, I close again this issue. Thanks Kyle for the test_asyncio fix! -- resolution: -> fixed status: open -> closed ___ Python tracker

[issue39812] Avoid daemon threads in concurrent.futures

Kyle Stanley added the comment: I attached a PR to bpo-40115 to address the refleak in test_asyncio: PR-19282. -- ___ Python tracker ___

[issue39812] Avoid daemon threads in concurrent.futures

Kyle Stanley added the comment: I'm currently busy today, but I'll have some time to explore a few different potential solutions tomorrow. If I can't find anything to fix it by the end of the day, I'll prepare a PR to revert PR-19149, and re-apply it later (after more thoroughly testing to

[issue39812] Avoid daemon threads in concurrent.futures

STINNER Victor added the comment: > I'll see if I can figure it out, but this one might be a bit tricky to > troubleshoot. Oh yes, I know that fixing such issue can be very tricky. For example, there is no threading._unregister_atexit() function. Maybe the callback stays alive after the

[issue39812] Avoid daemon threads in concurrent.futures

Kyle Stanley added the comment: > This change introduced a leak in test_asyncio: bpo-40115. Thanks for bringing attention to it Victor. It seems like a rather odd side effect, considering that PR-19149 had no C code and was internal to concurrent.futures and threading. I did not expect

[issue39812] Avoid daemon threads in concurrent.futures

STINNER Victor added the comment: (Oops typo) If buildbots remain red, we will *miss* other regressions which would make the situation even worse. -- ___ Python tracker ___

[issue39812] Avoid daemon threads in concurrent.futures

STINNER Victor added the comment: I reopen the issue since it introduced a regression on buildbots. If no fix is found soon (let's say in 1 or 2 days), I will revert the change to get more time to investigate. If buildbots remain red, we will be other regressions which would make the

[issue39812] Avoid daemon threads in concurrent.futures

STINNER Victor added the comment: > bpo-39812: Remove daemon threads in concurrent.futures (GH-19149) This change introduced a leak in test_asyncio: bpo-40115. -- nosy: +vstinner ___ Python tracker

[issue39812] Avoid daemon threads in concurrent.futures

Change by Antoine Pitrou : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue39812] Avoid daemon threads in concurrent.futures

Antoine Pitrou added the comment: New changeset b61b818d916942aad1f8f3e33181801c4a1ed14b by Kyle Stanley in branch 'master': bpo-39812: Remove daemon threads in concurrent.futures (GH-19149) https://github.com/python/cpython/commit/b61b818d916942aad1f8f3e33181801c4a1ed14b --

[issue39812] Avoid daemon threads in concurrent.futures

Change by Kyle Stanley : -- keywords: +patch pull_requests: +18510 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/19149 ___ Python tracker

[issue39812] Avoid daemon threads in concurrent.futures

Change by Kyle Stanley : -- assignee: -> aeros ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39812] Avoid daemon threads in concurrent.futures

Kyle Stanley added the comment: I spent some further time considering the solution to the problem, and I still think something like a `threading.register_atexit()` (see https://bugs.python.org/issue37266#msg362960) would be the most suitable. However, I'm not certain regarding the exact

[issue39812] Avoid daemon threads in concurrent.futures

New submission from Antoine Pitrou : Since issue37266 (which forbid daemon threads in subinterpreters), we probably want to forego daemon threads in concurrent.futures. This means we also need a way to run an atexit-like hook before non-daemon threads are joined on (sub)interpreter