Hi there, I'm a bit confused. In my scenario I a mixing threading with multiprocessing. Threading by itself would be nice, but for GIL reasons I need both, unfortunately. I've encountered a weird situation in which multiprocessing Process()es which are started in a new thread don't actually start and so they deadlock on join.
I've created a minimal example that demonstrates the issue. I'm running on x86_64 Linux using Python 3.9.5 (default, May 11 2021, 08:20:37) ([GCC 10.3.0] on linux). Here's the code: import time import multiprocessing import threading def myfnc(): print("myfnc") def run(result_queue, callback): result = callback() result_queue.put(result) def start(fnc): def background_thread(): queue = multiprocessing.Queue() proc = multiprocessing.Process(target = run, args = (queue, fnc)) proc.start() print("join?") proc.join() print("joined.") result = queue.get() threading.Thread(target = background_thread).start() start(myfnc) start(myfnc) start(myfnc) start(myfnc) while True: time.sleep(1) What you'll see is that "join?" and "joined." nondeterministically does *not* appear in pairs. For example: join? join? myfnc myfnc join? join? joined. joined. What's worse is that when this happens and I Ctrl-C out of Python, the started Thread is still running in the background: $ ps ax | grep minimal 370167 pts/0 S 0:00 python3 minimal.py 370175 pts/2 S+ 0:00 grep minimal Can someone figure out what is going on there? Best, Johannes -- https://mail.python.org/mailman/listinfo/python-list