Hi!, in short your code should work.
I think that the join-joined problem is just an interpretation problem.
In pseudo code the background_thread function does:
def background_thread()
# bla
print("join?")
# bla
print("joined")
When running this function in parallel using threads, you will probably
get a few "join?" first before receiving any "joined?". That is because
the functions are running in parallel.
The order "join?" then "joined" is preserved within a thread but not
preserved globally.
Now, I see another issue in the output (and perhaps you was asking about
this one):
join?
join?
myfnc
myfnc
join?
join?
joined.
joined.
So you have 4 "join?" that correspond to the 4 background_thread
function calls in threads but only 2 "myfnc" and 2 "joined".
Could be possible that the output is truncated by accident?
I ran the same program and I got a reasonable output (4 "join?", "myfnc"
and "joined"):
join?
join?
myfnc
join?
myfnc
join?
joined.
myfnc
joined.
joined.
myfnc
joined.
Another issue that I see is that you are not joining the threads that
you spawned (background_thread functions).
I hope that this can guide you to fix or at least narrow the issue.
Thanks,
Martin.
On Mon, Dec 06, 2021 at 12:50:11AM +0100, Johannes Bauer wrote:
Hi there,
I'm a bit confused. In my scenario I a mixing threading with
multiprocessing. Threading by itself would be nice, but for GIL reasons
I need both, unfortunately. I've encountered a weird situation in which
multiprocessing Process()es which are started in a new thread don't
actually start and so they deadlock on join.
I've created a minimal example that demonstrates the issue. I'm running
on x86_64 Linux using Python 3.9.5 (default, May 11 2021, 08:20:37)
([GCC 10.3.0] on linux).
Here's the code:
import time
import multiprocessing
import threading
def myfnc():
print("myfnc")
def run(result_queue, callback):
result = callback()
result_queue.put(result)
def start(fnc):
def background_thread():
queue = multiprocessing.Queue()
proc = multiprocessing.Process(target = run, args = (queue,
fnc))
proc.start()
print("join?")
proc.join()
print("joined.")
result = queue.get()
threading.Thread(target = background_thread).start()
start(myfnc)
start(myfnc)
start(myfnc)
start(myfnc)
while True:
time.sleep(1)
What you'll see is that "join?" and "joined." nondeterministically does
*not* appear in pairs. For example:
join?
join?
myfnc
myfnc
join?
join?
joined.
joined.
What's worse is that when this happens and I Ctrl-C out of Python, the
started Thread is still running in the background:
$ ps ax | grep minimal
370167 pts/0 S 0:00 python3 minimal.py
370175 pts/2 S+ 0:00 grep minimal
Can someone figure out what is going on there?
Best,
Johannes
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
