On 07Oct2022 20:16, Robin van der veer <robinvdv...@gmail.com> wrote:
If I have two processes communicating through a JoinableQueue, and I do the
following:
process 1:
queue.put(1) #unfished tasks = 1
queue.join() #block until unfished tasks = 0
print('hello')[/python]
process 2:
queue.get()
queue.task_done() #unfished tasks = 0
queue.put(1) #unfinished tasks 1[/python]
the unfished tasks refers to what is written in the documentation (
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.JoinableQueue.join
)
will 'hello' always be printed? Or is there a chance that the put in
process 2 executes before process 1 noticed that it should unblock?
I had to read this closely. Yes, the second `put(1)` could execute
before the `join()` commences (or tests), and the `hello` would be
blocked still.
It seems that the whole point of join() is that 'hello' should always be
printed, but I just want to make sure that I understand it correctly.
That's the purpose of using `join`, but you need to use it correctly.
The "some tasks are not completed" condition which `join` supports
doesn't fit what you're doing.
So yes, you're correct in your concern.
Maybe 2 queues would suit you better? Maybe not if they are common.
Maybe some kind of blocking counter, so that you could track task
counts? You'd need to make one, but something which allowed:
count = queue.put(1)
queue.wait_for(count)
print('hello')
and at the other end:
queue.get()
queue.task_done() # bumps the counter
count2 = queue.put(1)
Here, the counter would not be the internal "unfinished tasks" counter
but instead a distinct counter which always went up. Probably a pair:
tasks submitted by `put` and tasks completed by `task_done`. You could
subclass `JoinableQueue` and add implementations of these counters and
add a `wait_for(count)` method.
This amounts to assigning each "task" a unique id (the counter) and a
means to wait for that id.
Cheers,
Cameron Simpson <c...@cskk.id.au>
--
https://mail.python.org/mailman/listinfo/python-list
