[issue32937] Multiprocessing worker functions not terminating with a large number of processes and a manager

2018-02-26 Thread EricG
EricG <ericg...@gmail.com> added the comment: Making some further observations, when I set processes = 12, for example, I can see 12 separate python processes + 4 additional processes also created which I assume are setup for the manager and, perhaps, other purposes. Now, what makes t

[issue32937] Multiprocessing worker functions not terminating with a large number of processes and a manager

2018-02-26 Thread EricG
EricG <ericg...@gmail.com> added the comment: If I do: from queue import Queue messages = Queue() No messages are printed. I believe this is expected as a regular Queue cannot be shared between processes. It was a problem that the manager was designed to solve. I am using a

[issue32937] Multiprocessing worker functions not terminating with a large number of processes and a manager

2018-02-24 Thread EricG
EricG <ericg...@gmail.com> added the comment: I do plan to consume the messages on the queue, but only after all worker functions are complete...after pool.join() returns. Is this not ok? I can certainly spawn a thread on the main process which will consume the queue entries and inser