Re: multiprocessing, pool, queue length

2016-03-21 Thread Ian Kelly
On Mon, Mar 21, 2016 at 1:46 PM, Michael Welle wrote: > Wait on the result means to set a multiprocessing.Event if one of the > consumers finds the sentinel task and wait for it on the producer? Hmm, > that might be better than incrementing a counter. But still, it couples > the consumers and the

Re: multiprocessing, pool, queue length

2016-03-21 Thread Ian Kelly
On Mon, Mar 21, 2016 at 4:25 AM, Michael Welle wrote: > Hello, > > I use a multiprocessing pool. My producer calls pool.map_async() > to fill the pool's job queue. It can do that quite fast, while the > consumer processes need much more time to empty the job queue. Since the > producer can create