I would like to get feedback on an idea I had for simplifying the use
of queues with daemon consumer threads
Sometimes, I launch one or more consumer threads that wait for a task
to enter a queue and then work on the task. A recurring problem is that
I sometimes need to know if all of the tasks ha
Raymond Hettinger:
>There are some competing approaches. One is to attach sentinel objects
>to the end of the line as a way for consumer threads to know that they
>should shut down. Then a regular t.join() can be used to block until
>the consumers threads have shut-down. This approach is
>straig
Raymond Hettinger wrote:
> I would like to get feedback on an idea I had for simplifying the use
> of queues with daemon consumer threads
>
> Sometimes, I launch one or more consumer threads that wait for a task
> to enter a queue and then work on the task. A recurring problem is that
> I sometimes
Rene Pijlman wrote:
> >2) complicating the producer logic to append one sentinel for each consumer
> >when the data stream is done
>
> for i in range(self.numberOfThreads):
> self.workQueue.put(None)
Or, you could just put one sentinel in the Queue, and subclass the
Queue's _g
Carl Banks wrote:
> But yeah, something like an InterruptableQueue might be a nice thing to
> have.
Ok, I see now that InterruptableQueue wouldn't help the OP, though it
would have helped me in my situation, so it'd still be a good idea.
Carl Banks
--
http://mail.python.org/mailman/listinfo/pyt
Carl Banks:
>Rene Pijlman:
>> for i in range(self.numberOfThreads):
>> self.workQueue.put(None)
>
>Or, you could just put one sentinel in the Queue, and subclass the
>Queue's _get method not to take the sentinel out.
Ah yes, clever trick. But you'd have to worry about thread-sa
Raymond Hettinger wrote:
> I would like to get feedback on an idea I had for simplifying the use
> of queues with daemon consumer threads
>
> Sometimes, I launch one or more consumer threads that wait for a task
> to enter a queue and then work on the task. A recurring problem is that
> I sometime
Rene Pijlman wrote:
> Carl Banks:
> >Rene Pijlman:
> >> for i in range(self.numberOfThreads):
> >> self.workQueue.put(None)
> >
> >Or, you could just put one sentinel in the Queue, and subclass the
> >Queue's _get method not to take the sentinel out.
>
> Ah yes, clever trick. Bu
Carl Banks:
>Rene Pijlman:
>> Ah yes, clever trick. But you'd have to worry about thread-safety of your
>> subclass though.
>
>Queue worries about this for you. The Queue class only calls _get when
>it has the queue's mutex, so you can assume thread safety when
>subclassing it.
Ah yes, I overlook
