Veloz wrote:
On Mar 2, 1:18 pm, Raymond Hettinger <pyt...@rcn.com> wrote:
On Mar 2, 8:29 am, Veloz <michaelve...@gmail.com> wrote:
Hi all
I'm looking for a queue that I can use with multiprocessing, which has
a peek method.
I've seen some discussion about queue.peek but don't see anything in
the docs about it.
Does python have a queue class with peek semantics?
Am curious about your use case? Why peek at something
that could be gone by the time you want to use it.
val = q.peek()
if something_i_want(val):
v2 = q.get() # this could be different than val
Wouldn't it be better to just get() the value and return if you don't
need it?
val = q.peek()
if not something_i_want(val):
q.put(val)
Raymond
Yeah, I hear you. Perhaps queue is not the best solution. My highest
level use case is this: The user visits a web page (my app is a
Pylons app) and requests a "report" be created. The report takes too
long to create and display on the spot, so the user expects to visit
some url "later" and see if the specific report has completed, and if
so, have it returned to them.
At a lower level, I'm thinking of using some process workers to create
these reports in the background; there'd be a request queue (into
which requests for reports would go, each with an ID) and a completion
queue, into which the workers would write an entry when a report was
created, along with an ID matching the original request.
The "peek" parts comes in when the user comes back later to see if
their report has done. That is, in my page controller logic, I'd like
to look through the complete queue and see if the specific report has
been finished (I could tell by matching up the ID of the original
request to the ID in the completed queue). If there was an item in the
queue matching the ID, it would be removed.
It's since occurred to me that perhaps a queue is not the best way to
handle the completions. (We're ignoring the file system as a solution
for the time being, and focusing on in-memory structures). I'm
wondering now if a simple array of completed items wouldn't be better.
Of course, all the access to the array would have to be thread/process-
proof. As you pointed out, for example, multi-part operations such as
"is such-and-such an ID in the list? If so, remove it and return in"
would have to be treated atomically to avoid concurrency issues.
Any thoughts on this design approach are welcomed :-)
A set of completed reports, or a dict with the ID as the key? The
advantage of a dict is that the value could contain several bits of
information, such as when it was completed, the status (OK or failed),
etc. You might want to wrap it in a class with locks (mutexes) to ensure
it's threadsafe.
--
http://mail.python.org/mailman/listinfo/python-list
