On 01/25/10 03:50 PM, [email protected] wrote:
On Mon, Jan 25, 2010 at 03:34:00PM -0600, Shawn Walker wrote:
As for the generator bit, if the wrapper function does something like:
try:
return self.__generator_func()
except ...:
...
finally:
....
...it will be fine. There's no real difference between returning a
generator (as shown above) or calling the generator directly.
I might still be missing something here though.
Is this going to work properly with locking, though?
If __generator_func() needs to hold a lock as long as it is returning
results, won't releasing the lock in the finally statement cause the
lock to get dropped after the generator is returned to the caller?
That's not what's desired, I think, since we want the lock held while
the generator is running.
Yes, you're right, sorry. For the generator case, if you want to wrap
it, you have to iterate over its results to retain the try/finally
behaviour needed for locking. This is also true for a @contextmanager.
This also raises another interesting question about holding locks in
these functions: if the caller never runs the generator to completion,
at what point does python destroy the generator object? What happens to
the locks that are held? Does this mean that we need to drop and
re-acquire any locks around a yield statment? (I'm guessing the
answer is yes.)
Based on the python docs
(http://docs.python.org/reference/expressions.html), I'd say yes.
Execution for a generator can be paused indefinitely (technically until
the generator is garbage collected). This means that holding locks for
a generator function is problematic at best.
If the answer is yes, we don't really want to drop and re-acquire the
activity lock, and disable/enable cancel every time this function
yields.
I think this means we really want to do something like:
def remote_search_wrapper():
activity_lock.acquire()
enable_cancel()
try:
r = remote_search()
while r:
yield r
finally:
disable_cancel()
activity_lock.release()
However, I don't know whether this will cause all of remote_search()'s
results to be held in memory, or if this will just return True as long
as the function still has results to return. Is this documented
anywhere?
No, this will not cause all results to be held in memory (simple tests
of generators of generators with print statements show this). However,
after reading section 5.2.8 of python's expressions doc, I think we're
sort of stuck here (see above).
...
1821
This doesn't re-raise the exception once it's caught. This ought to be
safe.
The thought was that the origin may be invalid, but I suppose we
should assume that user input has been validated first.
Ah, so you mean that you're expecting another exception besides the
UnknownPublisher that's already being caught?
Yes; the code was originally written here before locking was in place.
It could be an invalid origin URI (syntactically) since remote_search
accepts the '-s' option.
--
Shawn Walker
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
