I'd like to propose that the Python community standardize on a
"deferred" object for asynchronous return values, modeled after the
well-thought-out Twisted Deferred class.
With more and more Python libraries implementing asynchronicity (for
example Futures -- PEP 3148), it's crucial to have a standard deferred
object in place so that code using a single asynchronous reactor can
interoperate with different asynchronous libraries.
I think a lot of people don't realize how much cooler and more elegant
it is to return a deferred object from an asynchronous function rather
than using a generic callback approach (where you pass a function
argument to the asynchronous function telling it where to call when the
asynchronous operation completes).
While asynchronous systems have been shown to have excellent scalability
properties, the callback-based programming style often used in
asynchronous programming has been criticized for breaking up the
sequential readability of program logic.
This problem is elegantly addressed by using Deferred Generators. Since
Python 2.5 added enhanced generators (i.e. the capability for "yield" to
return a value), the infrastructure is now in place to allow an
asynchronous function to be written in a sequential style, without the
use of explicit callbacks.
See the following blog article for a nice write-up on the capability:
http://blog.mekk.waw.pl/archives/14-Twisted-inlineCallbacks-and-deferredGenerator.html
Mekk's Twisted Deferred example:
@defer.inlineCallbacks
def someFunction():
a = 1
b = yield deferredReturningFunction(a)
c = yield anotherDeferredReturningFunction(a, b)
defer.returnValue(c)
What's cool about this is that between the two yield statements, the
Twisted reactor is in control meaning that other pending asynchronous
tasks can be attended to or the thread's remaining time slice can be
yielded to the kernel, yet this is all accomplished without the use of
multi-threading. Another interesting aspect of this approach is that
since it leverages on Python's enhanced generators, an exception thrown
inside either of the deferred-returning functions will be propagated
through to someFunction() where it can be handled with try/except.
Think about what this means -- this sort of emulates the "stackless"
design pattern you would expect in Erlang or Stackless Python without
leaving standard Python. And it's made possible under the hood by
Python Enhanced Generators.
Needless to say, it would be great to see this coolness be part of the
standard Python library, instead of having every Python asynchronous
library implement its own ad-hoc callback system.
James Yonan
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
