After seeing David Mertz's talk at PyCon 2012, "Coroutines, event loops, and the history of Python generators" [1], I got thinking again about Python's expressive power for asynchronous programming. Generators, particularly with the addition of 'yield from' and 'return' in PEP 380 [2], allow us to write code that is executed "bit by bit" but still reads naturally. There are a number of frameworks that take advantage of this ability, but each is a little different -- enough so that there's negligible code re-use between these frameworks. I think that's a shame.
I proposed a design PEP a while back [3] with the intent of defining a standard way of writing asynchronous code, with the goal of allowing code re-use and bringing users of the frameworks closer together. Ideally, we could have libraries to implement network protocols, database wrappers, subprocess execution, and so on, that would work in any of the available asynchronous frameworks. My proposal met with near-silence, and I didn't pursue it. Instead, I did what any self-respecting hacker would do - I wrote up a framework, uthreads [4], that implemented my idea. This was initially a simple trampoline scheduler, but I eventually refactored it to run atop Twisted, since that's what I use. To my knowledge, it's never been used. I'm considering re-drafting the PEP with the following changes: * De-emphasize the thread emulation aspects, and focus on code-portability issues: * callbacks vs. "blocking" calls (e.g., when accepting incoming connections on a socket, how is my code invoked?) * consistent access to primitives, regardless of framework (e.g., where's the function I call to branch execution?) * nested asynchronous methods * Account for PEP 380 (by making the StopIteration workarounds match PEP 380, and explicitly deprecating them after Python 3.3) * Look forward to a world with software transactional memory [5] by matching that API where appropriate As I get to work on the PEP, I'd like to hear any initial reactions to the idea. Dustin [1] https://us.pycon.org/2012/schedule/presentation/104/ [2] http://www.python.org/dev/peps/pep-0380 [3] http://code.google.com/p/uthreads/source/browse/trunk/microthreading-pep.txt [4] http://code.google.com/p/uthreads/ [5] https://bitbucket.org/pypy/pypy/raw/stm-thread/pypy/doc/stm.rst -- http://mail.python.org/mailman/listinfo/python-list
