On Sat, Aug 01, 2015 at 02:54:16PM +0000, paul_kon...@dell.com wrote: > > > On Jul 31, 2015, at 4:14 PM, Trent Nelson <tr...@snakebite.org> > > wrote: > > > > Hi folks, > > > > Bit off-topic, but just wanted to let people know about an > > experimental proof-of-concept fork of Python 3 I've been working on > > for the past couple of years called PyParallel: > > http://pyparallel.org. It essentially gets around the GIL > > limitations and allows Python code to run simultaneously in multiple > > threads from within a single interpreter/process. > > > > It's Windows only -- so I figured it might be of interest to this > > list. > > Impressive. Is the Windows only aspect because you used Windows > proprietary thread APIs? Or because there is something special about > Windows thread mechanisms that doesn’t carry over to, say, POSIX > pthread? I assume the former, in other words it should be adaptable > to standard thread architectures with some effort.
It's actually closer to the latter. I don't create a single thread with PyParallel -- I use the threadpool facilities exclusively. POSIX doesn't have anything like this -- the pthread abstraction is focused around the thread (the "worker"), not the work. (OS X has GCD, which comes close.) You *could* port PyParallel to POSIX, but you're either going to have to implement all the stuff Windows provides for free, which is a pretty big engineering effort. Even then, you're constrained by side-effects of the readiness-oriented I/O model (versus the completion-oriented model of Windows), so it becomes a lot harder to saturate all your I/O channels *and* peg all of your cores at 100% under load. (A typical multithreaded architecture will have a single thread dedicated to accept(), and then simply round-robin dispatch new FDs to a manual thread pool. A thread picks up the FD and adds it to its epoll/kqueue set -- that FD is now bound to that thread, which can quickly result in lop-sided resource allocation. The Windows I/O model revolves around the I/O request packet, which allows the "thread agnostic" separation between the I/O event (a write or read has completed) and the underlying thread used to handle the callback.) > paul Trent. _______________________________________________ python-win32 mailing list python-win32@python.org https://mail.python.org/mailman/listinfo/python-win32