which async framework? - a summary

2014-03-22 Thread Chris Withers
On 14/03/2014 00:36, Tim Chase wrote: On 2014-03-14 00:25, Chris Withers wrote: I've been pleasantly surprised by the succinct, well reasoned and respectful replies from each of the communities! As one who doesn't lurk on the other lists, is there a nice executive summary of their responses?

Re: which async framework?

2014-03-13 Thread Tim Chase
On 2014-03-14 00:25, Chris Withers wrote: > I've been pleasantly surprised by the succinct, well reasoned and > respectful replies from each of the communities! As one who doesn't lurk on the other lists, is there a nice executive summary of their responses? -tkc -- https://mail.python.org/m

Re: which async framework?

2014-03-13 Thread Chris Withers
On 11/03/2014 19:41, Terry Reedy wrote: I suspect I'll just end up cross-posting to the various mailing lists, Bad idea. Post separately if you must. > which I hope won't cause too much offence or kick off any flame wars. It would do both. Ye of little faith :-P I've been pleasantly surp

Re: which async framework?

2014-03-12 Thread Neal Becker
Grant Edwards wrote: > On 2014-03-11, Antoine Pitrou wrote: >> Sturla Molden gmail.com> writes: >>> >>> Chris Withers simplistix.co.uk> wrote: >>> > Hi All, >>> > >>> > I see python now has a plethora of async frameworks and I need to try >>> > and pick one to use from: >>> > >>> > - asyncio

Re: which async framework?

2014-03-12 Thread Ian Kelly
On Tue, Mar 11, 2014 at 5:38 PM, Chris Angelico wrote: > On Wed, Mar 12, 2014 at 10:18 AM, Marko Rauhamaa wrote: >> Chris Angelico : >> >>> What corner cases are there with threads that you don't have with >>> anything else? >> >> There are numerous. Here's one example: deadlocks due to two threa

Re: which async framework?

2014-03-11 Thread Chris Angelico
On Wed, Mar 12, 2014 at 10:18 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> What corner cases are there with threads that you don't have with >> anything else? > > There are numerous. Here's one example: deadlocks due to two threads > taking locks in a different order. The problem crops up nat

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Chris Angelico : > Yep. Now how is that not a problem when you use some other model, like > an event loop? The standard methods of avoiding deadlocks (like > acquiring locks in strict order) work exactly the same for all models, > and are just as necessary. I was simply saying that if you can wor

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Chris Angelico : > What corner cases are there with threads that you don't have with > anything else? There are numerous. Here's one example: deadlocks due to two threads taking locks in a different order. The problem crops up naturally with two intercommunicating classes. It can sometimes be ver

Re: which async framework?

2014-03-11 Thread Chris Angelico
On Wed, Mar 12, 2014 at 9:28 AM, Marko Rauhamaa wrote: > If you can't write your own event loop, you probably can't be trusted > with any multithreaded code, which has much more baffling corner cases. I'm not sure about that. Threads are generally easier to handle, because each one just does some

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Antoine Pitrou : > This is the usual assumption that high-level libraries are made of > useless cruft piled up by careless programmers. But there are actual > reasons why these frameworks have a significant amount of code, and > people who decide to ignore those reasons are simply bound to > reimp

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Gregory Ewing : > It's not "epoll function calls" that the coroutine style is intended > to replace, it's complex systems of chained callbacks. They're > supposed to make that kind of logic *easier* to follow. If you haven't > had that experience, it may be because you've only dealt with simple >

Re: which async framework?

2014-03-11 Thread Gregory Ewing
Sturla Molden wrote: Another thing is that co-routines and "yield from" statements just makes it hard to follow the logic of the program. I still have to convince myself that a library for transforming epoll function calls into co-routines is actually useful. It's not "epoll function calls" tha

Re: which async framework?

2014-03-11 Thread Sturla Molden
Antoine Pitrou wrote: > This is the usual assumption that high-level libraries are made of useless > cruft piled up by careless programmers. It often is the case, particularly in network programming. But in this case the programmer is Guido, so it doesn't apply. :) > What irks me with your r

Re: which async framework?

2014-03-11 Thread Terry Reedy
On 3/11/2014 3:53 AM, Chris Withers wrote: On 10/03/2014 21:57, Terry Reedy wrote: I'd like to be able to serve the rest of the web api using a pyramid wsgi app if possible, and I'd like to be able to write the things that process requests in and validation out in a synchronous fashion, most lik

Re: which async framework?

2014-03-11 Thread Chris Angelico
On Wed, Mar 12, 2014 at 5:43 AM, Antoine Pitrou wrote: > This is the usual assumption that high-level libraries are made of useless > cruft piled up by careless programmers. But there are actual reasons > why these frameworks have a significant amount of code, and people who > decide to ignore tho

Re: which async framework?

2014-03-11 Thread Antoine Pitrou
Sturla Molden gmail.com> writes: > > Antoine Pitrou pitrou.net> wrote: > > > Yes, why use a library when you can rewrite it all yourself? > > This assumes something equivalent to the library will have to be written. > But if it can be replaced with something very minimalistic it is just > bloa

Re: which async framework?

2014-03-11 Thread Sturla Molden
Antoine Pitrou wrote: > Yes, why use a library when you can rewrite it all yourself? This assumes something equivalent to the library will have to be written. But if it can be replaced with something very minimalistic it is just bloat. I would also like to respond that the select module and pywi

Re: which async framework?

2014-03-11 Thread Chris Angelico
On Wed, Mar 12, 2014 at 3:01 AM, Antoine Pitrou wrote: > Yes, why use a library when you can rewrite it all yourself? > Actually, you should probably issue system calls to the kernel directly, > the libc is overrated (as is portability, I suppose). It's a trade-off, of course. I am fluent in over

Re: which async framework?

2014-03-11 Thread jkn
Hi Grant On Tuesday, 11 March 2014 16:52:18 UTC, Grant Edwards wrote: [...] > > And don't bother with device drivers for the network adapters either. > Just map their PCI regions in to user-space and twiddle the reigisters > directly! ;) > > [I do that when testing PCI boards with C code, and

Re: which async framework?

2014-03-11 Thread Grant Edwards
On 2014-03-11, Antoine Pitrou wrote: > Sturla Molden gmail.com> writes: >> >> Chris Withers simplistix.co.uk> wrote: >> > Hi All, >> > >> > I see python now has a plethora of async frameworks and I need to try >> > and pick one to use from: >> > >> > - asyncio/tulip >> > - tornado >> > - twi

Re: which async framework?

2014-03-11 Thread Giampaolo Rodola'
On Tue, Mar 11, 2014 at 11:17 AM, Sturla Molden wrote: > Chris Withers wrote: > > Hi All, > > > > I see python now has a plethora of async frameworks and I need to try > > and pick one to use from: > > > > - asyncio/tulip > > - tornado > > - twisted > > Looking at Tornado's examples on the web I

Re: which async framework?

2014-03-11 Thread Antoine Pitrou
Sturla Molden gmail.com> writes: > > Chris Withers simplistix.co.uk> wrote: > > Hi All, > > > > I see python now has a plethora of async frameworks and I need to try > > and pick one to use from: > > > > - asyncio/tulip > > - tornado > > - twisted > > I'd go for using iocp, epoll and kqueue/

Re: which async framework?

2014-03-11 Thread Antoine Pitrou
Chris Withers simplistix.co.uk> writes: > > The protocols are all financial (do we really not have a pure-python FIX > library?!) but none are likely to have existing python implementations. If you are mostly writing protocol implementations (aka parsers and serializers), then you should consid

Re: which async framework?

2014-03-11 Thread Oscar Benjamin
On 11 March 2014 11:54, Marko Rauhamaa wrote: > Ian Kelly : > >> eventlet has 115k downloads from PyPI over the last month. gevent has >> 143k. Twisted has 147k. Tornado has 173k. >> >> I'd say that a lot of Python users are already doing non-blocking >> network I/O, in one form or another. > > Th

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Ian Kelly : > eventlet has 115k downloads from PyPI over the last month. gevent has > 143k. Twisted has 147k. Tornado has 173k. > > I'd say that a lot of Python users are already doing non-blocking > network I/O, in one form or another. There aren't so many network developers in the world. That m

Re: which async framework?

2014-03-11 Thread Ian Kelly
On Tue, Mar 11, 2014 at 4:58 AM, Marko Rauhamaa wrote: > Sturla Molden : > >> I'd go for using iocp, epoll and kqueue/kevent directly. Why bother to >> learn a framework? You will find epoll and kqueue/kevent in the select >> module and iocp in pywin32. > > You beat me to it. > > However, I'm hopi

Re: which async framework?

2014-03-11 Thread Sturla Molden
Marko Rauhamaa wrote: > Now, I've taken a brief look at the new asyncio and it looks as if it > has everything one would hope for (and then some). You'd still need to > supply the protocol implementations yourself. Tulip (the new async module) is nice. But I am a bit confused as to how it combin

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Sturla Molden : > I'd go for using iocp, epoll and kqueue/kevent directly. Why bother to > learn a framework? You will find epoll and kqueue/kevent in the select > module and iocp in pywin32. You beat me to it. However, I'm hoping asyncio will steer the Python faithful away from blocking threads

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Sturla Molden : > Looking at Tornado's examples on the web I find this: > > [...] > > (1) This was written by some Java guys. I have written several Python async "frameworks" starting from select.epoll(). It's only a handful of lines of code (plus an AVL tree implementation for timers). Then, I'

Re: which async framework?

2014-03-11 Thread Sturla Molden
Chris Withers wrote: > Hi All, > > I see python now has a plethora of async frameworks and I need to try > and pick one to use from: > > - asyncio/tulip > - tornado > - twisted I'd go for using iocp, epoll and kqueue/kevent directly. Why bother to learn a framework? You will find epoll and kqu

Re: which async framework?

2014-03-11 Thread Sturla Molden
Chris Withers wrote: > Hi All, > > I see python now has a plethora of async frameworks and I need to try > and pick one to use from: > > - asyncio/tulip > - tornado > - twisted Looking at Tornado's examples on the web I find this: tornado.ioloop.IOLoop.instance().start() This single line

Re: which async framework?

2014-03-11 Thread Chris Withers
On 10/03/2014 21:57, Terry Reedy wrote: I'd like to be able to serve the rest of the web api using a pyramid wsgi app if possible, and I'd like to be able to write the things that process requests in and validation out in a synchronous fashion, most likely spinning off a thread for each one. If

Re: which async framework?

2014-03-10 Thread Terry Reedy
On 3/10/2014 4:38 PM, Chris Withers wrote: Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted From my side, I'm looking to experimentally build a network testing tool that will need to speak a fair few ne

which async framework?

2014-03-10 Thread Chris Withers
Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted From my side, I'm looking to experimentally build a network testing tool that will need to speak a fair few network protocols, both classic tcp and mul