Re: [Twisted-Python] which async framework?
> The protocols are all financial (do we really not have a pure-python FIX > library?!) but none are likely to have existing python implementations. > > How should I pick between the options? What would people recommend and why? > This might be a starting point for the FIX stuff https://pypi.python.org/pypi/fixlib/0.5 Tim Hughes mailto:thug...@thegoldfish.org ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] which async framework?
On Wed, 12 Mar 2014 09:53:07 +0100 Laurens Van Houtven <_...@lvh.io> wrote: > > A common criticism of Twisted is that it "takes over" your codebase. I am > speaking at PyCon in about a month to demonstrate that that isn't true. (It > just looks that way, because once people use it, they don't want to go > back... ;-)) Or sometimes they do. Regards Antoine. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] which async framework?
On 12/03/2014 08:53, Laurens Van Houtven wrote: Twisted has a thing called Cyclone, which I hear (but that's only hearsay) gives you Tornado's API on top of twisted, so you get all of the stuff below for free. From my side, I'm looking to experimentally build a network testing tool that will need to speak a fair few protocols, both classic tcp and multicast-based, and have a web api living on top of it that most likely will have a websocket for pumping data to the browser. It'll also need to write out JUnit-compatible xml results, but that's like the easiest bit ;-) I don't know which protocols you are interested in specifically; Aiming low (insert sarcastic look here), here's the first few: http://cdn.batstrading.com/resources/participant_resources/BATS_Europe_FIX_Specification.pdf http://cdn.batstrading.com/resources/participant_resources/BATS_Europe_Binary_Order_Entry_Specification.pdf http://cdn.batstrading.com/resources/participant_resources/BATS_Europe_PITCH_Specification.pdf http://cdn.batstrading.com/resources/participant_resources/BATS_Europe_MC_PITCH_Specification.pdf Twisted has a third party project called txsockjs which works excellently, both by itself and in the context of other (HTTP) resources. sockjs is a protocol which is basically "websockets, damnit", even when the consumer is a bad browser like IE6. Thankfully, I only have to support sane browsers... Twisted comes with a threadpool-backed WSGI server. (When I say threadpool-backed I mean that the WSGI requests are handled in threads; the IO itself of course comes from the Twisted reactor). Yeah, I remember that, and I remember liking it :-) I find twisted to be a great tool for writing protocol implementations. I have written tools for querying all sorts of gnarly proprietary protocols over all sorts of gnarly transports (packet radio; it's totally a thing), and more recently for doing crazy things like multiplexing stream transports over existing RPC protocols. (Like, you see a local port come up, and that actually creates a virtual stream connection over an existing RPC thing to some virtual server on the other end of the wire: https://www.youtube.com/watch?v=W_jEIvugwes). Cool! Twisted has had so many people write so many protocols in it that also the testing tools (MemoryReactor, StringTransport) are great. Especially if you are writing something very close to a wire protocol you will undoubtedly enjoy those amenities. There's also tons of composable things for receiving delimited lines, nestrings, etc. It's hard to tell what you will be looking for since I don't know details about your protocol, but having written more than a few protocol implementations I'm going to wager a guess and say Twisted has it or a third party thing for twisted has it. Yep, certainly sounds good. I guess I have concerns about discoverability and making sure I'm picking the right things to use rather than re-implementing things when I don't need to. What's the best way for me to find things I should be using? I guess my fallback position is to ask lots of clueless questions on this list and hope I don't annoy to many people, is that viable? Twisted, emphatically and without reservation, for all the above reasons. It's stable. All the stuff you need has been tried and tested extensively in production. It runs on PyPy, and usually a damn sight faster than on CPython, too. Didn't know about the PyPy thing, that will be interesting if I ever hit performance problems... A common criticism of Twisted is that it "takes over" your codebase. I am speaking at PyCon in about a month to demonstrate that that isn't true. (It just looks that way, because once people use it, they don't want to go back... ;-)) Gutted I can't make it to PyCon this year, look forward to watching the video! Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] which async framework?
> From my side, I'm looking to experimentally build a network testing tool that > will need to speak a fair few protocols, both classic tcp and multicast-based, Tornado is a _Web_ framework. A requirement for UDP, and multicast UDP, narrows suitable choices to Twisted and asyncio. Twisted is around much longer, and also has a broader focus .. lots of protocols implemented on top of TCP/IP. Asyncio is opiniated towards a co-routine based style. Twisted probably more towards "classical" Deferred based style, though you can go co-routine style as well. > and have a web api living on top of it that most likely will have a websocket > for pumping data to the browser. It'll also need to write out JUnit-compatible If you need WebSocket anyway, you can directly compare Twisted and asyncio programming with these examples: https://github.com/tavendo/AutobahnPython/tree/master/examples/twisted/websocket/echo https://github.com/tavendo/AutobahnPython/tree/master/examples/asyncio/websocket/echo Disclaimer: I am affiliated with that stuff. > I'd like to be able to serve the rest of the web api using a pyramid wsgi app > if Twisted can act as a WSGI host, and that can be combined with WebSocket also (in one server, and both services running on one port). /Tobias ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] which async framework?
Hi Chris, It will probably not surprise you that I like Twisted :-) On Wed, Mar 12, 2014 at 8:28 AM, Chris Withers wrote: > So, I see python now has a plethora of async frameworks and I need to try > and pick one to use from: > > - asyncio/tulip > - tornado > - twisted > Twisted has a thing called Cyclone, which I hear (but that's only hearsay) gives you Tornado's API on top of twisted, so you get all of the stuff below for free. > From my side, I'm looking to experimentally build a network testing tool > that will need to speak a fair few protocols, both classic tcp and > multicast-based, and have a web api living on top of it that most likely > will have a websocket for pumping data to the browser. It'll also need to > write out JUnit-compatible xml results, but that's like the easiest bit ;-) > I don't know which protocols you are interested in specifically; later on in the e-mail you mention that it's *probably* not already extant. Twisted comes with a huge number of protocols already implemented. It has TCP support (duh) as well as an interface for UDP multicast. Twisted has a third party project called txsockjs which works excellently, both by itself and in the context of other (HTTP) resources. sockjs is a protocol which is basically "websockets, damnit", even when the consumer is a bad browser like IE6. > I'd like to be able to serve the rest of the web api using a pyramid wsgi > app if possible, and I'd like to be able to write the things that process > requests in and validation out in a synchronous fashion, most likely > spinning off a thread for each one. > Twisted comes with a threadpool-backed WSGI server. (When I say threadpool-backed I mean that the WSGI requests are handled in threads; the IO itself of course comes from the Twisted reactor). > The protocols are all financial (do we really not have a pure-python FIX > library?!) but none are likely to have existing python implementations. > I find twisted to be a great tool for writing protocol implementations. I have written tools for querying all sorts of gnarly proprietary protocols over all sorts of gnarly transports (packet radio; it's totally a thing), and more recently for doing crazy things like multiplexing stream transports over existing RPC protocols. (Like, you see a local port come up, and that actually creates a virtual stream connection over an existing RPC thing to some virtual server on the other end of the wire: https://www.youtube.com/watch?v=W_jEIvugwes). Twisted has had so many people write so many protocols in it that also the testing tools (MemoryReactor, StringTransport) are great. Especially if you are writing something very close to a wire protocol you will undoubtedly enjoy those amenities. There's also tons of composable things for receiving delimited lines, nestrings, etc. It's hard to tell what you will be looking for since I don't know details about your protocol, but having written more than a few protocol implementations I'm going to wager a guess and say Twisted has it or a third party thing for twisted has it. Even though I mostly write Clojure now, I still write my networking stuff in Twisted. Macros are cool. Eleven years worth of battle-tested code is better. That said, tulip is nice in that you can also write protocol implementations that will look similar :-) > How should I pick between the options? What would people recommend and why? > Twisted, emphatically and without reservation, for all the above reasons. It's stable. All the stuff you need has been tried and tested extensively in production. It runs on PyPy, and usually a damn sight faster than on CPython, too. A common criticism of Twisted is that it "takes over" your codebase. I am speaking at PyCon in about a month to demonstrate that that isn't true. (It just looks that way, because once people use it, they don't want to go back... ;-)) hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] which async framework?
Hi All, Apologies for the cross-post, but this is a "which framework" question so seemed the most constructive way. Not interested in religious debates, just trying to pick the best tool for the job and didn't get much of a useful response from python-list... So, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted From my side, I'm looking to experimentally build a network testing tool that will need to speak a fair few protocols, both classic tcp and multicast-based, and have a web api living on top of it that most likely will have a websocket for pumping data to the browser. It'll also need to write out JUnit-compatible xml results, but that's like the easiest bit ;-) I'd like to be able to serve the rest of the web api using a pyramid wsgi app if possible, and I'd like to be able to write the things that process requests in and validation out in a synchronous fashion, most likely spinning off a thread for each one. The protocols are all financial (do we really not have a pure-python FIX library?!) but none are likely to have existing python implementations. How should I pick between the options? What would people recommend and why? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
