Re: [Twisted-Python] Twisted WebSockets Branch
Hi Rob, if your main concern is robust WebSocket on Twisted, why not use Autobahn (https://pypi.python.org/pypi/autobahn) ? Autobahn supports wrapping arbitrary stream-based protocols on top of WebSocket, with full support for advanced features like WebSocket compression and auto-fragmentation plus Twisted endpoints: https://github.com/tavendo/AutobahnPython/tree/master/examples/twisted/websocket/wrapping Note: Wrapping arbitrary stream-based protocols in WebSocket requires proper WebSocket subprotocol negotiation and handling. Autobahn follows the scheme established by https://github.com/kanaka/websockify for which you find e.g. JS libraries running in a browser. Autobahn also supports running WebSocket as a resource in a Twisted Web resource tree: https://github.com/tavendo/AutobahnPython/blob/master/examples/twisted/websocket/echo_site/server.py Disclosure: I wrote this stuff. So I am biased;) Glyph at one point mentioned that he likes to see "basic WebSocket support" built into Twisted. Bringing the branch to the full level of Autobahn rgd. features, compliance, robustness etc will be a lot of work though. Merging Autobahn into Twisted is unlikely to happen / doesn't make sense for different reasons I guess. /Tobias Von: twisted-python-boun...@twistedmatrix.com [mailto:twisted-python-boun...@twistedmatrix.com] Im Auftrag von Rob Oakes Gesendet: Freitag, 22. August 2014 04:24 An: twisted-python@twistedmatrix.com Cc: roa...@gurulabs.com Betreff: [Twisted-Python] Twisted WebSockets Branch Dear Twisted Developers, As a quick introduction, my name is Rob Oakes. I'm the lead developer for a company called Guru Labs. I'm writing for two reasons: # Reason 1: Thank You First and foremost, I'm writing to express thanks for creating an excellent framework. We use Twisted extensively in our infrastructure, and it is typically our go-to tool for any sort of custom server. The other two reasons have to do with some questions about the development of Twisted. Before I dive into those, however, let me provide some background. For the better part of a year, we've been using some code out of the websockets (twisted.web.websockets.WebsocketsResource) branch to wrap a trio of custom protocols we use in one of our web based products. I know that the code is still pre-release, but we've generally found it to be stable and work very well. # Reason Two: Websockets Development This brings me to the second reason I'm writing. Over the past year, it seems as though the development on the Twisted websockets branch has stalled. We would like to unstall it. For this reason, Guru Labs would be interested in: 1.) Contracting with one of the core Twisted devs of Twisted who might be interested in finishing the implementation of the websockets wrappers (resolving the issues described in ticket 4173) or another party. We are happy to pay hourly rates, set a bounty, make a donation to the Twisted project ... whatever. A stable implementation of websockets available in Twisted Web is a priority to us, and if it's a matter of money, we are happy to throw money where it might be needed. (If this of interest, please contact me off-list at roa...@gurulabs.com<mailto:roa...@gurulabs.com>.) or 2.) Assigning a Guru Labs developer (probably me) who might complete the work. I've been studying the issues which are still open (specifically 4173), and the various branches associated with websocket development (websocket-4173-3 and websocket-4173-4) and it seems that most of the major concerns are related to the manner in which websocket connections are closed. Assuming that someone else doesn't step forward ... I've already merged the most recent version of trunk with these branches, but I've found myself with several questions on how to best continue with the work. a. Which of the two websocket branches should be used as the basis for new work? On GitHub, websocket-4173-4 is marked as closed with a note telling the contributor to see the contribution guidelines. There are also several additions to the code which do not follow Twisted conventions (as I understand them). The last set of commits also seem to come from approximately the same time. websocket-4173-4 includes code which attempts to resolve issues noted in 4173 that is not present in 4173-3, but there aren't really any comments to determine if this should be incorporated or not. At this point, I'm really not familiar enough with the code to draw my own conclusion. b. In general, the protocol wrapper seems to work quite well. However when merged with the most recent version of trunk (post twisted 14), I've been seeing frequent unhandled deferred errors. The most common is: "twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost" It happens when: * Connecti
Re: [Twisted-Python] Twisted WebSockets Branch
On 2014-08-21 22:24, Rob Oakes wrote: > a. Which of the two websocket branches should be used as the basis for new > work? > > On GitHub, websocket-4173-4 is marked as closed with a note telling the > contributor to see the contribution guidelines. There are also several > additions to the code which do not follow Twisted conventions (as I > understand them). The last set of commits also seem to come from > approximately the same time. > > websocket-4173-4 includes code which attempts to resolve issues noted in 4173 > that is not present in 4173-3, but there aren't really any comments to > determine if this should be incorporated or not. At this point, I'm really > not familiar enough with the code to draw my own conclusion. I'm not sure what you mean by "the branch is marked as closed" - AFAIK GitHub has no concept of closing branches. Plus GitHub is just a mirror. If -4 is the latest revision in Subversion then that's the latest version of the code. -Itamar ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Twisted WebSockets Branch
Dear Twisted Developers, As a quick introduction, my name is Rob Oakes. I'm the lead developer for a company called Guru Labs. I'm writing for two reasons: # Reason 1: Thank You First and foremost, I'm writing to express thanks for creating an excellent framework. We use Twisted extensively in our infrastructure, and it is typically our go-to tool for any sort of custom server. The other two reasons have to do with some questions about the development of Twisted. Before I dive into those, however, let me provide some background. For the better part of a year, we've been using some code out of the websockets (twisted.web.websockets.WebsocketsResource) branch to wrap a trio of custom protocols we use in one of our web based products. I know that the code is still pre-release, but we've generally found it to be stable and work very well. # Reason Two: Websockets Development This brings me to the second reason I'm writing. Over the past year, it seems as though the development on the Twisted websockets branch has stalled. We would like to unstall it. For this reason, Guru Labs would be interested in: 1.) Contracting with one of the core Twisted devs of Twisted who might be interested in finishing the implementation of the websockets wrappers (resolving the issues described in ticket 4173) or another party. We are happy to pay hourly rates, set a bounty, make a donation to the Twisted project ... whatever. A stable implementation of websockets available in Twisted Web is a priority to us, and if it's a matter of money, we are happy to throw money where it might be needed. (If this of interest, please contact me off-list at roa...@gurulabs.com.) or 2.) Assigning a Guru Labs developer (probably me) who might complete the work. I've been studying the issues which are still open (specifically 4173), and the various branches associated with websocket development (websocket-4173-3 and websocket-4173-4) and it seems that most of the major concerns are related to the manner in which websocket connections are closed. Assuming that someone else doesn't step forward ... I've already merged the most recent version of trunk with these branches, but I've found myself with several questions on how to best continue with the work. a. Which of the two websocket branches should be used as the basis for new work? On GitHub, websocket-4173-4 is marked as closed with a note telling the contributor to see the contribution guidelines. There are also several additions to the code which do not follow Twisted conventions (as I understand them). The last set of commits also seem to come from approximately the same time. websocket-4173-4 includes code which attempts to resolve issues noted in 4173 that is not present in 4173-3, but there aren't really any comments to determine if this should be incorporated or not. At this point, I'm really not familiar enough with the code to draw my own conclusion. b. In general, the protocol wrapper seems to work quite well. However when merged with the most recent version of trunk (post twisted 14), I've been seeing frequent unhandled deferred errors. The most common is: "twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost" It happens when: * Connections are closed from the server, using transport.loseConnection * Connections do not transmit a "close" frame prior to disconnecting * To reiterate, this issue only started appearing after merging the websocket code with Twisted 14. The previous version of Twisted we were using (Twisted 12.3 worked flawlessly.) No error is raised if the client correctly closes the connection or when using non-browser based clients (like the Python ws4py websocket client). We mostly see the exception when the objects are garbage collected (based upon the deferred documentation at http://twistedmatrix.com/documents/14.0.0/core/howto/defer.html). Put another way, we see a whole string of errors upon stopping the reactor. Despite the exception, we don't see any errors in the browser client. Also somewhat frustratingly, the traceback isn't terribly helpful. This is a fairly routine example: ```error Unhandled error in Deferred: Traceback (most recent call last): Failure: twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost. ``` Technically, the error is probably appropriate, as they appear when the connection is lost in a non-clean fashion. I am a little concerned, though, in that I haven't found a good way to catch or suppress the error. Moreover, it doesn't seem like the defferreds are getting garbage collected which seems like a memory leak waiting to happen. Can someone clarify if this is: * intended behavior, and if so, what might be a strategy I can use for managing the error in my wrapped protocols * which part of the websocket code I should
