Re: [Twisted-Python] naming question before we end up committed...
+0 on “logger” (top-level names have a history of whimsy) +1 on making it top-level hth lvh signature.asc Description: Message signed with OpenPGP using GPGMail ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] GPRS/Modem protocol through USB support
Hi Bram Simon, I’ll answer both e-mails in one go :-) On 22 Sep 2014, at 16:57, Bram Van Steenlandt b...@diomedia.be wrote: I'm using a gsm modem for sending sms messages, the modem is rs232 so I have to use and ftdi adapter. You can use these with pyserial in a thread in twisted. Huh! Interesting; what drove that choice, given twisted’s serial support? op 22-09-14 16:01, Simon Andrieu schreef: Could you help me to know if GPRS/3G/Modem protocols through USB are supported with Twisted please? And how it is possible to use this interface? Depends what that thing exposes. Usually it’s a character device, so you end up whistling some bytes. However, if you are *just* interested in a very standard Internet connection setup, it is probably much easier to use the existing OS tools for connecting to the internet using that model, than to write something using Twisted. Twisted would be interesting if you want to do interesting things beyond that; e.g. talk to the device and ask it for all sorts of e.g. telemetry data, data that is not already exposed in some standard, easy to consume way. hth lvh signature.asc Description: Message signed with OpenPGP using GPGMail ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] GPRS/Modem protocol through USB support
On 22 Sep 2014, at 17:31, Bram Van Steenlandt b...@diomedia.be wrote: No specific reason actually, I didn't even know twisted had support for a serial port. This being said: ftdio_sio can be a bit buggy at times, esp when using multiple devices on the same pc, ftdi on freebsd osx isn't very stable either. I use this interface selector system where the serial connection can be setup with pyserial,libftdi (which works much better for ftdi adapters) or a tcp socket (which uses a so called device server with ethernet and rs232 onboard). Doing it with twisted seems more complicated to me (but I could be wrong), in my case it makes a lot of sense to wait for the data, I poll plc's over rs485 and those kind of things. Use cases vary :-) In a wait-dosomething-wait scenario, I find @inlineCallbacks to typically yield virtually the same code you ended up with. When I was doing this, there was some pretty complex chatter back and forth, and I was multiplexing several connections over a single RS232 device, so Twisted certainly came in handy. In your case: don’t fix what ain’t broken, of course :) Also as I've said, there are sometimes a lot of hardware issues (usb bus resets and stuff, users that disconnect usb-ports and plug them back in) which can be difficult to deal with. How long does twisted has this serial port support actually ? Well over a decade: Mon Dec 16 04:04:25 2002 to be precise. See svn rev trunk@4528. hth lvh signature.asc Description: Message signed with OpenPGP using GPGMail ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] inlineCallbacks and current exceptions: leaky abstraction or bug?
Honored twistedeers, Consider the following (blocking) decorator, which runs a function in a transaction: def _with_transaction(f): def decorated(self, *args, **kwargs): conn = self.engine.connect() txn = conn.begin() try: result = f(self, conn, *args, **kwargs) except: txn.rollback() raise else: txn.commit() return return decorated Where I to translate this logic verbatim to @inlineCallbacks, I get: def _with_transaction(f): @inlineCallbacks def decorated(self, *args, **kwargs): conn = yield self.engine.connect() txn = yield conn.begin() try: result = yield f(self, conn, *args, **kwargs) except: yield txn.rollback() raise else: yield txn.commit() returnValue(result) return decorated However, there’s a bug here! In the except clause: there’s an (implicit) current exception, to be re-raised by the bare raise statement. Unfortunately, when doing yield txn.rollback(), that conveniently eats said exception. Of course, there’s a fairly simple workaround involving catching BaseException and capturing the exception instance explicitly. I’m wondering if this is just a leaky abstraction, or if I should report it as a bug in @inlineCallbacks? cheers lvh signature.asc Description: Message signed with OpenPGP using GPGMail ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] [ANN] thimble 0.1.0
On Mon, May 19, 2014 at 12:19 AM, Glyph gl...@twistedmatrix.com wrote: I think this is the sort of thing that should move into Twisted itself, eventually. Thoughts? I agree! There's a few things I'd like to work out with the freedom to break things of a v0.x.x semver version before that happens though. Notably, this thing assumes that all attribute accesses are thread-safe and synchronous. That's often true, but hey, this is Python, so you get to do whatever you want when you access an attribute (and @property even makes it easy to do so). Also because right now I'm suggesting a pool with a singled thread for synchronized access to an object that isn't thread safe; especially if you *know* the object isn't thread safe you probably want something to take care of attribute access too :-) hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] [ANN] thimble 0.1.0
Hi everyone! thimble is a library that makes it easy to wrap blocking objects with a thread pool to give you an async API. In particular, it lets you selectively wrap blocking methods, and takes an explicit reactor and thread pool, making it at least slightly harder to do the obvious but potentially bad thing and tricky to test thing (global reactor state, using the reactor thread pool). (It's a thimble because it lets you play with threads without pricking yourself :-)) This is a small library: it doesn't really fix any big engineering problems, and that's okay. It just removes a bunch of boilerplate I've seen in many a Twisted software project, usually boilerplate that's implemented using the suboptimal methods described above. The README should run you through it real quick: https://pypi.python.org/pypi/thimble This is just an early release. I expect that the documentation isn't good enough to get best practices out the door, particularly in setting up thread pools. I think it's pretty decent and probably already worth a look, though :-) Thanks to Rackspace for letting me write it open source on company time :-) hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here
Hi! First of all, thank you so much for working on this. Secondly, provided we don't find any regressions in pre5, option 2 seems the least footgunny to me :) hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Serial interface for software program
Hi Anatoly, Do you know about manhole? I made a video that also demos axiom, but it should show you what manhole does :) https://www.youtube.com/watch?v=3-UZiO-AnLc Essentially it's repl in a running process. hth lvh On Mon, May 5, 2014 at 4:43 AM, anatoly techtonik techto...@gmail.comwrote: Hello, network hackers, Recently I've got a Raspberry Pi and a friend showed me its serial interface and helped to setup it. I was stunned. What I saw - you open serial terminal, and you jump in into the middle of boot section. Just open a terminal and you see what's going on inside of R.Pi. Any time. That's was an intro. =) For a long time I wanted the same interface for debugging software programs. When I was hacking on Spyder I found that my human brain is too limited to squeeze details of dynamic of interaction of objects at run-time. Spyder is written in Python with Qt, and it is a parallel application much like any Twisted app is. But I wanted to push the progress, I have to deal with complexity, so I badly needed to be able to draw a video of interaction at run-time. That was the problem I tried to solve. And for the first step I needed a way to connect to Spyder at run-time and see what's going on. I didn't know how to call that way before, but now I have a rather accurate wording - a serial interface for an application. That is the background for the question I am trying to ask. What is the best way to implement such interface considering the following properties: 1. it needs to be simple (so that you can quickly create python script that reads the info) 2. it needs to be universal (so that script for one app worked for other) 3. it should be asynchronous (connected terminal should not degrade performance) 4. it needs to be reliable (at least detecting missing packets) 5. and cross-platform (and pure python) Why Twisted? For the first I think that this problem is actual for people who are dealing with complexity of dynamic and interconnected systems built with the help of Twisted. For the second I believe it can only be solved or analyzed by people skilled in interprocess and internetwork communications. One button test: 1. program behaves weird 2. user hits the button 3. terminal pops up 4. shows a lot of data for a running program I am interested to know. 1. how the button should discover the program network port seems cool, but what about two parallel programs? what about different programs that provide this interface? 2. how terminal receives the data i see it as line based - one line - one event, everything is text 3. how to make it fast TCP clearly won't here 4. how to send the data what if two threads (or parallel code) produce serial data? who should run the server that serves connected terminals, and how other thread will send message to this server? you can not run two TCP servers on the same port on the app side, so is there an alternative? how to detect missing data in a stream? how to avoid dealing with incomplete packets and low level network details? Thanks. I'd like to get back to hacking on Spyder one day, but this is a stumbling block for me every time I think about it or any program of compared complexity. -- anatoly t. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] txtorcon v0.9.2
Thank you for all your work on this project :) On Thu, Apr 24, 2014 at 2:27 PM, meejah mee...@meejah.ca wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I am happy to announce that txtorcon v0.9.2 is now available. This release adds a few minor bug-fixes and a few API enhancements. Full details: * add on_disconnect callback for TorControlProtocol (no more monkey-patching Protocol API) * add age() method to Circuit * add time_created property to Circuit * don't incorrectly listen for NEWDESC events in TorState * add .flags dict to track flags in Circuit, Stream * build_circuit() can now take hex IDs (as well as Router instances) * add unique_name property to Router (returns the hex id, unless Named then return name) * add location property to Router * TorState.close_circuit now takes either a Circuit ID or Circuit instance * TorState.close_stream now takes either a Stream ID or Stream instance * support both GeoIP API versions * more test-coverage * small patch from enriquefynn improving tor binary locating * strip OK lines in TorControlProtocol ( https://github.com/meejah/txtorcon/issues/8) * use TERM not KILL when Tor launch times out ( https://github.com/meejah/txtorcon/pull/68) from hellais * Unit-test coverage now at 98% sha256 sums for the distribution files: 93e934f83e3fc6fcf40e76f7c9c28459af04205fb912d384aaacb7ac5269bb8f dist/txtorcon-0.9.2-py2-none-any.whl fe90743cdc453002ad046aa6556b611b4e85b813ff92865769d3d27712c2ca47 dist/txtorcon-0.9.2.tar.gz There are also signatures on github and txtorcon.readthedocs.org You may download from github or the hidden service: https://github.com/meejah/txtorcon/releases/tag/v0.9.2 https://github.com/meejah/txtorcon/releases/download/v0.9.2/txtorcon-0.9.2-py2-none-any.whl https://github.com/meejah/txtorcon/releases/download/v0.9.2/txtorcon-0.9.2-py2-none-any.whl.asc https://github.com/meejah/txtorcon/releases/download/v0.9.2/txtorcon-0.9.2.tar.gz https://github.com/meejah/txtorcon/releases/download/v0.9.2/txtorcon-0.9.2.tar.gz.asc http://timaq4ygg2iegci7.onion/txtorcon-0.9.2.tar.gz http://timaq4ygg2iegci7.onion/txtorcon-0.9.2.tar.gz.asc Source code: https://github.com/meejah/txtorcon/archive/v0.9.2.tar.gz Thanks, meejah -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iQEcBAEBAgAGBQJTWVf5AAoJEMJgKAMSgGmn07kIAKSwjBck76dyN1lWJj0fRl/f BevLpnp+rb+ge5hBAeKfsVTYciDBZeSo8/fE44wTNh5Qj9HEhFopVC4WF61rIFU+ 4IkpnFvfmVEd8Iu1vqQ/hFmP1jrvT8T+nTbaTGkcoCSPI+GyXkbxLqcl0Fncq51M M0OIRphyWA7EK3YoZ2Q1BOEIwsN0pwERYUhU0CGS45L7OZmyw86RXTMBZpBnNXrD 5VjQdpx8fvrV2iCRXi/k/e2Jy/xqs8o0I2+o9M6WrBiGCs5S9YbjsAKzRb7dsaBZ RlRAdKUjyzkquPl4K8E5ocDToB1hIGvqCSp7s11a5rq5T/jUiDMYrjkxIXH1Yg4= =W7+f -END PGP SIGNATURE- ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted at PyCon 2014
I've tentatively put an open space down: 1900 - 2100 522A on Friday. If that screws up some people's schedules lemme know. On Wed, Apr 9, 2014 at 6:42 AM, Itamar Turner-Trauring ita...@itamarst.orgwrote: PyCon is upon us; here's some of the Twisted-related talks and events this year (I may be missing a few, and this year I'm only mentioning stuff that will specifically mentions Twisted): Tutorials Aurynn Shaw will be teaching a Twisted tutorial today - https://us.pycon.org/2014/schedule/presentation/58/ Talks Ashwini Oruganti will be talking about how she implemented the happy eyeballs algorithm (really! aka RFC 6555) - https://us.pycon.org/2014/schedule/presentation/156/ Stacey Sern will present an introduction to Twisted - https://us.pycon.org/2014/schedule/presentation/189/ Laurens van Houtven will be talking about ways to use Twisted from other frameworks and concurrency models - https://us.pycon.org/2014/schedule/presentation/138/ Hynek Schlawack will tell you that SSL in Twisted has some issues (but getting better literally by the day) - https://us.pycon.org/2014/schedule/presentation/144/ Narahari Allamraju will be reviewing different options for messaging layers, from low-level systems like Twisted to high-level systems like RabbitMQ - https://us.pycon.org/2014/schedule/presentation/282/ Other We'll be sprinting after the conference, please join in! ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Getting the options object from the service
Hi Jonas, I strongly prefer to pass the options around (through the Service objects you're making) rather than relying on the global state of the plugin instance. hth lvh On Mon, Mar 17, 2014 at 2:02 PM, Jonas Brunsgaard jonas.brunsga...@gmail.com wrote: Hello Everybody What is the prefered way to get the options object from the servicemaker when using twistd plugins. I guess i could access the object via from twisted.plugins.my_plugin import myServiceMaker options = myServicemMaker.options But is there a better way to do it? Somthing like from somwhere import get_service_config() options = get_service_config() All the best -- Brunsgaard ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Simple way to halt all timers?
Hi Russell, The way to do this is to pass the reactor as a parameter, and use a Clock instead of the regular parameter for unit testing purposes. See also: https://twistedmatrix.com/documents/current/core/howto/trial.html ; that has a complete example of how to write unit tests that use Clock. That way your unit tests will be fast, deterministic, and also very clean :-) hth lvh On Thu, Mar 13, 2014 at 10:09 PM, Russell E. Owen ro...@uw.edu wrote: Is there a simple way to cancel all Twisted timers (pending reactor.callLater calls)? The reason I want to do this is to clean up a unit test without getting dirty reactor complaints. The system is a server that will rarely be shut down and does not have to shut down gracefully (i.e. I don't care if the reactor is dirty). But I want the unit tests to run cleanly, so for now I've been keeping a reference to each timer that I start, so my test cleanup code can manually cancel it. It's a lot of fuss, and makes timers harder to use than they should be. I admit that in many cases I need a reference to the timer anyway, since I may have to cancel it for reasons that matter to the server. But I hate having to do it for *every* timer. If there was a simple way to tell the reactor to cancel any outstanding callLater calls, the code would be much cleaner. -- Russell ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] which async framework?
Hi Chris, It will probably not surprise you that I like Twisted :-) On Wed, Mar 12, 2014 at 8:28 AM, Chris Withers ch...@simplistix.co.ukwrote: So, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted Twisted has a thing called Cyclone, which I hear (but that's only hearsay) gives you Tornado's API on top of twisted, so you get all of the stuff below for free. From my side, I'm looking to experimentally build a network testing tool that will need to speak a fair few protocols, both classic tcp and multicast-based, and have a web api living on top of it that most likely will have a websocket for pumping data to the browser. It'll also need to write out JUnit-compatible xml results, but that's like the easiest bit ;-) I don't know which protocols you are interested in specifically; later on in the e-mail you mention that it's *probably* not already extant. Twisted comes with a huge number of protocols already implemented. It has TCP support (duh) as well as an interface for UDP multicast. Twisted has a third party project called txsockjs which works excellently, both by itself and in the context of other (HTTP) resources. sockjs is a protocol which is basically websockets, damnit, even when the consumer is a bad browser like IE6. I'd like to be able to serve the rest of the web api using a pyramid wsgi app if possible, and I'd like to be able to write the things that process requests in and validation out in a synchronous fashion, most likely spinning off a thread for each one. Twisted comes with a threadpool-backed WSGI server. (When I say threadpool-backed I mean that the WSGI requests are handled in threads; the IO itself of course comes from the Twisted reactor). The protocols are all financial (do we really not have a pure-python FIX library?!) but none are likely to have existing python implementations. I find twisted to be a great tool for writing protocol implementations. I have written tools for querying all sorts of gnarly proprietary protocols over all sorts of gnarly transports (packet radio; it's totally a thing), and more recently for doing crazy things like multiplexing stream transports over existing RPC protocols. (Like, you see a local port come up, and that actually creates a virtual stream connection over an existing RPC thing to some virtual server on the other end of the wire: https://www.youtube.com/watch?v=W_jEIvugwes). Twisted has had so many people write so many protocols in it that also the testing tools (MemoryReactor, StringTransport) are great. Especially if you are writing something very close to a wire protocol you will undoubtedly enjoy those amenities. There's also tons of composable things for receiving delimited lines, nestrings, etc. It's hard to tell what you will be looking for since I don't know details about your protocol, but having written more than a few protocol implementations I'm going to wager a guess and say Twisted has it or a third party thing for twisted has it. Even though I mostly write Clojure now, I still write my networking stuff in Twisted. Macros are cool. Eleven years worth of battle-tested code is better. That said, tulip is nice in that you can also write protocol implementations that will look similar :-) How should I pick between the options? What would people recommend and why? Twisted, emphatically and without reservation, for all the above reasons. It's stable. All the stuff you need has been tried and tested extensively in production. It runs on PyPy, and usually a damn sight faster than on CPython, too. A common criticism of Twisted is that it takes over your codebase. I am speaking at PyCon in about a month to demonstrate that that isn't true. (It just looks that way, because once people use it, they don't want to go back... ;-)) hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Issues stemming from CVE-2014-1912?
Hi Dustin, This exploit appears to be specific to how received data is written to the already existing buffer, so the _into forms of recv,recvfrom. Even if we assume there's a parallel export for regular recv_into and not just recvfrom_into (which hasn't been shown), Twisted never calls either of the _into forms. As a result, it looks like we're unaffected. hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] maybe video
Hi Kevin, Thanks for sharing, always nice to see people doing useful things with Twisted :-) lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Deadlocks when launching processes - how to investigate?
Hi Orestis, Since creating processes involves syscalls, I would expect the tool of choice to debug this on OS X to be dtruss. I wish I could help on this endeavor but right now I'm focusing on fighting e-mail. (Brace yourself. PyCon is coming.) hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] positioning framework
Hi Kasun! I'm lvh, the primary author of the ticket. On Wed, Feb 12, 2014 at 3:19 AM, Kasun Thennakoon tmka...@gmail.com wrote: It took long time to go through the ticket[1] and understands, still i couldn't find any issue with it, instead its working perfectly for me. and thanks to the twisted positioning NMEA adapter, it become easy to me to write a adapter for the device(MVT380 device[2]) which i'm using in my project[3]. Awesome! Glad to hear it's working so well for you. IMHO, 1. isn't it good to use items() rather than iteritems() since python 3.x has deprecated iteritems(), There are many more Python 3 issues, I suspect. Both of those would work; I'm personally a bigger fan of using things like six' iteritems(d); which does d.items() on 3.x and d.iteritems() on 2.x, since you guarantee that you have identical semantics (modulo all the fancy things you can do with views that you can't do with iterators). I think there's a few cases where I don't really want *all* the items (although, of course, premature optimization is the root of all evil :)). 2. In nmea._UNIT_CONVERTERS dictionary mapper, can't see any usage of key 'K' which convert Km/h unit to meters/s because it has a FIXER for speedInKnots but not for speedInKMh, i think in $GPVTG NMEA string type (it is not defined in _SENTENCE_CONTENTS) string they use text 'K' to indicates that speed over ground is in kilometers/hour. That's there for forward compatibility as well as compatibility with really bad GPSes that use illegal units. The fix for that lives in: def _fixUnits So it is independent from any particular unit. That just blindly indexes into that dict; so if a GPS says something is in kph, it will convert to m/s, even when according to the spec kph isn't a valid unit for that. Also as you mentioned for some *other* sentences it *is* a valid unit, but that's okay, we can just leave it there and then later add GPVTG support and get units conversion for free! other than those suggestions,its works perfectly well. i wonder why this positioning module didn't merged with the trunk yet,and i'll be happy if i can help to get it merged. The ticket is up for review, and has been for a long time. Since you have apparently reviewed the ticket and used the code, would you mind giving it a review so it can (finally!) get into Twisted itself? Review instructions are here: https://twistedmatrix.com/trac/wiki/ReviewProcess Some of the review comments so far have asked for extensive changes. I am reminded of this e-mail by Glyph: https://twistedmatrix.com/pipermail/twisted-python/2014-January/027894.html ... which is quite long, but I think the bottom line is that you should just trust the review process and do that :) hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] how to hire a Twisted developer to implement the feature?
Hi Laurent, Apologies for brevity; I am on my phone. I am a Twisted core developer, and the author of minitrue, a software package that uses the proxy parts of Twisted. I am currently available for contracting. hth lvh On Feb 7, 2014 5:12 PM, Laurent Domenech-Cabaud l...@ldom.net wrote: Hello, We're looking to hire (contract) someone with a good knowledge of Twisted, ideally a dev of this great project. What we need is fairly simple: implement an HTTPS transparent proxy to work with a basic HTTP proxy (based on the example from the doc). The HTTPS proxy will just have to pass the data to/from the destination server (untouched, but some filtering will be added by us on the destination host name) and the HTTP proxy will do a little more filtering (this portion is not part of the work to be done). Please let me know. I'm at your disposal for more details. If it's not the proper place to ask, please accept my apologies. Cheers, Laurent ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Python Events Calendar (Now with Twitter feed)
Hi Twisters, Marc-Andre Lemburg from the PSF has been taking care of a Python events calendar for a while. It now has a twitter feed making it easier to find new events. If you're doing any Python-related events, such as of course Twisted sprints, or maybe Python user group meetups, please don't hesitate to contact the PSF to get your event added. M.-A.'s e-mail as been forwarded below. hth lvh -- Forwarded message -- From: M.-A. Lemburg m...@python.org Date: Wed, Jan 29, 2014 at 11:20 AM Subject: [psf-members-ann] ANN: Python Events Calendar - Now with Twitter feed To: PSF Members Announcements psf-members-annou...@python.org [Please help spread the word by forwarding to other relevant mailing lists, user groups, etc. world-wide; thanks :-)] ANNOUNCING Python Events Calendars - Now with Twitter feed maintained by the Python Software Foundation (PSF) and a group of volunteers INTRODUCTION As some of you may know, the PSF has put together a team of volunteers who are maintaining a central Python events calendar. We currently have two calendars in place: * Python Events Calendar - meant for conferences and larger gatherings focusing on Python or a related technology (in whole or in part) * Python User Group Calendar - meant for user group events and other smaller local events The calendars are displayed on http://pycon.org/ and in a smaller version in the sidebar of the http://python.org/ website. You can subscribe to the calendars using iCal and RSS feeds and also embed the calendar widgets on your sites. Please see our wiki page for details: https://wiki.python.org/moin/PythonEventsCalendar The calendars are open to the world-wide Python community, so you can have local user group events, as well as regional and international conference events added to the calendars. NEWS We have now created a Twitter feed for the calendars, which you can follow to get updates on all newly added events: https://twitter.com/pythonevents The tweets come with links to the event listings, which you can add to your own Google calendars with a single click. ADDING EVENTS If you want to have entries added to those calendars, please write to eve...@python.org and include the following information: * Name of the event * Type of the event (conference, bar camp, user group, etc) * Focus on Python and approximate size * URL * Location and country * Date and time (if relevant) For recurring events, please also include a description of the recurrence in a way that's compatible and supported by Google calendars. MORE INFORMATION More information on the calendars, the URLs, feed links, IDs, embedding, etc. is available on the wiki: https://wiki.python.org/moin/PythonEventsCalendar Enjoy, -- Marc-Andre Lemburg Director Python Software Foundation http://www.python.org/psf/ ___ psf-members-announce mailing list psf-members-annou...@python.org https://mail.python.org/mailman/listinfo/psf-members-announce ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Protocol for socket.io
Hi Andrew, SockJS is a different project from socket.io; it just tries to solve a similar problem (WebSockets everywhere). If you literally need socket.io and not a similar thing in the same problem space, then SockJS will not help you. You can read about the spec for SockJS here: https://github.com/sockjs/sockjs-protocol hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Protocol for socket.io
On Sun, Jan 26, 2014 at 10:46 AM, Glyph gl...@twistedmatrix.com wrote: It would be great if someone built a package to do this (more interop is always better), but I think interest has generally fizzled on Socket.IO implementations in the past because SockJS does pretty much exactly the same thing, and as Laurens said, our friends over at Desert Bus for Hope maintain great Twisted support for it https://github.com/DesertBus/sockjs-twisted https://pypi.python.org/pypi/txsockjs. socket.io adds a few things like RPC support and pubsub, both those are things that should be spoken over some transport, not part of some transport, and are trivial to layer over an existing browser transport. I like SockJS because it's literally just websockets damnit!. Also, it doesn't use Flash, which has excellent benefits including no gnarly XML files to server and not having to wait three seconds to figure out if a connection works or not (yes, really). Also: protip, speak SockJS over TLS exclusively. It's not even a security thing (except yes, that too), but if you do end up speaking websockets you'd be amazed how much Enterprise Quality Web Middleware Server Hardware Firewall(TM) screws up websocket connections. TLS, OTOH, they just leave be. hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Protocol for socket.io
On Sun, Jan 26, 2014 at 10:57 AM, Glyph gl...@twistedmatrix.com wrote: Having had the life-transforming experience of supporting customers using *Government-*Quality Web Middleware Server Hardware Firewall™, in a pre-websockets world, with a two-way web application... No, no I probably wouldn't. Hah; okay, fair enough: that was just a generic warning to those of us who have not had to stare into the Black Goat of the Rack with a Thousand Permanently Quiet Sockets... cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Protocol for socket.io
Hi Tobias, Hm. I can't find it in the current version. Either I am misremembering, or they expect you to use the pubsub thing as an RPC mechanism somehow (i.e. the room name is the procedure name). hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Protocol for socket.io
Hi Andrew, No idea about socket.io, but if you're free to pick whatever you want, and you just want websockets, damnit! then sockjs is actually better, and has excellent twisted support. hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Limit headers by size and not by number in twisted.web
Hi Adi, I'm assuming this is somewhat related to http://homakov.blogspot.be/2014/01/cookie-bomb-or-lets-break-internet.html:) I don't know of any mechanisms to limit cookie size. It's probably a good feature to have, and perhaps even enable by default. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Limit headers by size and not by number in twisted.web
Hi Adi, Awesome! Thank you for your research contribution. lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Announcing pyOpenSSL 0.14a3
Hi JP, FWIW, clarent appears to work fine against twisted and pyopenssl trunk. Is there a significant difference between the alpha and trunk (so that I should also test against the alpha specifically)? merlyn blows up, but that appears to be only because Axiom uses unsignedID, which has been removed from twisted apparently :) hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Announcing pyOpenSSL 0.14a3
On Tue, Jan 21, 2014 at 2:26 PM, exar...@twistedmatrix.com wrote: Hi Laurens, Right now pyOpenSSL trunk and the release branch only differ in the version number they report. There are some important differences between a2 and a3 though (so if you tested trunk before I announced a3 you might not have tested the latest code - of course, I hope those changes only fixed things, not introduced any new bugs ;). Yep, I've been testing against trunk pretty much continuously over the last few days, so I think we're good. Anyway, all systems green AFAICT, at least no PyOpenSSL bugs.. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted and symlink
Hi Toph, On Tue, Jan 21, 2014 at 4:14 PM, Toph Bei Fong toph...@yahoo.de wrote: i was trying to port Python, Twisted, etc.. to VxWorks. Ooh, exciting :) My Problem now is: twisted/python/lockfile.py (import symlink). VxWorks doenst support symlink. My Questions are: * For what is the symlink used? Symlinks are used for lockfiles. * Can i disable them? Well, maybe. Is there some way to create an atomic filesystem lock on VxWorks? * Is there an old version without symlink? Probably not a reasonable version, no. Is there any other alternative or solution for the symlink? Sure. That module implements an alternative for Windows. I would guess that you would need one for VxWorks as well. hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Better usage of Twisted on multi core processor
Hi Sumanth, The first answer by JP Calderone (exarkun) on the following stackoverflow question may be of some assistance. http://stackoverflow.com/questions/10077745/twistedweb-on-multicore-multiprocessor This answer is from 2012; the good news is that the good API that JP talks about in his answer is now available in released Twisteds IIUC :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Maximum number of SSL connections to the server
Hi Sumanth, Still sounds like an open file problem. What platform? How did you set the number of files higher that 256? Take a look at /etc/security/limits.conf :-) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Maximum number of SSL connections to the server
Hi Sumanth, You probably want to look at the limits launchd is imposing. Consider: launchctl limit maxfiles 5000 5000 You can make this permanent by pumping that line (minus launchctl) into /etc/launchd.conf. hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Maximum number of SSL connections to the server
On Mon, Jan 13, 2014 at 3:19 PM, exar...@twistedmatrix.com wrote: select() has a hard limit that is often 1024. Make sure you're not using the select()-based reactor (which I think is probably the default on OS X). Jean-Paul What is the A-grade reactor on OS X? I thought it was cfreactor, but the docs appear to brand it as a GUI reactor (that doesn't mean it can't be better than select, of course); perhaps it's kqueuereactor but I hear kqueue is kind of gimpy on OS X compared to FreeBSD (but I have no idea what that actually entails). cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] ISSLTransport.getPeerCertificate returning None?
On Tue, Dec 10, 2013 at 9:41 AM, Phil Mayers p.may...@imperial.ac.ukwrote: Unless you set the appropriate verify options on the server side too, the client cert isn't available for inspection. A-ha! But I want to do TOFU-POP; the certs are generated by the client, and there's little point in me signing them (I don't need to be able to verify that I ever signed them somewhere else). The context factory I got from PrivateCertificate.options() doesn't seem to do what I want, so I wrote my own SSL ContextFactory like it's 1999; I used VERIFY_PEER with a callback that always claims it validated: class ContextFactory(object): def getContext(self): ctx = Context(TLSv1_METHOD) ctx.use_certificate_file(cert.pem) ctx.use_privatekey_file(key.pem) ctx.set_verify(VERIFY_PEER, _verify) return ctx def _verify(connection, x509, errorNumber, errorDepth, returnCode): Always pretend the certificate verified. return True Does that sound about right, or is there an easier way to do this? (Yeah, yeah, don't hardcode paths etc :)) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] ISSLTransport.getPeerCertificate returning None?
Somehow, JP's e-mail did not make it into my inbox :-( Anyway, using the context factory that I got from ssl.PrivateCertificate.options() (which I think is CertificateOptions), I wasn't able to inspect the peer certificate. That's on 13.2. Using the context factory I pasted in a previous e-mail, it's now working great :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] ISSLTransport.getPeerCertificate returning None?
Hi! I'm trying to authenticate a client by looking at their SSL certificate. I'm calling it from within an AMP responder (so, there's bytes going over the TLS transport already), and it's still None, so this is unrelated to calling it in connectionMade. That leads me to believe my client is doing something wrong... I've followed the SSL howto on how to write a client that sends a client cert: https://twistedmatrix.com/documents/current/core/howto/ssl.html#auto9 My client is here: https://gist.github.com/lvh/d8d9f0b530d07a087da4 ... but in the server's AMP protocol, self.transport.getPeerCertificate() is still None :/ Any idea what's going on? cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Unit testing AMP responder method registration
On Wed, Dec 4, 2013 at 9:02 AM, exar...@twistedmatrix.com wrote: https://bazaar.launchpad.net/~game- hackers/game/trunk/view/head:/ game/test/test_network.py#L418 Thanks! I'll see how I can incorporate this. It does seem like the obvious test. Right now this is more spread out in my tests: I currently test the locator directly and the command separately. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Unit testing AMP responder method registration
Hi! I'm trying to write code using TDD and AMP. I'm trying to figure out how to write a unit test for: @MyCommand.responder Specifically, I would assume that test: 1. Tests that the responder locator has a responder for MyCommand; 2. That responder matches the decorated method (1) is easy, (2) not so much. Since usually the responder locator gives me this nasty doit function that isn't introspectable (unless it.func_closure[0].cell_contents.im_func counts as introspection), I'm not sure how to write that unit test. I do have end-to-end tests for this, obviously: I would like to also have regular tests :) thanks in advance lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Some thoughts hacking with AMP (was: something about multiplexing and accessing protocols)
On Fri, Nov 22, 2013 at 8:00 PM, Glyph gl...@twistedmatrix.com wrote: On Nov 22, 2013, at 4:10 AM, Laurens Van Houtven _...@lvh.io wrote: Also, I do really really want the protocol and not the transport. This is because I want to pass a reference to the protocol around so that later I can call callRemote on it. That I can also get the transport is mostly just gravy so that I can return nice things for my fake transport's getHost/getPeer. Except that maybe your protocol is just a BinaryBoxProtocol, and has no callRemote method. Or maybe it's actually HTTP and feeding things to AMP after some deserialization pass, like via JSON (aren't you even doing this already in some other code?). Is there even a protocol visible to this code in that case? Yep and yep: - I guess I mean IBoxReceiver? The thing with callRemote on it ;) - I'm doing that, and that's what I hope to do again here :) - Maybe there should be a new API that passes the proto (and actually means proto ;)) I still think that before providing this new mechanism we need *some* way of declaring that we expect more from the protocol. Yep! I think I have some code up (or will have some code up soon, depending on when you read this email) that does have sort-of working multiplexed transports: Cool. For what it's worth, I hacked together something in docs/examples that works. I'll see what I have to do to get it to work over a browser, and see what I can expose as API :) -glyph ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Some thoughts hacking with AMP (was: something about multiplexing and accessing protocols)
Hi, Some thoughts while hacking: - The argument having access to the responder locator and then eventually it going to a responder function is kind of useless, usually. The responder function already has a reference to the locator, usually called self :) - The locator object can be shared between different connections, so a hack using the current status quo where you get the proto using the responder locator is... well, hacky. Also, I do really really want the protocol and not the transport. This is because I want to pass a reference to the protocol around so that later I can call callRemote on it. That I can also get the transport is mostly just gravy so that I can return nice things for my fake transport's getHost/getPeer. So, to reiterate: - I think all the docstrings should reflect the real situation. I guess I need to file a ticket for that. - Maybe there should be a new API that passes the proto (and actually means proto ;)) I think I have some code up (or will have some code up soon, depending on when you read this email) that does have sort-of working multiplexed transports: https://github.com/lvh/txampext When done, this will be available from pypi as txampext==0.0.6. If anyone is interested in this, the next big feature mark will be integrating _habnabit's AMP producer stuff, so that it's not just transports, but IConsumer/IProducer transports as well. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] how to change an qt4 applicaion using twisted?
On Sun, Nov 17, 2013 at 11:10 AM, yangyouxiu yangyou...@gmail.com wrote: Thank you very much. Is it reasonable to bulid only one protocol to deal with datas tranports between client and server ？ If in this way, i believe i have to write lots of case to deal with the message, such as to call differnent method,it's impressive. Is there any suggestions? It might be. For simple client-server interactions using messages, it may be a much better idea to use some existing machinery instead. Many APIs use REST. Twisted provides the tools to do REST, but it also provides other (IMHO better) mechanisms. For example: AMP is a message-based asynchronous protocol. Alternatively, twisted provides PB, which allows you to share objects over the wire. If you like the mental model of a client and a server sending messages to each other, AMP is probably your thing :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Error unmerging r40589
Hi,
I tried to review #2053. Unfortunately, the patch had a wrong name and a
wrong topfile (the author was convinced they were working on 2503). I
noticed the wrong patch name, but missed the wrong topfile. So, I'm trying
to unmerge that revision (40589) from trunk using svn merge -c -40589 ..
That part works, but a pre-commit hook is preventing me from committing,
because it insists that I'm not deleting a topfile, even though it looks to
me like I am:
trunk ❯ svn
status
⏎
A +twisted/test/test_import.py
D twisted/topfiles/2503.removal
trunk ❯ svn commit -m Unmerge remove-test-import-2053
Reopens: #2053
This branch accidentally had the wrong topfile.
Adding twisted/test/test_import.py
Deleting twisted/topfiles/2503.removal
svn: E165001: Commit failed (details follow):
svn: E165001: Commit blocked by pre-commit hook (exit code 1) with output:
Must remove a ticket.{feature,bugfix,doc,removal,misc} file for re-opened
tickets. For further details, refer to
http://twistedmatrix.com/trac/wiki/ReviewProcess#Newsfiles
thanks in advance
lvh
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Error unmerging r40589
Hi everyone, This was resolved. The error message is (of course) incorrect; the issue relates to trying to reopen a different ticket from the topfile you're removing. I've filed a ticket on twisted's trac integration: https://bugs.launchpad.net/twisted-trac-integration/+bug/1248491 cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Bytes vs unicode in twisted.python.logfile's python3 porting
Hi Glyph, On Wed, Oct 30, 2013 at 1:22 AM, Glyph gl...@twistedmatrix.com wrote: The new logging branch (destined for review Real Soon Now) is a bit more rigorous about sensible behavior with respect to encoding, so we can tighten up this behavior and make it work in more cases without breaking compatibility. What you're doing here basically makes sense though. Okay, great :-) I specialcased Windows so that the test would use NUL instead of /dev/null. That appears to make the win7 bot happy. I've resubmitted it for review, we'll see what happens :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Limiting cipher options for SSH/SFTP
Hi Ray! This seems like a clear failure of the documentation. Please file a ticket so that it can be alleviated at some point :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted 13.2.0pre1 Testing Feedback
Hi Hawkowl, Thanks for managing the release! My stuff works fine :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Bytes vs unicode in twisted.python.logfile's python3 porting
Hi everyone, I'm working on #6749 for porting t.p.logfile to python3. I'm dealing with some test failures, which you can see from the buildbot here: http://buildbot.twistedmatrix.com/builders/python-3.3-tests/builds/1602/steps/shell/logs/stdio I have pasted the relevant bit into a gist here: https://gist.github.com/lvh/7174766 I think what's happening is that LogFile.write should take native strings (since that's what log.msg takes). However, I'm opening all files in binary mode, since that's on the reviewer checklist (point 8) for the Python 3 porting plan. Should I not open the files in binary mode? cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] AutobahnPython 0.6.3 - WebSocket compression and more
Congratulations! Please keep the announcements coming. If I get a chance, I'll try to apply the recent attacks by Rizzo et al. on TLS compression and the compressed stream over TLS equivalent by Prado et al., since I like compression but I also send credentials over TLS :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Help for Twisted + Thread + Message Queue
Hi Stefano, It's probably a good idea to avoid thinking in threads until they come up as a potential solution. Some questions: 1) What are the latency requirements? Is it okay for everything else to halt while you do that computation? What are the latency requirements for the computation? Is the data useless if you don't get it in the next n seconds? 2) What kind of machine is it running on? Are multiple cores available, and would it make sense to use them? 3) Do you require a lot of shared state with the process talking to the modem in order to make the long computation? The easiest thing to do is probably deferToThread, but you have to be careful not to touch any state you're not supposed to touch. The second easiest thing is probably Ampoule, a process pool, or maybe just regular AMP, if you only want a regular process. Those would have the benefit that the reading process is never doing anything hard, so latency should stay predictable. hth lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Help for Twisted + Thread + Message Queue
On Fri, Oct 4, 2013 at 10:59 AM, Stefano Danzi s.da...@hawai.it wrote: Hi Laurens, many thanks for yours considerations. 1) I don't have problems about latency but I can't halt all during the computation. Why not? I can defer the computation for minutes if I attach to data the timestamp related to when data arrives. Sure, gettimeofday is a thing, that shouldn't be a problem. 2) The monster machine that have to do all is an embedded pc based on 1000MHz AMD Geode NX1500 Okay, so single core. 3) The long computation need only the data. In extremis I could write data on a file or DB and compute using another program. I would imagine you do that anyway for persistence reasons with the results :) I bit better explanation of scenario: The radiomodem take unexpected data from a network of varius sensors. The sensors could be reachable or not. The only way to know when the sensor is reachable is to wait fo unexpected data. If I need to send data to a specific sensor I have to do it immediatly after it send me unexpected data, wait also 1 second could be sufficient to have the sensor no longer reachable. I can't send data to sensor and after do computation because meanwhile could arrive data from another sensor. Mainly it is the reason because I think to defer long computation to another thread.. Okay, so try with threads first, and hope that the scheduler lets your reactor thread work often enough. If that doesn't work, do the computation in a subprocess, and hope that nice works well enough :) lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] AMP Argument.toBox's proto argument is a locator, not the proto?
On Tue, Oct 1, 2013 at 12:59 AM, Glyph gl...@twistedmatrix.com wrote: Most of the code I can think of that wants to use that really wants the * transport* rather than the protocol, Yes, but having the protocol would also immediately give you access to the transport, and, from what I understand in most cases of AMP, also everything else :) but nothing within AMP itself actually uses those arguments; in fact, searching the usual suspects (epsilon, vertex) I can't even find any Arguments that use the 'proto' argument for anything useful. I suppose it's too late to get proto to actually mean proto and not boxSender? It would definitely be a backwards-incompatible change, and I do actually have some code that somehow relied on it being the boxSender (actually, I think I saved that code in txampext, mostly, except I renamed that thing proto). If I recall, I believe the idea behind it was to allow an AMP responder within Vertex to return the peer's IP address back to the peer, from within an authenticated AMP route that (because it was a route) wasn't necessarily connected directly to the transport (and therefore couldn't just do self.transport.getPeer()). Ironically I don't think it'll actually work for that now :-). When we pull the authentication logic in from http://bazaar.launchpad.net/~divmod-dev/divmod.org/trunk/view/head:/Epsilon/epsilon/ampauth.py, you might write a responder that's interested in authentication information that lives in some other relation to the protocol. I wrote very similar deep-in-AMP auth logic once, and did look at that code (but ended up not using it because I use TLS, so I don't need hand-rolled challenge response or OTP systems). When you say other relation to the protocol, does that mean it can be the protocol because the protocol will have some kind of reference to it? So in order to fix fromBox/toBox, we need to do a fix that firms up that contract and perhaps exposes more than a Protocol object. The *recommended* API should be more or less like what ExposingArgument is doing - specify an Argument that asks for a particular attribute of the transport or the protocol or the authentication context or whatever, the implementation details may involve other lower-level public APIs. That still sounds like it can be done by making proto actually the proto ;-) So, basically, the question is if proto being the locator is a bug that I can fix, or an interface that I can't. My contributions to AMP have been more of the defect-findy kind, but I could certainly turn them more into the code-contributy kind. I imagine I'm not the first person to want tests for command classes ( https://github.com/lvh/txampext/blob/master/txampext/commandtests.py) or a nested AMP box ( https://github.com/lvh/txampext/blob/master/txampext/nested.py). That would be cool. And, you know, that auth thing I said :-). If I can change proto to mean actually the protocol not something else then that seems plenty easy to add, and it would definitely be cool if people don't have to mess with this nonsense themselves for something as ostensibly simple as having access to the protocol :-) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] AMP Argument.toBox's proto argument is a locator, not the proto?
Hi everyone, I think I've hit one of those cases where AMP really seems to want everything (locator, receiver, sender) to be an instance of t.p.amp.AMP :-( I've written some code that tries to multiplex stream transports over AMP: https://github.com/lvh/txampext/blob/multiplexing/txampext/multiplexing.py The repo contains an example server and client, which demonstrate the issue: https://github.com/lvh/txampext/blob/multiplexing/docs/examples/multiplexing_client.py https://github.com/lvh/txampext/blob/multiplexing/docs/examples/multiplexing_server.py In order to do some of this multiplexing, I need access to the protocol instance inside the responder on the server side. Fortunately, I already had some code that exposed box senders (after a lot of advice from Glyph). I modified it to expose the protocol as well: https://github.com/lvh/txampext/blob/multiplexing/txampext/exposed.py#L41 However, it turns out fromBox gets called with the *responder locator* as the proto argument, not the actual protocol. The server has a pudb call that makes it easy (?!) to trace this down. The CommandLocator class, inside doit (a function defined in _wrapWithSerialization) passes self to command.parseArguments: https://twistedmatrix.com/trac/browser/trunk/twisted/protocols/amp.py#L1015 This is the part where I think the contract is broken, since parseArguments claims to want the protocol (well, it says it wants the AMP protocol, which, subclassing everything, is also all of the things, of course), but receives the responder locator. What am I doing wrong? Is this a bug? confusedly, lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Clean pb solution for two-way object sync?
Hi Daniel, If you're interested in PB, you may also be interested in Foolscap, the object-capability extension to PB. Foolscap lives at: http://foolscap.lothar.com/trac Feature overview: http://foolscap.lothar.com/trac/wiki/FoolscapFeatures cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] run python application with twistd -y ifinger.tac, error occurs. please help to check
Hi Charles, Yes, twistd doesn't appear to work on py3k. Not being able to import _preamble is normal (once Twisted has been installed), but _preamble does assume sys.exc_clear exists, which isn't true on 3.x. I couldn't find any ticket on this. Perhaps you should file it (and maybe fix it!) :) It's my understanding that the new except clause semantics are expected to take care of this problem. Can someone comment as to why this was put there in the first place, and, specifically, why it can't just be pass? Is there some logging system that looks at the current exception or something when twistd (or some other script) starts? cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Passing additional arguments to errback
On Thu, Sep 5, 2013 at 12:00 PM, Maciej Wasilak wasi...@gmail.com wrote: Laurens, You don't have to do it from in there. You can do .addErrback(handleErrors, request), since it's all the same request object, right? Aaargh! I see the problem now. I wrote everything as part of the Protocol class (DD - DeferredsDictionaries), when I should have extracted request functionality into separate class. Then I can save request body as a class member between callback and errback. Thanks!!! Well, yes, you should, but still keep in mind that you can actually just pass the request like so: .addErrback(handleErrors, request) You don't really have to save the request itself anywhere as an attribute. 1. It seems self.endpoint is a t.w.s.Site object. That's kind of confusing, since twisted has an endpoint concept that's one step removed from a Site (endpoints connect or listen with factories, a Site is a factory). You might want to reconsider that name :) Endpoint is the official name in draft: http://tools.ietf.org/html/draft-ietf-core-coap-18 I'll rename it to coap_endpoint to avoid confusion 2. It's kind of strange to start with defer.succeed() and then start making a callback chain IMHO, but it's not wrong, really. I've recently understood callback chaining and I really like the idea. I guess it's this old proverb about having a hammer, and seeing nails everywhere. I'll try to come up with something better :) . No problem. Case closed - thank you very much! Glad to have helped. Regards Maciek cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Passing additional arguments to errback
Cześć Maciek :) In general, you can pass extra arguments when you call addCallback(s) or addErrback. They will get passed to the callback. However, as a side note to that code example, do you understand the difference between .addCallbacks(cb, eb) and: .addCallback(cb).addErrback(eb) and: .addErrback(eb).addCallback(cb) ... Also, keep in mind that you only errback when there is an issue setting up the connection. If the server successfully responds with an error (say, a 404 Not Found, or something), the callback will be called with the response object. So, your question doesn't make a lot of sense to me: if the errback gets called, there's not really a response! Also, if you want to do scraping with Twisted, consider looking at Scrapy, a fully-featured web scraper that uses Twisted internally. pozdrawiam lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Encouraging New Reviewers (was: Re: something about Github, as all threads about fostering community now start on that subject)
On Tue, Aug 27, 2013 at 6:07 AM, Glyph gl...@twistedmatrix.com wrote: I think that's a fine rule, but it has one gigantic problem: we don't have a list of committers anywhere. Making this list - and, critically, associating VCS handle (svn.twistedmatrix.com login) with Trac handle, for those cases where it differs - would really help new contributors figure out whether there's You got cut off here. I feel like if someone could hack up a Trac report, or an extension to the high scores page, that addressed the first point, it would be a lot easier to deal with the second and third ones. So, if anyone reading this would like to do that, I would be forever in your debt. What would be the extension to the score board? lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Interesting plot on new author contributions post-Github for several Python packages
IIUC, Tom does most of his contributing through Github. That probably means that it's a well-supported process that has most of the kinks ironed out: after all, if Tom hasn't caught them, I would guess a new contributor probably wouldn't either :-) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Interesting plot on new author contributions post-Github for several Python packages
On Tue, Aug 27, 2013 at 2:14 PM, exar...@twistedmatrix.com wrote: On 09:54 am, _...@lvh.io wrote: IIUC, Tom does most of his contributing through Github. That probably means that it's a well-supported process that has most of the kinks ironed out: after all, if Tom hasn't caught them, I would guess a new contributor probably wouldn't either :-) Or it means Tom's workflow is fine and he's learned all the issues he needs to avoid tripping over. That sounds like what I mean. I don't understand the difference. I am saying that Tom's workflow is fine and knows where the hard parts are. Also, given the amount of time that he (usually) gets to spend on Twisted and his responsiveness on the IRC channel, I think it counts as well-supported. You didn't quote anything in the message you replied to, so I'll just guess that this is an email meant as a +1 please switch to github. Please correct that impression if it is mistaken. No. I am suggesting that there is a way to contribute using Github that isn't totally broken, and someone has already figured out what that way is, and that someone is in a position to explain it to others. I'm saying we don't need to switch to github, because apparently you can already use github if you want to, everything else staying the same. It's possible that this way of using github does not please the would-be new reviewers sufficiently, but I'm saying that if someone suggests we move to github, I'd like to make it clear to them that you can already use github, and that they should probably at least formulate their arguments in the shape of the current way that I can use github is bad because ... :) Thanks, Jean-Paul cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Advice Request: Under what circumstances should I use AMP's Command Response field?
Hi Burak, I think you're doing fine. Distributed systems are just kind of hard :-) It sounds like your fundamentally building an eventually consistent distributed database. We have a few of those already: it might be significantly less work to just use one of them. I suppose it depends why you're trying to make it distributed. Is this about reliability in the face of e.g. hardware failure, or is this about being able to disseminate data when someone tries to stop you from doing so? Are you trying to protect against byzantine failures too? That said, you might want to consider how you communicate posts. Six months worth of posts is a lot. Even with ten posts per day, you'd end up with ~10*30*6 = 1800 hash values. The digest size of BLAKE2 is variable, but if you're using 512 bit digests, that's 64 bytes, or 112.5 kibibytes for the whole thing. That's probably more than you want to send in a single message. While you can't rely on synchronized clocks (in the wall-clock time sense) in a distributed system, you *can* rely on timestamps of your immutable messages. You could send only message id's in the preceding time window, for example. You can use hash chains to guarantee that the boards share the same history. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Advice Request: Under what circumstances should I use AMP's Command Response field?
Hi Burak, On Tue, Aug 20, 2013 at 12:21 PM, Burak Nehbit bu...@nehbit.net wrote: Hi there, I was meaning to ask you all about the 'proper' way to use AMP in regards to two–way communication. The problem I am facing is quite simple, I am building a simple peer–to–peer network in which the user: a) When sent a handshake, returns a handshake, b) When asked for fingerprints of posts in the local, returns it, and then asks for the same thing, and finally, c) If any of the fingerprints do not match posts, request those posts with the fingerprint. The only non–symmetric step I have in my protocol is the last one. My question is, should I send my responses over the same AMP command as a response to the command requested, or should I just return an ACK (a boolean value of true, to confirm receipt) and then initiate a separate AMP request from local to the same remote? Keep in mind that AMP commands are always ack'ed without you needing to specify any kind of value, unless you set requiresResponse = False on the Command class. That would mean I would need to handle my open–connections pool by myself, to prevent posts from a non–initiated machine being accepted into the local machine, but I do need to implement that anyway due to the complexity of my business rules. I don't understand this part. It sounds like an authentication problem, no? At that point, it seems that I am (Am I?) just using AMP as a remote method header I don't strictly need, and I could just use LineReceiver—all my data is text—to do my bidding. It's usually correct that you can do things with LineReceiver, but it's also usually true that AMP is doing stuff for you ;) Some more information about your system would be useful. Are posts append-only? Would it make sense to have a given serialization of them? If so, a hash chain would allow you to sync with just one hash value. A consensus protocol might help you figure out what that serialization is. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted as a House for Sale
Hi Matt, Loved the story, excited about the prospects. Just one nitpick: after one of Glyph's rants on gender-neutral words, I got tripped up by Ronnie not being referred to as they ;-) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Multiplexing streams with Conch
On Tue, Aug 6, 2013 at 2:14 AM, exar...@twistedmatrix.com wrote: This is still sort of incomprehensible to me. For example, you haven't even said whether you're trying to develop the server side of this, the client side, both, or what. I'm definitely trying to write the server. I'm hoping to be able to get out from writing the client, and have people use any common SSH client. (That part was in the original mail: I'm asking if this would be possible to do so with regular SSH clients, or if I'd have to write that side as well, implying the part I'm definitely writing is the server :-)) And why would a server ever try to make a TCP connection to a port on the client? Perhaps you can lay out your goals in a bit more detail, skipping over fewer of the other important details? Imagine a stream transport inside the server. It's a StringTransport, but it's simulating a TCP transport between two hosts on a real network. I'd like my client to be able to observe and modify the traffic on that transport. To me, the obvious way to do that seemed to be to cut the transport in half into two transports, and put the client on both of the new ends: fake host 1 -- fake host 2 becomes fake host 1 client fake host 2 Since the client is a real machine and fake host 1 and fake host 2 are simulated hosts that live in the server, each at an end of a StringTransport, these new transports will have to be communicated between the server and the client, so multiplexing streams seemed relevant. This is obviously possible if I write both server and client (and if I did, I probably wouldn't use Conch), but I was wondering if it'd be possible using regular SSH clients. About the only part I understand is multiplexing, to which the answer is probably, SSH does support multiple logical channels over a single TCP connection. Okay. I know conch also supports port forwarding, so I guess the command line tool would be a good start there. Jean-Paul cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Multiplexing streams with Conch
Hi, Sorry, when editing and cutting my post I got rid of something important :-) When you log in, you get greeted by some terminal app, say something written with Urwid or so. In addition to that, on the client, you can start accepting connections on some TCP port, and at some point the server will connect to that. That stream should be multiplexed inside SSH. Also, The client should be able to make connections on localhost, which are also multiplexed and actually turn into transports inside the server. Would that be possible? cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] ugh open-ssl and twisted python
Hi Gary, On Sat, Jul 27, 2013 at 6:08 PM, gary clark burslem2...@yahoo.com wrote: Hello, This problem has been driving me crazy for a couple of days. I'm hoping someone can shed some light on this. I have a twisted server its using open-ssl (using certificates) and runs on linux and communicates great with a windows client thats send messages every second. However when I send data back to the client sometimes it doesnt receive the packet and there is no indication of failure at the client whats more I dont see any failure on the twisted server end. When you say packet, does that mean actual IP packet that you observed leaving the server, or do you mean the thing I passed to transport.write? Is there any way I can tell what could cause this failure? It just look like the transport.write(data) fails just dont know why it would when receiving works like a charm. So, it fails silently? Keep in mind that transport.write working doesn't necessarily mean that anything on the other end has successfully read it: that's why good RPC mechanisms like AMP have positive acknowedgement :) Any help on this would relieve the pain I'm feeling right now. How are these two servers connected? Can you reliably reproduce the problem? Does it persist with other transports? Could you shed some more light on what you mean by sometimes? cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Best way to trigger a future connection with data
Hi Nick On Mon, Jul 22, 2013 at 10:40 AM, Nick Johnson nick.john...@ed.ac.ukwrote: Hi, Firstly, thanks for this gist, I had done a few experiments using endpoints and I think this is definitely the way to go for this code. Welcome :) As to the questions: source and destination are parameters for the job and might change between runs (a function I didn't include for brevity handles computation of these). Interval was to be the time passed to LoopingCall and type_req was another job parameter. I agree that, having looked at the gist, trying to pack everything into one Protocol was not the best way to go and using a separate protocol for each type of communication (ie, getjob, retrievejob) is more sensible. For what it's worth: a protocol implementing all of these might make even more sense if you have some functions as the high level API (like the ones I wrote in the gist): the functions could call high level methods on the protocol that cause it to do certain things. As an example, consider the IRCClient protocol: https://twistedmatrix.com/documents/current/api/twisted.words.protocols.irc.IRCClient.html ... which has methods like join, leave, say, message... Thanks for helping me out with this, Twisted is slowly starting to make sense now. Cheers, -Nick. On 19/07/13 14:52, Laurens Van Houtven wrote: Hi Nick, Okay, question and code review time. Why are source and destination arguments to the protocol? Can't they just access it on the factory? It seems that the factory initiates many connections with the same parameters. Is that true? Does it only ever make sense to use the factory to fire many requests? Anyway, the biggest issue seems to be that you're stuck on trying to do everything with one protocol; it might make total sense for you to have a job-queueing and a job-getting protocol :) Can you explain what the interval and type_req arguments are, and why they're passed to the factory? cheers lvh -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] SQL ORM for Twisted PostgreSQL?
On Fri, Jul 19, 2013 at 9:06 AM, Hynek Schlawack h...@ox.cx wrote: How would you feel about packaging it up on PyPI so people can try it out effortlessly? What do Apple’s licenses say about that? Yes, I’m volunteering. It seems it's released under the ASL2. I don't know if Apple prevents *its* employees from doing anything in particular, but it seems like third party contributors are free to do with it as they please (within the limits of the license, of course). cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Best way to trigger a future connection with data
On Fri, Jul 19, 2013 at 10:19 AM, Nick Johnson nick.john...@ed.ac.ukwrote: Thanks lvh, I did have to override the buildProtocol method in the Factory but I then set Protocol.factory to be equal to the Factory (ie, myprotocol.factory=self). You could (perhaps should) do this by calling ClientFactory.buildProtocol(self, addr). I'm still stuck however with what to do when I get more complex than this simple case. For example, I use a callingLoop to call multiple connections with a 0.1 second interval to launch jobs and each of those connections does as mentioned by setting up a future connection to retrieve the output, say 10 seconds later. There is going to be some overlap, ie I might have launched 100 new jobs before the first one fires it's task.callLater. Storing state in the factory class doesn't work in this case because each new connection wont know whether to initiate a job or retrieve output as I cannot pass it this extra information. I don't understand why not. Could you elaborate? Either way, it seems to me that the API should be: d = scheduleJob() d.addCallback(getJob) That is: only get the job once it has been scheduled. getJob would probably have to be split up into something that delays (consider delayLater), and something that actually gets the job. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Best way to trigger a future connection with data
Hi Nick, Okay, question and code review time. Why are source and destination arguments to the protocol? Can't they just access it on the factory? It seems that the factory initiates many connections with the same parameters. Is that true? Does it only ever make sense to use the factory to fire many requests? Anyway, the biggest issue seems to be that you're stuck on trying to do everything with one protocol; it might make total sense for you to have a job-queueing and a job-getting protocol :) Can you explain what the interval and type_req arguments are, and why they're passed to the factory? cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Best way to trigger a future connection with data
Hi Nick, You're pretty much there already. Instantiate a ClientFactory that holds all the necessary state. By default, your protocol will have access to that state through its factory attribute (unless you override the Factory's buildProtocol method). cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] How do you write a setup.py that installs plugins?
Hi,
How are you supposed to write a setup.py that installs twistd plugins?
I have this piece of tribal knowledge:
from setuptools.command import egg_info
def _top_level(name):
return name.split('.', 1)[0]
def _hacked_write_toplevel_names(cmd, basename, filename):
names = map(_top_level, cmd.distribution.iter_distribution_names())
pkgs = dict.fromkeys(set(names) - set([twisted]))
cmd.write_file(top-level names, filename, '\n'.join(pkgs) + '\n')
egg_info.write_toplevel_names = _hacked_write_toplevel_names
---
But I don't actually know what's actually required, and I don't actually
know where this is documented (if at all). Plus, it looks pretty
unsanitary, and appears to introduce a dependency to setuptools.
So, how am I supposed to do this?
Thanks in advance
lvh
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] If the errbacks of a canceled Deferred are called with error other than CancelledError, is this acceptable?
On Tue, Jun 18, 2013 at 8:22 PM, Glyph gl...@twistedmatrix.com wrote: I would say that if we want to percolate this information up to the caller, there should be a ConnectingCancelled exception that is a subtype of the previous exception type. Doesn't that mean we'll have many subclasses that mean that something was cancelled? If I didn't take backwards compatibility into account, I would say that composing the original exception into a new CancellationError (or something) exception would be preferable. Would you agree that it would be preferable? (Again, not taking compatibility into account -- I'm trying to get compatibility vs niceness of API to face off against each other. Personally, I think it's enough of a change in functionality to warrant a chance in ways a function can fail, but there's no point in even having that argument if there's no consensus that the composed way would even be better...) After all, if it's interesting that the operation was cancelled, presumably it's interesting *at what stage* the operation is cancelled. IIUC that would work the same with composition as inheritance :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] 13.1.0 and #6499
This was resolved :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Difficulty restarting/shutting down a twisted server - ports not closing
Hi Paul The documentation covers how to use Twisted with Tk (and many other event loops). Here's the link you want: https://twistedmatrix.com/documents/current/core/howto/choosing-reactor.html#auto16 Hope that helps :) lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] New registrations to this mailing list broken?
Hi, Sending this hoping that it will in fact make it through ;) Lynn Root (rougelynn) mentioned a few minutes ago on the IRC channel that she was unable to join the Twisted mailing list: 09:25 roguelynn hey glyph + tomprince - I'm trying to get on the twisted mailing this but see this error http://glui.me/?i=7ckizft82fokgnt/2013-05-28_at_11.22.18_AM.png/ The short version of that image (just in case it goes down or something) is that Mailman is complaining that the CGI app is run with group t-web, and it was expecting to run as www-data. Solutions include: change the group to be www-data, or re-run the configuration script with flag --with-cgi-gid=t-web Thanks! lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] New registrations to this mailing list broken?
Oops, just read that this is a know issue. My bad :) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] How does Twisted handle multiple request at the same time ?
On Thu, May 23, 2013 at 9:46 AM, abhishek unnikrishnan thecreator...@gmail.com wrote: hi guys Hi Abhishek! i'm new to twisted and asynchronous programming . i have created a rpc server using twisted and xmlrpc packages . i wanted to know what will happen if two or more clients give an rpc call at the same time. how will the server handle this situation. The tricky part is in the at the same time. Twisted's reactor thread only does one thing at a time, and that's okay: both connections get accepted and some data gets read from them (and eventually written to them). The important part is that you do these things non-blockingly (or, synonymously, asynchronously): it's okay to only do one thing at a time as long as each thing you do takes very little time. The second connection doesn't have to wait long before the connection gets opened and data gets read, because the operations that happen before it don't take long. This pattern repeats throughout asynchronous programming: it's okay to only decode one bit of JSON at a time, as long as doing that doesn't take a long time (i.e. block), preventing anything else from happening in the immediate future. Does that clarify things? and also wanted to know is there a way for the server to call the client in an rpc scenario . That depends on your RPC protocol. Many allow this (often called peer to peer), many are explicitly client-server and only allow one-way calls. Even protocols that do support full-duplex communication don't necessarily have clients that can take method calls very often. AMP, my personal favorite, explicitly supports two-way RPC. It comes with twisted, and you can read about it here: http://amp-protocol.net/ Alternatively, I think JSON-RPC is full-duplex too. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] t.test.proto_helpers.StringTransportWithDisconnection.connectionLost
On Wed, May 22, 2013 at 1:47 PM, Adi Roiban a...@roiban.ro wrote: Thanks. Did that. http://twistedmatrix.com/trac/ticket/6521 Awesome, thank you :) I have a test which always fails when running `trial twisted.test` but pass if executed as an individual tests this is due to a failure which is only raised at garbage collection. Wait, related to this problem? I would expect the test to just try and use the thing that comes out as a Failure, and check that it's of the appropriate type... I don't understand how GC comes in to play. Could you provide said test? cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] How do I write AMP Command definitions TTD-edly?
Hi again, Sorry to drag up this old post, but I have another question. On Mon, Mar 4, 2013 at 5:17 PM, exar...@twistedmatrix.com wrote: Chris and I wrote all the AMP-using code for game TDD. See game/test/test_network.py in lp:game. I've implemented this in txampext.commandtests: https://github.com/lvh/txampext/blob/master/txampext/commandtests.py There's one more issue I have though. The lp:game code (and now txampext) deals with arguments and responses, but not exceptions. Did you have something pleasant for that, or should I just hack the the equivalent thing together for AMP exceptions as well? cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] GSOC 2013 Project Idea for Twisted Matrix
On Sat, Apr 20, 2013 at 10:10 AM, Chitrank Dixit chitrankdi...@gmail.comwrote: Good Evening Hi! :-) Twisted Matrix is a great module for Networking and I would like to work with this project. I want to propose Twisted a new Idea of Promoting the project to Porting to Python 3 (with backward compatibility). Porting benefits by grabbing the Python 3 users and would prepare the Twisted for the Future. Helping the Python 3 porting effort sounds like a GSoC project to me :) You might not finish it (there's a lot of work, and some of the problems are very tricky), but it's definitely an area I think deserves some full-time attention. Are you familiar with the Twisted contribution process? It is my understanding that we ask GSoC students to find and fix a single, easy (usually easy literally means minutes of actual work, once your system is set up for Twisted development) to qualify, so that they're already familiar with what they'd need to do. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] use twisted get ftp's version
Are you only interested in the version, or would you also like to actually do FTP? On Wed, Apr 10, 2013 at 6:20 PM, yuyan zhang z858570...@gmail.com wrote: i use nc command bash-3.2# nc 83.85.85.238 21 220 (vsFTPd 2.0.5) get the version of the ftp but i want to use twisted to do this,and i don't know the function can anyone help me? Thank you very much. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Implementing a new stream transport
On Tue, Apr 9, 2013 at 5:58 AM, Phil Budne p...@ultimate.com wrote: Yes, and I implemented the ITransport methods. The first bump was that I was missing registerProducer and unregisterProducer, which are defined as part of IConsumer. But after that I had another problem (which I can't reproduce at this moment). Huh. If the interface you're passing it to claims it needs an ITransport, and an ITransport didn't work, that's a bug. The consumer/producer interfaces are somewhat related to transports, but definitely distinct. Could you paste some code that you tried illustrating the problem? (You mentioned something involving Sites and Protocols, but a Site is a ServerFactory.) -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Why is this epytext docstring bad?
Hi, I'm trying to get twisted.positioning landed, but it's blocking on an invalid epytext docstring. I've stared at this docstring a few times in the past weeks, but that didn't help. Failing buildbot log: https://buildbot.twistedmatrix.com/builders/documentation/builds/3141/steps/api-documentation/logs/stdio Docstring: https://twistedmatrix.com/trac/browser/branches/positioning-3926-2/twisted/positioning/nmea.py?rev=37997#L256 If someone could tell me what's wrong with it, that'd be much appreciated :) -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Why is this epytext docstring bad?
Cool, thanks, I'll try to debug it this way. Why doesn't the buildbot produce similar output? IIUC, pydoctor produces similarly useful output, but the output from the buildbot wasnt' useful in debugging this. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Implementing a new stream transport
On Apr 8, 2013 9:44 PM, Phil Budne p...@ultimate.com wrote: As a quick crock, I made my transport a subclass of twisted.internet.tcp.Server, and got away with it, but I'd like to do the right thing. I'd be delighted to find out that mixins exist that would solve my problem, but I'd be happy to know what interfaces I need to implement, or even better where I find find documentation on how to implement a new transport, especially one that isn't file-descriptor based. Are you familiar with interfaces? ITransport sounds like what you want. Thanks! Phil ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Release questions
On Fri, Apr 5, 2013 at 3:12 PM, Tristan Seligmann mithra...@mithrandi.netwrote: DSA keys larger than 1024 bit(?) are non-standard, but I think the bigger issue is that DSA only supports 160-bit hashes; larger hashes will be truncated, which means you don't gain much by using SHA-256/SHA-512/etc. instead of SHA-1. DSA2 can handle larger hashes, but there's no real reason to use DSA2 when RSA is so widespread. I think this is the reason the defaults are changing (were changed?) in GnuPG. As far as I can tell, the defaults are currently beyond plain old DSA. It appears this support has been in GnuPG since 2006 and in PGP since at least then, I would guess that it's been commonly available since 2008 or so. I'm not sure *why* it's DSA now, but I'm hoping the GPG people had good reasons :) (I'm guessing it's adoption was sufficient that by the time we considered changing the default the reasons were no longer valid) For DSA, GPG selects the first hash algorithm in your digest preference list that is of size = q. q is the hash size for your DSA key, which is hardcoded into the key. The defaults in GPG are: q = 160 for 1024 bit keys (i.e. plain old DSA) q = 224 for 1024 to 2048 bit keys q = 256 bit for 2048 to 3072 bit keys You can use gpg --list-packets to view this, but the output is a little obscure. It's easier to use pgpdump, which, for my key produces (truncated output): Public Key Packet(tag 6)(1198 bytes) Ver 4 - new Public key creation time - Sun Oct 14 13:56:19 UTC 2012 Pub alg - DSA Digital Signature Algorithm(pub 17) DSA p(3072 bits) - ... DSA q(256 bits) - ... DSA g(3068 bits) - ... DSA y(3071 bits) - ... The important bit is the q value: 256. Combined with my digest algorithm preferences, that means I'll get SHA-256 (first hash of sufficient size), not truncated (since 256 == 256 ;)). I guess this is drifting off-topic though... Sure thing :D I wasn't trying to argue for any particular algorithm, but simply that: - if you have a recent key of maximum allowable size for whatever the default was on your system, you're probably fine - signing a bunch of SHA sums is fine, provided it's SHA-256 or better, preferably SHA-512 (SHA-3 wouldn't be wrong, but isn't commonly supported yet, and SHA-2 has withstood attempts to break it so far better than anyone anticipated, so we're good). cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Release questions
On Fri, Apr 5, 2013 at 4:47 PM, Laurens Van Houtven _...@lvh.cc wrote: As far as I can tell, the defaults are currently beyond plain old DSA. Whoops, that's not entirely correct, as exarkun was nice enough to point out to me. Apparently, it's beyond plain old DSA *only if you ask for DSA*. The default, at least for plain old GPG, is to get you RSA/RSA. I could've sworn that the default the GPG Keychain Access tool thing from MacGPG was DSA, because I consciously didn't pick anything in particular and somehow ended up with a 3072 bit DSA key. The rest of my e-mail still counts though :) -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Release questions
On Wed, Apr 3, 2013 at 6:14 PM, Thomas Hervé the...@free.fr wrote: * Glyph mumbled something about sha sums of the release files, instead of md5. Should we pursue that? We may need to update some trac integration code. Depends, what's the goal of the checksums? If it's we want people to be able to check that the tarball they have is in fact the release and not something tainted by patches or malware, perhaps we either should have a Twisted signing key, or have the release manager sign the release instead (especially since we have a lot of signatures since PyCon :)). -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Release questions
On Wed, Apr 3, 2013 at 10:51 PM, exar...@twistedmatrix.com wrote: The question relates to step 4 beneath Cut the tarballs installers: http://twistedmatrix.com/trac/wiki/ReleaseProcess#Cutthetarballsinstallers The checksums are intended to let people verify their download was neither accidentally corrupted nor intentionally tampered with. Is the accidental corruption thing a real risk? I thought that was the point of, say, TCP checksums :) Perhaps I'm just mistaken as to how often his happens in the wild... I think the original motivation for signing some checksums instead of signing the release artifacts was something like: * gpg is a pain to use, signing one thing is nicer than signing 30 things * lots of people do not care about cryptographic concerns here, and the checksum is good enough for them Okay, fair enough. I'm a little worried about the I don't care about the cryptography part, if a user is consciously choosing that, fine; but what if they think they're doing something (verifying the integrity of the Twisted release) when in fact not doing that at all? Perhaps that's even rarer than the accidental corruption thing I so quickly dismissed just now, though ;-) As for gpg being a pain to use, `ls | xargs -n 1 gpg --sign` seems to work for me provided you have gpg-agent (and have it configured to not need a signature every time). Is gpg-agent something we don't want to require from release managers? Generating and signing a single document containing checksums of all the files is less work for the release manager and offers both possible audiences some value. Perhaps it's a round-about way to achieve those goals, though. Is there something simpler that we could do that wouldn't make releases harder or kick sand in the eyes of people just trying to make sure their ethernet card didn't hiccup? Probably not, the current thing seems pretty easy, right? If I understand correctly, the only complaint is that MD5 sucks. So if we upgrade that to SHA-256/512 (SHA-3 would be nice, but plenty of people don't have access to it yet on the command line...), that'd do it. I don't think there is anything wrong with a hash sum file, I'm just concerned that the reasons for *not* having or verifying signatures might not be that great. Jean-Paul -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted.cred interface deficiences
Okay, cool. I definitely agree that IUsernamePassword.checkPassword is dumb and support deprecating it. On Monday, April 1, 2013, Shell wrote: It actually might be the appropriate thing already. There's a couple of possible reasons for renaming; one is that the password might not be hashed but the credentials object wants to insert additional logic (exarkun's statement in IRC) anyway, but technically that's just hashing using the identity function, so it should be OK. The other is that the credentials object might want to collect the plaintext password, and perform some logic based on that *before* collecting something which can be compared to something derived from the plaintext password from the other end of a protocol. I don't know of any protocols which do this off the top of my head, but I have a suspicion that at least a couple exist, and they'd fit perfectly into this interface. If none do, then keeping it with the same name sounds fine to me. Cameron On 1 April 2013 22:12, Laurens Van Houtven _...@lvh.cc wrote: Why does IUsernameHashedPassword have to be renamed? It sounds like it's the appropriate thing already. On Mon, Apr 1, 2013 at 10:55 PM, Shell cam.t...@gmail.com wrote: The twisted.cred.IUsernamePassword interface declares: * IUsernamePassword.username - The username associated with these credentials. * IUsernamePassword.password - The password associated with these credentials. * IUsernamePassword.checkPassword(password) - Validate these credentials against the correct password. The issue is that the interface (according to exarkun) allows you to implement checkPassword() to do things other than the obvious password == self.password. Now, this is an issue because Twisted then explicitly supports (again, according to exarkun) two different uses of this interface by the credentials checker: * Call the checkPassword() method, passing it the correct password * Just take the password out and do whatever you want with it (which is necessary in any secure system) Now, imagine I write a version of checkPassword() in a library which does something security-centric (what would this be? shouldn't it be part of the checker?), assuming that it'll be used by a credentials checker which calls checkPassword(). Except... then, a library user uses it with a credentials checker which checks the password itself. Now they're skipping over my security-centric code! So I have to tell my library users that they have to use my library with a credentials checker which makes sure to call checkPassword(), not just one which accepts the correct interface. IUsernamePassword's docstring claims that the stored password must be reversible to plaintext to be compared with the password, which implies that taking the password out and doing other things is incorrect, unlike what exarkun suggests. In this case, exposing password in the interface makes little sense. In addition, twisted.cred.checkers.FilePasswordDB apparently ignores this docstring entirely already ( http://twistedmatrix.com/trac/browser/tags/releases/twisted-12.3.0/twisted/cred/checkers.py#L238 ). I propose that IUsernamePassword should be split into at least two interfaces: * IUsernamePassword, with only username and password, no methods, which allows password to be used in any way * Another interface, which only defines username and checkPassword() - possibly just a rename of IUsernameHashedPassword, which declares a similar interface However, this has the issue that any credential checker which can use the second interface would also be able to use an IUsernamePassword here if there were an adapter between the two, but support for this would have to go into every credential checker which supports the second interface at present. Maybe the Portal could automatically search for adapters if it can't find a direct match? Thanks, Cameron ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- Sent from Gmail Mobile ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Current callback best practices
I use closures unless I use the same callback all the time. On Apr 2, 2013 1:04 PM, Hynek Schlawack h...@ox.cx wrote: Hi, I’ve asked this one already at the Twisted dinner (thanks again for it, it was probably my PyCon highlight) and got various answers from people sitting around me. I’m still indecisive so I’d like to gather even more opinions[1]. I already tried to ask on IRC but my time zone is just not very #twisted friendly so I got zero replies. My question can be simplified to: Closures yes or no? They seem *really* handy since they allow to have some data present without handing it through all the time. Eg in my cred checker, I can refer to the user name from the closure instead of passing it around all the time – making the code much cleaner. Also most current examples and callbacks-are-hard-FUD-rebuttals seem to use them. OTOH, private methods like `_cbPrintResult` are nicer to test individually. If “yes, closures”: Still using cb/eb prefixes? I don’t see them very often in recent examples. What about addBoth handlers? Bonus points for the preferred style for the Twisted code base. Thanks in advance thank you for your time, Hynek [1]: I’d be glad if people from the dinner replied again too so I/we have it in written form, my memory tends to be rather sloppy. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twisted.cred interface deficiences
Why does IUsernameHashedPassword have to be renamed? It sounds like it's the appropriate thing already. On Mon, Apr 1, 2013 at 10:55 PM, Shell cam.t...@gmail.com wrote: The twisted.cred.IUsernamePassword interface declares: * IUsernamePassword.username - The username associated with these credentials. * IUsernamePassword.password - The password associated with these credentials. * IUsernamePassword.checkPassword(password) - Validate these credentials against the correct password. The issue is that the interface (according to exarkun) allows you to implement checkPassword() to do things other than the obvious password == self.password. Now, this is an issue because Twisted then explicitly supports (again, according to exarkun) two different uses of this interface by the credentials checker: * Call the checkPassword() method, passing it the correct password * Just take the password out and do whatever you want with it (which is necessary in any secure system) Now, imagine I write a version of checkPassword() in a library which does something security-centric (what would this be? shouldn't it be part of the checker?), assuming that it'll be used by a credentials checker which calls checkPassword(). Except... then, a library user uses it with a credentials checker which checks the password itself. Now they're skipping over my security-centric code! So I have to tell my library users that they have to use my library with a credentials checker which makes sure to call checkPassword(), not just one which accepts the correct interface. IUsernamePassword's docstring claims that the stored password must be reversible to plaintext to be compared with the password, which implies that taking the password out and doing other things is incorrect, unlike what exarkun suggests. In this case, exposing password in the interface makes little sense. In addition, twisted.cred.checkers.FilePasswordDB apparently ignores this docstring entirely already ( http://twistedmatrix.com/trac/browser/tags/releases/twisted-12.3.0/twisted/cred/checkers.py#L238 ). I propose that IUsernamePassword should be split into at least two interfaces: * IUsernamePassword, with only username and password, no methods, which allows password to be used in any way * Another interface, which only defines username and checkPassword() - possibly just a rename of IUsernameHashedPassword, which declares a similar interface However, this has the issue that any credential checker which can use the second interface would also be able to use an IUsernamePassword here if there were an adapter between the two, but support for this would have to go into every credential checker which supports the second interface at present. Maybe the Portal could automatically search for adapters if it can't find a direct match? Thanks, Cameron ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] is twisted compatible with pickle?
While everything the people above me have said is correct, I would suggest
AMP as an alternative. It would allow you to send most basic Python data
types in your messages.
On Mar 29, 2013 2:32 AM, succer...@tiscali.it succer...@tiscali.it
wrote:
I have made 2 application:
The client extract data from a sql server (10k lines), and send every line
pickled to a collector server via socket.
The server uses twisted and receive every line, unpikle it and store the
data in another sql server.
Everytime i start sending data from client to server, in the first 200
line (everytime a different line) **the server** throws an exception:
SOMETIMES it something like:
Traceback (most recent call last):
File collector2.py, line 81, in dataReceived
self.count,account = pickle.loads(data)
File /usr/lib/python2.6/pickle.py, line 1374, in loads
return Unpickler(file).load()
File /usr/lib/python2.6/pickle.py, line 858, in load
dispatch[key](self)
File /usr/lib/python2.6/pickle.py, line 1138, in load_pop
del self.stack[-1]
IndexError: list assignment index out of range
But it's NOT every time the same. Printing my exception i red:
Exception: pop from empty list
Exception: list index out of range
Exception: '
Exception: list assignment index out of range
Another strange errors is:
File /usr/lib/python2.6/pickle.py, line 1124, in find_class
__import__(module)
exceptions.ImportError: No module named ond'
for i in listaSAI:
crm={}
try:
crm['uid']=i[0]
except:
crm['uid']=None
try:
crm['type_cond']=i[01]
except:
crm['type_cond']=None
try:
crm['population_id']=i[2]
except:
crm['population_id']=None
try:
crm['product_id']=i[3]
except:
crm['product_id']=None
try:
crm['a_id']=i[4]
except:
crm['a_id']=None
try:
crm['status']=i[5]
except:
crm['status']=None
#time.sleep(0.001)
serialized = pickle.dumps((count,crm))
#print sent num, count, crm
s.sendall(serialized)
count += 1
And my server:
def dataReceived(self, data):
try:
self.count,account = pickle.loads(data)
except Exception as e:
print Eccezione:, e
print self.count+1
print data
print traceback.print_exc()
Printing the data in my client tells me that everything it's ok.
*If i try to slow down the process of sending using time.sleep(0.01) in my
client, EVERYTHING IS FINE, and no exception are raised.*
What can i do to debug my code?
p.s.
I suspect that exceptions.ImportError: No module named ond' refers to
type_cond key in crm.
Invita i tuoi amici e Tiscali ti premia! Il consiglio di un amico vale più
di uno spot in TV. Per ogni nuovo abbonato 30 € di premio per te e per lui!
Un amico al mese e parli e navighi sempre gratis:
http://freelosophy.tiscali.it/
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted dinner pictures
I think that last year's GSoC student came in not knowing any Twisted and was a speaker this year that picked up a release manager hat just to get me to merge my 5 year old branch is a testament to: - open source is awesome - sprints are awesome - pycon is awesome (even if my phone insists on correcting it to toxin) - ashfall is awesome On Mar 25, 2013 7:08 PM, Hynek Schlawack h...@ox.cx wrote: Hi, As you may have noticed, I'm building a little page to show off that PyCon wasn't about donglegate. http://thisispycon.com I would love to post about our little dinner and would be delighted if anyone having pics AND quotes could forward them to me. Same goes for sprints etc, flood me! Cheers, Hynek Sent from my phone. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted dinner pictures
Yep: https://farm9.staticflickr.com/8391/8589643930_67282142ac_h.jpg https://farm9.staticflickr.com/8099/8588545611_3001043027_h.jpg Sorry for not being coherent: codeine-containing cough drugs + jetlag == bad On Tue, Mar 26, 2013 at 4:42 PM, Hynek Schlawack h...@ox.cx wrote: Great, now make it coherent and quotable please. :) And the others: please help out too! I won’t name people I know about having pictures because they may not want me to share them with the world – but know that I’m sad. ;) P.S. Does anyone have a picture of “Go Ashwini”? Am 26.03.2013 um 16:11 schrieb Laurens Van Houtven _...@lvh.cc: I think that last year's GSoC student came in not knowing any Twisted and was a speaker this year that picked up a release manager hat just to get me to merge my 5 year old branch is a testament to: - open source is awesome - sprints are awesome - pycon is awesome (even if my phone insists on correcting it to toxin) - ashfall is awesome On Mar 25, 2013 7:08 PM, Hynek Schlawack h...@ox.cx wrote: Hi, As you may have noticed, I'm building a little page to show off that PyCon wasn't about donglegate. http://thisispycon.com I would love to post about our little dinner and would be delighted if anyone having pics AND quotes could forward them to me. Same goes for sprints etc, flood me! Cheers, Hynek Sent from my phone. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] TODAY @ 4PM in Rm 202: BoF/Meetup session at PyCon USA 2013
I won't be able to make it :( My talk is from 4:15 PM onwards. On Sat, Mar 16, 2013 at 9:36 AM, Bradley M. Kuhn bk...@sfconservancy.orgwrote: I've set up a Conservancy BoF/Meetup session today at 16:00 (4PM) in Room 202 (which is on floor 2A, you can find it on the map). The meeting will last one hour, and is directly after the PyPy BoF session (and in the same room). Agenda for the earlier 4PM discussion is open, but can include: * Questions/concerns that existing Conservancy projects has. * Questions/discussion from existing Conservancy applicants or projects considering offers for membership. * A chance for different Representatives / Project Leadership Committess / contractors from different Conservancy projects to meet each other! I look forward to seeing you there! -- Bradley M. Kuhn, Executive Director, Software Freedom Conservancy ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted BoF session
Nah; I mean, you could, but it'd only be for advertisement reasons, but I'm guessing all the usual suspects will show up anyway. It doesn't help me much in terms of admin, I just picked a room that looks like it'd be big enough for the amount of people I'd expect :) On Fri, Mar 15, 2013 at 12:17 AM, Glyph gl...@twistedmatrix.com wrote: On Mar 14, 2013, at 3:56 PM, Laurens Van Houtven _...@lvh.cc wrote: I'm organizing a Twisted BoF session/open space. I've tried to get a combination of a) nice room b) no collisions with anyone's talks. I have it down for 7-9 in Great America Meeting Room 2 (that's next to the room where Itamar's tutorial was). I'm in charge of open spaces, so I can probably cater to any objections I get before late tonight. Is there anywhere we should sign up? ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Twisted BoF session
I'm organizing a Twisted BoF session/open space. I've tried to get a combination of a) nice room b) no collisions with anyone's talks. I have it down for 7-9 in Great America Meeting Room 2 (that's next to the room where Itamar's tutorial was). I'm in charge of open spaces, so I can probably cater to any objections I get before late tonight. -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted BoF session
By the way, this is *Friday* On Thursday, March 14, 2013, Laurens Van Houtven _...@lvh.cc wrote: I'm organizing a Twisted BoF session/open space. I've tried to get a combination of a) nice room b) no collisions with anyone's talks. I have it down for 7-9 in Great America Meeting Room 2 (that's next to the room where Itamar's tutorial was). I'm in charge of open spaces, so I can probably cater to any objections I get before late tonight. -- cheers lvh -- Sent from Gmail Mobile ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] How do I write AMP Command definitions TTD-edly?
Awesome, thanks! On Mar 6, 2013 5:18 PM, exar...@twistedmatrix.com wrote: On 5 Mar, 09:46 am, _...@lvh.cc wrote: That looks awesome. I can't find a license for it. The code was already MIT licensed. It's more redundantly so, now. Jean-Paul I'm publishing some third-party helpers for twisted's amp implementation as I need them myself (presumably I'll make a formal release announcement at the Pycon sprints). Do you mind if I copy this code? I can't seem to find a license. (txampext is ISC-licensed, which I assume is congruent to your sensibilities :)) On Mon, Mar 4, 2013 at 5:17 PM, exar...@twistedmatrix.com wrote: On 03:34 pm, _...@lvh.cc wrote: Hi, I have no issue writing AMP responders TTD-edly, they work pretty much the way any other code would. However, I did notice that the only way I'd notice if my Command definitions themselves were broken were if my functional tests start failing. Chris and I wrote all the AMP-using code for game TDD. See game/test/test_network.py in lp:game. Jean-Paul It would be nice if I could write unit test driven Command specifications, as well. The quickest I could come up with was to call makeArguments/makeResponse with a few known-good/known-bad dictionaries, and see if it blows up. Is there a smarter way to do it? -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
