[Twisted-Python] Is there a 'development mode ' for twisted ?
HI, there I'm new to twisted, after reading the documents from twisted websites, I got some questions: 1. Is there a 'development mode ' for twisted ? like django, you can see your changes without restarting server. 2. Can I use django and twisted together? actually, I want to use django's ORM inside twisted server, so that I can manipulate data easily. I've try 'sob.py', but I can not figure it out how to use it . On the other hand, django is synchronous, twisted is asynchronous, how to use them together if we can ? 3. How to suppress logs that I do not need ? there are many logs in twisted.log file, as shown in the follow: 2013-09-05 07:04:07+0800 [SSHChannel session (0) on SSHService ssh-connection on ClientCommandTransport,client] remote eof 2013-09-05 07:04:07+0800 [SSHChannel session (0) on SSHService ssh-connection on ClientCommandTransport,client] unhandled request for Any suggestions would be appreciated. Thanks. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Asynchronously reading posted data
On Sep 4, 2013, at 10:48 AM, Paul Wiseman wrote: > I'm still pretty new to twisted and feel I'm slowly getting the hang of it, > enough to realise that this line of code is probably going to block and not > do me any favours, the line is the first line in a resource.Resource > render_POST. > > json_request = json.loads(request.content.read()) > > The resource.Resource is a child of another resource which is passed to > server.Site which is passed to internent.TCPServer. > > The problem is I can't work out how I can read the post data from the request > in an async way. > > I have a feeling I need to implement a protocol, I'm guessing a LineReceiver > but I can't figure out how I'd tie that in with my current solution or > specifically how the LineReceiver would even read asynchronously to be > honest.. > > Maybe the read is fine? I need the whole post data to do anything useful I > guess as I can't string to a json decoder that I'm aware of. Just it will > block everything up while I read, which shouldn't be long but I guess I'm > bound to the speed of the person posting. > > Thanks all! If you're parsing a JSON object, you're going to be representing the whole thing in memory at the end of the interaction regardless, even if you parsed it and buffered it in an event-driven way. That means you need to keep this data relatively small no matter what; if it's arbitrarily large, you are going to start swapping anyway. So probably, just doing the blocking parse is fine. You might be able to save a *little* memory by parsing it as it comes in, but you're also going to have to write your own JSON parser, which is probably going to take more programmer time than you will ever save in execution time by this marginal memory reduction :-). Better would be to spend that effort enforcing stringent resource limits so that you will give up reading before you ever get to the parse in the case where it's big enough to cause a problem. -glyph ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Asynchronously reading posted data
I'm still pretty new to twisted and feel I'm slowly getting the hang of it, enough to realise that this line of code is probably going to block and not do me any favours, the line is the first line in a resource.Resource render_POST. json_request = json.loads(request.content.read()) The resource.Resource is a child of another resource which is passed to server.Site which is passed to internent.TCPServer. The problem is I can't work out how I can read the post data from the request in an async way. I have a feeling I need to implement a protocol, I'm guessing a LineReceiver but I can't figure out how I'd tie that in with my current solution or specifically how the LineReceiver would even read asynchronously to be honest.. Maybe the read is fine? I need the whole post data to do anything useful I guess as I can't string to a json decoder that I'm aware of. Just it will block everything up while I read, which shouldn't be long but I guess I'm bound to the speed of the person posting. Thanks all! Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Passing additional arguments to errback
Cześć Maciek :) In general, you can pass extra arguments when you call addCallback(s) or addErrback. They will get passed to the callback. However, as a side note to that code example, do you understand the difference between .addCallbacks(cb, eb) and: .addCallback(cb).addErrback(eb) and: .addErrback(eb).addCallback(cb) ... Also, keep in mind that you only errback when there is an issue setting up the connection. If the server successfully responds with an error (say, a 404 Not Found, or something), the callback will be called with the response object. So, your question doesn't make a lot of sense to me: if the errback gets called, there's not really a response! Also, if you want to do scraping with Twisted, consider looking at Scrapy, a fully-featured web scraper that uses Twisted internally. pozdrawiam lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Can't check elements without uri in twisted.words.xish.domish.Element
G'day, in the method "elements" of twisted.words.xish.domish.Element, the function "generateElementsQNamed" is called event if the uri if None, so the uri is checked against None instead of not checked at all. I think the function "generateElementsNamed" should be called instead, it looks like a bug for me. Can you confirm ? cheers Goffi ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Passing additional arguments to errback
Dear list,
I've found such a code example on "Stack Overflow" regarding errbacks:
___
class YourExample(object):
def your_example(self):
self.agent = Agent(reactor, pool=pool)
self.deferred = self.agent.request(
'GET',
self.url,
Headers({'User-Agent': ['Mozilla/5.0']})
)
self.deferred.addCallback(self.gotResponse).addErrback(self.gotBadResponse)
def gotBadResponse(self,raised):
"""you might have cleanup code here, or mark the url as bad in the
database, or something similar"""
pass
__
Normally only Failure object is passed to gotBadResponse() . I would
like to pass full response body to it - is it possible?
Do I have to encapsulate the response body inside Failure object?
Best regards
Maciek
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
