Re: [Twisted-Python] Twisted in a multicore environment

2012-07-12 Thread Ray Cote
Hi Augusto: You don't mention the protocol you're implementing in Twisted, but if is is HTTP... The way we address this problem with an HTTP protocol is to put multiple copies of Twisted behind nginx. On two and four core machines, we set the number of nginx worker processes equal to the nu

[Twisted-Python] How to best log server/client interactions from a test?

2012-07-12 Thread Dan Milstein
'lo all, I've been doing some work with the twisted.mail.imap4 module, and its associated unit tests (in order to learn the code more, I'm tackling a few of the outstanding tickets). When running those tests, I'd often like to log all the server/client interaction. I managed to do so by way o

Re: [Twisted-Python] Logging observers and threads

2012-07-12 Thread Tobias Oberstein
> libuv is a giant pile of new, unaudited C code that didn't even have a passing > test suite until a month or two ago. Please feel free to write a reactor for > it; >From what I've read, libuv has designed it's API around the pattern A. "do operation and get notified when done" instead of B.

Re: [Twisted-Python] Logging observers and threads

2012-07-12 Thread Glyph
Le Jul 12, 2012 à 6:47 AM, exar...@twistedmatrix.com a écrit : > I think it was a mistake to try to make it possible to use > twisted.python.log free-threaded. I think we should think about getting > rid of this claimed feature. Perhaps it could be replaced with a more > explicit, probably mo

Re: [Twisted-Python] Logging observers and threads

2012-07-12 Thread Glyph
Le Jul 12, 2012 à 8:26 AM, Tobias Oberstein a écrit : >> As far as I know, all of the implementations of POSIX AIO are relatively low >> quality. They include a number of tricky, low-level constraints (eg, char* >> to >> write must be page aligned) as well as some sad functional limitations s

Re: [Twisted-Python] A few simple questions

2012-07-12 Thread exarkun
On 04:02 pm, _...@lvh.cc wrote: >On Thursday, July 12, 2012, Russell E. Owen wrote: >>I'm adding Twisted support to some existing communication code and I >>have a few questions which I hope are simple. I've been reading the >>API >>documentation, but if the answers are there I'm missing them. >>

Re: [Twisted-Python] A few simple questions

2012-07-12 Thread Laurens Van Houtven
On Thursday, July 12, 2012, Russell E. Owen wrote: > I'm adding Twisted support to some existing communication code and I > have a few questions which I hope are simple. I've been reading the API > documentation, but if the answers are there I'm missing them. > > What happens if one tries to write

[Twisted-Python] A few simple questions

2012-07-12 Thread Russell E. Owen
I'm adding Twisted support to some existing communication code and I have a few questions which I hope are simple. I've been reading the API documentation, but if the answers are there I'm missing them. What happens if one tries to write to a protocol.transport that is not connected or in an er

Re: [Twisted-Python] Logging observers and threads

2012-07-12 Thread Tobias Oberstein
> As far as I know, all of the implementations of POSIX AIO are relatively low > quality. They include a number of tricky, low-level constraints (eg, char* to > write must be page aligned) as well as some sad functional limitations such as > blocking instead of being asynchronous if too many AIOs

Re: [Twisted-Python] Logging observers and threads

2012-07-12 Thread Justin Venus
For logging could we look at implementing posix aio on platforms that support it? I would be willing to help code that. Justin On Jul 12, 2012 8:51 AM, wrote: > On 11:32 am, ita...@itamarst.org wrote: > >On 07/12/2012 05:17 AM, Tristan Seligmann wrote: > >>3) log observers should expect to be i

Re: [Twisted-Python] Logging observers and threads

2012-07-12 Thread exarkun
On 11:32 am, ita...@itamarst.org wrote: >On 07/12/2012 05:17 AM, Tristan Seligmann wrote: >>3) log observers should expect to be inovked in non-reactor threads, >>making this a Mantissa bug. >In theory this is the case - the logging howto says "The observer needs >to be thread safe if you anticipat

Re: [Twisted-Python] Twisted in a multicore environment

2012-07-12 Thread Corbin Simpson
On Thu, Jul 12, 2012 at 08:10:45AM +0200, Vincent Bernat wrote: > Is there something similar to Ampoule but allowing to spawn processes > across different servers? Something that would provide > "deferToRemoteAMPProcess". AFAIK, not exactly. You can pretty easily craft this yourself, though. The

Re: [Twisted-Python] Logging observers and threads

2012-07-12 Thread Itamar Turner-Trauring
On 07/12/2012 05:17 AM, Tristan Seligmann wrote: > 3) log observers should expect to be inovked in non-reactor threads, > making this a Mantissa bug. > In theory this is the case - the logging howto says "The observer needs to be thread safe if you anticipate using threads in your program." The A

[Twisted-Python] Logging observers and threads

2012-07-12 Thread Tristan Seligmann
When an error occurs in a database connection thread, adbapi invokes log.err from that thread, which in turns causes log observers to be invoked in this thread. Mantissa's TracebackCollector has a log observer which logs items as Items in an Axiom store; this means that the log observer tries to ru