Re: [Twisted-Python] Multi-reactor architecture

2009-11-12 Thread Kevin Horn
On Thu, Nov 12, 2009 at 12:53 PM, wrote: > > You right, distributed systems architecture. > > Probably I need to rephrase my question more precisely: how to build > distributed system architecture with Twisted technology only ? > > As you mentioned, even reverse proxy could be a another twisted p

Re: [Twisted-Python] Multi-reactor architecture

2009-11-12 Thread vitaly
You right, distributed systems architecture. Probably I need to rephrase my question more precisely: how to build distributed system architecture with Twisted technology only ? As you mentioned, even reverse proxy could be a another twisted process. Quoting "Reza Lotun" : >> How reverse prox

Re: [Twisted-Python] Multi-reactor architecture

2009-11-12 Thread Reza Lotun
> How reverse proxy knows to balance between couple of twistd processes? > Do you have may be an example(or doc link) of such proxy that can > distributing connections to each twistd process? This is a classic distributed systems architecture. A reverse proxy can either something like haproxy, ngi

Re: [Twisted-Python] Amount of concurrent connections

2009-11-12 Thread Bokhan Artem
vit...@synapticvision.com пишет: > Sorry, forgot to mention, per second. > Amount of concurrent connections per second. Just for example, in my tests standard twisted http server can do about 1k dummy req/resp per second on one modern cpu core. > > Quoting "Eero Nevalainen" : > >> vit...@syna

Re: [Twisted-Python] Multi-reactor architecture

2009-11-12 Thread vitaly
How reverse proxy knows to balance between couple of twistd processes? Do you have may be an example(or doc link) of such proxy that can distributing connections to each twistd process? Quoting "Reza Lotun" : >> hi, >> what would be the right thing to start from in order to build >> multi-rea

Re: [Twisted-Python] Multi-reactor architecture

2009-11-12 Thread Reza Lotun
> hi, > what would be the right thing to start from in order to build > multi-reactor arch to handle thousands of concurrent connections? Why would you want multiple reactors? The only reason would be to have one per CPU core. For that the simplest thing would be to manually start n twistd process

Re: [Twisted-Python] Amount of concurrent connections

2009-11-12 Thread vitaly
Of course its also depend on frequency of r/r, but also on size of data. Or am I missing something? Quoting "Bokhan Artem" : > vit...@synapticvision.com пишет: >> hi, >> what would be expected amount(+-) of concurrent connections > > Why this should depend on size of responses/requests? > >

Re: [Twisted-Python] Amount of concurrent connections

2009-11-12 Thread vitaly
Sorry, forgot to mention, per second. Amount of concurrent connections per second. Quoting "Eero Nevalainen" : > vit...@synapticvision.com wrote: >> hi, >> what would be expected amount(+-) of concurrent connections for >> twisted reactor if single request to gateway eq 1Kb and response >

Re: [Twisted-Python] Amount of concurrent connections

2009-11-12 Thread Bokhan Artem
vit...@synapticvision.com пишет: > hi, > what would be expected amount(+-) of concurrent connections Why this should depend on size of responses/requests? for > twisted reactor if single request to gateway eq 1Kb and response back > 30Kb (I mean response data much more bigger than request)? (

[Twisted-Python] Multi-reactor architecture

2009-11-12 Thread vitaly
hi, what would be the right thing to start from in order to build multi-reactor arch to handle thousands of concurrent connections? Appreciate the help. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin

[Twisted-Python] Amount of concurrent connections

2009-11-12 Thread vitaly
hi, what would be expected amount(+-) of concurrent connections for twisted reactor if single request to gateway eq 1Kb and response back 30Kb (I mean response data much more bigger than request)? (1Gb network card on twisted machine) Appreciate the help. __

Re: [Twisted-Python] Proposal (almost) to switch from Lore to Sphinx

2009-11-12 Thread Kevin Horn
On Wed, Nov 11, 2009 at 11:22 PM, Glyph Lefkowitz wrote: Excellent. In that case (modulo several weeks of work on your part), I'm > sold. > Hooray! > Hmmm...I hadn't considered the Divmod stuff, but you're right. > > > I'm glad you think so. I do tend to be, especially about Twisted ;-). > >

Re: [Twisted-Python] Proposal (almost) to switch from Lore to Sphinx

2009-11-12 Thread Kevin Horn
On Thu, Nov 12, 2009 at 7:01 AM, wrote: > On 05:22 am, gl...@twistedmatrix.com wrote: > >On Nov 11, 2009, at 7:37 PM, Kevin Horn wrote: > > > >You're correct that lore cannot include partial external files. > > Actually, it has limited support for this. The PB documentation relies > on it (searc

[Twisted-Python] Thrift/Twisted reconnecting client .. code review?

2009-11-12 Thread David Koblas
I just got done putting together a reconnecting client (with queuing) for twisted. While it appears to work for me in practice, if anybody has a few minutes to do a quick code review that would be really handy. This is based on another thrift client, but in the end totally gutted and now hope

Re: [Twisted-Python] Using sqlalchemy in twisted.

2009-11-12 Thread Phil Mayers
> Is there a reason why multiprocessing could not be used? > > http://docs.python.org/library/multiprocessing.html#exchanging-objects-between-processes > Yes http://twistedmatrix.com/trac/ticket/3901 ___ Twisted-Python mailing list Twisted-Python@tw

Re: [Twisted-Python] Using sqlalchemy in twisted.

2009-11-12 Thread Eero Nevalainen
Peter Cai wrote: > If you Google "sqlalchemy twisted", you can see my post is on the top: > > http://groups.google.com/group/sqlalchemy/browse_thread/thread/c802749c9d0dd6 > > I've been busy for a long time so I didn't considered this question > anymore. But occasionally, I got some new ideas to

Re: [Twisted-Python] Proposal (almost) to switch from Lore to Sphinx

2009-11-12 Thread exarkun
On 05:22 am, gl...@twistedmatrix.com wrote: >On Nov 11, 2009, at 7:37 PM, Kevin Horn wrote: > >You're correct that lore cannot include partial external files. Actually, it has limited support for this. The PB documentation relies on it (search for "skipLines" in doc/core/howto). And if it were