I'm glad that we've all concluded on the need of another queuing layer. I'm not sure if this extra layer should be injected into the thrift framework or should be kept outside the framework as a mean of of the service logic.
Best Regards, Utku On Thu, Feb 11, 2010 at 9:15 PM, Ted Dunning <[email protected]> wrote: > It sounds like you need an extra queuing layer. The thrift call can insert > the request into the queue (and thereby close the socket). It should be > possible to insert 500 transactions per second into a large queue with no > difficulty at all. > > The real problem here is not the late close. It is the fact that you can't > handle your peak volume without a huge backlog. If you design a layer that > can absorb the backlog, then the problem will go away. You still have the > problem that if your peak is 4x higher than the rate that you can process, > then you are probably getting close to the situation where you can't keep > up > with your average load either. > > On Thu, Feb 11, 2010 at 11:10 AM, Utku Can Topçu <[email protected]> > wrote: > > > Ted, > > > > I sure agree with you, however having 500*x more threads for handling the > > concurrency, will eventually set the processing time for one request from > 4 > > seconds to 4*x seconds. > > At the end of the day the open connection count will be the same ;) > > > > Best, > > Utku > > > > On Thu, Feb 11, 2010 at 9:06 PM, Ted Dunning <[email protected]> > > wrote: > > > > > This is the key phrase that I was looking at. If you only allow 500x > > > concurrency and you are getting 500 transactions per second, then you > are > > > going to back up pretty seriously. > > > > > > On Thu, Feb 11, 2010 at 10:58 AM, Utku Can Topçu <[email protected]> > > > wrote: > > > > > > > > > time. Say each request needs 4 seconds to complete in 500 > > > concurrency. > > > > > > > > > > > > > > > -- > > > Ted Dunning, CTO > > > DeepDyve > > > > > > > > > -- > Ted Dunning, CTO > DeepDyve >
