On Thu, Apr 23, 2009 at 11:38 AM, Debacker <[email protected]> wrote:
> Thanks for your answer. I should have specified that I would use > non-blocking asynchronous I/O for the initial read. > Hence, I can do an async read of max 4096 bytes on 1000 connections, put > them in a poll, and make a thread wait a completition event of any of the > 1000 connections. > Once the completition event is fired, the thread pick one of the ready > socket, and pass it to a worker thread. The worker thread will begin > reading > the beginning of the request, if it is not received completely, we can read > it synchronously until complete, then make the call to the interface, and > then write the response. After that, the worker thread give the connection > back to the master thread, and wait a new job from the master thread. The > objective here, is to support "sleeping" connections, which don't send any > request for some a few seconds or minutes. This, in turn, make it possible > to support connection pools at the client-side, to reduce useless > connection > creation/destruction, and the lag of it. > > With the current thread-pooled server, 1000 connections will use 1000 > threads, even if connection are only used for 10% of the time. With my > system, I would use between 100 and 1000 threads instead. > > Another possibility would be to have one (or few) thread(s) reading > requests > from all connections at the same time (using aync I/O and epoll), using one > state machine per connection. Once the state machine indicates that the > parsing of the request is complete, we can pass the request to a thread of > the thread poll, get the response, I may be mistaken, but I believe this is what the THsHaServer does. The incoming requests are handled asynchronously using select() and when a complete request is received (framing is required to detect this) it is handed off to a thread in the invoker pool. I believe the THsHaServer is actually very close to what you desire. Also, if you're running on a system with a modern thread implementation, having a high number of threads isn't as bad as it once was*. You may be better off starting with something that's easily available and optimizing if necessary. * http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-is-not-faster-than.html Cheers, Joel and pass the response to another thread > which will write all response using async I/O concurrently again. That way, > we do async I/O all the time, and ony the implementation will be blocking. > But I think it is not possible to implement this using the current > architecture, that is to say, using the current Protocol classes. And the > only advantage over my first proposition would be to more efficiency for > connection which are very slow to transmit a request, which is not that > important. > > Laurent Debacker. > > On Thu, Apr 23, 2009 at 7:29 PM, David Reiss <[email protected]> wrote: > > > > I have noticed your non blocking server based on > > > > > > http://java.sun.com/javase/6/docs/api/java/nio/channels/spi/SelectorProvider.html > > , > > > but you use the framed protocol to read the whole packet completely > > before > > > calling the implementation. But this is not needed, as I said in the > > first > > > paragraph, we could just read the first chunk of bytes (i.e. 4096) of > > each > > > connection. This would required one read buffer per connection, but it > is > > > not a problem since the objective would be to reuse the connections. > > > > You have to read the entire request before handing off to the processor > > code. This is because the processor will attempt to deserialize the > > entire request. If it is deserializing from a buffer, it will reach the > > end and throw an exception, wasting valuable CPU time reading a request > > that is not complete. If it is deserializing directly from the socket > > (or a TBufferedTransport on the socket), it will block until the rest of > > the message arrives, which makes that thread useless. Does this make > > sense? > > > > --David > > >
