There were definitely bugs with AbstractSelector & SelectionKeys that could cause deadlocks to happen in rare scenarios. See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4744057 for one. Newer versions of the JDK have fixed this. There were also problems on OS X if using the default selector (kqueue) that caused the entire system to hang. I'm not positive if newer releases have fixed this, as we switched to not using kqueue (-Djava.nio.preferSelect=true).
Sam On 1/23/08, Oleg Kalnichevski <[EMAIL PROTECTED]> wrote: > > On Tue, 2008-01-22 at 22:23 -0800, David Koski wrote: > > Hi, > > > > Sadly, I can't claim I have personal knowledge of this being true. I > > have an application where I can get a reactor thread to hang like that > > and this description seemed to fit. I have been unable to make a unit > > test that reproduces this behavior however. > > > > I am using: > > > > java version "1.5.0_06" > > Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-113) > > Java HotSpot(TM) Server VM (build 1.5.0_06-68, mixed mode) > > > > Here is another reference: > > > > http://forum.java.sun.com/thread.jspa?threadID=459338&start=120 > > > > "register and interestOps will block if they are done from another > > thread while the select is active. That's why I use a list of things > > that must be done in the select loop." > > > > And I read this: > > > > > > http://java.sun.com/j2se/1.5.0/docs/api/java/nio/channels/SelectionKey.html > > > > Selection keys are safe for use by multiple concurrent threads. **The > > operations of reading and writing the interest set will, in general, > > be synchronized with certain operations of the selector. Exactly how > > this synchronization is performed is implementation-dependent: In a > > naive implementation, reading or writing the interest set may block > > indefinitely if a selection operation is already in progress**; in a > > high-performance implementation, reading or writing the interest set > > may block briefly, if at all. In any case, a selection operation will > > always use the interest-set value that was current at the moment that > > the operation began. > > > > The key here is the word 'naive'. I have a hard time believing Sun chose > a naive implementation for its JREs. Inability to modify interest > operations on the selection key from a different thread _utterly_ > defeats its purpose. > > NIO used to be (and still is) quite buggy. So, before I start working on > an ugly workaround I would like to be reasonably sure this is not a bug > in Sun's older JREs. I would also like to know to what extent different > platforms are affected. The JRE you are using is fairly out of date. > Could you please upgrade to the latest Java 1.5 release (1.5.0_14) and > see if the problem is still reproducible? Have you tried Java 1.6? What > OS are you on? > > Oleg > > > To me, that suggests that select + modify of the interest ops == bad, > > which agrees with what the various posts say. Again, I have no nio > > expertise, so my idea may be bunk. > > > > David Koski > > > > > > On Jan 22, 2008, at 2:22 PM, Oleg Kalnichevski wrote: > > > > > > > > On Tue, 2008-01-22 at 16:52 -0500, [EMAIL PROTECTED] wrote: > > >> Hi, > > >> > > >> Your suggestion worked well, but I think I ran into a problem with > > >> it (using A6, if it matters). I ended up with a hung IOReactor > > >> thread: > > >> > > >> Thread [I/O dispatcher 2] (Suspended) > > >> KQueueArrayWrapper.register0(int, int, int, int) line: not > > >> available [native method] > > >> KQueueArrayWrapper.setInterest(int, int) line: 99 > > >> KQueueSelectorImpl.putEventOps(SelectionKeyImpl, int) line: 179 > > >> SocketChannelImpl.translateAndSetInterestOps(int, > > >> SelectionKeyImpl) line: 733 > > >> SelectionKeyImpl.nioInterestOps(int) line: 87 > > >> SelectionKeyImpl.interestOps(int) line: 65 > > >> IOSessionImpl.clearEvent(int) line: 125 > > >> AsyncHTTPClient > > >> $ > > >> AsyncConnection > > >> (DefaultNHttpClientConnection).produceOutput(NHttpClientHandler) > > >> line: 183 > > >> AsyncHTTPClient > > >> $EventDispatch(DefaultClientIOEventDispatch).outputReady(IOSession) > > >> line: 102 > > >> AsyncHTTPClient$EventDispatch.outputReady(IOSession) line: 353 > > >> BaseIOReactor.writable(SelectionKey) line: 109 > > >> BaseIOReactor(AbstractIOReactor).processEvent(SelectionKey) line: > > >> 192 > > >> BaseIOReactor(AbstractIOReactor).processEvents(Set) line: 174 > > >> BaseIOReactor(AbstractIOReactor).execute() line: 137 > > >> BaseIOReactor.execute(IOEventDispatch) line: 69 > > >> AbstractMultiworkerIOReactor$Worker.run() line: 281 > > >> Thread.run() line: 613 > > >> > > >> The AsyncHTTPClient class is mine, but it is just very thin > > >> wrappers on the default implementations. Anyway, the thread hangs > > >> like this while trying to clear the write interest on the > > >> SelectionKey. I only seem to run into this at high (> 1000 per > > >> second) request rates. > > >> > > >> I think this may actually be related to waking up the the IOControl: > > >> > > >> conn.requestOutput(); > > >> > > >> this ends up here: > > >> > > >> > > >> public void setEvent(int op) { > > >> if (this.status == CLOSED) { > > >> return; > > >> } > > >> synchronized (this.key) { > > >> int ops = this.key.interestOps(); > > >> this.key.interestOps(ops | op); > > >> this.key.selector().wakeup(); > > >> } > > >> } > > >> > > >> However, several sites suggest that concurrent modification of > > >> SelectionKeys is a recipe for disaster: > > >> > > >> http://rox-xmlrpc.sourceforge.net/niotut/index.html > > >> > > >> As a result, if you plan to hang onto your sanity don't modify the > > >> selector from any thread other than the selecting thread. This > > >> includes modifying the interest ops set for a selection key, > > >> registering new channels with the selector, and cancelling existing > > >> channels. > > >> > > >> > > >> > > >> But that is _exactly_ what I am doing here. The thread that wants > > >> the connection is touching the SelectionKey and the IOReactor (I/O > > >> dispatcher) thread is also touching it. > > >> > > >> Am I understanding this correctly? > > >> > > > > > > David, > > > > > > The javadocs of the SelectonKey clearly states the class is threading > > > safe [1]: > > > > > > "... Selection keys are safe for use by multiple concurrent > > > threads..." > > > > > > One certainly should not attempt to access the selector from multiple > > > threads but it is completely new to me some people think this also > > > applies to the selection keys. Selection keys would be completely > > > pointless if they were not threading safe > > > > > > Can this be a JRE issue? What is the JRE you are using? > > > > > > Oleg > > > > > > [1] > > > http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/SelectionKey.html > > > > > > > > >> Thanks, > > >> David Koski > > >> > > >> > > >> -- Oleg Kalnichevski wrote : > > >> On Thu, 2007-07-12 at 14:10 -0700, David Koski wrote: > > >>> Hi, > > >>> > > >>> I have been reading NHttpClient and I think I finally understand how > > >>> it all works, but I am a bit stuck with how to use it. Let's say I > > >>> wanted to build something along the lines of a load balancer: many > > >>> incoming connections, many outgoing connections, most are idle or > > >>> waiting for a response. > > >>> > > >>> Focusing on the outgoing connections part, I want to have keep-alive > > >>> connections to a set of hosts, multiple connections per port. For > > >>> example: > > >>> > > >>> LB -> host1:80 > > >>> LB -> host1:80 > > >>> LB -> host1:80 > > >>> > > >>> LB -> host2:80 > > >>> LB -> host2:80 > > >>> > > >>> If I have a queue of operations I want to do, I can see how I might > > >>> use a series of ioReactor.connect() calls to create the connections > > >>> and have submitRequest() methods in my HttpRequestExecutionHandler > > >>> pull them off the queue and service them. > > >>> > > >>> The problem I am having is dealing with the steady state. I would > > >>> like to keep these connections around for a while (and indeed using > > >>> the DefaultConnectionReuseStrategy they are kept alive. However, > > >>> once > > >>> my queue drains and my submitRequest() method returns null, how do I > > >>> wake the handlers back up? I can see any way to get the reactor to > > >>> call back into my handler without opening a new connection. > > >>> > > >>> Am I missing something? Or going about this the wrong way? > > >>> > > >> > > >> Hi David > > >> > > >> Just invoke IOControl#requestOutput() (implemented by all NHttp > > >> connections) and it will cause the I/O reactor to fire up the > > >> NHttpClientHandler#requestReady() event, which you can use to > > >> submit a > > >> new request on that connection > > >> > > >> Hope this helps > > >> > > >> Oleg > > >> > > >>> Thanks, > > >>> David Koski > > >>> > > >>> --------------------------------------------------------------------- > > >>> To unsubscribe, e-mail: [EMAIL PROTECTED] > > >>> For additional commands, e-mail: [EMAIL PROTECTED] > > >>> > > >>> > > >> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: [EMAIL PROTECTED] > > >> For additional commands, e-mail: [EMAIL PROTECTED] > > >> > > >> > > >> -- > > >> This message was sent on behalf of [EMAIL PROTECTED] at > > >> openSubscriber.com > > >> http://www.opensubscriber.com/message/[EMAIL PROTECTED]/7128633.html > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: [EMAIL PROTECTED] > > >> For additional commands, e-mail: [EMAIL PROTECTED] > > >> > > >> > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
