On Mon, 2016-05-30 at 21:37 +0000, John Price wrote: > We're using the HttpAsyncClient to make calls from within our own server > application. Periodically (once a week or less), our service stops being > able to make requests and we see errors of the form: > > java.lang.IllegalStateException: Request cannot be executed; I/O reactor > status: STOPPED > at org.apache.http.util.Asserts.check(Asserts.java:46) > at > org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90) > at > org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123) > ... > > Digging further into the logs, it seems these begin after we see an > exception that I've included at the bottom but basically seems to boil down > to "I/O dispatch worker terminated abnormally" caused by "IOException: > Connection reset by peer" in the depths of SSLIOSession. > > I guess I've got two questions: first, is this a bug? I'm not sure how much > additional info I can provide into the root cause of the IOException since > it happens so infrequently, but I'll try to provide what I can if it's > relevant. > > Second, how should we be handling this issue? Some of the Apache > documentation and older mailing list posts talk about "restarting the I/O > Reactor" but I'm not clear what means in this context since I'm not > interacting with it directly. Should I just be throwing away my instance of > HttpAsyncClient and creating a new one? > > We're using: > httpclient 4.5.2 > httpasyncclient 4.1.1 > httpcore.nio4.4.4 > > John. > > org.apache.http.nio.reactor.IOReactorException: I/O dispatch worker > terminated abnormally > at > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:358) > > at > org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:191) > > at > org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] > Caused by: rx.exceptions.OnErrorNotImplementedException: Connection reset > by peer
The original 'java.io.IOException: Connection reset by peer' exception was re-thrown as OnErrorNotImplementedException from inside the FutureCallback#failed method, which killed the i/o dispatch thread and caused the i/o reactor to shut down. This issue needs to be addressed either in your application code or in the RX library. Hope this helps Oleg > at rx.Observable$27.onError(Observable.java:7923) > at rx.observers.SafeSubscriber._onError(SafeSubscriber.java:159) > at rx.observers.SafeSubscriber.onError(SafeSubscriber.java:120) > at > rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:71) > at OurCode.SessionManager.lambda$null$10(SessionManager.java:174) > at rx.Observable$10.onError(Observable.java:4407) > at > rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:65) > at rx.internal.operators.OperatorMap$1.onError(OperatorMap.java:48) > at > rx.internal.operators.OperatorMerge$MergeSubscriber.reportError(OperatorMerge.java:240) > at > rx.internal.operators.OperatorMerge$MergeSubscriber.checkTerminate(OperatorMerge.java:776) > at > rx.internal.operators.OperatorMerge$MergeSubscriber.emitLoop(OperatorMerge.java:537) > at > rx.internal.operators.OperatorMerge$MergeSubscriber.emit(OperatorMerge.java:526) > at > rx.internal.operators.OperatorMerge$MergeSubscriber.onError(OperatorMerge.java:250) > at rx.internal.operators.OperatorMap$1.onError(OperatorMap.java:48) > at > rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:71) > at rx.apache.http.ObservableHttp$2$1.failed(ObservableHttp.java:195) > at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:134) > at > org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:419) > > at > org.apache.http.nio.protocol.HttpAsyncRequestExecutor.exception(HttpAsyncRequestExecutor.java:152) > > at > org.apache.http.impl.nio.client.InternalIODispatch.onException(InternalIODispatch.java:76) > > at > org.apache.http.impl.nio.client.InternalIODispatch.onException(InternalIODispatch.java:39) > > at > org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:127) > > at > org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:164) > > at > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:339) > > at > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:317) > > at > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:278) > > at > org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106) > > at > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:590) > Caused by: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.8.0_65] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.8.0_65] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0_65] > at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.8.0_65] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) > ~[na:1.8.0_65] > at > org.apache.http.nio.reactor.ssl.SSLIOSession.receiveEncryptedData(SSLIOSession.java:449) > ~[antelope-server.jar:na] > at > org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:503) > ~[antelope-server.jar:na] > at > org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:122) > ~[antelope-server.jar:na] > ... 7 common frames omitted --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
