We're using the HttpAsyncClient to make calls from within our own server application. Periodically (once a week or less), our service stops being able to make requests and we see errors of the form:
java.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED at org.apache.http.util.Asserts.check(Asserts.java:46) at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90) at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123) ... Digging further into the logs, it seems these begin after we see an exception that I've included at the bottom but basically seems to boil down to "I/O dispatch worker terminated abnormally" caused by "IOException: Connection reset by peer" in the depths of SSLIOSession. I guess I've got two questions: first, is this a bug? I'm not sure how much additional info I can provide into the root cause of the IOException since it happens so infrequently, but I'll try to provide what I can if it's relevant. Second, how should we be handling this issue? Some of the Apache documentation and older mailing list posts talk about "restarting the I/O Reactor" but I'm not clear what means in this context since I'm not interacting with it directly. Should I just be throwing away my instance of HttpAsyncClient and creating a new one? We're using: httpclient 4.5.2 httpasyncclient 4.1.1 httpcore.nio4.4.4 John. org.apache.http.nio.reactor.IOReactorException: I/O dispatch worker terminated abnormally at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:358) at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:191) at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] Caused by: rx.exceptions.OnErrorNotImplementedException: Connection reset by peer at rx.Observable$27.onError(Observable.java:7923) at rx.observers.SafeSubscriber._onError(SafeSubscriber.java:159) at rx.observers.SafeSubscriber.onError(SafeSubscriber.java:120) at rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:71) at OurCode.SessionManager.lambda$null$10(SessionManager.java:174) at rx.Observable$10.onError(Observable.java:4407) at rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:65) at rx.internal.operators.OperatorMap$1.onError(OperatorMap.java:48) at rx.internal.operators.OperatorMerge$MergeSubscriber.reportError(OperatorMerge.java:240) at rx.internal.operators.OperatorMerge$MergeSubscriber.checkTerminate(OperatorMerge.java:776) at rx.internal.operators.OperatorMerge$MergeSubscriber.emitLoop(OperatorMerge.java:537) at rx.internal.operators.OperatorMerge$MergeSubscriber.emit(OperatorMerge.java:526) at rx.internal.operators.OperatorMerge$MergeSubscriber.onError(OperatorMerge.java:250) at rx.internal.operators.OperatorMap$1.onError(OperatorMap.java:48) at rx.internal.operators.OperatorDoOnEach$1.onError(OperatorDoOnEach.java:71) at rx.apache.http.ObservableHttp$2$1.failed(ObservableHttp.java:195) at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:134) at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:419) at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.exception(HttpAsyncRequestExecutor.java:152) at org.apache.http.impl.nio.client.InternalIODispatch.onException(InternalIODispatch.java:76) at org.apache.http.impl.nio.client.InternalIODispatch.onException(InternalIODispatch.java:39) at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:127) at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:164) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:339) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:317) at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:278) at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106) at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:590) Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.8.0_65] at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.8.0_65] at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[na:1.8.0_65] at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[na:1.8.0_65] at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[na:1.8.0_65] at org.apache.http.nio.reactor.ssl.SSLIOSession.receiveEncryptedData(SSLIOSession.java:449) ~[antelope-server.jar:na] at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:503) ~[antelope-server.jar:na] at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:122) ~[antelope-server.jar:na] ... 7 common frames omitted
