On Thu, Oct 5, 2017 at 7:11 PM, Eirik Lykken <eiriklyk...@protonmail.ch> wrote:
> > 05-Oct-2017 17:13:32.535 SEVERE > [https-openssl-nio2-94.229.64.230-443-exec-19701] > org.apache.coyote.AbstractProtocol$ConnectionHandler.process Error > reading request, ignored > java.lang.IllegalStateException > at org.apache.coyote.http2.Http2UpgradeHandler.fill(Http2Upgrad > eHandler.java:1314) > at org.apache.coyote.http2.Http2UpgradeHandler.fill(Http2Upgrad > eHandler.java:1290) > at org.apache.coyote.http2.Http2Parser.readConnectionPreface(Ht > tp2Parser.java:574) > at org.apache.coyote.http2.Http2UpgradeHandler.init(Http2Upgrad > eHandler.java:243) > at org.apache.coyote.http2.Http2UpgradeHandler.upgradeDispatch( > Http2UpgradeHandler.java:310) > at org.apache.coyote.http11.upgrade.UpgradeProcessorInternal.di > spatch(UpgradeProcessorInternal.java:54) > at org.apache.coyote.AbstractProcessorLight.process(AbstractPro > cessorLight.java:53) > at org.apache.coyote.AbstractProtocol$ConnectionHandler.process > (AbstractProtocol.java:868) > at org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.doRu > n(Nio2Endpoint.java:1693) > at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketPro > cessorBase.java:49) > at org.apache.tomcat.util.net.AbstractEndpoint.processSocket(Ab > stractEndpoint.java:946) > at org.apache.tomcat.util.net.SecureNio2Channel$HandshakeWriteC > ompletionHandler.completed(SecureNio2Channel.java:115) > at org.apache.tomcat.util.net.SecureNio2Channel$HandshakeWriteC > ompletionHandler.completed(SecureNio2Channel.java:108) > at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) > at sun.nio.ch.Invoker.invokeDirect(Invoker.java:157) > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.implWrite(UnixA > synchronousSocketChannelImpl.java:736) > at sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousS > ocketChannelImpl.java:382) > at sun.nio.ch.AsynchronousSocketChannelImpl.write(AsynchronousS > ocketChannelImpl.java:399) > at java.nio.channels.AsynchronousSocketChannel.write(Asynchrono > usSocketChannel.java:577) > at org.apache.tomcat.util.net.SecureNio2Channel.handshakeIntern > al(SecureNio2Channel.java:273) > at org.apache.tomcat.util.net.SecureNio2Channel.handshake(Secur > eNio2Channel.java:204) > at org.apache.tomcat.util.net.Nio2Endpoint$SocketProcessor.doRu > n(Nio2Endpoint.java:1671) > at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketPro > cessorBase.java:49) > at org.apache.tomcat.util.net.AbstractEndpoint.processSocket(Ab > stractEndpoint.java:946) > at org.apache.tomcat.util.net.SecureNio2Channel$HandshakeReadCo > mpletionHandler.completed(SecureNio2Channel.java:98) > at org.apache.tomcat.util.net.SecureNio2Channel$HandshakeReadCo > mpletionHandler.completed(SecureNio2Channel.java:91) > at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126) > at sun.nio.ch.Invoker$2.run(Invoker.java:218) > at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousCh > annelGroupImpl.java:112) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool > Executor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo > lExecutor.java:624) > at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.r > un(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:748) > I think the state of the handshake is most likely ok despite the weird looking stack. The code doing blocking SSL reads is complex also, it would be returning 0 bytes for a blocking read in this case. To examine the issue it would need to be reproduced. Although it might simply be an IO error that doesn't get reported as it should and you see that stack instead (but the result is then the same). > > This is our current Connector configuration: > > <Connector port="443" address="xxx" > protocol="org.apache.coyote.http11.Http11Nio2Protocol" > SSLEnabled="true" > maxThreads="1500" acceptCount="5000" scheme="https" secure="true" > defaultSSLHostConfigName="xxx" > hostName="xxx" > connectionTimeout="70000" > URIEncoding="UTF-8" > server="Apache" > enableLookups="false" > compression="on" compressionMinSize="2048" > compressableMimeType="text/html,text/xml,text/csv,text/css,t > ext/javascript,text/html,text/plain,application/javascript,a > pplication/x-javascript" > sslImplementationName="org.apache.tomcat.util.net.openssl.Op > enSSLImplementation" > > ><UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" > readTimeout="70000" writeTimeout="70000" maxConcurrentStreams="400" > maxConcurrentStreamExecution="400"/> > maxConcurrentStreamExecution="400" that's waaaaay too much, don't do that. > We have experimented with different parameters of the Http2Protocol > connector to improve performance and control the leak with > Http11NioProtocol connector but the exceptions are there with any of the > values or removing the extra parameters. > > We also tried Tomcat/9.0.0.M26 and the exception we got was: > > 22-Sep-2017 19:16:26.872 SEVERE > [https-openssl-nio2-94.229.64.230-443-exec-229] > org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() > for servlet [default] in context with path [] threw exception > java.lang.NullPointerException > at org.apache.coyote.http2.HpackEncoder$TableEntry.access$100(H > packEncoder.java:339) > at org.apache.coyote.http2.HpackEncoder.findInTable(HpackEncode > r.java:297) > at org.apache.coyote.http2.HpackEncoder.encode(HpackEncoder.jav > a:150) > at org.apache.coyote.http2.Http2UpgradeHandler.doWriteHeaders(H > ttp2UpgradeHandler.java:573) > at org.apache.coyote.http2.Http2AsyncUpgradeHandler.writeHeader > s(Http2AsyncUpgradeHandler.java:138) > at org.apache.coyote.http2.Stream.writeHeaders(Stream.java:411) > at org.apache.coyote.http2.StreamProcessor.prepareResponse(Stre > amProcessor.java:100) > ... > > I don't understand how this stack can happen. Rémy
