Hi,
So I have found a long term solution to our crash problem. We were using JSSE
for SSL, switching to APR and OpenSSL fixed the problems. So my findings are
this....
JSSE has a bug in it that can cause the Tomcat server to crash brought on by
SSL, Chrome and a form post of a specific amount of data. The server crashes
can be mitigated by starting Tomcat with
"-XX:CompileCommand=exclude,com/sun/crypto/provider/*.*". Instead of the server
crashing Chrome returns net::ERR_SSL_PROTOCOL_ERROR and you can actually catch
the error, the stack trace is below.
I have reported my findings to Oracle. They need to fix the bug, but for us the
best solution was just to move away from JSSE and switch to APR OpenSSL which
is the recommend solution to begin with.
Thanks,
Chad
07-Oct-2014 10:10:58.057 SEVERE [http-nio-443-exec-38]
org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for
servlet [Controller] in context with path [/mtg] threw exception
java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at com.sun.crypto.provider.GCTR.reset(GCTR.java:125)
at com.sun.crypto.provider.GCTR.doFinal(GCTR.java:116)
at
com.sun.crypto.provider.GaloisCounterMode.doLastBlock(GaloisCounterMode.java:343)
at
com.sun.crypto.provider.GaloisCounterMode.decryptFinal(GaloisCounterMode.java:511)
at
com.sun.crypto.provider.CipherCore.finalNoPadding(CipherCore.java:1023)
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:960)
at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:479)
at javax.crypto.CipherSpi.bufferCrypt(CipherSpi.java:830)
at javax.crypto.CipherSpi.engineDoFinal(CipherSpi.java:730)
at javax.crypto.Cipher.doFinal(Cipher.java:2416)
at sun.security.ssl.CipherBox.decrypt(Unknown Source)
at sun.security.ssl.EngineInputRecord.decrypt(Unknown Source)
at sun.security.ssl.SSLEngineImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLEngineImpl.readNetRecord(Unknown Source)
at sun.security.ssl.SSLEngineImpl.unwrap(Unknown Source)
at javax.net.ssl.SSLEngine.unwrap(Unknown Source)
at
org.apache.tomcat.util.net.SecureNioChannel.read(SecureNioChannel.java:439)
at
org.apache.tomcat.util.net.NioBlockingSelector.read(NioBlockingSelector.java:173)
at
org.apache.tomcat.util.net.NioSelectorPool.read(NioSelectorPool.java:251)
at
org.apache.tomcat.util.net.NioSelectorPool.read(NioSelectorPool.java:232)
at
org.apache.coyote.http11.InternalNioInputBuffer.fill(InternalNioInputBuffer.java:133)
at
org.apache.coyote.http11.InternalNioInputBuffer$SocketInputBuffer.doRead(InternalNioInputBuffer.java:177)
at
org.apache.coyote.http11.filters.IdentityInputFilter.doRead(IdentityInputFilter.java:110)
at
org.apache.coyote.http11.AbstractInputBuffer.doRead(AbstractInputBuffer.java:413)
at org.apache.coyote.Request.doRead(Request.java:459)
at
org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.java:338)
at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:395)
at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:363)
at
org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStream.java:190)
at org.apache.catalina.connector.Request.readPostBody(Request.java:3034)
at
org.apache.catalina.connector.Request.parseParameters(Request.java:2983)
at org.apache.catalina.connector.Request.getParameter(Request.java:1077)
at
org.apache.catalina.connector.RequestFacade.getParameter(RequestFacade.java:380)
at com.mtg.mtg.controller.Controller.doPost(Controller.java:41)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:506)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1566)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1523)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Unknown Source)
________________________________________
From: Mark Thomas <[email protected]>
Sent: Friday, October 03, 2014 1:50 PM
To: Tomcat Users List
Subject: Re: Tomcat JVM Crash
On 03/10/2014 19:38, Chad Maniccia wrote:
> Hi Mark,
>
> Thanks for replying. I actually reported this bug to Oracle before contacting
> this group. They contacted me once but then never replied again. I'd
> appreciate it if you could bring it to their attention again.
>
> https://bugs.openjdk.java.net/browse/JDK-8058284
Happy to do that once you have a repeatable test case. Frankly, without
one, I doubt this is going to get much attention.
Mark
>
> This bug is kind of elusive as a form that is crashing today might not crash
> tomorrow, I suspect it is because headers, cookies, session keys etc have
> changed. I'll see if I can reproduce it by creating a testing form.
>
> Can anyone tell me why this line causes the site to not crash?
>
> -XX:CompileCommand=exclude,com/sun/crypto/provider/*.*
>
> P.S.
> Igal thanks for your support.
> ________________________________________
> From: Mark Thomas <[email protected]>
> Sent: Friday, October 03, 2014 1:14 PM
> To: Tomcat Users List
> Subject: Re: Tomcat JVM Crash
>
> On 03/10/2014 17:11, Igal @ getRailo.org wrote:
>>> Whose problem is this: Google, Apache Tomcat, GoDaddy(SSL), or Oracle?
>>> regardless of whose fault this is, Tomcat should be patched so that it
>>> doesn't crash.
>
> The general position of the Tomcat developers is that we do *not* patch
> Tomcat to work around bugs in third party code.
>
> There have been exceptions in the past but - since this JVM bug as a
> workaround available - I very much doubt that Tomcat will be patched to
> avoid this (even if such a patch was possible which looks unlikely).
>
>> can you produce a reduced test case so that the good people at Tomcat
>> can reproduce it on their end and patch it?
>
> A reproducible test case is definitely a good thing but it needs to go
> to Oracle, not to the Tomcat devs.
>
> Note we do have some contacts with Oracle we can use to ensure a bug
> report gets in front of the right people.
>
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
