Hi all,
Some help is really appreciated, atleast it will be good to hear if some
one else is also facing slowness with the tomcat7 server.
I did some more investigation here and as mentioned earlier saw the
slowness with tomcat7 is more reproducible when there are some
cryptographic signature verifications. Turning on the
StuckThreadDetectionValve with 1 second threshold as shown below revealed
that there are warning logs with threads stuck showing up in the next run
of the test. This clearly shows that the slowness is due to the threads
getting into the blocked state. More are in the signature verification but
few are in other areas too.
<Valve
className="org.apache.catalina.valves.StuckThreadDetectionValve"
threshold="1" />
NOTE - the above test was performed on a low end windows machine. Since the
granularity of the valve threshold is in seconds was finding it hard to
reproduce the valve to spit out some logs on the high end server. But the
test shows that in the second run of the performance test which was run on
another machine, only then i see the warning logs on the tomcat server.
CATALINA.out contents showing the stuck threads ===>
Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context [/host-manager]
Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context [/docs]
Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context []
Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context [/Signature]
Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context [/examples]
Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig undeploy
INFO: Undeploying context [/manager]
Nov 07, 2015 11:01:13 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive
D:\apache-tomcat-7.0.64\webapps\Signature.war
Nov 07, 2015 11:01:16 AM org.apache.catalina.startup.TldConfig execute
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable
debug logging for this logger for a complete list of JARs that were scanned
but no TLDs were found in them. Skipping unneeded JARs during scanning can
improve startup time and JSP compilation time.
Nov 07, 2015 11:01:17 AM
com.sun.jersey.api.core.servlet.WebAppResourceConfig init
INFO: Scanning for root resource and provider classes in the Web app
resource paths:
/WEB-INF/lib
/WEB-INF/classes
Nov 07, 2015 11:01:18 AM com.sun.jersey.api.core.ScanningResourceConfig
logClasses
INFO: Root resource classes found:
class signature.NullEndPoint
class signature.SignatureEndPoint
Nov 07, 2015 11:01:18 AM com.sun.jersey.api.core.ScanningResourceConfig
logClasses
INFO: Provider classes found:
class org.codehaus.jackson.jaxrs.JsonMappingExceptionMapper
class org.codehaus.jackson.jaxrs.JacksonJaxbJsonProvider
class org.codehaus.jackson.jaxrs.JacksonJsonProvider
class org.codehaus.jackson.jaxrs.JsonParseExceptionMapper
Nov 07, 2015 11:01:18 AM
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.17 01/17/2013 03:31
PM'
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deployment of web application archive
D:\apache-tomcat-7.0.64\webapps\Signature.war has finished in 5,871 ms
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
D:\apache-tomcat-7.0.64\webapps\docs
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.TldConfig execute
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable
debug logging for this logger for a complete list of JARs that were scanned
but no TLDs were found in them. Skipping unneeded JARs during scanning can
improve startup time and JSP compilation time.
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deployment of web application directory
D:\apache-tomcat-7.0.64\webapps\docs has finished in 140 ms
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
D:\apache-tomcat-7.0.64\webapps\examples
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.TldConfig execute
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable
debug logging for this logger for a complete list of JARs that were scanned
but no TLDs were found in them. Skipping unneeded JARs during scanning can
improve startup time and JSP compilation time.
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deployment of web application directory
D:\apache-tomcat-7.0.64\webapps\examples has finished in 445 ms
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
D:\apache-tomcat-7.0.64\webapps\host-manager
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.TldConfig execute
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable
debug logging for this logger for a complete list of JARs that were scanned
but no TLDs were found in them. Skipping unneeded JARs during scanning can
improve startup time and JSP compilation time.
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deployment of web application directory
D:\apache-tomcat-7.0.64\webapps\host-manager has finished in 152 ms
Nov 07, 2015 11:01:19 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
D:\apache-tomcat-7.0.64\webapps\manager
Nov 07, 2015 11:01:20 AM org.apache.catalina.startup.TldConfig execute
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable
debug logging for this logger for a complete list of JARs that were scanned
but no TLDs were found in them. Skipping unneeded JARs during scanning can
improve startup time and JSP compilation time.
Nov 07, 2015 11:01:20 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deployment of web application directory
D:\apache-tomcat-7.0.64\webapps\manager has finished in 203 ms
Nov 07, 2015 11:01:20 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deploying web application directory
D:\apache-tomcat-7.0.64\webapps\ROOT
Nov 07, 2015 11:01:20 AM org.apache.catalina.startup.TldConfig execute
INFO: At least one JAR was scanned for TLDs yet contained no TLDs. Enable
debug logging for this logger for a complete list of JARs that were scanned
but no TLDs were found in them. Skipping unneeded JARs during scanning can
improve startup time and JSP compilation time.
Nov 07, 2015 11:01:20 AM org.apache.catalina.startup.HostConfig
deployDirectory
INFO: Deployment of web application directory
D:\apache-tomcat-7.0.64\webapps\ROOT has finished in 162 ms
Nov 07, 2015 11:06:07 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-55" (id=83) has been active for 1,397
milliseconds (since 11/7/15 11:06 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 1 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at sun.security.ec.ECDSASignature.verifySignedDigest(Native Method)
at sun.security.ec.ECDSASignature.engineVerify(ECDSASignature.java:309)
at java.security.Signature$Delegate.engineVerify(Signature.java:1192)
at java.security.Signature.verify(Signature.java:626)
at signature.SignatureEndPoint.verifyECDSASignature(Unknown Source)
at signature.SignatureEndPoint.verifySignature(Unknown Source)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:06:08 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-55" (id=83) was previously reported to
be stuck but has completed. It was active for approximately 1,468
milliseconds.
Nov 07, 2015 11:08:41 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-20" (id=48) has been active for 1,221
milliseconds (since 11/7/15 11:08 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 1 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at sun.security.ec.ECDSASignature.verifySignedDigest(Native Method)
at sun.security.ec.ECDSASignature.engineVerify(ECDSASignature.java:309)
at java.security.Signature$Delegate.engineVerify(Signature.java:1192)
at java.security.Signature.verify(Signature.java:626)
at signature.SignatureEndPoint.verifyECDSASignature(Unknown Source)
at signature.SignatureEndPoint.verifySignature(Unknown Source)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:08:44 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-20" (id=48) was previously reported to
be stuck but has completed. It was active for approximately 1,240
milliseconds.
Nov 07, 2015 11:09:16 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-27" (id=55) has been active for 1,930
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 1 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at sun.security.ec.ECDSASignature.verifySignedDigest(Native Method)
at sun.security.ec.ECDSASignature.engineVerify(ECDSASignature.java:309)
at java.security.Signature$Delegate.engineVerify(Signature.java:1192)
at java.security.Signature.verify(Signature.java:626)
at signature.SignatureEndPoint.verifyECDSASignature(Unknown Source)
at signature.SignatureEndPoint.verifySignature(Unknown Source)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:16 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-42" (id=70) has been active for 1,942
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 2 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at java.io.BufferedWriter.<init>(BufferedWriter.java:105)
at java.io.BufferedWriter.<init>(BufferedWriter.java:88)
at
com.sun.jersey.core.util.ReaderWriter.writeToAsString(ReaderWriter.java:188)
at
com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:128)
at
com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:88)
at
com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:58)
at
com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:306)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1479)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:16 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-71" (id=99) has been active for 1,024
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 3 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at java.util.Arrays.copyOfRange(Arrays.java:2694)
at java.lang.String.<init>(String.java:203)
at java.lang.String.substring(String.java:1913)
at java.net.URI$Parser.substring(URI.java:2850)
at java.net.URI$Parser.parseServer(URI.java:3228)
at java.net.URI$Parser.parseAuthority(URI.java:3136)
at java.net.URI$Parser.parseHierarchical(URI.java:3078)
at java.net.URI$Parser.parse(URI.java:3034)
at java.net.URI.<init>(URI.java:595)
at com.sun.jersey.api.uri.UriBuilderImpl.createURI(UriBuilderImpl.java:721)
at com.sun.jersey.api.uri.UriBuilderImpl._build(UriBuilderImpl.java:651)
at com.sun.jersey.api.uri.UriBuilderImpl.build(UriBuilderImpl.java:641)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:704)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:16 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-17" (id=45) has been active for 1,666
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 4 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at java.io.BufferedWriter.<init>(BufferedWriter.java:105)
at java.io.BufferedWriter.<init>(BufferedWriter.java:88)
at
com.sun.jersey.core.util.ReaderWriter.writeToAsString(ReaderWriter.java:188)
at
com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:128)
at
com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:88)
at
com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:58)
at
com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:306)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1479)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:17 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-30" (id=58) has been active for 1,647
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 5 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at java.io.BufferedWriter.<init>(BufferedWriter.java:105)
at java.io.BufferedWriter.<init>(BufferedWriter.java:88)
at
com.sun.jersey.core.util.ReaderWriter.writeToAsString(ReaderWriter.java:188)
at
com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:128)
at
com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:88)
at
com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:58)
at
com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:306)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1479)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:17 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-36" (id=64) has been active for 1,674
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 6 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
at sun.nio.cs.StreamEncoder.<init>(StreamEncoder.java:195)
at sun.nio.cs.StreamEncoder.<init>(StreamEncoder.java:175)
at sun.nio.cs.StreamEncoder.forOutputStreamWriter(StreamEncoder.java:68)
at java.io.OutputStreamWriter.<init>(OutputStreamWriter.java:133)
at
com.sun.jersey.core.util.ReaderWriter.writeToAsString(ReaderWriter.java:188)
at
com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:128)
at
com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:88)
at
com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:58)
at
com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:306)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1479)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:17 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-27" (id=55) was previously reported to
be stuck but has completed. It was active for approximately 1,942
milliseconds. There is/are still 5 thread(s) that are monitored by this
Valve and may be stuck.
Nov 07, 2015 11:09:17 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-71" (id=99) was previously reported to
be stuck but has completed. It was active for approximately 1,662
milliseconds. There is/are still 4 thread(s) that are monitored by this
Valve and may be stuck.
Nov 07, 2015 11:09:17 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-42" (id=70) was previously reported to
be stuck but has completed. It was active for approximately 2,699
milliseconds. There is/are still 3 thread(s) that are monitored by this
Valve and may be stuck.
Nov 07, 2015 11:09:17 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-30" (id=58) was previously reported to
be stuck but has completed. It was active for approximately 1,848
milliseconds. There is/are still 2 thread(s) that are monitored by this
Valve and may be stuck.
Nov 07, 2015 11:09:17 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-17" (id=45) was previously reported to
be stuck but has completed. It was active for approximately 2,495
milliseconds. There is/are still 1 thread(s) that are monitored by this
Valve and may be stuck.
Nov 07, 2015 11:09:17 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-36" (id=64) was previously reported to
be stuck but has completed. It was active for approximately 1,977
milliseconds.
Nov 07, 2015 11:09:47 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-12" (id=40) has been active for 2,278
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 1 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at sun.security.ec.ECDSASignature.verifySignedDigest(Native Method)
at sun.security.ec.ECDSASignature.engineVerify(ECDSASignature.java:309)
at java.security.Signature$Delegate.engineVerify(Signature.java:1192)
at java.security.Signature.verify(Signature.java:626)
at signature.SignatureEndPoint.verifyECDSASignature(Unknown Source)
at signature.SignatureEndPoint.verifySignature(Unknown Source)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:48 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-12" (id=40) was previously reported to
be stuck but has completed. It was active for approximately 2,285
milliseconds.
Nov 07, 2015 11:09:58 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-40" (id=68) has been active for 1,013
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 1 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at java.security.Security.getImpl(Security.java:695)
at
java.security.AlgorithmParameters.getInstance(AlgorithmParameters.java:146)
at sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:121)
at sun.security.x509.AlgorithmId.<init>(AlgorithmId.java:114)
at sun.security.x509.AlgorithmId.parse(AlgorithmId.java:381)
at sun.security.x509.X509Key.decode(X509Key.java:389)
at sun.security.x509.X509Key.decode(X509Key.java:403)
at sun.security.ec.ECPublicKeyImpl.<init>(ECPublicKeyImpl.java:68)
at sun.security.ec.ECKeyFactory.implGeneratePublic(ECKeyFactory.java:224)
at sun.security.ec.ECKeyFactory.engineGeneratePublic(ECKeyFactory.java:158)
at java.security.KeyFactory.generatePublic(KeyFactory.java:334)
at signature.SignatureEndPoint.verifyECDSASignature(Unknown Source)
at signature.SignatureEndPoint.verifySignature(Unknown Source)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:58 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-55" (id=83) has been active for 1,484
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 2 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at sun.security.ec.ECDSASignature.verifySignedDigest(Native Method)
at sun.security.ec.ECDSASignature.engineVerify(ECDSASignature.java:309)
at java.security.Signature$Delegate.engineVerify(Signature.java:1192)
at java.security.Signature.verify(Signature.java:626)
at signature.SignatureEndPoint.verifyECDSASignature(Unknown Source)
at signature.SignatureEndPoint.verifySignature(Unknown Source)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:58 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadDetected
WARNING: Thread "http-nio-8443-exec-14" (id=42) has been active for 1,491
milliseconds (since 11/7/15 11:09 AM) to serve the same request for
https://10.55.198.52:8443/Signature/authenticate and may be stuck
(configured threshold for this StuckThreadDetectionValve is 1 seconds).
There is/are 3 thread(s) in total that are monitored by this Valve and may
be stuck.
java.lang.Throwable
at sun.security.ec.ECDSASignature.verifySignedDigest(Native Method)
at sun.security.ec.ECDSASignature.engineVerify(ECDSASignature.java:309)
at java.security.Signature$Delegate.engineVerify(Signature.java:1192)
at java.security.Signature.verify(Signature.java:626)
at signature.SignatureEndPoint.verifyECDSASignature(Unknown Source)
at signature.SignatureEndPoint.verifySignature(Unknown Source)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
at
org.apache.catalina.valves.StuckThreadDetectionValve.invoke(StuckThreadDetectionValve.java:221)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Nov 07, 2015 11:09:58 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-40" (id=68) was previously reported to
be stuck but has completed. It was active for approximately 1,361
milliseconds. There is/are still 2 thread(s) that are monitored by this
Valve and may be stuck.
Nov 07, 2015 11:09:58 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-14" (id=42) was previously reported to
be stuck but has completed. It was active for approximately 1,493
milliseconds. There is/are still 1 thread(s) that are monitored by this
Valve and may be stuck.
Nov 07, 2015 11:09:58 AM
org.apache.catalina.valves.StuckThreadDetectionValve
notifyStuckThreadCompleted
WARNING: Thread "http-nio-8443-exec-55" (id=83) was previously reported to
be stuck but has completed. It was active for approximately 1,518
milliseconds.
Thanks,
Dimple Ranka
On Thu, Nov 5, 2015 at 12:32 PM, dimple ranka <dimplekra...@gmail.com>
wrote:
> Looks like attachments are not allowed so only sharing the contents of
> threadDump3.out
>
> 2015-11-04 22:55:58
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.79-b02 mixed mode):
>
> "http-nio-8443-exec-147" daemon prio=10 tid=0x00007f5c1c005000 nid=0x6241
> runnable [0x00007f5bd30ef000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:192)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> - locked <0x00000000fc2d9498> (a java.lang.Object)
> at
> org.apache.tomcat.util.net.SecureNioChannel.read(SecureNioChannel.java:443)
> at
> org.apache.coyote.http11.InternalNioInputBuffer.readSocket(InternalNioInputBuffer.java:436)
> at
> org.apache.coyote.http11.InternalNioInputBuffer.fill(InternalNioInputBuffer.java:795)
> at
> org.apache.coyote.http11.InternalNioInputBuffer.parseRequestLine(InternalNioInputBuffer.java:227)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:993)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
> - locked <0x00000000875077f0> (a
> org.apache.tomcat.util.net.SecureNioChannel)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - <0x0000000088a613f0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>
> "http-nio-8443-exec-146" daemon prio=10 tid=0x00007f5c18020800 nid=0x6240
> waiting on condition [0x00007f5c6e271000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-145" daemon prio=10 tid=0x00007f5c18037000 nid=0x623f
> waiting on condition [0x00007f5c6e372000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-144" daemon prio=10 tid=0x00007f5c1802f800 nid=0x623e
> waiting on condition [0x00007f5c6c89c000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-143" daemon prio=10 tid=0x00007f5c1800e000 nid=0x623d
> waiting on condition [0x00007f5bd3cfb000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-141" daemon prio=10 tid=0x00007f5c1c046000 nid=0x623c
> waiting on condition [0x00007f5c6c397000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-142" daemon prio=10 tid=0x00007f5c1803c000 nid=0x623b
> waiting on condition [0x00007f5bd2eed000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-140" daemon prio=10 tid=0x00007f5c1c011000 nid=0x623a
> waiting on condition [0x00007f5bd3dfc000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-139" daemon prio=10 tid=0x00007f5c18024000 nid=0x6239
> waiting on condition [0x00007f5bd34f3000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-138" daemon prio=10 tid=0x00007f5c1c00a800 nid=0x6238
> runnable [0x00007f5bd31ef000]
> java.lang.Thread.State: RUNNABLE
> at sun.security.provider.SHA.implDigest(SHA.java:103)
> at sun.security.provider.DigestBase.engineDigest(DigestBase.java:186)
> at
> java.security.MessageDigest$Delegate.engineDigest(MessageDigest.java:581)
> at java.security.MessageDigest.digest(MessageDigest.java:380)
> at com.sun.crypto.provider.TlsPrfGenerator.expand(TlsPrfGenerator.java:315)
> at
> com.sun.crypto.provider.TlsPrfGenerator.doTLS10PRF(TlsPrfGenerator.java:251)
> at
> com.sun.crypto.provider.TlsKeyMaterialGenerator.engineGenerateKey0(TlsKeyMaterialGenerator.java:135)
> at
> com.sun.crypto.provider.TlsKeyMaterialGenerator.engineGenerateKey(TlsKeyMaterialGenerator.java:89)
> at javax.crypto.KeyGenerator.generateKey(KeyGenerator.java:530)
> at
> sun.security.ssl.Handshaker.calculateConnectionKeys(Handshaker.java:1150)
> at sun.security.ssl.ServerHandshaker.clientHello(ServerHandshaker.java:677)
> at
> sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:167)
> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:901)
> at sun.security.ssl.Handshaker$1.run(Handshaker.java:841)
> at sun.security.ssl.Handshaker$1.run(Handshaker.java:839)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1273)
> - locked <0x00000000d723cc78> (a sun.security.ssl.SSLEngineImpl)
> at
> org.apache.tomcat.util.net.SecureNioChannel.tasks(SecureNioChannel.java:301)
> at
> org.apache.tomcat.util.net.SecureNioChannel.handshakeUnwrap(SecureNioChannel.java:359)
> at
> org.apache.tomcat.util.net.SecureNioChannel.handshake(SecureNioChannel.java:209)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1736)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
> - locked <0x00000000873beeb8> (a
> org.apache.tomcat.util.net.SecureNioChannel)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - <0x0000000088700840> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>
> "http-nio-8443-exec-137" daemon prio=10 tid=0x00007f5c1c042800 nid=0x6234
> waiting on condition [0x00007f5bd25e4000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-136" daemon prio=10 tid=0x00007f5c1c05b000 nid=0x6233
> waiting on condition [0x00007f5bd37f6000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-135" daemon prio=10 tid=0x00007f5c1c044000 nid=0x6232
> waiting on condition [0x00007f5bd2bea000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-134" daemon prio=10 tid=0x00007f5c18022800 nid=0x6231
> waiting on condition [0x00007f5c6c79b000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-133" daemon prio=10 tid=0x00007f5c1801b800 nid=0x6230
> waiting on condition [0x00007f5c6c296000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-132" daemon prio=10 tid=0x00007f5c1c01a800 nid=0x622f
> runnable [0x00007f5bd13d1000]
> java.lang.Thread.State: RUNNABLE
> at java.lang.Class.getEnclosingMethod0(Native Method)
> at java.lang.Class.getEnclosingMethodInfo(Class.java:964)
> at java.lang.Class.getEnclosingClass(Class.java:1137)
> at java.lang.Class.getSimpleBinaryName(Class.java:1282)
> at java.lang.Class.getSimpleName(Class.java:1174)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:93)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1511)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1442)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
> - locked <0x0000000087744de8> (a
> org.apache.tomcat.util.net.SecureNioChannel)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - <0x0000000087b269d0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>
> "http-nio-8443-exec-131" daemon prio=10 tid=0x00007f5c1802c000 nid=0x622e
> runnable [0x00007f5bd26e4000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.NativeThread.current(Native Method)
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:484)
> - locked <0x00000000fc2c6ad0> (a java.lang.Object)
> - locked <0x00000000fc2c6ac0> (a java.lang.Object)
> at
> org.apache.tomcat.util.net.SecureNioChannel.flush(SecureNioChannel.java:141)
> at
> org.apache.tomcat.util.net.SecureNioChannel.write(SecureNioChannel.java:526)
> at
> org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
> at
> org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:174)
> at
> org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:163)
> - locked <0x0000000087982f38> (a
> org.apache.coyote.http11.InternalNioOutputBuffer)
> at
> org.apache.coyote.http11.InternalNioOutputBuffer.flushBuffer(InternalNioOutputBuffer.java:242)
> at
> org.apache.coyote.http11.InternalNioOutputBuffer.flush(InternalNioOutputBuffer.java:94)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.action(AbstractHttp11Processor.java:801)
> at org.apache.coyote.Response.action(Response.java:172)
> at
> org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:363)
> at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:331)
> at
> org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:101)
> at
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.flush(WebComponent.java:315)
> at
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.flush(ContainerResponse.java:145)
> at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
> at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
> - locked <0x00000000d74f0a88> (a java.io.OutputStreamWriter)
> at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
> at java.io.BufferedWriter.flush(BufferedWriter.java:254)
> - locked <0x00000000d74f0a88> (a java.io.OutputStreamWriter)
> at
> com.sun.jersey.core.util.ReaderWriter.writeToAsString(ReaderWriter.java:191)
> at
> com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:128)
> at
> com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:88)
> at
> com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:58)
> at
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:306)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1479)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
> - locked <0x000000008786c748> (a
> org.apache.tomcat.util.net.SecureNioChannel)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - <0x0000000087b251e8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>
> "http-nio-8443-exec-129" daemon prio=10 tid=0x00007f5c18009800 nid=0x622d
> waiting on condition [0x00007f5bd19d8000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-130" daemon prio=10 tid=0x00007f5c1c016000 nid=0x622c
> waiting on condition [0x00007f5bd18d7000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-128" daemon prio=10 tid=0x00007f5c1801f000 nid=0x622b
> waiting on condition [0x00007f5c6d125000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-127" daemon prio=10 tid=0x00007f5c18001000 nid=0x6229
> waiting on condition [0x00007f5bd28e7000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-126" daemon prio=10 tid=0x00007f5c1c01c000 nid=0x6228
> waiting on condition [0x00007f5c6cca0000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-125" daemon prio=10 tid=0x00007f5c1c00c800 nid=0x6227
> runnable [0x00007f5bd36f5000]
> java.lang.Thread.State: RUNNABLE
> at sun.misc.Unsafe.unpark(Native Method)
> at java.util.concurrent.locks.LockSupport.unpark(LockSupport.java:152)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.unparkSuccessor(AbstractQueuedSynchronizer.java:662)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1263)
> at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:460)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:474)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-124" daemon prio=10 tid=0x00007f5c1800c800 nid=0x6226
> waiting on condition [0x00007f5bd2ceb000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-123" daemon prio=10 tid=0x00007f5c1c025000 nid=0x6225
> waiting on condition [0x00007f5bd3efd000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-122" daemon prio=10 tid=0x00007f5c1c051000 nid=0x6224
> waiting on condition [0x00007f5c6c69a000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-121" daemon prio=10 tid=0x00007f5c18027000 nid=0x6223
> waiting on condition [0x00007f5c6c195000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-120" daemon prio=10 tid=0x00007f5c1c041000 nid=0x6222
> waiting on condition [0x00007f5bd3af9000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-119" daemon prio=10 tid=0x00007f5c1c019000 nid=0x6220
> waiting on condition [0x00007f5bd35f4000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-118" daemon prio=10 tid=0x00007f5c1c03f800 nid=0x621f
> waiting on condition [0x00007f5c6c498000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-117" daemon prio=10 tid=0x00007f5c1c00e000 nid=0x621a
> waiting on condition [0x00007f5bd3ffe000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-116" daemon prio=10 tid=0x00007f5c1c061000 nid=0x6209
> waiting on condition [0x00007f5bd17d6000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-108" daemon prio=10 tid=0x00007f5c1805c000 nid=0x6201
> runnable [0x00007f5bd38f7000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-97" daemon prio=10 tid=0x00007f5c1c030800 nid=0x61e3
> waiting on condition [0x00007f5bd33f2000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-93" daemon prio=10 tid=0x00007f5c1c009000 nid=0x61de
> waiting on condition [0x00007f5bd27e6000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-90" daemon prio=10 tid=0x00007f5c1c04a000 nid=0x61db
> waiting on condition [0x00007f5bd16d5000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-87" daemon prio=10 tid=0x00007f5c1802a000 nid=0x61d8
> waiting on condition [0x00007f5bd20df000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-83" daemon prio=10 tid=0x00007f5c1c012800 nid=0x61c0
> waiting on condition [0x00007f5bd24e3000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7a088> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-69" daemon prio=10 tid=0x00007f5c1c02d800 nid=0x61b1
> runnable [0x00007f5c6cb9e000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)
> at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)
> at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:193)
> - locked <0x000000008591b0f0> (a java.lang.Object)
> at
> org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.wakeup(NioBlockingSelector.java:233)
> at
> org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.remove(NioBlockingSelector.java:312)
> at
> org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:136)
> at
> org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:174)
> at
> org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:163)
> - locked <0x000000008758bc78> (a
> org.apache.coyote.http11.InternalNioOutputBuffer)
> at
> org.apache.coyote.http11.InternalNioOutputBuffer.flushBuffer(InternalNioOutputBuffer.java:242)
> at
> org.apache.coyote.http11.InternalNioOutputBuffer.flush(InternalNioOutputBuffer.java:94)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.action(AbstractHttp11Processor.java:801)
> at org.apache.coyote.Response.action(Response.java:172)
> at
> org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:363)
> at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:331)
> at
> org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:101)
> at
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.flush(WebComponent.java:315)
> at
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.flush(ContainerResponse.java:145)
> at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
> at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
> - locked <0x00000000fc2898f0> (a java.io.OutputStreamWriter)
> at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
> at java.io.BufferedWriter.flush(BufferedWriter.java:254)
> - locked <0x00000000fc2898f0> (a java.io.OutputStreamWriter)
> at
> com.sun.jersey.core.util.ReaderWriter.writeToAsString(ReaderWriter.java:191)
> at
> com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:128)
> at
> com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:88)
> at
> com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:58)
> at
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:306)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1479)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
> - locked <0x00000000879aad60> (a
> org.apache.tomcat.util.net.SecureNioChannel)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - <0x0000000087b0c410> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>
> "http-nio-8443-exec-55" daemon prio=10 tid=0x00007f5c18038800 nid=0x6191
> waiting on condition [0x00007f5bd1bda000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000086b7fcc0> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(AbstractQueuedSynchronizer.java:894)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1221)
> at
> java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
> at
> java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:462)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:86)
> at org.apache.tomcat.util.threads.TaskQueue.poll(TaskQueue.java:32)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "Attach Listener" daemon prio=10 tid=0x00007f5c58001000 nid=0x618b waiting
> on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-exec-24" daemon prio=10 tid=0x00007f5c18017000 nid=0x6159
> runnable [0x00007f5bd3bf9000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.NativeThread.current(Native Method)
> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:484)
> - locked <0x00000000fc298138> (a java.lang.Object)
> - locked <0x00000000fc2981b8> (a java.lang.Object)
> at
> org.apache.tomcat.util.net.SecureNioChannel.flush(SecureNioChannel.java:141)
> at
> org.apache.tomcat.util.net.SecureNioChannel.write(SecureNioChannel.java:526)
> at
> org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
> at
> org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:174)
> at
> org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:163)
> - locked <0x0000000087a11498> (a
> org.apache.coyote.http11.InternalNioOutputBuffer)
> at
> org.apache.coyote.http11.InternalNioOutputBuffer.flushBuffer(InternalNioOutputBuffer.java:242)
> at
> org.apache.coyote.http11.InternalNioOutputBuffer.flush(InternalNioOutputBuffer.java:94)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.action(AbstractHttp11Processor.java:801)
> at org.apache.coyote.Response.action(Response.java:172)
> at
> org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:363)
> at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:331)
> at
> org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:101)
> at
> com.sun.jersey.spi.container.servlet.WebComponent$Writer.flush(WebComponent.java:315)
> at
> com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.flush(ContainerResponse.java:145)
> at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
> at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
> - locked <0x00000000fc2981c8> (a java.io.OutputStreamWriter)
> at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
> at java.io.BufferedWriter.flush(BufferedWriter.java:254)
> - locked <0x00000000fc2981c8> (a java.io.OutputStreamWriter)
> at
> com.sun.jersey.core.util.ReaderWriter.writeToAsString(ReaderWriter.java:191)
> at
> com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:128)
> at
> com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:88)
> at
> com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:58)
> at
> com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:306)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1479)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1391)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1381)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.logging.log4j.core.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:66)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:614)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1760)
> at
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1719)
> - locked <0x0000000086b80968> (a
> org.apache.tomcat.util.net.SecureNioChannel)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - <0x00000000875dc3d0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>
> "http-nio-8443-Acceptor-0" daemon prio=10 tid=0x00007f5c981ff000
> nid=0x613b runnable [0x00007f5c6d327000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> at
> sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
> - locked <0x000000008589b020> (a java.lang.Object)
> at
> org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:825)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-ClientPoller-1" daemon prio=10 tid=0x00007f5c98127800
> nid=0x613a runnable [0x00007f5c6d428000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> - locked <0x0000000086ca24c8> (a sun.nio.ch.Util$2)
> - locked <0x0000000086ca24d8> (a java.util.Collections$UnmodifiableSet)
> - locked <0x0000000086ca2480> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
> at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1198)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "http-nio-8443-ClientPoller-0" daemon prio=10 tid=0x00007f5c98125800
> nid=0x6139 runnable [0x00007f5c6d529000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.IOUtil.drain(Native Method)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)
> - locked <0x0000000086c81b80> (a java.lang.Object)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> - locked <0x0000000086c80960> (a sun.nio.ch.Util$2)
> - locked <0x0000000086c80950> (a java.util.Collections$UnmodifiableSet)
> - locked <0x0000000086b80758> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
> at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1198)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "ajp-bio-8009-AsyncTimeout" daemon prio=10 tid=0x00007f5c98124000
> nid=0x6138 waiting on condition [0x00007f5c6d62a000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.tomcat.util.net.JIoEndpoint$AsyncTimeout.run(JIoEndpoint.java:152)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "ajp-bio-8009-Acceptor-0" daemon prio=10 tid=0x00007f5c98122800 nid=0x6137
> runnable [0x00007f5c6d72b000]
> java.lang.Thread.State: RUNNABLE
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
> at java.net.ServerSocket.implAccept(ServerSocket.java:530)
> at java.net.ServerSocket.accept(ServerSocket.java:498)
> at
> org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:60)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:222)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon prio=10
> tid=0x00007f5c98120000 nid=0x6136 waiting on condition [0x00007f5c6d82c000]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1512)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "pool-2-thread-1" prio=10 tid=0x00007f5c38665800 nid=0x6135 waiting on
> condition [0x00007f5c6d92d000]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000000008637b008> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
> Locked ownable synchronizers:
> - None
>
> "Thread-5" daemon prio=10 tid=0x00007f5c385d1000 nid=0x6134 runnable
> [0x00007f5c6da2e000]
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.net.SocketInputStream.read(SocketInputStream.java:108)
> at redis.clients.util.RedisInputStream.fill(RedisInputStream.java:109)
> at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:45)
> at redis.clients.jedis.Protocol.process(Protocol.java:129)
> at redis.clients.jedis.Protocol.read(Protocol.java:200)
> at
> redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:285)
> at
> redis.clients.jedis.Connection.getRawObjectMultiBulkReply(Connection.java:230)
> at redis.clients.jedis.JedisPubSub.process(JedisPubSub.java:109)
> at redis.clients.jedis.JedisPubSub.proceed(JedisPubSub.java:103)
> at redis.clients.jedis.Jedis.subscribe(Jedis.java:2810)
> at
> redis.clients.jedis.JedisSentinelPool$MasterListener.run(JedisSentinelPool.java:244)
>
> Locked ownable synchronizers:
> - None
>
> "commons-pool-EvictionTimer" daemon prio=10 tid=0x00007f5c385bb000
> nid=0x6133 in Object.wait() [0x00007f5c6db2f000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000866bc048> (a java.util.TaskQueue)
> at java.util.TimerThread.mainLoop(Timer.java:552)
> - locked <0x00000000866bc048> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:505)
>
> Locked ownable synchronizers:
> - None
>
> "com.google.inject.internal.util.$Finalizer" daemon prio=10
> tid=0x00007f5c382b9800 nid=0x6132 in Object.wait() [0x00007f5c6e109000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000860179e8> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x00000000860179e8> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at com.google.inject.internal.util.$Finalizer.run(Finalizer.java:114)
>
> Locked ownable synchronizers:
> - None
>
> "NioBlockingSelector.BlockPoller-1" daemon prio=10 tid=0x00007f5c985da800
> nid=0x612e waiting for monitor entry [0x00007f5c6e473000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:88)
> - waiting to lock <0x000000008591b0f0> (a java.lang.Object)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> - locked <0x000000008591afc0> (a sun.nio.ch.Util$2)
> - locked <0x000000008591afd0> (a java.util.Collections$UnmodifiableSet)
> - locked <0x000000008591af78> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
> at
> org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:339)
>
> Locked ownable synchronizers:
> - None
>
> "GC Daemon" daemon prio=10 tid=0x00007f5c98559800 nid=0x612d in
> Object.wait() [0x00007f5c6e7bb000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000085c91a18> (a sun.misc.GC$LatencyLock)
> at sun.misc.GC$Daemon.run(GC.java:117)
> - locked <0x0000000085c91a18> (a sun.misc.GC$LatencyLock)
>
> Locked ownable synchronizers:
> - None
>
> "Service Thread" daemon prio=10 tid=0x00007f5c9809f800 nid=0x6127 runnable
> [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread1" daemon prio=10 tid=0x00007f5c9809d000 nid=0x6126
> waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "C2 CompilerThread0" daemon prio=10 tid=0x00007f5c9809a000 nid=0x6125
> waiting on condition [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "Signal Dispatcher" daemon prio=10 tid=0x00007f5c98098000 nid=0x6124
> runnable [0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "Finalizer" daemon prio=10 tid=0x00007f5c98077800 nid=0x6123 in
> Object.wait() [0x00007f5c6f3f1000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000858a6998> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x00000000858a6998> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
>
> Locked ownable synchronizers:
> - None
>
> "Reference Handler" daemon prio=10 tid=0x00007f5c98075800 nid=0x6122 in
> Object.wait() [0x00007f5c6f4f2000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000858a6530> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x00000000858a6530> (a java.lang.ref.Reference$Lock)
>
> Locked ownable synchronizers:
> - None
>
> "main" prio=10 tid=0x00007f5c98009000 nid=0x6118 runnable
> [0x00007f5c9e3bb000]
> java.lang.Thread.State: RUNNABLE
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at
> java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
> at java.net.ServerSocket.implAccept(ServerSocket.java:530)
> at java.net.ServerSocket.accept(ServerSocket.java:498)
> at org.apache.catalina.core.StandardServer.await(StandardServer.java:453)
> at org.apache.catalina.startup.Catalina.await(Catalina.java:777)
> at org.apache.catalina.startup.Catalina.start(Catalina.java:723)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:321)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:455)
>
> Locked ownable synchronizers:
> - None
>
> "VM Thread" prio=10 tid=0x00007f5c98071000 nid=0x6121 runnable
>
> "GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f5c9801e800 nid=0x6119
> runnable
>
> "GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f5c98020800 nid=0x611a
> runnable
>
> "GC task thread#2 (ParallelGC)" prio=10 tid=0x00007f5c98022000 nid=0x611b
> runnable
>
> "GC task thread#3 (ParallelGC)" prio=10 tid=0x00007f5c98024000 nid=0x611c
> runnable
>
> "GC task thread#4 (ParallelGC)" prio=10 tid=0x00007f5c98026000 nid=0x611d
> runnable
>
> "GC task thread#5 (ParallelGC)" prio=10 tid=0x00007f5c98028000 nid=0x611e
> runnable
>
> "GC task thread#6 (ParallelGC)" prio=10 tid=0x00007f5c98029800 nid=0x611f
> runnable
>
> "GC task thread#7 (ParallelGC)" prio=10 tid=0x00007f5c9802b800 nid=0x6120
> runnable
>
> "VM Periodic Task Thread" prio=10 tid=0x00007f5c980aa000 nid=0x6128
> waiting on condition
>
> JNI global references: 330
>
>
> On Thu, Nov 5, 2015 at 12:27 PM, dimple ranka <dimplekra...@gmail.com>
> wrote:
>
>> Hi Chris,
>>
>> I captured 3 threadumps with the below requests/second when the server is
>> already slow. We were using BIO before and had observed slowness even with
>> that. We switched to using NIO since it seemed a recommendation for
>> production environment.
>> --------------------------------------------------------------
>> *WITH NIO*
>> *--------------------------------------------------------------*
>> *RUN1 (threadDump1.out)*
>> Requests/second = 18849.1606557377
>>
>> *RUN2 **(threadDump2.out)*
>> Requests/second = 18943.43606557377
>>
>> *RUN3 **(threadDump3.out)*
>> Requests/second = 18894.21241830065
>> --------------------------------------------------------------
>>
>> The issue we are trying to address why does performance degrade by 50%
>> with subsequent runs.
>>
>> Thanks,
>> Dimple
>>
>> On Thu, Nov 5, 2015 at 1:31 AM, Christopher Schultz <
>> ch...@christopherschultz.net> wrote:
>>
>>> Dimple,
>>>
>>> On 11/3/15 11:24 PM, dimple ranka wrote:
>>> > Also forgot to mention that setting maxKeepAliveRequests to -1 did not
>>> > help. As expected from the connector documentation the default value
>>> for
>>> > this attribute is 100 and we have a 60 user test set up.
>>> >
>>> > On Wed, Nov 4, 2015 at 8:18 AM, dimple ranka <dimplekra...@gmail.com>
>>> wrote:
>>> >
>>> >> Hi Mark,
>>> >>
>>> >> Another observation to be noted here is that system CPU usage shoots
>>> up
>>> >> for subsequent runs, especially for later runs.
>>> >>
>>> >> We have been looking into this issue for couple of weeks now and it is
>>> >> clear in the same environment with the same setup it doesn't
>>> reproduce on
>>> >> tomcat6. The moment we deploy the web application in a tomcat7
>>> container
>>> >> the slowness with subsequent runs shows up.
>>>
>>> Can you take some thread dumps to find out what Tomcat is doing? High
>>> CPU usage and lower request throughput sounds like a Poller problem
>>> since you are using NIO.
>>>
>>> -chris
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>>
>>>
>>
>
