Hello all,
My tomcat (7.0.72) hosts several web aplications in the server (based in
linux 6.8).
There are many clients or 3rd party applications working as client to my
server (having tomcat and web applications).
There are instances when poorly designed client application can affect
severly to Tomcat. Connections/sessions not being reused or closed is one
of them.
My question is the way to prove/identify such symptoms of the 3rd party
applications.
I have a situation where all the applications and web/GUI access slows down
and tomcat shows as consuming 100% cpu
(even though overall CPU is less)
My diagnosis shows memory tests for tomcat failing (less than 100KB of free
heap left), And so i generated memory heap dump and thread dumps.
Below are the results. Based on below, does this qualify for a poorly
socket implemetation ? Any thoughts will be helpful.
Memory heap dump generated is of
Size: 787.3 MB Classes: 139k Objects: 19.3m Class Loader: 1.6k
Overview shows 580.9 MB occupied by remainder's.
Problem suspect:-
465 MB occupied by remainder
152.2 MB- leak suspect 1
6 instances of "com.sun.xml.bind.v2.runtime.JAXBContextImpl", loaded by
"org.apache.catalina.loader.WebappClassLoader @ 0xacc38e98" occupy
159,582,744 (19.33%) bytes.
91 MB- leak suspect 2
58 instances of "org.apache.catalina.loader.WebappClassLoader", loaded by
"java.net.URLClassLoader @ 0xa6b8e038" occupy 95,396,344 (11.56%) bytes
79.1 MB - leak suspect 3
4 instances of "com.rsa.sslj.x.aO", loaded by
"sun.misc.Launcher$ExtClassLoader @ 0xa6b763b0" occupy 82,968,424 (10.05%)
bytes.
In the thread dumps I see these threads repeatedly. I wonder these pointing
to com.rsa.sslj.x.
"http-bio-8443-exec-230" daemon prio=10 tid=0x1130a400 nid=0x411b runnable
[0x01be1000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:153)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at com.rsa.sslj.x.ap.c(Unknown Source)
at com.rsa.sslj.x.ap.a(Unknown Source)
at com.rsa.sslj.x.ap.b(Unknown Source)
at com.rsa.sslj.x.ap.b(Unknown Source)
at com.rsa.sslj.x.al.read(Unknown Source)
at
org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:519)
at
org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:504)
at
org.apache.coyote.http11.Http11Processor.setRequestLineReadTimeout(Http11Processor.java:168)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:998)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318)
- locked <0x8f1f68d8> (a org.apache.tomcat.util.net.SocketWrapper)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
"http-bio-8443-exec-232" daemon prio=10 tid=0x11143400 nid=0x411d runnable
[0x0943a000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:153)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at com.rsa.sslj.x.ap.c(Unknown Source)
at com.rsa.sslj.x.ap.a(Unknown Source)
at com.rsa.sslj.x.ap.a(Unknown Source)
- locked <0x8f52fd38> (a com.rsa.sslj.x.ap)
at com.rsa.sslj.x.ap.j(Unknown Source)
- locked <0x8f52fd38> (a com.rsa.sslj.x.ap)
at com.rsa.sslj.x.ap.i(Unknown Source)
- locked <0x8f52fd38> (a com.rsa.sslj.x.ap)
at com.rsa.sslj.x.aS.getSession(Unknown Source)
at
org.apache.tomcat.util.net.jsse.JSSESocketFactory.handshake(JSSESocketFactory.java:289)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:304)
- locked <0x8f53f438> (a org.apache.tomcat.util.net.SocketWrapper)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
-Th
