Webservice request handling consuming considerable CPU
------------------------------------------------------
Key: GERONIMO-4623
URL: https://issues.apache.org/jira/browse/GERONIMO-4623
Project: Geronimo
Issue Type: Bug
Security Level: public (Regular issues)
Affects Versions: 2.1.4, 2.1.3, 2.1.2
Environment: *OS*: Windows XP, Windows 2003 Server Multi-Core
*JRE*: 1.6.0.10
Reporter: Murty Dasari
I've a typical web-service that gets polled every N minutes, I noticed that
when it is servicing a request the CPU usage is hitting close to 80%. I expect
to have multiple clients to this web-service so it is going to be a problem.
Based on what I deduced, the following stack trace is trying to load several of
JAR files and not able to load/find them. This is resulting in lot of kernel
level calls thus CPU utilization.
---------- STACK of the thread processing a WS request and trying to load JARs
-------------
"http-0.0.0.0-8080-1" daemon prio=6 tid=0x437c7c00 nid=0x1700 runnable
[0x432ce000..0x432cfa14]
java.lang.Thread.State: RUNNABLE
at java.io.WinNTFileSystem.checkAccess(Native Method)
at java.io.File.canRead(File.java:691)
at
org.apache.geronimo.kernel.classloader.UrlResourceFinder.getClassPath(UrlResourceFinder.java:151)
at
org.apache.geronimo.kernel.classloader.UrlResourceFinder.findResource(UrlResourceFinder.java:95)
- locked <0x05420118> (a java.lang.Object)
at
org.apache.geronimo.kernel.classloader.JarFileClassLoader$3.run(JarFileClassLoader.java:179)
at java.security.AccessController.doPrivileged(Native Method)
at
org.apache.geronimo.kernel.classloader.JarFileClassLoader.findResource(JarFileClassLoader.java:177)
at
org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:580)
at
org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565)
at
org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565)
at
org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565)
at
org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565)
at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1168)
at
org.apache.catalina.loader.WebappClassLoader.getResourceAsStream(WebappClassLoader.java:1170)
at javax.xml.transform.SecuritySupport$4.run(SecuritySupport.java:94)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.xml.transform.SecuritySupport.getResourceAsStream(SecuritySupport.java:87)
at
javax.xml.transform.FactoryFinder.findJarServiceProvider(FactoryFinder.java:250)
at javax.xml.transform.FactoryFinder.find(FactoryFinder.java:223)
at
javax.xml.transform.TransformerFactory.newInstance(TransformerFactory.java:102)
at
org.apache.geronimo.axis2.WSDLQueryHandler.writeTo(WSDLQueryHandler.java:246)
at
org.apache.geronimo.axis2.WSDLQueryHandler.writeTo(WSDLQueryHandler.java:240)
at
org.apache.geronimo.axis2.WSDLQueryHandler.writeResponse(WSDLQueryHandler.java:164)
at
org.apache.geronimo.axis2.Axis2WebServiceContainer.processGETRequest(Axis2WebServiceContainer.java:321)
at
org.apache.geronimo.axis2.Axis2WebServiceContainer.doService2(Axis2WebServiceContainer.java:268)
at
org.apache.geronimo.axis2.Axis2WebServiceContainer.doService(Axis2WebServiceContainer.java:205)
at
org.apache.geronimo.axis2.Axis2WebServiceContainer.getWsdl(Axis2WebServiceContainer.java:161)
at
org.apache.geronimo.tomcat.TomcatEJBWebServiceContext$EJBWebServiceValve.invoke(TomcatEJBWebServiceContext.java:159)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:619)
---------------------
The following are files this java process is trying to read/load. (Output from
Windows process monitor). It seems to be doing in a loop as I see continuous
system calls to read these jar files repeatedly (multiple times per every
request)
---------------------------------------
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\bind\jaxb-xjc\2.0.5\activation.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\bind\jaxb-xjc\2.0.5\jaxb-api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\bind\jaxb-xjc\2.0.5\jaxb-impl.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\bind\jaxb-xjc\2.0.5\jsr173_1.0_api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\messaging\saaj\saaj-impl\1.3\activation.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\messaging\saaj\saaj-impl\1.3\saaj-api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\jaxb-api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\jaxb-impl.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\jaxws-api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\jsr173_api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\jsr181-api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\jsr250-api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\resolver.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\saaj-api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\saaj-impl.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\2.0\sjsxp.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\jaxb\lib\jaxb-api.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-rt\jaxb\lib\jaxb-impl.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-tools\2.0\jaxb-xjc.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-tools\2.0\jaxws-rt.jar
geronimo-tomcat6-javaee5-2.1.3\repository\com\sun\xml\ws\jaxws-tools\2.0\relaxngDatatype.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_cs.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_de_DE.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_es.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_fr.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_hu.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_it.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_ja_JP.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_ko_KR.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_pl.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_pt_BR.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_ru.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_zh_CN.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derby\10.4.1.3\derbyLocale_zh_TW.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_cs.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_de_DE.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_es.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_fr.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_hu.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_it.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_ja_JP.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_ko_KR.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_pl.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_pt_BR.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_ru.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_zh_CN.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbyclient\10.4.1.3\derbyLocale_zh_TW.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbynet\10.4.1.3\derby.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_cs.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_de_DE.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_es.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_fr.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_hu.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_it.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_ja_JP.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_ko_KR.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_pl.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_pt_BR.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_ru.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_zh_CN.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\derby\derbytools\10.4.1.3\derbyLocale_zh_TW.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\openejb\openejb-core\3.0\openejb-client-3.0.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\openejb\openejb-core\3.0\openejb-loader-3.0.jar
geronimo-tomcat6-javaee5-2.1.3\repository\org\apache\openejb\openejb-core\3.0\xbean-finder-3.4-r636442.jar
---------------------------------------
I can see references to these files in respective MANIFEST files, for e.g:
derby/10.4.1.3/derby-10.4.1.3.jar file has following libs in the class path
Class-Path: derbyLocale_cs.jar derbyLocale_de_DE.jar derbyLocale_es.jar
derbyLocale_fr.jar derbyLocale_hu.jar derbyLocale_it.jar derbyLocale_ja_JP.jar
derbyLocale_ko_KR.jar derbyLocale_pl.jar derbyLocale_pt_BR.jar
derbyLocale_ru.jar derbyLocale_zh_CN.jar derbyLocale_zh_TW.jar
Similarly, jarb-impl-2.0.5.jar has
Class-Path: jaxb-api.jar activation.jar jsr173_1.0_api.jar jaxb1-impl.
jar
------------------------
The following is the e-mail thread related to this issue on Geronimo Users Forum
http://www.nabble.com/Webservice-handling-consuming-considerable-CPU-%28Windows%29-tt23303330s134.html#a23399539
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
