[jira] Commented: (GERONIMO-4623) Webservice request handling consuming considerable CPU
[ https://issues.apache.org/jira/browse/GERONIMO-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713182#action_12713182 ] Murty Dasari commented on GERONIMO-4623: Thanks Ivan, I'll check with v2.1.4. You can close this issue as changes in class-loader seems more appropriate, if that doesn't work then I'll open a bug against that JIRA. cheers - Murty > Webservice request handling consuming considerable CPU > -- > > Key: GERONIMO-4623 > URL: https://issues.apache.org/jira/browse/GERONIMO-4623 > Project: Geronimo > Issue Type: Bug > Security Level: public(Regular issues) >Affects Versions: 2.1.2, 2.1.3, 2.1.4 > Environment: *OS*: Windows XP, Windows 2003 Server Multi-Core > *JRE*: 1.6.0.10 >Reporter: Murty Dasari >Assignee: Ivan > Fix For: 2.1.5, 2.2 > > > I've a typical web-service that gets polled every N minutes, I noticed that > when it is servicing a request the CPU usage is hitting close to 80%. I > expect to have multiple clients to this web-service so it is going to be a > problem. > Based on what I deduced, the following stack trace is trying to load several > of JAR files and not able to load/find them. This is resulting in lot of > kernel level calls thus CPU utilization. > -- STACK of the thread processing a WS request and trying to load > JARs - > "http-0.0.0.0-8080-1" daemon prio=6 tid=0x437c7c00 nid=0x1700 runnable > [0x432ce000..0x432cfa14] >java.lang.Thread.State: RUNNABLE > at java.io.WinNTFileSystem.checkAccess(Native Method) > at java.io.File.canRead(File.java:691) > at > org.apache.geronimo.kernel.classloader.UrlResourceFinder.getClassPath(UrlResourceFinder.java:151) > > at > org.apache.geronimo.kernel.classloader.UrlResourceFinder.findResource(UrlResourceFinder.java:95) > > - locked <0x05420118> (a java.lang.Object) > at > org.apache.geronimo.kernel.classloader.JarFileClassLoader$3.run(JarFileClassLoader.java:179) > > at java.security.AccessController.doPrivileged(Native Method) > at > org.apache.geronimo.kernel.classloader.JarFileClassLoader.findResource(JarFileClassLoader.java:177) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:580) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1168) > at > org.apache.catalina.loader.WebappClassLoader.getResourceAsStream(WebappClassLoader.java:1170) > > at javax.xml.transform.SecuritySupport$4.run(SecuritySupport.java:94) > at java.security.AccessController.doPrivileged(Native Method) > at > javax.xml.transform.SecuritySupport.getResourceAsStream(SecuritySupport.java:87) > > at > javax.xml.transform.FactoryFinder.findJarServiceProvider(FactoryFinder.java:250) > > at javax.xml.transform.FactoryFinder.find(FactoryFinder.java:223) > at > javax.xml.transform.TransformerFactory.newInstance(TransformerFactory.java:102) > > at > org.apache.geronimo.axis2.WSDLQueryHandler.writeTo(WSDLQueryHandler.java:246) > at > org.apache.geronimo.axis2.WSDLQueryHandler.writeTo(WSDLQueryHandler.java:240) > at > org.apache.geronimo.axis2.WSDLQueryHandler.writeResponse(WSDLQueryHandler.java:164) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.processGETRequest(Axis2WebServiceContainer.java:321) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.doService2(Axis2WebServiceContainer.java:268) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.doService(Axis2WebServiceContainer.java:205) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.getWsdl(Axis2WebServiceContainer.java:161) > > at > org.apache.geronimo.tomcat.TomcatEJBWebServiceContext$EJBWebServiceValve.invoke(TomcatEJBWebServiceContext.java:159) > > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > > at > org.apache.catalina.valves.AccessLog
[jira] Commented: (GERONIMO-4623) Webservice request handling consuming considerable CPU
[ https://issues.apache.org/jira/browse/GERONIMO-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712537#action_12712537 ] Ivan commented on GERONIMO-4623: Hi, Murty, I found the Geronimo version you version is 2.1.3, could you please try the new version 2.1.4, we did some classloader optimization. It will cache the failed found resources. Thanks. Ivan > Webservice request handling consuming considerable CPU > -- > > Key: GERONIMO-4623 > URL: https://issues.apache.org/jira/browse/GERONIMO-4623 > Project: Geronimo > Issue Type: Bug > Security Level: public(Regular issues) >Affects Versions: 2.1.2, 2.1.3, 2.1.4 > Environment: *OS*: Windows XP, Windows 2003 Server Multi-Core > *JRE*: 1.6.0.10 >Reporter: Murty Dasari >Assignee: Ivan > Fix For: 2.1.5, 2.2 > > > I've a typical web-service that gets polled every N minutes, I noticed that > when it is servicing a request the CPU usage is hitting close to 80%. I > expect to have multiple clients to this web-service so it is going to be a > problem. > Based on what I deduced, the following stack trace is trying to load several > of JAR files and not able to load/find them. This is resulting in lot of > kernel level calls thus CPU utilization. > -- STACK of the thread processing a WS request and trying to load > JARs - > "http-0.0.0.0-8080-1" daemon prio=6 tid=0x437c7c00 nid=0x1700 runnable > [0x432ce000..0x432cfa14] >java.lang.Thread.State: RUNNABLE > at java.io.WinNTFileSystem.checkAccess(Native Method) > at java.io.File.canRead(File.java:691) > at > org.apache.geronimo.kernel.classloader.UrlResourceFinder.getClassPath(UrlResourceFinder.java:151) > > at > org.apache.geronimo.kernel.classloader.UrlResourceFinder.findResource(UrlResourceFinder.java:95) > > - locked <0x05420118> (a java.lang.Object) > at > org.apache.geronimo.kernel.classloader.JarFileClassLoader$3.run(JarFileClassLoader.java:179) > > at java.security.AccessController.doPrivileged(Native Method) > at > org.apache.geronimo.kernel.classloader.JarFileClassLoader.findResource(JarFileClassLoader.java:177) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:580) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1168) > at > org.apache.catalina.loader.WebappClassLoader.getResourceAsStream(WebappClassLoader.java:1170) > > at javax.xml.transform.SecuritySupport$4.run(SecuritySupport.java:94) > at java.security.AccessController.doPrivileged(Native Method) > at > javax.xml.transform.SecuritySupport.getResourceAsStream(SecuritySupport.java:87) > > at > javax.xml.transform.FactoryFinder.findJarServiceProvider(FactoryFinder.java:250) > > at javax.xml.transform.FactoryFinder.find(FactoryFinder.java:223) > at > javax.xml.transform.TransformerFactory.newInstance(TransformerFactory.java:102) > > at > org.apache.geronimo.axis2.WSDLQueryHandler.writeTo(WSDLQueryHandler.java:246) > at > org.apache.geronimo.axis2.WSDLQueryHandler.writeTo(WSDLQueryHandler.java:240) > at > org.apache.geronimo.axis2.WSDLQueryHandler.writeResponse(WSDLQueryHandler.java:164) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.processGETRequest(Axis2WebServiceContainer.java:321) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.doService2(Axis2WebServiceContainer.java:268) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.doService(Axis2WebServiceContainer.java:205) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.getWsdl(Axis2WebServiceContainer.java:161) > > at > org.apache.geronimo.tomcat.TomcatEJBWebServiceContext$EJBWebServiceValve.invoke(TomcatEJBWebServiceContext.java:159) > > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > > at > org.apache.catalina.valves.AccessLogValve.i
[jira] Commented: (GERONIMO-4623) Webservice request handling consuming considerable CPU
[ https://issues.apache.org/jira/browse/GERONIMO-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712402#action_12712402 ] Ivan commented on GERONIMO-4623: Thanks, Murty. At the first glance at the invocation stack, the search behavior occurs in the method com.ibm.wsdl.xml.WSDLWriterImpl.getDocument.Not sure whether I could change its behavior. I will do some investigation on it. Ivan > Webservice request handling consuming considerable CPU > -- > > Key: GERONIMO-4623 > URL: https://issues.apache.org/jira/browse/GERONIMO-4623 > Project: Geronimo > Issue Type: Bug > Security Level: public(Regular issues) >Affects Versions: 2.1.2, 2.1.3, 2.1.4 > Environment: *OS*: Windows XP, Windows 2003 Server Multi-Core > *JRE*: 1.6.0.10 >Reporter: Murty Dasari >Assignee: Ivan > Fix For: 2.1.5, 2.2 > > > I've a typical web-service that gets polled every N minutes, I noticed that > when it is servicing a request the CPU usage is hitting close to 80%. I > expect to have multiple clients to this web-service so it is going to be a > problem. > Based on what I deduced, the following stack trace is trying to load several > of JAR files and not able to load/find them. This is resulting in lot of > kernel level calls thus CPU utilization. > -- STACK of the thread processing a WS request and trying to load > JARs - > "http-0.0.0.0-8080-1" daemon prio=6 tid=0x437c7c00 nid=0x1700 runnable > [0x432ce000..0x432cfa14] >java.lang.Thread.State: RUNNABLE > at java.io.WinNTFileSystem.checkAccess(Native Method) > at java.io.File.canRead(File.java:691) > at > org.apache.geronimo.kernel.classloader.UrlResourceFinder.getClassPath(UrlResourceFinder.java:151) > > at > org.apache.geronimo.kernel.classloader.UrlResourceFinder.findResource(UrlResourceFinder.java:95) > > - locked <0x05420118> (a java.lang.Object) > at > org.apache.geronimo.kernel.classloader.JarFileClassLoader$3.run(JarFileClassLoader.java:179) > > at java.security.AccessController.doPrivileged(Native Method) > at > org.apache.geronimo.kernel.classloader.JarFileClassLoader.findResource(JarFileClassLoader.java:177) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:580) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at > org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) > > at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1168) > at > org.apache.catalina.loader.WebappClassLoader.getResourceAsStream(WebappClassLoader.java:1170) > > at javax.xml.transform.SecuritySupport$4.run(SecuritySupport.java:94) > at java.security.AccessController.doPrivileged(Native Method) > at > javax.xml.transform.SecuritySupport.getResourceAsStream(SecuritySupport.java:87) > > at > javax.xml.transform.FactoryFinder.findJarServiceProvider(FactoryFinder.java:250) > > at javax.xml.transform.FactoryFinder.find(FactoryFinder.java:223) > at > javax.xml.transform.TransformerFactory.newInstance(TransformerFactory.java:102) > > at > org.apache.geronimo.axis2.WSDLQueryHandler.writeTo(WSDLQueryHandler.java:246) > at > org.apache.geronimo.axis2.WSDLQueryHandler.writeTo(WSDLQueryHandler.java:240) > at > org.apache.geronimo.axis2.WSDLQueryHandler.writeResponse(WSDLQueryHandler.java:164) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.processGETRequest(Axis2WebServiceContainer.java:321) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.doService2(Axis2WebServiceContainer.java:268) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.doService(Axis2WebServiceContainer.java:205) > > at > org.apache.geronimo.axis2.Axis2WebServiceContainer.getWsdl(Axis2WebServiceContainer.java:161) > > at > org.apache.geronimo.tomcat.TomcatEJBWebServiceContext$EJBWebServiceValve.invoke(TomcatEJBWebServiceContext.java:159) > > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > > at > org.apach
[jira] Commented: (GERONIMO-4623) Webservice request handling consuming considerable CPU
[ https://issues.apache.org/jira/browse/GERONIMO-4623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712294#action_12712294 ] Murty Dasari commented on GERONIMO-4623: Hi Ivan, Thanks for a quick turnaround on this issue. While the fix that you incorporated reduced the CPU utilization substantially it didn't completely eliminate the issue, it seems there is similar issue with WSDLQueryHandler::writeResponse method as well. Please see the stack trace below: "http-0.0.0.0-8080-3" daemon prio=6 tid=0x451dd400 nid=0xc20 runnable [0x46ace000..0x46acfc14] java.lang.Thread.State: RUNNABLE at java.io.WinNTFileSystem.checkAccess(Native Method) at java.io.File.canRead(File.java:691) at org.apache.geronimo.kernel.classloader.UrlResourceFinder.getClassPath(UrlResourceFinder.java:151) at org.apache.geronimo.kernel.classloader.UrlResourceFinder.findResource(UrlResourceFinder.java:95) - locked <0x0501abe8> (a java.lang.Object) at org.apache.geronimo.kernel.classloader.JarFileClassLoader$3.run(JarFileClassLoader.java:179) at java.security.AccessController.doPrivileged(Native Method) at org.apache.geronimo.kernel.classloader.JarFileClassLoader.findResource(JarFileClassLoader.java:177) at org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:580) at org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) at org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) at org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) at org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) at org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) at org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) at org.apache.geronimo.kernel.config.MultiParentClassLoader.getResource(MultiParentClassLoader.java:565) at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1168) at org.apache.catalina.loader.WebappClassLoader.getResourceAsStream(WebappClassLoader.java:1170) at javax.xml.parsers.SecuritySupport$4.run(SecuritySupport.java:96) at java.security.AccessController.doPrivileged(Native Method) at javax.xml.parsers.SecuritySupport.getResourceAsStream(SecuritySupport.java:89) at javax.xml.parsers.FactoryFinder.findJarServiceProvider(FactoryFinder.java:250) at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:223) at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:123) at com.ibm.wsdl.xml.WSDLWriterImpl.getDocument(Unknown Source) at com.ibm.wsdl.xml.WSDLWriterImpl.getDocument(Unknown Source) at org.apache.geronimo.axis2.WSDLQueryHandler.writeResponse(WSDLQueryHandler.java:128) at org.apache.geronimo.axis2.Axis2WebServiceContainer.processGETRequest(Axis2WebServiceContainer.java:321) at org.apache.geronimo.axis2.Axis2WebServiceContainer.doService2(Axis2WebServiceContainer.java:268) at org.apache.geronimo.axis2.Axis2WebServiceContainer.doService(Axis2WebServiceContainer.java:205) at org.apache.geronimo.axis2.Axis2WebServiceContainer.getWsdl(Axis2WebServiceContainer.java:161) at org.apache.geronimo.tomcat.TomcatEJBWebServiceContext$EJBWebServiceValve.invoke(TomcatEJBWebServiceContext.java:159) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) Is it something that is possible to fix as well? The line numbers in WSDLQueryHandler might not match with the ASF SVN as I manually added your changes to test the fix. best regards - mdasari > Webservice request handling consuming considerable CPU > -- > > Key: GERONIMO-4623 > URL: https://issues.apache.org/jira/browse/GERONIMO-4623 > Project: Geronimo >