We're upgrading our application servers from Jboss3.2.4 to JBoss4.0.1.
Currently, our web application runs fine (fine == 50k page requests and no
crashes, GC working properly).
However, after the upgrade to 4.0.1, the application still serves content but
at about 25k page requests (25/second) it kills the server with an
OutOfMemoryError.
I would normally consider this to be a developer memory leak issue, but the
codebase is IDENTICAL between the two server deployments. Running top on our
redhat machines, I watch the mem usage rise to 1.6GB and then the problems
start. It does not appear to be garbage collecting, which I guess it wouldn't
if it still had active objects (or so it thinks).
===========================================
The question is: Is something different about the XML/XSLT libaries between
jboss3 and jboss4 that would either cause an inconsistency or introduce a
memory leak in our deployment? If so, what would the appropriate jar/lib to
update to?
===========================================
System Knowledge:
I threw JMemprof up on there and I'm not really getting a great amount of
information. The peculiar piece of it is that the highest number of objects
seem to be of type "unknown" (gee, thanks).
Checking the method hierarchy, there are a LOT of xalan related method calls.
The stack traces ALWAYS come from the xalan libraries. Here's an example:
javax.xml.transform.TransformerException: java.lang.OutOfMemoryError
at
org.apache.xalan.extensions.ExtensionHandlerJavaPackage.callFunction(ExtensionHandlerJavaPackage.java:403)
at
org.apache.xalan.extensions.ExtensionHandlerJavaPackage.callFunction(ExtensionHandlerJavaPackage.java:426)
at
org.apache.xalan.extensions.ExtensionsTable.extFunction(ExtensionsTable.java:220)
at
org.apache.xalan.transformer.TransformerImpl.extFunction(TransformerImpl.java:437)
at
org.apache.xpath.functions.FuncExtFunction.execute(FuncExtFunction.java:199)
at org.apache.xpath.XPath.execute(XPath.java:268)
at
org.apache.xalan.templates.ElemVariable.getValue(ElemVariable.java:279)
at
org.apache.xalan.templates.ElemVariable.execute(ElemVariable.java:247)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at org.apache.xalan.templates.ElemChoose.execute(ElemChoose.java:127)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at org.apache.xalan.templates.ElemChoose.execute(ElemChoose.java:127)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at org.apache.xalan.templates.ElemChoose.execute(ElemChoose.java:127)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemElement.constructNode(ElemElement.java:338)
at org.apache.xalan.templates.ElemElement.execute(ElemElement.java:287)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at org.apache.xalan.templates.ElemChoose.execute(ElemChoose.java:127)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at org.apache.xalan.templates.ElemChoose.execute(ElemChoose.java:127)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at org.apache.xalan.templates.ElemChoose.execute(ElemChoose.java:127)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at org.apache.xalan.templates.ElemChoose.execute(ElemChoose.java:127)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at org.apache.xalan.templates.ElemIf.execute(ElemIf.java:161)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemLiteralResult.execute(ElemLiteralResult.java:682)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.templates.ElemTemplate.execute(ElemTemplate.java:393)
at
org.apache.xalan.templates.ElemCallTemplate.execute(ElemCallTemplate.java:247)
at
org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at
org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at
org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at
org.apache.xalan.transformer.TransformerImpl.applyTemplateToNode(TransformerImpl.java:2202)
at
org.apache.xalan.transformer.TransformerImpl.transformNode(TransformerImpl.java:1276)
at
org.apache.xalan.transformer.TransformerImpl.transform(TransformerImpl.java:673)
at
org.apache.xalan.transformer.TransformerImpl.transform(TransformerImpl.java:1192)
at
org.apache.xalan.transformer.TransformerImpl.transform(TransformerImpl.java:1170)
at com.mycompany.SpecificResponse.applyXslt(Unknown Source)
at com.mycompany.SpecificServlet.service(Unknown Source)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:75)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:186)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)
at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at
org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at
org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:66)
at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:150)
at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:54)
at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
at
org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
at
org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:534)
Caused by: java.lang.OutOfMemoryError
Obviously I changed the company name and class names.
I've tried a number of XSLT transformation optimizations:
1) static the TransformerFactory instance
2) hashmap cache the Templates objects so they can be used again, if possible
3) tFactory.setAttribute
("http://xml.apache.org/xalan/features/incremental";,
java.lang.Boolean.TRUE);
as was suggested by:
http://xml.apache.org/xalan-j/dtm.html#incremental
Most of these were already in our codebase before the switch.
===========================================
Any help would be most appreciated.
===========================================
-ian
View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3900935#3900935
Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=3900935
-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
JBoss-user mailing list
JBoss-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jboss-user
