Hello, A) Version info of my system:
Server version: Apache Tomcat/7.0.53 Server built: May 1 2014 10:53:10 Server number: 7.0.53.0 OS Name: Linux OS Version: 3.0.58-0.6.2.1.5158.0.PTF-default Architecture: amd64 JVM Version: 1.6.0_75-b31 Deployed app server: provisioningagent-web da-00:/opt/tomcat/webapps # ls -trl total 36480 -rwxr-x--- 1 mmas mmas 10444010 Jul 7 16:53 enumprovisioning-web.war -rwxr-x--- 1 mmas mmas 26827266 Jul 7 17:03 provisioningagent-web.war drwxr-xr-x 7 mmas mmas 480 Sep 24 09:53 enumprovisioning-web drwxr-xr-x 9 mmas mmas 980 Sep 24 09:53 provisioningagent-web da-00:/opt/tomcat/webapps # B) Issue: Below exception happens randomly (it does not happen all the time, but often): Jul 10, 2014 6:15:19 AM org.apache.catalina.loader.WebappClassLoader loadClass INFO: Illegal access: this web application instance has been stopped already. Could not load com.sun.org.apache.xerces.internal.impl .dv.dtd.DTDDVFactoryImpl. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact. java.lang.IllegalStateException at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1600) at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559) at com.sun.org.apache.xerces.internal.utils.ObjectFactory.findProviderClass(ObjectFactory.java:358) at com.sun.org.apache.xerces.internal.utils.ObjectFactory.newInstance(ObjectFactory.java:303) at com.sun.org.apache.xerces.internal.utils.ObjectFactory.newInstance(ObjectFactory.java:289) at com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory.getInstance(DTDDVFactory.java:60) at com.sun.org.apache.xerces.internal.impl.dv.DTDDVFactory.getInstance(DTDDVFactory.java:45) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.<init>(XML11Configuration.java:538) at com.sun.org.apache.xerces.internal.parsers.XIncludeAwareParserConfiguration.<init>(XIncludeAwareParserConfiguration.java:125) at com.sun.org.apache.xerces.internal.parsers.XIncludeAwareParserConfiguration.<init>(XIncludeAwareParserConfiguration.java:86) at com.sun.org.apache.xerces.internal.parsers.DOMParser.<init>(DOMParser.java:133) at com.sun.org.apache.xerces.internal.parsers.DOMParser.<init>(DOMParser.java:117) at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.<init>(DocumentBuilderImpl.java:115) at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(DocumentBuilderFactoryImpl.java:72) at com.ericsson.messaging.oe.impl.bpmanagement.bpfaultmgt.ActiveAlarmList.update(ActiveAlarmList.java:597) at com.ericsson.messaging.oe.impl.bpmanagement.bpfaultmgt.ActiveAlarmList.read(ActiveAlarmList.java:823) at com.ericsson.messaging.oe.impl.bpmanagement.bpfaultmgt.SAFaultDispatc her$ActiveAlarmListVerificationTask.run(SAFaultDispatcher.java:223) at java.util.TimerThread.mainLoop(Timer.java:512) at java.util.TimerThread.run(Timer.java:462) We have done a lot of searches on the internet and found that many others have similar issue but we couldn't find a post which explains the root cause and how to prevent it to happen. Example: http://www.devconnectprogram.com/forums/posts/list/1641.page http://apache-xml-project.6118.n7.nabble.com/Could-not-load-DTDDVFactoryImpl-xerces-jar-td1020.html The most strange part is that it does not happen all the time, but sometimes (often though). We did verify the jar file (/opt/java/jre/lib/rt.jar) which contains com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl. It is always there, and the class can be correctly loaded when this does not happen. C) More info Here are the relevant code: import javax.xml.parsers.DocumentBuilderFactory; private static DocumentBuilderFactory factory; factory = DocumentBuilderFactory.newInstance(); try { DocumentBuilder docBuilder = factory.newDocumentBuilder(); } catch (Exception e) {} We stumbled into this page http://www.databasesandlife.com/java-always-explicitly-specify-which-xml-parser-to-use/ The conclusion of the page is that: Never do the following: DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance(); Because the new instance you get may not be what you want due to the system properties might have been changed by other apps deployed in the same container. Yes this is exactly what our code ActiveAlarmList.java was doing! So we thought that this might be the problem as we do "DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();" in our code. However further investigation reveals that it's not that simple. We printed out the actual classes our code wants to load (under the condition when all is working fine - the exception not occruing), they are: com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl Comparing these classes with the ones in the stack trace: at com.sun.org.apache.xerces.internal.parsers.DOMParser.<init>(DOMParser.java:117) at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.<init>(DocumentBuilderImpl.java:115) at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl.newDocumentBuilder(DocumentBuilderFactoryImpl.java:72) at com.ericsson.messaging.oe.impl.bpmanagement.bpfaultmgt.ActiveAlarmList.update(ActiveAlarmList.java:597) We can see that the classes match. This means that when the exception happened, we were NOT loading the wrong DocumentBuilderFactoryImpl/DocumentBuilderImpl class, hence we did not have the issue explained by the page - getting the wrong Factory class; we actually did have the right Factory class. So what's explained in that page is NOT the root cause of our problem. Our problem is deeper: something happened that a few steps down in the trace: com.sun.org.apache.xerces.internal.utils.ObjectFactory.findProviderClass fails to load the com.sun.org.apache.xerces.internal.impl.dv.dtd.DTDDVFactoryImpl class (which most of the times it can). Any help would be greatly appreciated! Cheng --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org