Hi,
I'm running Tomcat 6.0.16 on a 32 bit Debian Etch using Java 6 update
12. The problem I have is that occasionally after I restart Tomcat, the
startup fails with NoClassDefFoundError or ClassNotFoundException for
every servlet listener that's being added from web.xml.
If I restart Tomcat a couple of times, the exceptions mysteriously
vanish and startup succeeds. I've seen the exact same exceptions
happening when starting up another Tomcat instance I manage, which is a
Tomcat 6.0.18 on a 64 bit Debian Lenny also using Java 6 update 12.
The Tomcat 6.0.16 instance produces the following strack trace when
starting up:
Jul 23, 2009 11:32:41 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal
performance in production environments was not found on the
java.library.path:
/opt/jdk1.6.0_12/jre/lib/i386/server:/opt/jdk1.6.0_12/jre/lib/i386:/opt/jdk1.6.0_12/jre/../lib/i386:/usr/java/packages/l
ib/i386:/lib:/usr/lib
Jul 23, 2009 11:32:41 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 506 ms
Jul 23, 2009 11:32:41 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Jul 23, 2009 11:32:41 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.16
Jul 23, 2009 11:32:42 PM org.apache.catalina.core.StandardContext
addApplicationListener
INFO: The listener
"com.xmx.cyberdyme.system.settings.ApplicationSettings" is already
configured for this context. The duplicate definition has been ignored.
Jul 23, 2009 11:32:42 PM org.apache.catalina.core.StandardContext
addApplicationListener
INFO: The listener "org.quartz.ee.servlet.QuartzInitializerListener" is
already configured for this context. The duplicate definition has been
ignored.
Jul 23, 2009 11:32:42 PM org.apache.catalina.core.StandardContext
listenerStart
SEVERE: Exception sending context initialized event to listener instance
of class com.xmx.cyberdyme.system.settings.ApplicationSettings
java.lang.NoClassDefFoundError:
com/xmx/cyberdyme/system/settings/AbstractWebApplicationSettings
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1275)
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1206)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at
com.xmx.cyberdyme.system.settings.ApplicationSettings.contextInitialized(ApplicationSettings.java:52)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4350)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
at
org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:924)
at
org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:887)
at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1147)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at
org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at
org.apache.catalina.core.StandardService.start(StandardService.java:516)
at
org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.lang.ClassNotFoundException:
com.xmx.cyberdyme.system.settings.AbstractWebApplicationSettings
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 41 more
Jul 23, 2009 11:32:42 PM org.quartz.simpl.SimpleThreadPool initialize
INFO: Job execution threads will use class loader of thread: main
Jul 23, 2009 11:32:42 PM org.quartz.core.QuartzScheduler <init>
INFO: Quartz Scheduler v.1.5.2 created.
Jul 23, 2009 11:32:42 PM org.quartz.plugins.xml.JobInitializationPlugin
initialize
INFO: Registering Quartz Job Initialization Plug-in.
Jul 23, 2009 11:32:42 PM org.quartz.simpl.RAMJobStore initialize
INFO: RAMJobStore initialized.
Jul 23, 2009 11:32:42 PM org.quartz.impl.StdSchedulerFactory instantiate
INFO: Quartz scheduler 'DefaultQuartzScheduler' initialized from the
specified file : 'quartz.properties' from the class resource path.
Jul 23, 2009 11:32:42 PM org.quartz.impl.StdSchedulerFactory instantiate
INFO: Quartz scheduler version: 1.5.2
Jul 23, 2009 11:32:42 PM org.quartz.xml.JobSchedulingDataProcessor
processFile
INFO: Parsing XML file: com/xmx/cyberdyme/resources/jobs/systemJobs.xml
with systemId: com/xmx/cyberdyme/resources/jobs/systemJobs.xml
validating: false validating schema:
jar:file:/cyberdyme/www/moon_2.0.4_u4/WEB-INF/lib/quartz-1.5.2.jar!/org/quartz/xml/job_scheduling_data_1_5
.xsd
Jul 23, 2009 11:32:42 PM org.apache.commons.digester.Digester endElement
SEVERE: End event threw error
java.lang.NoClassDefFoundError:
com/xmx/cyberdyme/system/scheduling/AbstractSimpleJob
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1275)
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1206)
at
org.apache.commons.beanutils.converters.ClassConverter.convert(ClassConverter.java:118)
at
org.apache.commons.beanutils.ConvertUtilsBean.convert(ConvertUtilsBean.java:428)
at
org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:1002)
at
org.apache.commons.beanutils.BeanUtils.setProperty(BeanUtils.java:313)
at
org.apache.commons.digester.BeanPropertySetterRule.end(BeanPropertySetterRule.java:197)
at
org.apache.commons.digester.Digester.endElement(Digester.java:1067)
at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:601)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1774)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2930)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
at
com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at
com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
at org.apache.commons.digester.Digester.parse(Digester.java:1572)
at
org.quartz.xml.JobSchedulingDataProcessor.processFile(JobSchedulingDataProcessor.java:430)
at
org.quartz.xml.JobSchedulingDataProcessor.processFileAndScheduleJobs(JobSchedulingDataProcessor.java:499)
at
org.quartz.xml.JobSchedulingDataProcessor.processFileAndScheduleJobs(JobSchedulingDataProcessor.java:485)
at
org.quartz.plugins.xml.JobInitializationPlugin.processFile(JobInitializationPlugin.java:401)
at
org.quartz.plugins.xml.JobInitializationPlugin.start(JobInitializationPlugin.java:375)
at
org.quartz.core.QuartzScheduler.startPlugins(QuartzScheduler.java:1919)
at org.quartz.core.QuartzScheduler.start(QuartzScheduler.java:397)
at org.quartz.impl.StdScheduler.start(StdScheduler.java:147)
at
org.quartz.ee.servlet.QuartzInitializerListener.contextInitialized(QuartzInitializerListener.java:145)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4350)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
at
org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:924)
at
org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:887)
at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1147)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
Caused by: java.lang.ClassNotFoundException:
com.xmx.cyberdyme.system.scheduling.AbstractSimpleJob
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Jul 23, 2009 11:32:42 PM org.apache.catalina.core.StandardContext
listenerStart
SEVERE: Exception sending context initialized event to listener instance
of class org.quartz.ee.servlet.QuartzInitializerListener
java.lang.NoClassDefFoundError:
com/xmx/cyberdyme/system/scheduling/AbstractSimpleJob
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1275)
at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1206)
at
org.apache.commons.beanutils.converters.ClassConverter.convert(ClassConverter.java:118)
at
org.apache.commons.beanutils.ConvertUtilsBean.convert(ConvertUtilsBean.java:428)
at
org.apache.commons.beanutils.BeanUtilsBean.setProperty(BeanUtilsBean.java:1002)
at
org.apache.commons.beanutils.BeanUtils.setProperty(BeanUtils.java:313)
at
org.apache.commons.digester.BeanPropertySetterRule.end(BeanPropertySetterRule.java:197)
at
org.apache.commons.digester.Digester.endElement(Digester.java:1067)
at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:601)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1774)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2930)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
at
com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at
com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
at org.apache.commons.digester.Digester.parse(Digester.java:1572)
at
org.quartz.xml.JobSchedulingDataProcessor.processFile(JobSchedulingDataProcessor.java:430)
at
org.quartz.xml.JobSchedulingDataProcessor.processFileAndScheduleJobs(JobSchedulingDataProcessor.java:499)
at
org.quartz.xml.JobSchedulingDataProcessor.processFileAndScheduleJobs(JobSchedulingDataProcessor.java:485)
at
org.quartz.plugins.xml.JobInitializationPlugin.processFile(JobInitializationPlugin.java:401)
at
org.quartz.plugins.xml.JobInitializationPlugin.start(JobInitializationPlugin.java:375)
at
org.quartz.core.QuartzScheduler.startPlugins(QuartzScheduler.java:1919)
at org.quartz.core.QuartzScheduler.start(QuartzScheduler.java:397)
at org.quartz.impl.StdScheduler.start(StdScheduler.java:147)
at
org.quartz.ee.servlet.QuartzInitializerListener.contextInitialized(QuartzInitializerListener.java:145)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4350)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
at
org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:924)
at
org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:887)
at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1147)
at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at
org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at
org.apache.catalina.core.StandardService.start(StandardService.java:516)
at
org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.lang.ClassNotFoundException:
com.xmx.cyberdyme.system.scheduling.AbstractSimpleJob
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 64 more
Jul 23, 2009 11:32:42 PM org.apache.catalina.core.StandardContext start
SEVERE: Error listenerStart
Jul 23, 2009 11:32:42 PM org.apache.catalina.core.StandardContext start
SEVERE: Context [] startup failed due to previous errors
Jul 23, 2009 11:32:42 PM org.quartz.core.QuartzScheduler shutdown
INFO: Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED shutting down.
Jul 23, 2009 11:32:42 PM org.quartz.core.QuartzScheduler standby
INFO: Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED paused.
Jul 23, 2009 11:32:42 PM org.quartz.core.QuartzScheduler shutdown
INFO: Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED shutdown complete.
Jul 23, 2009 11:32:42 PM org.apache.catalina.core.StandardContext
listenerStop
As mentioned, the exceptions go away after a couple of restarts, which
may indicate a race condition of some sort. The exceptions seem to
happen exclusively on the production servers and never on my localhost
(where I start up Tomcat via Eclipse).The main difference between my
localhost and the production server is that my localhost uses a near
stock Tomcat, while the production servers have a modified server.xml,
and an Apache web server in front of it. The server.xml of one of those
production servers is as follows:
<Server port="8005" shutdown="SHUTDOWN">
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
<!--Initialize Jasper prior to webapps are loaded. Documentation at
/docs/jasper-howto.html -->
<Listener className="org.apache.catalina.core.JasperListener" />
<!-- JMX Support for the Tomcat server. Documentation at
/docs/non-existent.html -->
<Listener
className="org.apache.catalina.mbeans.ServerLifecycleListener" />
<Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<GlobalNamingResources>
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<Service name="Catalina">
<Connector port="8009"
protocol="AJP/1.3"
redirectPort="443"
backlog="200"
maxSpareThreads="50"
maxThreads="800"
minSpareThreads="10"
connectionTimeout="20000"
/>
<Engine name="Catalina" defaultHost="localhost">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
<Host name="localhost" appBase="webapps"
workDir="work/Catalina/cyberdyme"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
pattern="combined"
prefix="cybername-access"
resolveHosts="false"
suffix=".log"
rotatable="true"
fileDateFormat="-yyyy.MM.dd.HH"/>
</Host>
</Engine>
</Service>
</Server>
Does anyone have any ideas how to get rid of these exceptions? I've come
to the point where I barely dare to restart my Tomcat instances anymore,
for fear of them not coming back up.
Thanks in advance for all help.
--
It's a cult. If you've coded for any length of time, you've run across someone
from this warped brotherhood. Their creed: if you can write complicated code,
you must be good.
