Dear Mark,
> We need to find out what is triggering the second, unwanted
> initialization. You'll need to add some logging to your app to capture
> the relevant stack trace. When you have it, post it here.
taking remote access, I was able to extract a (hope so) relevant stack trace
from the logs. You can see the start of init runnning with one of the (in
comparison to TC6 new) pooled deployer threads. And then the interfering and
failing catalina-exec-64. From the stack, we see that this is my http call via
the managers proxy to the MBean.
And now, you Gurus can start to imagine what's happen internally between the
two marks - i.e. what make the invocation of
org.apache.tomcat.util.modeler.BaseModelMBean.getAttribute() resulting in a
call of javax.servlet.GenericServlet.init() , which sounds -- with my low
knowlegde -- to me to be the standard servlet entry point and this will raise
the issue.
By the way: I would expect that even the container state machine should not
allow to call this init() twice, because it should be for sure already in a
state called "starting" or what else.
First thing you see is the undeployment. Please ignore the well-known problems
with the dangling threads. You may advance to the next paragraph and take a
look at it later.
20160713-161415.617 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [HostConfig]
Undeploying context []
20160713-161415.618 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [[/]] Destroying
Spring FrameworkServlet 'webservices'
20160713-161415.627 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [[/]] Closing
Spring root WebApplicationContext
20160713-161415.701 WARN
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] appears to have started a
thread named [AsyncAppender-Dispatcher-
Thread-51] but has failed to stop it. This is very likely to create a memory
leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:548)
java.lang.Thread.run(Thread.java:745)
20160713-161415.702 ERROR
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] created a ThreadLocal with
key of type [org.apache.xmlbeans.impl.
schema.SchemaTypeLoaderImpl$1] (value
[org.apache.xmlbeans.impl.schema.SchemaTypeLoaderImpl$1@1542aa40]) and a value
of type [java.util.ArrayList] (value [[java.lang.ref.SoftReference@15ade4fe]])
but fa
iled to remove it when the web application was stopped. Threads are going to be
renewed over time to try and avoid a probable memory leak.
20160713-161415.703 ERROR
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] created a ThreadLocal with
key of type [org.apache.axis.utils.XML
Utils.ThreadLocalDocumentBuilder] (value
[org.apache.axis.utils.XMLUtils$ThreadLocalDocumentBuilder@26a7136c]) and a
value of type [com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl]
(value [c
om.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl@5cd78952]) but
failed to remove it when the web application was stopped. Threads are going to
be renewed over time to try and avoid a probable
memory leak.
20160713-161415.703 ERROR
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] created a ThreadLocal with
key of type [org.apache.xmlbeans.impl.
store.Locale$1] (value [org.apache.xmlbeans.impl.store.Locale$1@330e7103]) and
a value of type [java.lang.ref.SoftReference] (value
[java.lang.ref.SoftReference@28c8179f]) but failed to remove it when t
he web application was stopped. Threads are going to be renewed over time to
try and avoid a probable memory leak.
20160713-161415.704 ERROR
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] created a ThreadLocal with
key of type [org.apache.xmlbeans.impl.
store.CharUtil$1] (value [org.apache.xmlbeans.impl.store.CharUtil$1@36ea5d99])
and a value of type [java.lang.ref.SoftReference] (value
[java.lang.ref.SoftReference@5617df0f]) but failed to remove it wh
en the web application was stopped. Threads are going to be renewed over time
to try and avoid a probable memory leak.
20160713-161415.704 ERROR
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] created a ThreadLocal with
key of type [org.apache.xmlbeans.XmlBe
ans$1] (value [org.apache.xmlbeans.XmlBeans$1@16d13aa7]) and a value of type
[java.lang.ref.SoftReference] (value [java.lang.ref.SoftReference@7ecda30d])
but failed to remove it when the web application
was stopped. Threads are going to be renewed over time to try and avoid a
probable memory leak.
20160713-161415.704 ERROR
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] created a ThreadLocal with
key of type [java.lang.ThreadLocal] (v
alue [java.lang.ThreadLocal@72a87425]) and a value of type
[org.springframework.security.core.context.SecurityContextImpl] (value
[org.springframework.security.core.context.SecurityContextImpl@ffffffff:
Null authentication]) but failed to remove it when the web application was
stopped. Threads are going to be renewed over time to try and avoid a probable
memory leak.
20160713-161415.704 ERROR
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] created a ThreadLocal with
key of type [java.lang.ThreadLocal] (v
alue [java.lang.ThreadLocal@72a87425]) and a value of type
[org.springframework.security.core.context.SecurityContextImpl] (value
[org.springframework.security.core.context.SecurityContextImpl@ffffffff:
Null authentication]) but failed to remove it when the web application was
stopped. Threads are going to be renewed over time to try and avoid a probable
memory leak.
20160713-161415.704 ERROR
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] created a ThreadLocal with
key of type [java.lang.ThreadLocal] (v
alue [java.lang.ThreadLocal@72a87425]) and a value of type
[org.springframework.security.core.context.SecurityContextImpl] (value
[org.springframework.security.core.context.SecurityContextImpl@ffffffff:
Null authentication]) but failed to remove it when the web application was
stopped. Threads are going to be renewed over time to try and avoid a probable
memory leak.
20160713-161415.704 ERROR
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] created a ThreadLocal with
key of type [java.lang.ThreadLocal] (v
alue [java.lang.ThreadLocal@72a87425]) and a value of type
[org.springframework.security.core.context.SecurityContextImpl] (value
[org.springframework.security.core.context.SecurityContextImpl@ffffffff:
Null authentication]) but failed to remove it when the web application was
stopped. Threads are going to be renewed over time to try and avoid a probable
memory leak.
Now, the redeploy starts ...
20160713-161416.470 INFO [portal.dnb.de-startStop-7] [] [HostConfig] Deploying
web application archive /data/srv/test/webapps/portal.dnb.de/ROOT.war
20160713-161425.487 INFO [portal.dnb.de-startStop-7] [] [[/]] No Spring
WebApplicationInitializer types detected on classpath
20160713-161425.576 INFO [portal.dnb.de-startStop-7] [] [[/]] Initializing
Spring root WebApplicationContext
20160713-161427.061 INFO [catalina-exec-64] [] [[/]] Initializing Spring
FrameworkServlet 'webservices'
20160713-161427.340 ERROR [catalina-exec-64] [] [[/]] StandardWrapper.Throwable
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'opacControllerConfig': Injection of autowired dependencies failed;
nested exception is org.springframework.beans.factory.BeanCreationException:
Could not autowire field: private de.ddb.portal.opac.Opac
de.ddb.portal.mvc.configuration.OpacControllerConfig.opac; nested exception is
org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying
bean of type [de.ddb.portal.opac.Opac] found for dependency: expected at least
1 bean which qualifies as autowire candidate for this dependency. Dependency
annotations:
{@org.springframework.beans.factory.annotation.Autowired(required=true)}
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:292)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1185)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:703)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
at
org.springframework.web.servlet.FrameworkServlet.configureAndRefreshWebApplicationContext(FrameworkServlet.java:658)
at
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:624)
at
org.springframework.web.servlet.FrameworkServlet.createWebApplicationContext(FrameworkServlet.java:672)
at
org.springframework.web.servlet.FrameworkServlet.initWebApplicationContext(FrameworkServlet.java:543)
at
org.springframework.web.servlet.FrameworkServlet.initServletBean(FrameworkServlet.java:484)
at
org.springframework.web.servlet.HttpServletBean.init(HttpServletBean.java:136)
----> at javax.servlet.GenericServlet.init(Unknown Source)
at org.apache.catalina.core.StandardWrapper.initServlet(Unknown Source)
at org.apache.catalina.core.StandardWrapper.loadServlet(Unknown Source)
at org.apache.catalina.core.StandardWrapper.allocate(Unknown Source)
at org.apache.catalina.core.StandardWrapper.isSingleThreadModel(Unknown
Source)
at sun.reflect.GeneratedMethodAccessor178.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
---> at org.apache.tomcat.util.modeler.BaseModelMBean.getAttribute(Unknown
Source)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at org.apache.catalina.mbeans.MBeanDumper.dumpBeans(Unknown Source)
at org.apache.catalina.manager.JMXProxyServlet.listBeans(Unknown Source)
at org.apache.catalina.manager.JMXProxyServlet.doGet(Unknown Source)
at javax.servlet.http.HttpServlet.service(Unknown Source)
at javax.servlet.http.HttpServlet.service(Unknown Source)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(Unknown
Source)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(Unknown Source)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(Unknown
Source)
at
org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(Unknown Source)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Unknown Source)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(Unknown
Source)
at org.apache.catalina.core.StandardWrapperValve.invoke(Unknown Source)
at org.apache.catalina.core.StandardContextValve.invoke(Unknown Source)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(Unknown
Source)
at org.apache.catalina.core.StandardHostValve.invoke(Unknown Source)
at org.apache.catalina.valves.ErrorReportValve.invoke(Unknown Source)
at org.apache.catalina.valves.RequestFilterValve.process(Unknown Source)
at org.apache.catalina.valves.RemoteAddrValve.invoke(Unknown Source)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(Unknown
Source)
at org.apache.catalina.core.StandardEngineValve.invoke(Unknown Source)
at org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(Unknown
Source)
at org.apache.catalina.ha.tcp.ReplicationValve.invoke(Unknown Source)
at org.apache.catalina.connector.CoyoteAdapter.service(Unknown Source)
at org.apache.coyote.http11.AbstractHttp11Processor.process(Unknown
Source)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Unknown
Source)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(Unknown
Source)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(Unknown
Source)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(Unknown Source)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.beans.factory.BeanCreationException: Could not
autowire field: private de.ddb.portal.opac.Opac
de.ddb.portal.mvc.configuration.OpacControllerConfig.opac; nested exception is
org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying
bean of type [de.ddb.portal.opac.Opac] found for dependency: expected at least
1 bean which qualifies as autowire candidate for this dependency. Dependency
annotations:
{@org.springframework.beans.factory.annotation.Autowired(required=true)}
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:508)
at
org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:87)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:289)
... 60 more
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No
qualifying bean of type [de.ddb.portal.opac.Opac] found for dependency:
expected at least 1 bean which qualifies as autowire candidate for this
dependency. Dependency annotations:
{@org.springframework.beans.factory.annotation.Autowired(required=true)}
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.raiseNoSuchBeanDefinitionException(DefaultListableBeanFactory.java:1103)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:963)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:858)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:480)
... 62 more
[skipped the same with catalina-exec-53]
20160713-161430.289 INFO [portal.dnb.de-startStop-7] [] [[/]] Initializing
Spring FrameworkServlet 'webservices'
20160713-161430.941 INFO [portal.dnb.de-startStop-7] [] [HostConfig]
Deployment of web application archive
/data/srv/test/webapps/portal.dnb.de/ROOT.war has finished in 14,471 ms
Here a part of the log where the reloading runs undisturbed. Please ignore the
well-known problems with the unstopped threads.
20160713-105355.050 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [HostConfig]
Reloading context []
20160713-105355.050 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [StandardContext]
Reloading Context with name [] has started
20160713-105355.051 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [[/]] Destroying
Spring FrameworkServlet 'webservices'
20160713-105355.060 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [[/]] Closing
Spring root WebApplicationContext
20160713-105355.122 WARN
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] []
[WebappClassLoaderBase] The web application [ROOT] appears to have started a
thread named [AsyncAppender-Dispatcher-
Thread-18] but has failed to stop it. This is very likely to create a memory
leak. Stack trace of thread:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:548)
java.lang.Thread.run(Thread.java:745)
20160713-105400.037 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [[/]] No Spring
WebApplicationInitializer types detected on classpath
20160713-105400.131 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [[/]] Initializing
Spring root WebApplicationContext
20160713-105404.643 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [[/]] Initializing
Spring FrameworkServlet 'webservices'
20160713-105405.480 INFO
[ContainerBackgroundProcessor[StandardEngine[Catalina]]] [] [StandardContext]
Reloading Context with name [] is completed
greetings
Guido
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
