On 13.07.2016 15:20, Jäkel, Guido wrote:
Dear Developers,
I have an elaborate issue migrating from Tomcat 6.0.41 to 8.0.36 (currently
using java 8.0.66 on Gentoo Linux, but this is not of importance I think): It
is related to the build-in manager application used as a jmxproxy. And to
initialization of the Spring framework during an application lifecycle. But in
core, I think it's not related to the manager app and it's proxy feature
itself. But to some MBean or lifecycle implementation details.
First of all: In principle, all works well. If i start the tomcat from the
scratch, all applications will proper initialize and work as expected. If I
restart any application by different triggers (manager, touching the WAR or a
sensitive file inside the docroot and so on), it will be undeployed and
deployied again - all as usual.
But if there is a certain request to some MBean while the App is starting,
there will be a request to the application (via catalina-exec). And then, as a
race condition, the initialization of the Spring framework, esp. the bean
wiring will fail with lots of exceptions because this is not thread-safe.
If this additional (failing) request is processed fast enough, the application
will be usable due to initialization via the normal servlet lifecycle request.
But if it's as a race condition in the other way round, the application will be
not usable afterwards.
The point now is: This requests to the MBeans are done requently (ever 10s) by
our monitoring system. And about 90% of the automatic deployments done by the
CI system now will fail by this race condition in spite of the deployed apps
are well-done. Without any modification, the same application is able to start
by chance of no concurrent request to the MBean or (of course) by disabling the
Monitoring or factical preventing it's request also by a restart of the whole
tomcat, because the connectors are disabled until all apps are up as you'll
know.
This never happens with tomcat 6 used for many years. Therefore, there must be
some regression with an MBean that will allow to route a request to a
not-fully-started application. I happens with all our well-established
applications which use the Spring framework -- regardess if deployed via the CI
because of normal, slightly modification or even with unchanged versions
running in production on Tomcat6. I can't give information about the behavior
on Tomcat7.
I think I was able to narrow it down to a specific MBean readout: If I send the request
".../manager/jmxproxy?qry=Catalina:j2eeType=Servlet,WebModule=*,*" before the Deployer
logs "[StandardContext] Reloading Context with name [foo] is completed", then it will
trigger an unwanted, additional thread.
It sounds like it points into the right direction because the MBean related to
the Servlet is offering e.g. information about the containers lifecycle state.
May you fix it or did you see any workaround? If you need any more details,
please ask about. Thank you in advance for your support.
Hi.
The explanation above, as well as the underlying issue, are way over my level of knowledge
of Tomcat and/or Java.
But, as a possible (and possibly temporary) workaround, and focusing on this
bit :
"Without any modification, the same application is able to start by chance of no
concurrent request to the MBean or (of course) by disabling the Monitoring or factical
preventing it's request also by a restart of the whole tomcat, because the connectors are
disabled until all apps are up as you'll know."
So it would appear that if the Connectors are disabled, the monitoring system is not able
to reach Tomcat, and the problem does not occur then.
So would it not be possible to create some little piece of software, which would
temporarily "suspend" or "disable" the Connectors, while the application update is taking
place, and then re-enable them when it's done ?
Of course, if the application update is also going though the Connectors, this may be a
problem..
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
