Re: Parallel deploy with /manager/text servlet
Thanks Remy, I now see that Mark has already been working on this and
committed 7 patches to resolve this issue. I see that these commits were
made right after the release of 9.0.43 (Feb. 1st while 9.0.43 was
release on Jan. 28th)
https://github.com/apache/tomcat/commit/ee6069392752f930c64e160a6a08a9904ffea008
I assume these fixes will be available on 9.0.44 right? I'm unsure
whether they were just applied to the 10.x branch or backported to the
9.x branch as well.
Thanks
*Manuel Dominguez Sarmiento*
On 11/02/2021 10:11, Manuel Dominguez Sarmiento wrote:
Sorry it took so long to get back to this. @Mark I've just filed bug
65136 as requested:
https://bz.apache.org/bugzilla/show_bug.cgi?id=65136
*Manuel Dominguez Sarmiento*
On 07/01/2021 05:40, Mark Thomas wrote:
On 06/01/2021 21:48, Manuel Dominguez Sarmiento wrote:
Hi, our system consists on about 80+ webapps on different servers.
Multiple webapps are deployed per Tomcat server remotely using the
manager and host-manager webapps.
This has been working fine for years, however some servers now have 5+
webapps and they are taking a long time to start. We invoke the deploy
command on the manager webapp in parallel for each webapp, in order to
attempt a reduction in total server startup time. However the speedup is
very modest. The servers have powerful Xeon multi-core servers so a
parallel speedup should be possible.
Looking at the source code for
org.apache.catalina.manager.ManagerServlet we see that the following
method is synchronized which would explain the behaviour we are seeing:
protected synchronized void deploy
(PrintWriter writer, String config, ContextName cn,
String tag, boolean update, HttpServletRequest request,
StringManager smClient) {
Is there any real reason why this needs to be synchronized? Tomcat can
deploy webapps in parallel when backgroundProcessorDelay is set to a
positive number, but this is not useful when remotely deploying webapps
using the manager webapp.
Any ideas? Thanks
Some svn archaeology uncovered this commit comment:
"Known issue: locking occurs on an uploaded WAR, for reasons which
elude me right now."
The method level sync looks like an attempt to ensure that there aren't
parallel attempts to deploy a web application to the same path. I can't
think of any other reason. But locking on, essentially, the virtual
host, seems overly broad. A lock on the deployment path would be sufficient.
There are also calls to isServiced(), addServiced() and removeServiced()
which look to be trying to do the same thing which have their own issues
(not atomic, uses a List rather than a Set).
We do need to ensure that we don't get parallel attempts to service the
same web application. There is scope for all sorts of bad things to
happen if that is possible.
Overall, I think this can be fixed but it looks like some API changes
are going to be required which makes things a little more complicated.
It would be worth you opening a Bugzilla issue for this to make sure it
doesn't get forgotten about. The issues with the unsynchronized
deployment methods are sufficient - in my view - to class this as a bug
rather than an enhancement.
Mark
-
To unsubscribe, e-mail:users-unsubscr...@tomcat.apache.org
For additional commands, e-mail:users-h...@tomcat.apache.org
Re: Parallel deploy with /manager/text servlet
Sorry it took so long to get back to this. @Mark I've just filed bug
65136 as requested:
https://bz.apache.org/bugzilla/show_bug.cgi?id=65136
*Manuel Dominguez Sarmiento*
On 07/01/2021 05:40, Mark Thomas wrote:
On 06/01/2021 21:48, Manuel Dominguez Sarmiento wrote:
Hi, our system consists on about 80+ webapps on different servers.
Multiple webapps are deployed per Tomcat server remotely using the
manager and host-manager webapps.
This has been working fine for years, however some servers now have 5+
webapps and they are taking a long time to start. We invoke the deploy
command on the manager webapp in parallel for each webapp, in order to
attempt a reduction in total server startup time. However the speedup is
very modest. The servers have powerful Xeon multi-core servers so a
parallel speedup should be possible.
Looking at the source code for
org.apache.catalina.manager.ManagerServlet we see that the following
method is synchronized which would explain the behaviour we are seeing:
protected synchronized void deploy
(PrintWriter writer, String config, ContextName cn,
String tag, boolean update, HttpServletRequest request,
StringManager smClient) {
Is there any real reason why this needs to be synchronized? Tomcat can
deploy webapps in parallel when backgroundProcessorDelay is set to a
positive number, but this is not useful when remotely deploying webapps
using the manager webapp.
Any ideas? Thanks
Some svn archaeology uncovered this commit comment:
"Known issue: locking occurs on an uploaded WAR, for reasons which
elude me right now."
The method level sync looks like an attempt to ensure that there aren't
parallel attempts to deploy a web application to the same path. I can't
think of any other reason. But locking on, essentially, the virtual
host, seems overly broad. A lock on the deployment path would be sufficient.
There are also calls to isServiced(), addServiced() and removeServiced()
which look to be trying to do the same thing which have their own issues
(not atomic, uses a List rather than a Set).
We do need to ensure that we don't get parallel attempts to service the
same web application. There is scope for all sorts of bad things to
happen if that is possible.
Overall, I think this can be fixed but it looks like some API changes
are going to be required which makes things a little more complicated.
It would be worth you opening a Bugzilla issue for this to make sure it
doesn't get forgotten about. The issues with the unsynchronized
deployment methods are sufficient - in my view - to class this as a bug
rather than an enhancement.
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Parallel deploy with /manager/text servlet
On 06/01/2021 21:48, Manuel Dominguez Sarmiento wrote:
> Hi, our system consists on about 80+ webapps on different servers.
> Multiple webapps are deployed per Tomcat server remotely using the
> manager and host-manager webapps.
>
> This has been working fine for years, however some servers now have 5+
> webapps and they are taking a long time to start. We invoke the deploy
> command on the manager webapp in parallel for each webapp, in order to
> attempt a reduction in total server startup time. However the speedup is
> very modest. The servers have powerful Xeon multi-core servers so a
> parallel speedup should be possible.
>
> Looking at the source code for
> org.apache.catalina.manager.ManagerServlet we see that the following
> method is synchronized which would explain the behaviour we are seeing:
> protected synchronized void deploy
> (PrintWriter writer, String config, ContextName cn,
> String tag, boolean update, HttpServletRequest request,
> StringManager smClient) {
>
> Is there any real reason why this needs to be synchronized? Tomcat can
> deploy webapps in parallel when backgroundProcessorDelay is set to a
> positive number, but this is not useful when remotely deploying webapps
> using the manager webapp.
>
> Any ideas? Thanks
Some svn archaeology uncovered this commit comment:
"Known issue: locking occurs on an uploaded WAR, for reasons which
elude me right now."
The method level sync looks like an attempt to ensure that there aren't
parallel attempts to deploy a web application to the same path. I can't
think of any other reason. But locking on, essentially, the virtual
host, seems overly broad. A lock on the deployment path would be sufficient.
There are also calls to isServiced(), addServiced() and removeServiced()
which look to be trying to do the same thing which have their own issues
(not atomic, uses a List rather than a Set).
We do need to ensure that we don't get parallel attempts to service the
same web application. There is scope for all sorts of bad things to
happen if that is possible.
Overall, I think this can be fixed but it looks like some API changes
are going to be required which makes things a little more complicated.
It would be worth you opening a Bugzilla issue for this to make sure it
doesn't get forgotten about. The issues with the unsynchronized
deployment methods are sufficient - in my view - to class this as a bug
rather than an enhancement.
Mark
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Parallel deploy with /manager/text servlet
Hi, our system consists on about 80+ webapps on different servers.
Multiple webapps are deployed per Tomcat server remotely using the
manager and host-manager webapps.
This has been working fine for years, however some servers now have 5+
webapps and they are taking a long time to start. We invoke the deploy
command on the manager webapp in parallel for each webapp, in order to
attempt a reduction in total server startup time. However the speedup is
very modest. The servers have powerful Xeon multi-core servers so a
parallel speedup should be possible.
Looking at the source code for
org.apache.catalina.manager.ManagerServlet we see that the following
method is synchronized which would explain the behaviour we are seeing:
protected synchronized void deploy
(PrintWriter writer, String config, ContextName cn,
String tag, boolean update, HttpServletRequest request,
StringManager smClient) {
Is there any real reason why this needs to be synchronized? Tomcat can
deploy webapps in parallel when backgroundProcessorDelay is set to a
positive number, but this is not useful when remotely deploying webapps
using the manager webapp.
Any ideas? Thanks
*Manuel Dominguez Sarmiento*
