Hi.
I believe that what Philippe mentions below is somewhat different : in his configuration,
there is apparently a front-end httpd server, which communicates with Tomcat via AJP and
the tomcat AJP Connector.
In such a case, the "mod_jk" or "mod_proxy_ajp" connector module inside of httpd, creates
a series of connections to the back-end tomcat, and keeps these connections alive in a
pool of connections, according to rules which are different, than what applies to your
case (which consists of direct HTTP connections from browsers, to the tomcat HTTP connector).
So Philippe's case introduces probably an additional level of complexity into
the equation.
semi-graphically :
smith case :
client <- HTTP -> tomcat HTTP connector - tomcat thread
Philippe's case :
client <- HTTP -> apache httpd + ajp module <- AJP -> tomcat AJP connector -
tomcat thread
The real question is :
The tomcat 8.0 HTTP Connector documentation
(http://tomcat.apache.org/tomcat-8.0-doc/config/http.html) mentions a series of
attributes/parameters (maxThreads, minSpareThreads, connectionTimeout,
keepAliveTimeout,..) which, together, would lead one to believe that tomcat is managing
the number of threads dynamically (increasing the number of threads (up to maxThreads)
when needed to process simultaneous requests, and decreasing the number of threads again
(down to minSpareThreads) when less requests are being processed).
According to observations however (and to Chris' comments), that does not seem to be the
case : once the number of threads has grown to a certain number, it never goes down back
down again.
It has to be said that the above interpretation was mine, and the current documentation of
the HTTP Connector never says explicitly that, on its own, it manages the number of
threads dynamically. But it *does* mention a minSpareThreads parameter.
On the other hand, when you use an Executor
(http://tomcat.apache.org/tomcat-8.0-doc/config/executor.html), then the documentation
states explicitly that the number of threads *is* dynamically managed, up and down.
I found something else which seems to explain the riddle :
http://tomcat.apache.org/migration-6.html#Connector_thread_pools
That section explicitly says that since tomcat 6.0, for a Connector without an Executor,
the number of threads always grows, and never decreases. And it also explicitly says that
the Connector's "minSpareThreads" attribute will be ignored.
So in fact the only thing wrong, is the online documentation for the Connectors : the
minSpareThreads attribute should be removed (since it is anyway ignored).
That seems to have been an oversight ever since tomcat 6.0.
As far as I am concerned thus, the mystery is solved.
One question which is a bit left open is :
What - if any - is the real advantage/disadvantage of having perhaps maxThreads idle
threads, as opposed to using an Executor to manage the number of threads dynamically ?
But that is probably the kind of question to which the appropriate answer is : "it
depends.."
On 18.01.2017 07:33, smith wrote:
Thanks, Philippe
But we never monitored tomcat busy thread count high (we monitored one minutes
interval through nagios to get tomcat manager result, not high than 10). This
is strange
-----Original Message-----
From: Philippe Busque [mailto:pbus...@mediagrif.com]
Sent: Monday, January 16, 2017 8:09 PM
To: users@tomcat.apache.org
Subject: Re: Re: FW: tomcat 8080 thread not reduced
We're having a similar issues with our numberous Tomcat instances.
Our connector config look like this.
<Connector port="8910" protocol="AJP/1.3" maxThreads="200"
enableLookups="false" redirectPort="8443" acceptCount="100"
connectionTimeout="20000" URIEncoding="UTF-8" />
Sometime, the number of active connection would jump very high (up to 190), due
to some external issues (database lock, etc) and threads would accumulate.
Even though a connectionTimeout is set, and therefor set keepAliveTimeout as
the same value, threads are never released once the problem is resolved until
Tomcat is restarted. We would end up with maybe 5-10 busy workers, but 190
idle workers/ threads
I think the issue is related to how the StandardThreadExecutor is implemented.
The StandardThreadExecutor is a front for the default Java ThreadPoolExecutor.
If I'm not mistaken, ThreadPoolExecutor is distributing work in round robin
fashion among all defined workers, rather than sticking to the core threads.
As a result, should a website has any constant traffic (Apache AJP ping, load
balancer monitoring, normal traffic, etc), all thread will be hit at least once
within the configured keepAliveTimeout, reseting it. So unless the
keepAliveTimeout is set to a very low value, which defeat the purpose, thread
will never be released .
This is what I've come to suspect from looking at the StandardThreadExecutor,
but never really had the opportunity to do deeper test with load.
But from Tomcat 6 to tomcat 8, we were never able to decrease the number of
'idle' workers back from the highest value it had reached.
Le 2017-01-16 à 05:24, André Warnier (tomcat) a écrit :
On 16.01.2017 11:10, smith wrote:
We has same problem on dev env that no any traffic to the serive,
Ah. That is /new/ information, which may change the suggestions below.
It looks like you should really find out what these threads are doing,
probably by doing a few thread dumps.
See here e.g. :
http://stackoverflow.com/questions/18573411/tomcat-thread-dump
Again : we do not know your application, so we can only make guesses
based on the information that you provide.
will try on dev first
-----Original Message-----
From: André Warnier (tomcat) [mailto:a...@ice-sa.com]
Sent: Monday, January 16, 2017 10:08 AM
To: users@tomcat.apache.org
Subject: Re: FW: tomcat 8080 thread not reduced
On 16.01.2017 09:50, smith wrote:
Busy one is process customer request, do not know what non-busy one
is doing, always keep 120 for many days. I don't think 20s timeout
will not cause so long connection
-smith
And did you actually try it ?
We do not know your website or your application, so we cannot tell
how many clients there are, what these clients are really requesting,
how many requests each client is sending before going away, etc.
KeepAlive means that when a client has sent its /last/ request and
received the response, one thread is going to remain "not free" (but
doing nothing) for the duration of the KeepAlive timeout. This thread
will keep waiting, for KeepAliveTimeout seconds, just in case the
client would still send another request (which it may never do,
depending on the application).
Imagine that your application is so that the average client
- connects to your site
- sends a single HTTP request, which gets processed in 0.1 s
- receives the response
- and then goes away
and that the above sequence happens once every second, from different
clients.
After one second, there will be one thread waiting for another 19
seconds before becoming free (and potentially destroyed or re-used).
After 2 seconds, there will be 2 such threads. After 3 seconds, 3
threads. And so on. After 20 seconds, the first thread will be freed,
but there will be 19 other threads still waiting, and one new thread
just created.
If everything stays perfectly regular like that, your will have
/permanently/ 20 threads in existence, even if the minimum is 10.
If you change the above so that there is a new client every 0.5 s,
you will have permanently 40 threads (of which only 2 maximum are
really doing something).
The point is : KeepAlive is not "bad", and in some cases having a
relatively long KeepAliveTimeout is the right thing to do. Also,
having a high number of threads sitting idle is not necessarily a
problem.
Your own scenario is probably not like the above perfectly regular
and irrealistic one above. But there may be a perfectly logical
reason why you have so many threads on average, and I am just trying
to give you ideas for finding out the reason.
-----Original Message-----
From: André Warnier (tomcat) [mailto:a...@ice-sa.com]
Sent: Monday, January 16, 2017 8:33 AM
To: users@tomcat.apache.org
Subject: Re: FW: tomcat 8080 thread not reduced
On 16.01.2017 03:41, Smith Hua wrote:
actually there is not much busy threads, less tahn 10,so i think
this parameter may has nothing to do with this
It depends on what you call "busy". What are the busy ones doing ?
and what are the non-busy ones doing ?
--
从myMail的Android专用app所发送 星期一, 16 一月 2017, 03:01上午 +08:00
发件人 André Warnier (tomcat) a...@ice-sa.com :
Hi.
I can find nothing really wrong in your configuration below.
But, what happens if in this section :
> <Connector port="8080" protocol="HTTP/1.1"
> maxThreads="300" connectionTimeout="20000"
> redirectPort="8443" />
you change the connectionTimeout to 3000 (= 3 seconds, instead of
the above 20 seconds) ?
Do you still see the number of threads remaining at the maximum ?
See :
http://tomcat.apache.org/tomcat-8.0-doc/config/http.html#Standard_
Im
plementation
--> connectionTimeout
and the fact that it is also the default for keepAliveTimeout
On 14.01.2017 07:30, smith wrote:
The server.xml:
<?xml version='1.0' encoding='utf-8'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more
contributor license agreements. See the NOTICE file
distributed with
this work for additional information regarding copyright
ownership.
The ASF licenses this file to You under the Apache License,
Version 2.0
(the "License"); you may not use this file except in
compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software
distributed under the License is distributed on an "AS IS"
BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
express or implied.
See the License for the specific language governing
permissions and
limitations under the License.
-->
<!-- Note: A "Server" is not itself a "Container", so you may not
define subcomponents such as "Valves" at this level.
Documentation at /docs/config/server.html
-->
<Server port="8005" shutdown="SHUTDOWN">
<Listener
className="org.apache.catalina.startup.VersionLoggerListener" />
<!-- Security listener. Documentation at
/docs/config/listeners.html
<Listener
className="org.apache.catalina.security.SecurityListener" />
-->
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener
className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
<!-- Prevent memory leaks due to use of particular
java/javax APIs-->
<Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
/>
<Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener"
/>
<Listener
className="org.apache.catalina.core.ThreadLocalLeakPreventionListener"
/>
<!-- Global JNDI resources
Documentation at /docs/jndi-resources-howto.html
-->
<GlobalNamingResources>
<!-- Editable user database that can also be used by
UserDatabaseRealm to authenticate users
-->
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated
and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<!-- A "Service" is a collection of one or more
"Connectors" that share
a single "Container" Note: A "Service" is not itself
a "Container",
so you may not define subcomponents such as "Valves"
at this level.
Documentation at /docs/config/service.html
-->
<Service name="Catalina">
<!--The connectors can use a shared executor, you can
define one or more named thread pools-->
<!--
<Executor name="tomcatThreadPool"
namePrefix="catalina-exec-"
maxThreads="150" minSpareThreads="4"/>
-->
<!-- A "Connector" represents an endpoint by which
requests are received
and responses are returned. Documentation at :
Java HTTP Connector: /docs/config/http.html
(blocking & non-blocking)
Java AJP Connector: /docs/config/ajp.html
APR (HTTP/AJP) Connector: /docs/apr.html
Define a non-SSL HTTP/1.1 Connector on port 8080
-->
<Connector port="8080" protocol="HTTP/1.1"
maxThreads="300" connectionTimeout="20000"
redirectPort="8443" />
<!-- A "Connector" using the shared thread pool-->
<!--
<Connector executor="tomcatThreadPool"
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
-->
<!-- Define a SSL HTTP/1.1 Connector on port 8443
This connector uses the NIO implementation that
requires the JSSE
style configuration. When using the APR/native
implementation, the
OpenSSL style configuration is required as described
in the APR/native
documentation -->
<!--
<Connector port="8443"
protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true"
scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />
-->
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009" protocol="AJP/1.3"
redirectPort="8443" />
<!-- An Engine represents the entry point (within
Catalina) that processes
every request. The Engine implementation for Tomcat
stand alone
analyzes the HTTP headers included with the request,
and passes them
on to the appropriate Host (virtual host).
Documentation at /docs/config/engine.html -->
<!-- You should set jvmRoute to support load-balancing
via AJP ie :
<Engine name="Catalina" defaultHost="localhost"
jvmRoute="jvm1">
-->
<Engine name="Catalina" defaultHost="localhost">
<!--For clustering, please take a look at documentation
at:
/docs/cluster-howto.html (simple how to)
/docs/config/cluster.html (reference documentation)
-->
<!--
<Cluster
className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
-->
<!-- Use the LockOutRealm to prevent attempts to guess
user passwords
via a brute-force attack -->
<Realm className="org.apache.catalina.realm.LockOutRealm">
<!-- This Realm uses the UserDatabase configured in
the global JNDI
resources under the key "UserDatabase". Any edits
that are performed against this UserDatabase are
immediately
available for use by the Realm. -->
<Realm
className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
</Realm>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true">
<!-- SingleSignOn valve, share authentication between
web applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve
className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using
pattern="common" -->
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
prefix="localhost_access_log" suffix=".txt"
pattern="%h,%t,%m,%U,%H,%s,%B,%D,%{User-Agent}i" />
<Context path="" allowLinking="true"
crossContext="true" docBase="/****/t" sessionCookieName="****" />
</Host>
</Engine>
</Service>
</Server>
-----Original Message-----
From: André Warnier (tomcat) [mailto:a...@ice-sa.com]
Sent: Friday, January 13, 2017 10:42 AM
To: users@tomcat.apache.org
Subject: Re: FW: tomcat 8080 thread not reduced
On 13.01.2017 09:38, smith wrote:
From: smith [mailto:smith....@zoom.us]
Sent: Tuesday, January 10, 2017 9:57 AM
To: 'users'
Subject: tomcat 8080 thread not reduced
Hi,
We have installed Apache Tomcat/8.0.14, and found that after one
period of time, the thread count for 8080(our port published)
goes to 120 and never reduced even the busy count is only 3-4.
Why? Tomcat8 not reduced the thread pool even the thread is
idle, and the minSpareThreads for tomcat8 default is only 10.
When will the thread reduce?
Best regards
Smith
Hi.
Please copy/paste your complete server.xml configuration file
(confidential things removed), so that we could have a useful
look at it.
Please edit *only* the confidential things, not entire sections.
Often, the issue is in the details.
-----------------------------------------------------------------
----
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-----------------------------------------------------------------
----
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
------------------------------------------------------------------
--- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
--------------------------------------------------------------------
- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
--------------------------------------------------------------------
- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Ce message et les fichiers d’accompagnement transmis avec celui-ci s’adressent
expressément au(x) destinataire(s) et peuvent contenir des renseignements
confidentiels et privilégiés. Si vous recevez ce message par erreur, veuillez
en aviser immédiatement l’expéditeur par courrier électronique. Veuillez
également ne pas en prendre connaissance et en supprimer toutes les copies
immédiatement. Technologies Interactives Mediagrif Inc. et ses filiales
n’acceptent aucune responsabilité à l’égard des opinions exprimées dans le
message ou des conséquences de tout virus informatique qui pourrait être
transmis avec ce message. Ce message fait également l’objet d’un copyright. Il
est interdit d’en reproduire, adapter ou transmettre quelque partie que ce soit
sans le consentement écrit du détenteur du copyright.
This email and any files transmitted with it are solely intended for the use of
the addressee(s) and may contain information that is confidential and
privileged. If you receive this email in error, please advise us by return
email immediately. Please also disregard the contents of the email, delete it
and destroy any copies immediately. Mediagrif Interactive Technologies Inc. and
its subsidiaries do not accept liability for the views expressed in the email
or for the consequences of any computer viruses that may be transmitted with
this email. This email is also subject to copyright. No part of it should be
reproduced, adapted or transmitted without the written consent of the copyright
owner.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
