Damian Traverso wrote:
Hi,
we have one Apache-2.2.13 running mod_proxy_ajp + mod_proxy_balancer,
connected to (3) Tomcat-6.0.20 instances under Fedora release 8. We
are experiencing some issues with high CPU load on the Tomcat side,
and Apache starts logging errors like these
[Fri Oct 30 14:47:43 2009] [error] (70007)The timeout specified has
expired: ajp_ilink_receive() can't receive header
[Fri Oct 30 14:47:43 2009] [error] ajp_read_header: ajp_ilink_receive failed
[Fri Oct 30 14:47:43 2009] [error] (120006)APR does not understand
this error code: proxy: read response failed from
On Tomcat side we are using the APR Native Connector, with this settings:
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443"
maxThreads="130" backlog="10" connectionTimeout="30000"
keepAliveTimeout="30000" enableLookups="false" />
Apache MPM conf:
ServerLimit 100
<IfModule mpm_worker_module>
StartServers 30
MaxClients 2500
MinSpareThreads 50
MaxSpareThreads 100
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
mod_proxy_ajp conf:
<Proxy balancer://jogacraque>
BalancerMember ajp://xxx.xxx.xxx.xxx:8009 route=app01 ttl=30 max=2
BalancerMember ajp://xxx.xxx.xxx.xxx:8009 route=app02 ttl=30 max=2
BalancerMember ajp://xxx.xxx.xxx.xxx:8009 route=app03 ttl=30 max=2
</Proxy>
Apache specs:
Server version: Apache/2.2.13 (Unix)
Server built: Sep 29 2009 11:12:35
Server's Module Magic Number: 20051115:23
Server loaded: APR 1.3.8, APR-Util 1.3.9
Compiled using: APR 1.3.8, APR-Util 1.3.9
Architecture: 32-bit
Server MPM: Worker
threaded: yes (fixed thread count)
forked: yes (variable process count)
I've beeng testing different settings for mod_proxy_ajp, but I am not
sure what is the exact relation between the max option for mod_proxy
and ThreadsPerChild. Should I set max to 1? I understand I would have
as many connections to each Tomcat box as
max*(MaxClients/ThreadsPerChild), is this correct?
BTW, is the total number of connections that mod_proxy_ajp will
forward to Tomcat related with MaxThreads ? I don't fully understand
how APR connector works in this case.
Hi.
No great specialist here, but I'll go by the numbers above and give you
my interpretation.
MaxClients 2500
on your webserver/balancer indicates how many requests you are willing
to accept and process at the same time, at the Apache level.
ThreadsPerChild 25
means that, if you ever reach these 2500 simultaneous requests, Apache
will create 100 children processes to handle them, since each child is
limited to 25 threads, and 2500 / 25 threads -> 100 children.
(Which seems to match the ServerLimit. Good.)
It is so with the mod_jk connector, and probably also with mod_proy_ajp,
that the connector will automatically calculate how many connections to
make to Tomcat, based on the number of threads/processes above.
At the Tomcat level, the AJP Connector says
maxThreads="130"
and you have 3 Tomcats, thus for a total maximum number of threads of
130 X 3 = 390.
One Tomcat thread processes one request at a time.
So the Tomcats, together, can process 390 requests at a time.
There seems to be a mismatch between the number of requests you accept
in Apache (2500), and the number of threads available to process them in
Tomcat (390). At the limit, you could have 390 requests being processed
in Tomcat, and 2110 waiting to get a free Tomcat thread.
That is probably what you are seeing with this log message :
> [Fri Oct 30 14:47:43 2009] [error] (70007)The timeout specified has
> expired: ajp_ilink_receive() can't receive header
In other words, probably Apache/mod_proy_ajp got tired of waiting.
On the other hand, you indicate that the poor Tomcats are already
struggling to cope with the load. Increasing their number of threads is
thus unlikely to help much.
So it looks like you need more Tomcats.
Or else reduce the MaxClients in Apache.
Or optimise your Tomcat webapps.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
" from the digest: users-digest-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
