-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Don,
On 12/30/2010 10:46 PM, Don Hill wrote:
> From what I can tell it appears that the HTTPD server is stalling at
> MaxClients.
So you have too many clients for your httpd server? Well, if you have
resources to spare (memory and CPU), go ahead and increase MaxClients.
> This usually happens after some mailing and the site gets hit really
> hard with users, upward of 3000 active sessions being handle by 2
> HTTPD servers and 4 Tomcat instances.
The number of sessions is not really relevant unless you store lots of
stuff in the session. The number of simultaneous /requests/ is more
relevant.
> Since we are using prefork/mod_jk it seems there could also be some
> high CPU usage.
There's only one way to find out: measure it. Frankly, there shouldn't
be too much difference between the prefork and worker MPMs. If you are
using the "event" MPM (which is like Tomcat's NIO connector) you may see
better performance for CPU-bound requests.
> I have done extensive profiling from a heap perspective using jstat
> and jmap. I think I have the JVM tuned pretty good at this point.
Good.
> The Tomcat severs also had maxthreads for each JVM set really high
> like 1000 on the AJP/1.3 connector. I brought each JVM down to 200
> max threads.
If Tomcat's MaxThreads is greater than httpd's MaxClients / N where N is
the number of Tomcat nodes, then you are configuring threads that won't
be used.
So, if you have 2 clustered Tomcat instances each with 1000 MaxThreads,
that's 2000 simultaneous requests that the TC cluster can handle. If you
have httpd's MaxClients set to 256, then you have 2000 - 256 = 1744
unused threads across two JVMs taking up memory and never doing anything
for you.
> Currently there are 2 VHOSTS on each machine with one HTTPD
> instance. We are load balancing via a F5 Load balancer. The LB
> manages load accros all 4 VHOST. Each VHOST uses one worker for to
> talk to one dedicated tomcat JVM.
So, you've got something like this?
/----> Tomcat A1 (httpd vhost 1)
httpd A <
/ \----> Tomcat A2 (httpd vhost 2)
F5 <
\ /----> Tomcat B1 (httpd vhost 1)
httpd B <
\----> Tomcat B2 (httpd vhost 2)
Are all Tomcats running the same webapp(s)?
> Besides doing upgrades to Tomcat/HTTPD servers do you think there are
> any configuration issues or can you suggest a better layout for this
> env.
You'd have to give us more information. For instance, if you are limited
to 256 MaxClients for your httpd processes, you might want to launch
more than 1 httpd process on the nodes you have. Otherwise, you might be
able to handle much more load than your connection limit allows.
> I was thinking of getting rid of the vhosts and use a balancer config
> in the mod_jk.config to manage the load to the Tomcat JVMs.
You could even take httpd out of the equation and just use the F5 to
load balance directly to the Tomcat instances. I suspect there's no
limit on Tomcat's MaxThreads like there is on httpd's MaxClients.
> This way the load could be balanced based on the Tomcat usage and not
> the VHOST usage as some of the load could be coming from static
> content and messing up the LB's balancing ability.
You could even configure the F5 to recognize requests for static content
and route them to the httpd instances.
> I was also thinking that we should switch to worker mpm instead of
> the prefork.
As always, test for your configuration. You may find that you don't get
much benefit. Unless you can observe a measurable improvement, it's
better not to change something in a production configuration.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk0eXAoACgkQ9CaO5/Lv0PDftgCgpQOlGuoHYMzhACg4l8JptwzC
yEcAnjT4gmdNXZaFtuw4TmGX3zsCneVd
=A5nw
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
