I have successfully and consistently run 2.25 million servlet transactions out of 45 million+ requests through a hardware balancer across 8 clusters of 2 jvms for a total of 16 tomcat instances running on two different server grade machines in ~1 hour with little or no errors. Each tomcat instance uses ~160 busy threads and most transactions were under 2 seconds. The boxes ran at about 50% utilized.
When I try to run the same test against the same setup but with two apache 2.2.4 instances running on a third box, I can successfully proxy to and balance across the clusters, but once the volume ramps-up I run into many issues. Now, tomcat easily uses more than 1000 busy threads. I get different issues with mod_proxy_ajp and mod_proxy_http. With mod_proxy_ajp, ~5 to 10 minutes into the test mod_proxy_ajp transaction times go through the roof and the apache error log complains about: [Wed May 23 05:44:43 2007] [error] server reached MaxClients setting, consider raising the MaxClients setting [Wed May 23 05:48:20 2007] [error] (70014)End of file found: ajp_ilink_receive() can't receive header [Wed May 23 05:48:20 2007] [error] ajp_read_header: ajp_ilink_receive failed [Wed May 23 05:48:20 2007] [error] (120006)APR does not understand this error code: proxy: read response failed from (null) (xxx.xxx.xxx.xxx) [Wed May 23 05:48:25 2007] [error] (104)Connection reset by peer: ajp_ilink_receive() can't receive header With mod_proxy_http everything seems to run fine until about 30 minutes into the test the apache error logs complain about: [Wed May 23 02:41:07 2007] [error] server reached MaxClients setting, consider raising the MaxClients setting [Wed May 23 03:18:41 2007] [error] [client xxx.xxx.xxx.xxx] proxy: error reading status line from remote server xxx.xxx.xxx.xxx, referer: http://xxx.xxx.xxx.xxx..... [Wed May 23 03:18:41 2007] [error] [client xxx.xxx.xxx.xxx] proxy: Error reading from remote server returned by /tf/tad/accountSearch, referer: http://xxx.xxx.xxx.xxx.... [Wed May 23 03:18:42 2007] [error] [client xxx.xxx.xxx.xxx] proxy: error reading status line from remote server xxx.xxx.xxx.xxx, referer: http://xxx.xxx.xxx.xxx.... [Wed May 23 03:18:42 2007] [error] [client xxx.xxx.xxx.xxx] proxy: Error reading from remote server returned by /tf/tad/loadPageFragment, referer: http://xxx.xxx.xxx.xxx.... [Wed May 23 03:18:42 2007] [error] [client xxx.xxx.xxx.xxx] proxy: error reading status line from remote server xxx.xxx.xxx.xxx, referer: http://xxx.xxx.xxx.xxx..... [Wed May 23 03:18:42 2007] [error] [client xxx.xxx.xxx.xxx] proxy: Error reading from remote server returned by /tf/tad/accountSearch, referer: http://xxx.xxx.xxx.xxx.... [Wed May 23 03:18:42 2007] [error] [client xxx.xxx.xxx.xxx] (70007)The timeout specified has expired: proxy: error reading response If I let the mod_proxy_http test run it eventually brings down the two application servers to the point where I have to reboot them. Here are my apache settings for mod_proxy_http (mod_proxy_ajp is the same except it uses the ajp protocol): <Proxy balancer://mybalancer> BalancerMember http://1.2.3.4:8201/tf/tad smax=400 max=500 route=b1 redirect=a1 BalancerMember http://1.2.3.5:8201/tf/tad smax=400 max=500 route=a1 redirect=b1 BalancerMember http://1.2.3.4:8202/tf/tad smax=400 max=500 route=b2 redirect=a2 BalancerMember http://1.2.3.5:8202/tf/tad smax=400 max=500 route=a2 redirect=b2 BalancerMember http://1.2.3.4:8203/tf/tad smax=400 max=500 route=b3 redirect=a3 BalancerMember http://1.2.3.5:8203/tf/tad smax=400 max=500 route=a3 redirect=b3 BalancerMember http://1.2.3.4:8204/tf/tad smax=400 max=500 route=b4 redirect=a4 BalancerMember http://1.2.3.5:8204/tf/tad smax=400 max=500 route=a4 redirect=b4 BalancerMember http://1.2.3.4:8205/tf/tad smax=400 max=500 route=b5 redirect=a5 BalancerMember http://1.2.3.5:8205/tf/tad smax=400 max=500 route=a5 redirect=b5 BalancerMember http://1.2.3.4:8206/tf/tad smax=400 max=500 route=b6 redirect=a6 BalancerMember http://1.2.3.5:8206/tf/tad smax=400 max=500 route=a6 redirect=b6 BalancerMember http://1.2.3.4:8207/tf/tad smax=400 max=500 route=b7 redirect=a7 BalancerMember http://1.2.3.5:8207/tf/tad smax=400 max=500 route=a7 redirect=b7 BalancerMember http://1.2.3.4:8208/tf/tad smax=400 max=500 route=b8 redirect=a8 BalancerMember http://1.2.3.5:8208/tf/tad smax=400 max=500 route=a8 redirect=b8 </Proxy> ProxyPass /my/servlet balancer://mybalancer stickysession=JSESSIONID nofailover=Off maxattempts=2 lbmethod=byrequests <IfModule mpm_prefork_module> StartServers 512 ServerLimit 1024 MaxClients 1024 MinSpareServers 512 MaxSpareServers 512 MaxRequestsPerChild 512 </IfModule> Server version: Apache/2.2.4 (Unix) Server built: May 14 2007 12:26:06 Compiled in modules: core.c mod_authn_file.c mod_authn_default.c mod_authz_host.c mod_authz_groupfile.c mod_authz_user.c mod_authz_default.c mod_auth_basic.c mod_include.c mod_filter.c mod_log_config.c mod_env.c mod_setenvif.c mod_proxy.c mod_proxy_connect.c mod_proxy_ftp.c mod_proxy_http.c mod_proxy_ajp.c mod_proxy_balancer.c prefork.c http_core.c mod_mime.c mod_status.c mod_autoindex.c mod_asis.c mod_cgi.c mod_negotiation.c mod_dir.c mod_actions.c mod_userdir.c mod_alias.c mod_so.c Hardware/OS setup for all 3 boxes: Number of CPUs per Server: 4 Dual-core AMD Opteron Memory per Server: 16 gigabyte OS: Suse Enterprise Linux (SLES) Version 10.2 Is this a bug or am I configuring this wrong? Any configuration suggestions would be appreciated Thanks, J ____________________________________________________________________________________Got a little couch potato? Check out fun summer activities for kids. http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz --------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: [EMAIL PROTECTED] " from the digest: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
