-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Big Papa,
On 11/29/17 12:06 AM, TurboChargedDad . wrote: > So.. Thank you for those help me understand the NIO vs BIO in > tomcat 7.. So now all you have to do is upgrade to Tomcat 8.0 or, even better, Tomcat 8.5 :) > I made those changes things have improved quite a bit. I am still > experiencing some weirdness that I have tried to understand but > can't get a handle on it. Can you expand on the "weirdness"? I see you have some more details below but I think you could be more specific. > Quick overview.. --Proxies-- Apache Proxies (2) - The end user > terminates SSL at the proxy/edge The proxies use HTTPS/SSL to > reverse proxy back to the tomcat server. --/Proxies-- > > PXY1 & 2 configs for prefork mode. <IfModule prefork.c> > StartServers 30 MinSpareServers 15 MaxSpareServers 30 ServerLimit > 400 MaxClients 400 MaxRequestsPerChild 4000 </IfModule> If you want high performance, you have to abandon the prefork model and move to event. Some modules (e.g. mod_php IIRC) don't work properly with the event model. Think about using your lb with PHP running on another server as Jim Riggs suggests[1]. You may get better performance, stability, and fault-tolerance. > --Tomcat server-- (1) Apache terminates SSL over the top of Tomcat > on the same server. Reverse proxies to the tomcat server using NIO > AJP connectors. --/Tomcat server-- Above you say that you are using HTTPS/SSL to connect httpd -> Tomcat. If you are using AJP then this is not true. So which is it? Are you using HTTP or AJP as your protocol? > Tomcat apache prefork mode config: <IfModule prefork.c> > StartServers 8 MinSpareServers 5 MaxSpareServers 20 > ServerLimit 800 MaxClients 800 MaxRequestsPerChild > 4000 </IfModule> What does "Tomcat apache prefork mode" mean? The above is an httpd configuration, not a Tomcat one. > Typical vhost config for a given tenant would look like this.. > <someuser.conf> <VirtualHost 10.10.10.26:443 > <http://10.10.10.26:443/>> ServerAdmin ad...@company.com > <mailto:ad...@company.com> ServerName somewhere.somedomain.com > <http://somewhere.somedomain.com/> ProxyPass / > ajp://localhost:8126/ retry=3 Okay, now you are using AJP. I think there's definitely some confusion here as to what is being configured with what. > Typical tomcat connector thread config : <Connector port="8126" > protocol="org.apache.coyote.ajp.AjpNioProtocol" > redirectPort="8443" maxThreads="300" /> If this is the only <Connector> in Tomcat, then you are 100% using AJP and not HTTP as your protocol. Using NIO is the best practice here IMO. > We are operating a multi-tenant environment. As of right now, we > have somewhere around 20 tomcat instances on a large machine of > which only a handful are "busy". Good. > It used to be that when any one of them experienced a blocking > issue. Every one of them went down. All of their AJP connector > threads would rise until the system because tomcat was > unresponsive. That would be a capacity-planning problem with the httpd proxies. You probably didn't do your math correctly. > So far that appears for the most part to be addressed... Good. Maybe your math is better but it may still be wrong. > However... When an issue is experienced. The site(s) > experiencing the issue(s) going down doesn't seem to bring down any > of the other sites. (w00t! w00t!) Good. > But the httpd connections for each site all still climb together. That shouldn't happen (of course!). > (Please see attached graph) Again no outage is experienced buy as > demonstrated by the graph attached to this message. Attachments are stripped. Either post your graph elsewhere or describe it in words. > That graph is from zabbix using a custom metric that checks every > 3 mins.. It does the following for each virtual host / tomcat > instances > > For user25 : UserParameter=somewebsite.constats,sudo -tt > /bin/netstat -ntp | grep EST | grep httpd | grep ':8125' | wc -l > UserParameter=somewebsite2.constats,sudo -tt /bin/netstat -ntp | > grep EST | grep httpd | grep ':8126' | wc -l > > So there is virtually no way they can be getting mixed up. Not to > mention that there are a few that do not experience a rise in > connections. So the "Weirdness" is that your AJP connection count on the httpd proxy instances increases across all web servers (or all workers?). What does mod_proxy's status page say for *each worker*? THAT'S what you need to compare, not just the total number of connections/threads on the proxy. - -chris -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQJRBAEBCAA7FiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAloeyX0dHGNocmlzQGNo cmlzdG9waGVyc2NodWx0ei5uZXQACgkQHPApP6U8pFgAZQ/+OyyDIEaWzgF5zG1o amUGjCUackktehlpW9STa5kRhIj9REYT4Cql64Cwqvw8ciZVQXAOsYJBACXFKcfa fvegRQ03YeLy9LDXhPtsx4Nr+qT17ySiFo/MckEIkxCR9mBbFokUb1bVes9kkYQu yJjQ7AV8SWDWKGdAkbRk4WTuJ23bvRwZ2g4MNb0sDg5dJEQIOY7JYhlFJQLPm/1a Yeeo/xRMLfY4FBI0zpA1DAXEwiLyXup4SOztHnoxbK5h0YgrRGMOKvAwZXs5/u/2 NbiqCnsA80OzUrSXd5sDBYzsuR2yOfnnUMcUJh6LmL8XG1Fh+QeJdCDM2KCC0hJL HSyqEVfl9EyIXj/Bu0DzU6lL8z5lhxoEYeYBE+pMfJKZ3Skb3EHLImuI/Fwv/+hk T13o1irPIiqq0N0pTvjLtbJbapNZ9Nz8aIzTBLR3TRY6Cf79jI49QRHBYMsth7e/ 2Ub+WmkbphcoaC1oJEpXu8fbcSuDrdTAzZ9VSvUFsqpUw5b/9OSB2DjKx5GtnHzR f8bEg682xw8VKlOV7EsD4RvyILmFqisrrTAfE8/3F3IGMYqXV8Is+8BtdScvUgT5 MzaLVVyT650pv1NJxBjz/UF8BdhJ2Rpj0r5jhqB64Q1F1ZuNO6aLI2qEwRNgwYMV IoMgNCcqcB03pYbpuYxHKoX5Cfc= =imbH -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
