Hi Jamie, when this happens you can do a thread-dump (kill -3 pid on Linux platforms) and you would see if there is a lock on JDBC objects, or anything else synchronized (from the Collections like Hashtable). Not easy for beginners to understand a dump, but worth learning.
Very often an application will slow down because Garbage Collection, and the GC thread will be the only one working at that time. Please use the verbose settings to have all your Tomcats log GC activity. This is useful for both live and after time analysis. You won't monitor mod_proxy, this has no sense. If you suspect a network flood, do a 'netstat -an' to see how many connections are used by your Apache (both ESTABLISHED and TIME_WAIT). However, you might need to set timeout and flushpacket on your proxypass directive when your application has such requirements (see HTTPD doc for this settings). To know if you're flooded, you can do a 'ps -eaf | grep [h]ttp' and you would see many httpd process, it is similar to the netstat result more or less. If you enjoy live monitoring, you need to have a look on Christopher's presentation ( http://events.linuxfoundation.org/sites/events/files/slides/Monitoring%20Apache%20Tomcat%20with%20JMX.pdf ) who posts many answers to this mailing list. regards 2015-10-07 22:20 GMT+02:00 Jamie Jackson <jamieja...@gmail.com>: > Hi Folks, > > I had a server crash on Friday, and I had the following symptoms: > > Response time worsened, and when I looked at my real-time monitoring (an > app called FusionReactor), I saw that the app *seemed* to be > single-threading (I only saw one request process at a time), then after a > few minutes, even though resources still looked good, the app no longer > seemed to receive any requests. > > My first guess was that somehow, connections were slowly getting clogged > until there were none left, then my server was completely unresponsive. > > I front Tomcat (7.0.59) with Apache HTTPD, and connect with mod_proxy. (The > end app is built in Lucee, and is running on RHEL 6.7 and Oracle Java > 1.7.0_76.) > > My instinct is to monitor the connection (pools?) in, I guess, Tomcat and > mod_proxy, but: > > > 1. I don't know if that's sane. > 2. I have no clue how one would go about that. > > I could use some input. > > Thanks, > Jamie >
