About 10 hours ago, my website started getting really slow. My MRTG graph shows the server's bandwidth usage dropping to 1/4 of its previous amount:
http://lina.aaanime.net/~pmak/dropoff.png
When I got on the server, the load average was above 7. The shell prompt response time was very slow, the webpages took forever to load, but the CPU was over 90% idle, the swap used was less than the physical memory free, and the si/so/bi/bo values in "vmstat 1" were mostly 0.
I couldn't figure out why it was lagging so badly. Then on a whim, I tried "rm -rf" on the StateDir. An empty StateDir was re-created by Apache::ASP, and then the lag problems mysteriously evaporated.
Any ideas on how I can prevent this problem from reoccurring? (Damn, I should have saved the StateDir for further analysis instead of deleting it.)
I have never seen this before. My advice would be to write a program to monitor your system load & send you an alert when its over a certain value. I would use the 5 minute load value & not 1 minute as 1 minute can be fairly volatile.
Yes, next time, please save the StateDir for possibly reproducing this. It may have been something like your site getting slammed by a bot/spider, and then the StateDir having to cleanup all the garbage sessions &/or deal with really huge internal databases tracking timeouts & such. Maybe it was some quirk to do with SDBM_File used for the internal database that is used with the session manager. I am not sure we will ever know now.
Regards,
Josh ________________________________________________________________ Josh Chamas, Founder phone:925-552-0128 Chamas Enterprises Inc. http://www.chamas.com NodeWorks Link Checker http://www.nodeworks.com
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
