>-----Original Message----- >From: RAY, DAVID [mailto:david....@tccd.edu] >Sent: Tuesday, June 30, 2015 12:10 PM >To: us...@tomcat.apache.org >Subject: AIX OS Patch Breaks Apache/Tomcat > >Hello. > >I am running Apache 2.2.29 and Tomcat 7.0.59 with tomcat connector(mod_jk) >version 1.2.40 on AIX version 7.1 server. Started having problems this >morning after AIX OS was patched to AIX 7.1 TL 03 SP 04 and openssh to >>6.0.0.6108. WebAdvisor runs fine immediately after apache is started or >restarted. However its response slowing down. AIX server CPU steadily >increases and approaches 100% after running for 5 or 10 minutes under heavy >>load.. Seeing http process accumulate. Not seeing much traffic at all in >Tomcat server status. Recompiled apache, tomcat connector, and Tomcat native. > Still no luck. Later determined there is a bug with openssh 6.0.0.6.108 >and >tried 6.0.0.6109. Recompiled again. Still no improvement. Just booted from >Clone backup taken before the AIX patch and everything runs fine, no >problems. Anyone else have this experience with AIX >patches/updates? Not >really much to see in error logs. Tomcat is configured to run with java6 32 >bit: > >IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc-32 >jvmap3260sr16-20141216_227499 (JIT enabled, AOT enabled) >J9VM - 20141216_227499 >JIT - r9_20140523_64469ifx3 >GC - GA24_Java6_SR16_20141216_1020_B227499) >JCL - 20141216_01 > > >Thank you. >David
We have determined that that this problem is triggered by Apache server-status. I have always used server-status to monitor our production system, no problem. I rarely use server-status to monitor our test system because there is nothing to see there due to extremely light traffic. But after the AIX patch, server-status gradually causes the sever CPU to be maxed out. This problem was just duplicated on our test server. The CPU drops back to normal for that server when server-status is stopped. Example of server-status URL I use in my IE 11 browser: https://website.com/server-status?refresh=5 Has anyone else had this experience? I guess for now, I will just have to stop using server-status to monitor Apache so that we can patch our AIX system. Interesting how server-status is actually showing the gradual CPU increase. When this snapshot was taken, server-status had just moved from line 9 to line 10, where the W is. The 53.55 at line 9 is what the CPU usage was when this snapshot was taken: 0-0 17039448 0/13/13 _ 48.65 52 371 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/13/13 _ 49.14 47 305 0.0 0.10 0.10 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/13/13 _ 49.64 41 297 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/13/13 _ 50.46 35 554 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/13/13 _ 51.58 23 380 0.0 0.12 0.12 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/13/13 _ 51.04 29 373 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/13/13 _ 52.19 17 399 0.0 0.14 0.14 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/13/13 _ 52.93 11 528 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/13/13 _ 53.55 5 438 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 W 40.99 0 0 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 43.98 104 256 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 41.50 133 355 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 42.01 127 338 0.0 0.10 0.10 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/11/11 _ 42.47 122 237 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 42.93 116 325 0.0 0.10 0.10 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 43.45 110 320 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 44.41 99 264 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 44.91 93 368 0.0 0.12 0.12 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 46.99 70 378 0.0 0.08 0.08 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 45.48 87 365 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 45.91 82 256 0.0 0.10 0.10 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 48.06 58 254 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 46.41 76 294 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 0-0 17039448 0/12/12 _ 47.53 64 368 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 David
