On 12/9/11 9:34 AM, Win Htin wrote: > Hi folks, > > I have a particular process which is very latency sensitive (in the > milliseconds range). The network team has determined the > response/round-trip time between their monitoring device and that > particular process can go up to 4x above the norm. Looking through all > sorts of standard linux commands and output from monitoring tool > (Nagios), I can see the CPU usage never exceeds more than 20% and > app/user memory usage was below 50% during the so called slowness > windows. > > This issue pops up only a handful of times each day and since it is > very transient in nature, it is extremely difficult to determine what > is causing this intermittent slowness. Any idea what sort of tools > might be able to help me pinpoint the issue? > > The users are not complaining about other processes running on the > same server because those other processes are comparatively not that > latency sensitive. > > I am running RHEL 5.7 (64bit) with kernel revision 2.6.18-274.3.1.el5.
When the process is responding slowly, is it paged out? Have you tuned vm.swappiness for your server load to reduce the changes of processes getting paged out in favor of disk caching? The default vm.swappiness may not be ideal for your workload. A good write-up on Linux Memory Management is here: http://tldp.org/LDP/tlk/mm/memory.html You may want to look at /proc/<pid>/status to see what's going on with that process. http://www.kernel.org/doc/man-pages/online/pages/man5/proc.5.html /Brian/ -- Brian Long | | Corporate Security Programs Org . | | | . | | | . ' ' C I S C O _______________________________________________ rhelv5-list mailing list rhelv5-list@redhat.com https://www.redhat.com/mailman/listinfo/rhelv5-list
