Hello The problem appears several times ,however I could not capture the top output .My script is as follows code. I check the sys cpu usage whether it exceed 30%.the other metric information can be dumpped successfully except the top . Would you like to check my script that I am not able to figure out what is wrong.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------- #!/bin/bash while : do sysusage=$(mpstat 2 1 | grep -A 1 "%sys" | tail -n 1 | awk '{if($6 < 30) print 1; else print 0;}' ) if [ $sysusage -eq 0 ];then #echo $sysusage #perf record -o perf$(date +%Y%m%d%H%M%S).data -a -g -F 1000 sleep 30 file=$(date +%Y%m%d%H%M%S) top -n 2 >> top$file.data iotop -b -n 2 >> iotop$file.data iostat >> iostat$file.data netstat -an | awk '/^tcp/ {++state[$NF]} END {for(i in state) print i,"\t",state[i]}' >> netstat$file.data fi sleep 5 done You have new mail in /var/spool/mail/root ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2016-03-08 21:39 GMT+08:00 YouPeng Yang <yypvsxf19870...@gmail.com>: > Hi all > Thanks for your reply.I do some investigation for much time.and I will > post some logs of the 'top' and IO in a few days when the crash come again. > > 2016-03-08 10:45 GMT+08:00 Shawn Heisey <apa...@elyograg.org>: > >> On 3/7/2016 2:23 AM, Toke Eskildsen wrote: >> > How does this relate to YouPeng reporting that the CPU usage increases? >> > >> > This is not a snark. YouPeng mentions kernel issues. It might very well >> > be that IO is the real problem, but that it manifests in a non-intuitive >> > way. Before memory-mapping it was easy: Just look at IO-Wait. Now I am >> > not so sure. Can high kernel load (Sy% in *nix top) indicate that the IO >> > system is struggling, even if IO-Wait is low? >> >> It might turn out to be not directly related to memory, you're right >> about that. A very high query rate or particularly CPU-heavy queries or >> analysis could cause high CPU usage even when memory is plentiful, but >> in that situation I would expect high user percentage, not kernel. I'm >> not completely sure what might cause high kernel usage if iowait is low, >> but no specific information was given about iowait. I've seen iowait >> percentages of 10% or less with problems clearly caused by iowait. >> >> With the available information (especially seeing 700GB of index data), >> I believe that the "not enough memory" scenario is more likely than >> anything else. If the OP replies and says they have plenty of memory, >> then we can move on to the less common (IMHO) reasons for high CPU with >> a large index. >> >> If the OS is one that reports load average, I am curious what the 5 >> minute average is, and how many real (non-HT) CPU cores there are. >> >> Thanks, >> Shawn >> >> >