Is there a way to see where the cpu time is spent when it isn't in userland ? I took one of our affected systems and killed everything on it as well as disabling pf.
bmr1.brh# ps aux USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 1 0.0 0.0 324 296 ?? Is 1Mar10 0:00.02 /sbin/init root 8898 0.0 0.0 708 1200 ?? Is 1Mar10 0:00.02 /usr/sbin/sshd root 29797 0.0 0.1 3424 2468 ?? Is 8:29AM 0:00.11 sshd: thehoff [priv] (sshd) thehoff 27836 0.0 0.1 3396 1912 ?? S 8:29AM 0:00.03 sshd: theh...@ttyp0 (sshd) thehoff 4730 0.0 0.0 480 408 p0 Is 8:29AM 0:00.00 -ksh (ksh) root 23806 0.0 0.0 476 460 p0 S 8:29AM 0:00.01 -ksh (ksh) root 15249 0.0 0.0 276 276 p0 R+/1 8:53AM 0:00.00 ps -aux root 25718 0.0 0.0 408 736 C0 Is+ 1Mar10 0:00.00 /usr/libexec/getty std.9600 ttyC0 root 30984 0.0 0.0 300 736 C1 Is+ 1Mar10 0:00.00 /usr/libexec/getty std.9600 ttyC1 root 7406 0.0 0.0 256 740 C2 Is+ 1Mar10 0:00.00 /usr/libexec/getty std.9600 ttyC2 root 1736 0.0 0.0 336 728 C3 Is+ 1Mar10 0:00.00 /usr/libexec/getty std.9600 ttyC3 root 1371 0.0 0.0 440 736 C5 Is+ 1Mar10 0:00.00 /usr/libexec/getty std.9600 ttyC5 bmr1.brh# load averages: 0.08, 0.09, 0.08 08:52:43 12 processes: 11 idle, 1 on processor CPU0 states: 0.0% user, 0.0% nice, 0.0% system, 0.2% interrupt, 99.8% idle CPU1 states: 0.0% user, 0.0% nice, 8.1% system, 0.0% interrupt, 91.9% idle Memory: Real: 5220K/351M act/tot Free: 2916M Swap: 0K/8197M used/tot PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND 29797 root 2 0 3424K 2468K idle netio 0:00 0.00% sshd 27836 thehoff 2 0 3396K 1912K sleep/0 select 0:00 0.00% sshd 1 root 10 0 324K 296K idle wait 0:00 0.00% init 8898 root 2 0 708K 1200K idle select 0:00 0.00% sshd 23806 root 18 0 476K 460K sleep/0 pause 0:00 0.00% ksh 32058 root 28 0 712K 1420K onproc/1 - 0:00 0.00% top 4730 thehoff 18 0 480K 408K idle pause 0:00 0.00% ksh 25718 root 3 0 408K 736K idle ttyin 0:00 0.00% getty 1736 root 3 0 336K 728K idle ttyin 0:00 0.00% getty 30984 root 3 0 300K 736K idle ttyin 0:00 0.00% getty 1371 root 3 0 440K 736K idle ttyin 0:00 0.00% getty 7406 root 3 0 256K 740K idle ttyin 0:00 0.00% getty I suspect that there is some device in there that is being polled in some funky way as this only happens on these specific boxes. Even the fujitsu box marked as "kaputt" with red marker pen works as charm. /T