I wanted to post a closing message for this thread.
I resolved the system contention on this Solaris VM - although it was not by
way of Dtrace.
Turns out the VMWare settings in the vmx file for this Solaris VM were not
optimal:
> memsize = "2048" (old file)
> sched.mem.max = "256" (old file) - (If sched.mem.max is smaller than memsize,
> the balloon driver can start consuming memory (especially if the Guest
> Operating system application has peaky memory usage). However, this setting
> can cause the balloon driver to retain it's hold on memory continuously, even
> if the Guest Operating System requires it again. This causes Guest Operating
> System to start swapping and will slow down considerably.)
Now I recognize the vmware-memctld process consuming so much CPU was a red
flag for this.
Once the two settings were brought into line (by using VC and checking
Memory resources "unlimited") the VM functioned 100x better (responsiveness,
workload throughput etc_
Thanks
On 1/21/08 2:16 PM, "Brendan Gregg - Sun Microsystems" <[EMAIL PROTECTED]>
wrote:
> On Mon, Jan 21, 2008 at 01:55:36PM -0800, Fletcher Cocquyt wrote:
>> Followup - this system has a lot of kernel activity and I/O - (top typically
>> shows CPU > 50% kernel) - but the hotkernel blorked with this (eventhough
>> load avg was only ~2 and command line is responsive):
>>
>> [EMAIL PROTECTED]:~ 1:41pm 114 # ./hotkernel
>> Sampling... Hit Ctrl-C to end.
>> dtrace: processing aborted: Abort due to systemic unresponsiveness
>
> The system is so busy DTrace has decided to play it safe and abort...
>
> Based on a few hunches, try these:
>
> - interstat 1
> look for a network driver burning CPU
>
> - pidpersec.d from the DTraceToolkit
> (or "sar -c 1 100" if DTrace won't behave)
> look for lots of short lived processes
>
> - "procsystime -coT" from the DTraceToolkit
> look for frequent syscalls burning CPU time
>
> - dtrace -n 'profile-101 { @[stack(5)] = count(); }'
> (this has a slower profile rate than hotuser)
> look for hot kernel stacks
>
> Brendan
_______________________________________________
dtrace-discuss mailing list
dtrace-discuss@opensolaris.org
