I wanted to post a closing message for this thread. I resolved the system contention on this Solaris VM - although it was not by way of Dtrace. Turns out the VMWare settings in the vmx file for this Solaris VM were not optimal:
> memsize = "2048" (old file) > sched.mem.max = "256" (old file) - (If sched.mem.max is smaller than memsize, > the balloon driver can start consuming memory (especially if the Guest > Operating system application has peaky memory usage). However, this setting > can cause the balloon driver to retain it's hold on memory continuously, even > if the Guest Operating System requires it again. This causes Guest Operating > System to start swapping and will slow down considerably.) Now I recognize the vmware-memctld process consuming so much CPU was a red flag for this. Once the two settings were brought into line (by using VC and checking Memory resources "unlimited") the VM functioned 100x better (responsiveness, workload throughput etc_ Thanks On 1/21/08 2:16 PM, "Brendan Gregg - Sun Microsystems" <[EMAIL PROTECTED]> wrote: > On Mon, Jan 21, 2008 at 01:55:36PM -0800, Fletcher Cocquyt wrote: >> Followup - this system has a lot of kernel activity and I/O - (top typically >> shows CPU > 50% kernel) - but the hotkernel blorked with this (eventhough >> load avg was only ~2 and command line is responsive): >> >> [EMAIL PROTECTED]:~ 1:41pm 114 # ./hotkernel >> Sampling... Hit Ctrl-C to end. >> dtrace: processing aborted: Abort due to systemic unresponsiveness > > The system is so busy DTrace has decided to play it safe and abort... > > Based on a few hunches, try these: > > - interstat 1 > look for a network driver burning CPU > > - pidpersec.d from the DTraceToolkit > (or "sar -c 1 100" if DTrace won't behave) > look for lots of short lived processes > > - "procsystime -coT" from the DTraceToolkit > look for frequent syscalls burning CPU time > > - dtrace -n 'profile-101 { @[stack(5)] = count(); }' > (this has a slower profile rate than hotuser) > look for hot kernel stacks > > Brendan _______________________________________________ dtrace-discuss mailing list dtrace-discuss@opensolaris.org