On Mon, Jan 26, 2009 at 01:13:32PM -0800, Jonathan Adams wrote:
> On Sun, Jan 25, 2009 at 12:36:49PM -0800, Matt Stupple wrote:
> > [Also posted on tools-discuss, but no response at present.]
> >
> > I'm looking at a pstack dump from a hanging process in a live
> > environment. The theory is that the process is stuck in a file I/O
> > call due to NFS issues, but the suspect thread (78) shows *no* call
> > stack in the pstack output (other threads are showing 'correct' call
> > stacks):
> >
> > ----------------- lwp# 78 --------------------------------
> > ----------------- lwp# 79 --------------------------------
>
> > Is this possibly due to the fact that the thread is stuck in the
> > kernel inside an I/O call? Is there any way to get more information
> > (run pstack as root?)
>
> This indicates that the thread could not be stopped, so the userland
> state couldn't be retrieved. This will also effect gcore.
>
> There are two ways to get the kernel and userland stack traces:
>
> 1) mdb
> If you only need the kernel thread trace, you can do:
>
> 0tpid::pid2proc | ::walk thread t | ::print kthread_t t_tid | \
> ::grep '#(.-0ttid)' | ::map '<t' | ::findstack -v
> ^^^ pid ^^^ lwpid
> 2) dtrace
> Something like:
>
> dtrace -n 'sched:::off-cpu/pid == pid && tid == tid/
^^^ ^^^
replace these with the decimal pid and tid, of course.
> {stack(20); ustack(20); exit(0); }'
>
> Will give you both the kernel and user stack trace. If your thread
> is spinning in the kernel without ever going to sleep, you can do:
>
> dtrace -n 'profile:::profile-997,sched:::off-cpu
> /pid == pid && tid == tid/ {stack(20); ustack(20); exit(0); }'
>
> Cheers,
> - jonathan
>
>
> > I also have a gcore-generated core for the process which I've been
> > trying to debug - it gives the same output when I run pstack on the
> > core - but I've got some symbol mismatches between dev and prod at the
> > moment...
> > --
> > This message posted from opensolaris.org
> > _______________________________________________
> > observability-discuss mailing list
> > observability-discuss at opensolaris.org
