> > If it's true that some goroutines are simply looping, then the only > way to get their current PC is to crash the program with SIGQUIT and > look at the stack trace. The runtime doesn't have any way to ask a > goroutine what the currently executing PC is.
I suggest using pstack to capture the stacks of all threads of you can. > Because it is external ut is not subject to the stopping bias of the > internal go facilities. Thanks for the suggestions. I'll set up something external that sends sigquit during the stall and add an external gstack in a loop. On Tue, Aug 27, 2019 at 7:30 AM Robert Engels <reng...@ix.netcom.com> wrote: > I suggest using pstack to capture the stacks of all threads of you can. > Because it is external ut is not subject to the stopping bias of the > internal go facilities. > > > On Aug 27, 2019, at 9:05 AM, Ian Lance Taylor <i...@golang.org> wrote: > > > >> On Mon, Aug 26, 2019 at 2:12 PM Michael Andersen <mich...@steelcode.com> > wrote: > >> > >> I was concerned that perhaps the stacks that I dumped after the > scheduling stall did not reflect the actual PC's of the goroutines during > the stall, so I modified schedtrace to dump out g.sched.pc and g.m.vdsoPC > for each G currently on the P at the time schedtrace prints. > >> > >> It just occurred again a few minutes ago. Previously, all the P's were > running the problematic code, so other things weren't scheduled. This time > it appears that only a few P's (4 of 8) were running the problematic code > but the GC was trying to stop the world so even though there are P's with > nothing on them, nothing else ran. All the PC's on the problematic > goroutines (those with status=_Prunning) were static for the 3 seconds they > stalled. They all had the same PC's: > >> > >> g.sched.pc was 0x43389f which maps to: Line 302 of > "/usr/local/go/src/runtime/proc.go" starts at address 0x43389f > <runtime.gopark+239> and ends at 0x4338b7 <runtime.gopark+263>. > >> g.m.vdsoPC was 0x450c68 which maps to: Line 17 of > "/usr/local/go/src/runtime/timestub.go" starts at address 0x450c63 > <time.now+51> and ends at 0x450c8d <time.now+93>. > >> > >> In case there is more useful information that I missed, I attached the > log output. > > > > g.sched.pc doesn't tell us much about the current state. It tells us > > the PC value of the goroutine the last time the goroutine blocked. > > What you are seeing is that goroutine's block in gopark. That is not > > surprising. What we don't know is what the goroutine is currently > > doing. Similarly, vdsoPC tells us the PC value the last time the > > goroutine went into a VDSO call. Having that be time.now is not > > surprising. It doesn't tell us anything about what the goroutine is > > doing now. Also note that vdsoPC is only meaningful if vdsoSP != 0. > > > > > >> I'm trying to make a simpler program that reproduces this, but thus far > running time.Now in a tight loop across multiple goroutines isn't causing > it. Any ideas what I could try next? > > > > If it's true that some goroutines are simply looping, then the only > > way to get their current PC is to crash the program with SIGQUIT and > > look at the stack trace. The runtime doesn't have any way to ask a > > goroutine what the currently executing PC is. > > > > Ian > > > > -- > > You received this message because you are subscribed to the Google > Groups "golang-nuts" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to golang-nuts+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/CAOyqgcU1pvzbun3488exV%3D%2BhTauSxd8JpBh0Oo76inVafHu-rA%40mail.gmail.com > . > > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAPLpPrv-V6EQUBPVoGb%2BrP9TtGzFDwpu6ZfOP-26OUspkWJGVQ%40mail.gmail.com.
