On Mon, May 12, 2014 at 02:06:49AM +0530, Srivatsa S. Bhat wrote: > Today the smp-call-function code just prints a warning if we get an IPI on > an offline CPU. This info is sufficient to let us know that something went > wrong, but often it is very hard to debug exactly who sent the IPI and why, > from this info alone. > > In most cases, we get the warning about the IPI to an offline CPU, immediately > after the CPU going offline comes out of the stop-machine phase and reenables > interrupts. Since all online CPUs participate in stop-machine, the information > regarding the sender of the IPI is already lost by the time we exit the > stop-machine loop. So even if we dump the stack on each CPU at this point, > we won't find anything useful since all of them will show the stack-trace of > the stopper thread. So we need a better way to figure out who sent the IPI and > why. > > To achieve this, when we detect an IPI targeted to an offline CPU, loop > through > the call-single-data linked list and print out the payload (i.e., the name > of the function which was supposed to be executed by the target CPU). This > would give us an insight as to who might have sent the IPI and help us debug > this further. > > Signed-off-by: Srivatsa S. Bhat <srivatsa.b...@linux.vnet.ibm.com> > --- > > kernel/smp.c | 18 ++++++++++++++---- > 1 file changed, 14 insertions(+), 4 deletions(-) > > diff --git a/kernel/smp.c b/kernel/smp.c > index 06d574e..f864921 100644 > --- a/kernel/smp.c > +++ b/kernel/smp.c > @@ -185,14 +185,24 @@ void generic_smp_call_function_single_interrupt(void) > { > struct llist_node *entry; > struct call_single_data *csd, *csd_next; > + static bool warned; > + > + entry = llist_del_all(&__get_cpu_var(call_single_queue)); > + entry = llist_reverse_order(entry); > > /* > * Shouldn't receive this interrupt on a cpu that is not yet online. > */ > - WARN_ON_ONCE(!cpu_online(smp_processor_id())); > - > - entry = llist_del_all(&__get_cpu_var(call_single_queue)); > - entry = llist_reverse_order(entry); > + if (unlikely(!cpu_online(smp_processor_id()) && !warned)) { > + warned = true; > + WARN_ON(1);
More details may be better: WARN_ONCE(1, "IPI on offline CPU"); > + /* > + * We don't have to use the _safe() variant here > + * because we are not invoking the IPI handlers yet. > + */ > + llist_for_each_entry(csd, entry, llist) > + pr_warn("SMP IPI Payload: %pS \n", csd->func); Payload is kind of vague. How about "IPI func %pS sent on offline CPU". > + } > > llist_for_each_entry_safe(csd, csd_next, entry, llist) { > csd->func(csd->info); > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/