On Mon, Jan 12, 2015 at 11:48:28AM +0000, Stoidner, Christoph wrote:
> Hi Paul,
>
> > You got stack traces with the stall warnings, correct? If so, please look
> > at them and at Documentation/RCU/stallwarn.txt and see if the kernel is
> > looping somewhere inappropriate.
>
> Yes and no. I have a stack trace, but it is not generated by a stall warning.
> More
> precise: I can never see any stall warning. The reason is that the system
> freezes
> when it is about to output such a warning. Instead the stack trace is
> generated
> by gdb and JTAG hardware debugging, when freezing has occurred.
>
> So I am not sure if there is really a CPU-stall condition or it is just a
> misrepresented
> stall detection. However, outputting a stall warning leads to system freeze.
> The
> warning is never seen.
Two things to try:
1. alt-sysreq-t to get all tasks' stacks, or
2. disable RCU CPU stall warnings and see if the hangs go away.
Hmmm... Are you by chance pushing all dmesg through a serial console?
> > I am not familiar with the low-level ARM kernel code, but the stack below
> > leads me to suspect that your kernel is interrupting itself to death or
> > is improperly handling interrupts.
>
> The stack trace must be read from bottom to top. The repetitive occurrence of
> "__irq_svc () at arch/arm/kernel/entry-armv.S:202" on bottom of stack trace
> is
> caused by the stack frame of the interrupt context. This is completely legal
> and
> also the case in normal situations. Instead the problem is on the top of the
> stack
> trace, in function rcu_print_task_stall(). The loop rcutree_plugin.h in line
> 528
> never ends:
>
> static int rcu_print_task_stall(struct rcu_node *rnp)
> {
> ...
> ...
>
> list_for_each_entry_continue(t, &rnp->blkd_tasks, rcu_node_entry) {
> printk(KERN_CONT " P%d", t->pid);
> ndetected++;
> }
>
> ...
> ...
> }
>
> That means list_for_each_entry_continue () never ends since
> rcu_node_entry.next
> seems to point to it-self but not to rnp->blkd_tasks. I have no idea how this
> can
> happen.
It is not supposed to happen, and I haven't heard of it happening
anywhere else. I do hold the appropriate lock across that code.
One thing to try would be to add a counter and break out of the loop
after (say) 10 iterations. Is that a change you are comfortable making?
> One more thing: Just for testing I have now enabled CONFIG_TINY_PREEMPT_RCU.
> Until now the problem has not occurred anymore. Do you have any idea what
> makes
> the differences here?
Any number of things, including that I am not sure that your version
of CONFIG_TINY_PREEMPT_RCU correctly detects RCU CPU stalls. ;-)
Please note that CONFIG_TINY_PREEMPT_RCU was removed a few versions ago.
Thanx, Paul
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
