At Mon, 23 Mar 2015 10:35:41 +0100, Takashi Iwai wrote: > > At Mon, 23 Mar 2015 10:02:52 +0100, > Takashi Iwai wrote: > > > > At Fri, 20 Mar 2015 19:16:53 +0100, > > Denys Vlasenko wrote: > > > Takashi, are you willing to reproduce the panic one more time, > > > with this patch? I would like to see whether oops messages > > > are more informative with it. > > > > It can't be applied to 4.0-rc5, unfortunately. > > > > arch/x86/kernel/entry_64.S: Assembler messages: > > arch/x86/kernel/entry_64.S:1725: Error: no such instruction: > > `alloc_pt_gpregs_on_stack' > > arch/x86/kernel/entry_64.S:1716: Error: invalid operands (*UND* and *UND* > > sections) for `+' > > scripts/Makefile.build:294: recipe for target 'arch/x86/kernel/entry_64.o' > > failed > > I pulled tip tree on top of 4.0-rc5, built with your patch and now > succeeded to get a better message: > > kvm: zapping shadow pages for mmio generation wraparound > kvm [5126]: vcpu0 disabled perfctr wrmsr: 0xc1 data 0xffff > Exception on user stack 00007ffd22c23ef0: RSP: 0018:00007ffd22c23f28 > EFLAGS: 00010006 > RIP: 0010:[<ffffffff8162681d>] [<ffffffff8162681d>] > netlink_attachskb+0x1d/0x1d0 > PANIC: double fault, error_code: 0x0 > CPU: 1 PID: 10819 Comm: cc1 Tainted: G W 4.0.0-rc5-debug1+ #2 > Hardware name: Dell Inc. OptiPlex 9010/0M9KCM, BIOS A12 01/10/2013 > task: ffff8800d1b34b10 ti: ffff8800d1b30000 task.ti: ffff8800d1b30000 > RIP: 0010:[<ffffffff8162681d>] [<ffffffff8162681d>] > netlink_attachskb+0x1d/0x1d0 > RSP: 0018:00007ffd22c23f28 EFLAGS: 00010006 > RAX: 0000000000000000 RBX: 0000000000000005 RCX: 00000000c0000101 > RDX: 0000000000000000 RSI: 0000000000000001 RDI: 00007ffd22c23ef0 > RBP: 0000000000000ea7 R08: 0000000000001ea7 R09: ffffffffffffffff > R10: 000000000309dbf8 R11: 0000000000000246 R12: 0000000000000001 > R13: 0000000000000000 R14: 0000000003026e40 R15: 000000000309cd50 > FS: 00007f89c83c2800(0000) GS:ffff88021d240000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 000000000000016d CR3: 00000000d90a0000 CR4: 00000000001427e0 > Stack: > 0000000000000ea7 0000000000000000 0000000003099c10 0000000000000ea7 > 0000000000000ea7 0000000000000001 0000000003099c10 0000000000000ea7 > 0000000000c84696 0000000003099c88 00007f0122c23fb8 000000000302f610 > Call Trace: > <UNK> > Code: > 10 75 ee f0 ff 42 6c 48 89 d0 5d c3 66 90 0f 1f 44 00 00 55 48 89 e5 41 56 > 41 55 49 89 d5 41 54 49 89 f4 53 48 89 fb 48 83 ec 30 <8b> 87 68 01 00 00 39 > 87 9c 01 00 00 7c 25 48 8b 87 88 04 00 00 > Kernel panic - not syncing: Machine halted. > CPU: 1 PID: 10819 Comm: cc1 Tainted: G W 4.0.0-rc5-debug1+ #2 > Hardware name: Dell Inc. OptiPlex 9010/0M9KCM, BIOS A12 01/10/2013 > 0000000000000000 ffff8800d1b33e28 ffffffff816f80d2 0000000000000000 > ffffffff81a22f81 ffff8800d1b33ea8 ffffffff816f2358 00000000000058d7 > 0000000000000008 ffff8800d1b33eb8 ffff8800d1b33e58 ffff8800d1b33ea8 > Call Trace: > [<ffffffff816f80d2>] dump_stack+0x4c/0x6e > [<ffffffff816f2358>] panic+0xc0/0x1f3 > [<ffffffff81046e65>] df_debug+0x35/0x40 > [<ffffffff81003fe7>] do_double_fault+0x87/0x100 > [<ffffffff81004167>] do_userpsace_rsp_in_kernel+0x107/0x140 > [<ffffffff8162681d>] ? netlink_attachskb+0x1d/0x1d0 > [<ffffffff81703ca6>] userpsace_rsp_in_kernel+0x36/0x40 > [<ffffffff8162681d>] ? netlink_attachskb+0x1d/0x1d0 > > > So, it seems hitting in netlink_attachskb(). > I'd need to check whether this consistently hits there or just at > random.
I managed to reproduce the bug two more times, and all three show the very same stack trace like the above. So, it's well reproducible. I'm really puzzled now. We have a few pieces of information: - git bisection pointed the commit 96b6352c1271: x86_64, entry: Remove the syscall exit audit and schedule optimizations and reverting this "fixes" the problem indeed. Even just moving two lines LOCKDEP_SYS_EXIT DISABLE_INTERRUPTS(CLBR_NONE) at the beginning of ret_from_sys_call already fixes. (Of course I can't prove the fix but it stabilizes for a day without crash while usually I hit the bug in 10 minutes in full test running.) - Another piece is that the bug happens only when a KVM is running. The kernel ran without problem over days with similar tasks (compiling kernel, etc) when no KVM was used. - And now I get the trace as above, pointing netlink_attachskb(). I have a difficulty to imagine how all these pieces fit into a single picture. Is something already screwed up before that? Takashi -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/