Re: Oops in trace_hardirqs_on (powerpc)
On Wed, 2010-12-22 at 21:42 -0500, Steven Rostedt wrote: > On Sun, 2010-12-19 at 14:27 +0100, Jörg Sommer wrote: > > Hi Steven, > > > > > Did you've fixed this problem? The bug report is still marked as open. > > https://bugzilla.kernel.org/show_bug.cgi?id=16573 > > > > I just posted a patch to that BZ. I have it here below too. Could you > see if it fixes you problem. I only fixed the one place that you > reported, it may need more fixes (and in that case a macro to do the > work). > > I hit the same bug on my ppc64 box, and have a fix for that, that I'll > post to LKML tomorrow. Here's my official: Signed-off-by: Steven Rostedt -- Steve > -- Steve > > diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S > index ed4aeb9..915cc03 100644 > --- a/arch/powerpc/kernel/entry_32.S > +++ b/arch/powerpc/kernel/entry_32.S > @@ -879,7 +879,18 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_47x) >*/ > andi. r10,r9,MSR_EE > beq 1f > + /* > + * Since the ftrace irqsoff latency trace checks CALLER_ADDR1, > + * which is the stack frame here, we need to force a stack frame > + * in case we came from user space. > + */ > + stwur1,-32(r1) > + mflrr0 > + stw r0,4(r1) > + stwur1,-32(r1) > bl trace_hardirqs_on > + lwz r1,0(r1) > + lwz r1,0(r1) > lwz r9,_MSR(r1) > 1: > #endif /* CONFIG_TRACE_IRQFLAGS */ > ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
Hi Steven, Steven Rostedt hat am Wed 22. Dec, 21:42 (-0500) geschrieben: > On Sun, 2010-12-19 at 14:27 +0100, Jörg Sommer wrote: > > Did you've fixed this problem? The bug report is still marked as open. > > https://bugzilla.kernel.org/show_bug.cgi?id=16573 > > > > I just posted a patch to that BZ. I have it here below too. Could you > see if it fixes you problem. I only fixed the one place that you > reported, it may need more fixes (and in that case a macro to do the > work). > > I hit the same bug on my ppc64 box, and have a fix for that, that I'll > post to LKML tomorrow. > > -- Steve > > diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S > index ed4aeb9..915cc03 100644 > --- a/arch/powerpc/kernel/entry_32.S > +++ b/arch/powerpc/kernel/entry_32.S > @@ -879,7 +879,18 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_47x) >*/ > andi. r10,r9,MSR_EE > beq 1f > + /* > + * Since the ftrace irqsoff latency trace checks CALLER_ADDR1, > + * which is the stack frame here, we need to force a stack frame > + * in case we came from user space. > + */ > + stwur1,-32(r1) > + mflrr0 > + stw r0,4(r1) > + stwur1,-32(r1) > bl trace_hardirqs_on > + lwz r1,0(r1) > + lwz r1,0(r1) > lwz r9,_MSR(r1) > 1: > #endif /* CONFIG_TRACE_IRQFLAGS */ This patch eliminates the oopses. Bye, Jörg. -- Der Klügere gibt so lange nach bis er der Dumme ist. signature.asc Description: Digital signature http://en.wikipedia.org/wiki/OpenPGP ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
On Sun, 2010-12-19 at 14:27 +0100, Jörg Sommer wrote: > Hi Steven, > > Did you've fixed this problem? The bug report is still marked as open. > https://bugzilla.kernel.org/show_bug.cgi?id=16573 > I just posted a patch to that BZ. I have it here below too. Could you see if it fixes you problem. I only fixed the one place that you reported, it may need more fixes (and in that case a macro to do the work). I hit the same bug on my ppc64 box, and have a fix for that, that I'll post to LKML tomorrow. -- Steve diff --git a/arch/powerpc/kernel/entry_32.S b/arch/powerpc/kernel/entry_32.S index ed4aeb9..915cc03 100644 --- a/arch/powerpc/kernel/entry_32.S +++ b/arch/powerpc/kernel/entry_32.S @@ -879,7 +879,18 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_47x) */ andi. r10,r9,MSR_EE beq 1f + /* +* Since the ftrace irqsoff latency trace checks CALLER_ADDR1, +* which is the stack frame here, we need to force a stack frame +* in case we came from user space. +*/ + stwur1,-32(r1) + mflrr0 + stw r0,4(r1) + stwur1,-32(r1) bl trace_hardirqs_on + lwz r1,0(r1) + lwz r1,0(r1) lwz r9,_MSR(r1) 1: #endif /* CONFIG_TRACE_IRQFLAGS */ ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
On Mon, 2010-12-20 at 15:43 -0500, Steven Rostedt wrote: > Anyway, it looks like this only affects 32 bit PPC as I can't reproduce > it with my 64 bit one. And also, unfortunately, my 32bit ppc got taken > from me by my kids, so I can't test it on that either. Spoke too soon, I just triggered it on 64bit. I'll look into it. Thanks! -- Steve ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
On Sun, 2010-12-19 at 14:27 +0100, Jörg Sommer wrote: > Hi Steven, > > Steven Rostedt hat am Mon 27. Sep, 21:58 (-0400) geschrieben: > > On Mon, 2010-09-27 at 14:50 +0200, Jörg Sommer wrote: > > > Hello Steven, > > > > > > Steven Rostedt hat am Wed 22. Sep, 15:44 (-0400) geschrieben: > > > > Sorry for the late reply, but I was on vacation when you sent this, and > > > > I missed it while going through email. > > > > > > > > Do you still have this issue? > > > > > > No. I've rebuild my kernel without TRACE_IRQFLAGS and the problem > > > vanished, as expected. The problem is, that in some cases the stack is > > > only two frames deep, which causes the macro CALLER_ADDR1 makes an > > > invalid access. Someone told me, there a workaround for the problem on > > > i386, too. > > > > > > % sed -n 2p arch/x86/lib/thunk_32.S > > > * Trampoline to trace irqs off. (otherwise CALLER_ADDR1 might crash) > > > > Yes, I remember that problem. When I get back from Tokyo, I'll tried to > > remember to fix it. > > Did you've fixed this problem? The bug report is still marked as open. > https://bugzilla.kernel.org/show_bug.cgi?id=16573 > Ah, this email got lost in the hundreds I had when I got back from Tokyo, sorry about that again :-( Anyway, it looks like this only affects 32 bit PPC as I can't reproduce it with my 64 bit one. And also, unfortunately, my 32bit ppc got taken from me by my kids, so I can't test it on that either. I'll look to see if I can write up a patch. Perhaps you could test it for me. Thanks, -- Steve ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
Hi Steven, Steven Rostedt hat am Mon 27. Sep, 21:58 (-0400) geschrieben: > On Mon, 2010-09-27 at 14:50 +0200, Jörg Sommer wrote: > > Hello Steven, > > > > Steven Rostedt hat am Wed 22. Sep, 15:44 (-0400) geschrieben: > > > Sorry for the late reply, but I was on vacation when you sent this, and > > > I missed it while going through email. > > > > > > Do you still have this issue? > > > > No. I've rebuild my kernel without TRACE_IRQFLAGS and the problem > > vanished, as expected. The problem is, that in some cases the stack is > > only two frames deep, which causes the macro CALLER_ADDR1 makes an > > invalid access. Someone told me, there a workaround for the problem on > > i386, too. > > > > % sed -n 2p arch/x86/lib/thunk_32.S > > * Trampoline to trace irqs off. (otherwise CALLER_ADDR1 might crash) > > Yes, I remember that problem. When I get back from Tokyo, I'll tried to > remember to fix it. Did you've fixed this problem? The bug report is still marked as open. https://bugzilla.kernel.org/show_bug.cgi?id=16573 Regards, Jörg. -- Begebenheit aus dem wahren Leben: Mediziner: ICEs sind die weißen Züge. Mathematiker: Das ist falsch. Jeder ICE ist zwar weiß, aber nicht alle weißen Züge sind ICEs. signature.asc Description: Digital signature http://en.wikipedia.org/wiki/OpenPGP ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
On Mon, 2010-09-27 at 14:50 +0200, Jörg Sommer wrote: > Hello Steven, > > Steven Rostedt hat am Wed 22. Sep, 15:44 (-0400) geschrieben: > > Sorry for the late reply, but I was on vacation when you sent this, and > > I missed it while going through email. > > > > Do you still have this issue? > > No. I've rebuild my kernel without TRACE_IRQFLAGS and the problem > vanished, as expected. The problem is, that in some cases the stack is > only two frames deep, which causes the macro CALLER_ADDR1 makes an > invalid access. Someone told me, there a workaround for the problem on > i386, too. > > % sed -n 2p arch/x86/lib/thunk_32.S > * Trampoline to trace irqs off. (otherwise CALLER_ADDR1 might crash) Yes, I remember that problem. When I get back from Tokyo, I'll tried to remember to fix it. Thanks! -- Steve ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
Hello Steven, Steven Rostedt hat am Wed 22. Sep, 15:44 (-0400) geschrieben: > Sorry for the late reply, but I was on vacation when you sent this, and > I missed it while going through email. > > Do you still have this issue? No. I've rebuild my kernel without TRACE_IRQFLAGS and the problem vanished, as expected. The problem is, that in some cases the stack is only two frames deep, which causes the macro CALLER_ADDR1 makes an invalid access. Someone told me, there a workaround for the problem on i386, too. % sed -n 2p arch/x86/lib/thunk_32.S * Trampoline to trace irqs off. (otherwise CALLER_ADDR1 might crash) Bye, Jörg. -- Angenehme Worte sind nie wahr, wahre Worte sind nie angenehm. signature.asc Description: Digital signature http://en.wikipedia.org/wiki/OpenPGP ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Oops in trace_hardirqs_on (powerpc)
Sorry for the late reply, but I was on vacation when you sent this, and I missed it while going through email. Do you still have this issue? -- Steve On Sat, 2010-08-07 at 01:31 +0200, Jörg Sommer wrote: > Hi, > > I've built my 2.6.35 with tracing support and now, I'm getting > continuously oops'. It seems to happen on high process activity. > > [ 52.336371] device eth0 entered promiscuous mode > [ 52.347616] device eth0 left promiscuous mode > [ 55.240663] Unable to handle kernel paging request for data at address > 0xbfaf4a24 > [ 55.248289] Faulting instruction address: 0xc00aad98 > [ 55.255562] Oops: Kernel access of bad area, sig: 11 [#1] > [ 55.262588] PowerMac > [ 55.269606] last sysfs file: > /sys/devices/pci:00/:00:10.0/graphics/fb0/radeonbl0/brightness > [ 55.277111] Modules linked in: fuse snd_powermac option usb_wwan usbserial > ecb b43 snd_aoa_i2sbus snd_pcm_oss > [ 55.302368] NIP: c00aad98 LR: c001771c CTR: c003dba0 > [ 55.310738] REGS: e3211e70 TRAP: 0300 Not tainted (2.6.35) > [ 55.319122] MSR: 1032 CR: 22f88f42 XER: 2000 > [ 55.327650] DAR: bfaf4a24, DSISR: 4000 > [ 55.335954] TASK = e3245bc0[1929] 'sh' THREAD: e321 > [ 55.336144] GPR00: e3211f20 e3245bc0 e3245bc0 c000b944 > 003a1040 > [ 55.344859] GPR08: bfaf4a20 c05e c0614d18 c061 10033368 > 10018520 10007c2c > [ 55.353723] GPR16: 10007c30 bfecaa10 101d8304 > 10019c28 bfecbfab > [ 55.362438] GPR24: bfecaa08 10019c58 06d1 c063be80 bfeca9a0 > 0ffebff4 e3211f20 > [ 55.378913] NIP [c00aad98] trace_hardirqs_on+0x5c/0x124 > [ 55.386856] LR [c001771c] restore+0x10/0x6c > [ 55.394527] Call Trace: > [ 55.401878] [e3211f20] [10019c58] 0x10019c58 (unreliable) > [ 55.409437] [e3211f40] [c001771c] restore+0x10/0x6c > [ 55.417065] --- Exception: c00 at 0xff23c88 > [ 55.417071] LR = 0xff23c54 > [ 55.432267] Instruction dump: > [ 55.439808] 800a005c 70090002 418200c8 7ca6 70008000 408200bc 3d20c05e > 838a0058 > [ 55.447730] 81096f98 2f88 811f 8108 <83680004> 41be009c > 816b4d18 90096f98 > [ 55.455722] ---[ end trace 547f1189532873f7 ]--- > [ 390.022834] EXT4-fs (dm-0): mounted filesystem with ordered data mode. > Opts: (null) > > [ 507.793120] lo: Disabled Privacy Extensions > [ 518.228969] eth0: no IPv6 routers present > [ 737.593898] Unable to handle kernel paging request for data at address > 0x0004 > [ 737.593927] Faulting instruction address: 0xc00aad98 > [ 737.593957] Oops: Kernel access of bad area, sig: 11 [#2] > [ 737.593967] PowerMac > [ 737.593976] last sysfs file: > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor > [ 737.593992] Modules linked in: ppp_async crc_ccitt ipv6 ppp_generic slhc > fuse snd_powermac option usb_wwan usb > [ 737.594132] NIP: c00aad98 LR: c001771c CTR: c003dba0 > [ 737.594148] REGS: e685de70 TRAP: 0300 Tainted: G D (2.6.35) > [ 737.594159] MSR: 1032 CR: 2442 XER: 2000 > [ 737.594187] DAR: 0004, DSISR: 4000 > [ 737.594198] TASK = e30b3780[3322] 'zsh-beta' THREAD: e685c000 > [ 737.594208] GPR00: e685df20 e30b3780 e30b3780 c000b944 > 003e5f00 > [ 737.594240] GPR08: c05e c0614d18 c061 100b4ee8 > 10092dec > [ 737.594271] GPR16: 100bb400 100916fc bfbda1b0 bfbda4ec > > [ 737.594303] GPR24: 100b 100bae50 0cea c063be80 bfbd9e60 > 0fe64ff4 e685df20 > [ 737.594362] NIP [c00aad98] trace_hardirqs_on+0x5c/0x124 > [ 737.594379] LR [c001771c] restore+0x10/0x6c > [ 737.594388] Call Trace: > [ 737.594402] [e685df20] [100bae50] 0x100bae50 (unreliable) > [ 737.594421] [e685df40] [c001771c] restore+0x10/0x6c > [ 737.594432] Instruction dump: > [ 737.594442] 800a005c 70090002 418200c8 7ca6 70008000 408200bc 3d20c05e > 838a0058 > [ 737.594473] 81096f98 2f88 811f 8108 <83680004> 41be009c > 816b4d18 90096f98 > [ 737.594514] ---[ end trace 547f1189532873f8 ]--- > [ 737.919108] Unable to handle kernel paging request for data at address > 0x0003 > [ 737.919137] Faulting instruction address: 0xc00aad98 > [ 737.919168] Oops: Kernel access of bad area, sig: 11 [#3] > [ 737.919179] PowerMac > [ 737.919187] last sysfs file: > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor > [ 737.919203] Modules linked in: ppp_async crc_ccitt ipv6 ppp_generic slhc > fuse snd_powermac option usb_wwan usb > [ 737.919342] NIP: c00aad98 LR: c001771c CTR: > [ 737.919358] REGS: e6d15e70 TRAP: 0300 Tainted: G D (2.6.35) > [ 737.919369] MSR: 1032 CR: 2442 XER: > [ 737.919397] DAR: 0003, DSISR: 4000 > [ 737.919409] TASK = e30b3780[3350] 'zsh-beta' THREAD: e6d14000 > [ 737.919419] GPR00: e6d15f20 e30b3780 e30b3780 c000b944 > 0065df00 0008 > [ 737.919451] GP
