Re: Dynamic-ftrace not working in PlayStation3
> OK, I've been able to reproduce it. It looks like the assumptions of the > trampoline that I've made is not correct if the size of the module is over > some limit. I had to make usb-core into a module, and I see the same > bug. > > I'll take a deeper look into it. Thanks ! I've been a bit swamped lately and didn't get a chance to dig into that one .. yet. Let me know if you are stuck tho. Cheers Ben. ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Re: Dynamic-ftrace not working in PlayStation3
On Sat, 31 Jan 2009, Arnd Bergmann wrote: > On Saturday 31 January 2009, Geoff Levand wrote: > > > > > > So I take it that the above showed that the code worked for some? > > > > In my trials it blows up on the first load_module() call, and for my > > config that was usbcore: > > I looked into it some more with Remis yesterday, and we got ftrace > working by building usbcore and alsa into the kernel. Incidentally, > usbcore is the largest module in this installation (~3MB), which > indicates that it is in some way either related to the size of the > module. > > > This ptr of 5f666f725f726573 seems bogus. I think this code is not > > working correctly: > > This is an ascii string "_for_res", probably part of > usb_lock_device_for_reset. We saw the same behaviour, but > a different string. OK, I've been able to reproduce it. It looks like the assumptions of the trampoline that I've made is not correct if the size of the module is over some limit. I had to make usb-core into a module, and I see the same bug. I'll take a deeper look into it. -- Steve ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Re: Dynamic-ftrace not working in PlayStation3
On Sun, 1 Feb 2009, Remis wrote: > Complementing what Arnd said, we got the dynamic ftrace to work. Just > changed 4 things in the *menuconfig*. > Before these were built as modules (this is the default I guess): > CONFIG_USB=m > CONFIG_SOUND=m > CONFIG_SND=m > CONFIG_SND_USB_AUDIO=m > > Then got them built into the kernel: > CONFIG_USB=y > CONFIG_SOUND=y > CONFIG_SND=y > CONFIG_SND_USB_AUDIO=y > > And everything worked just fine. > In my config when I also tried to build the *CONFIG_SND_PS3* into the > kernel, the kernel did not boot (probably another bug !? :-). JFYI, I always use CONFIG_SND_PS3=y in my PS3 kernels, so it should work. Probably it conflicts with something else I have not enabled... With kind regards, Geert Uytterhoeven Software Architect Sony Techsoft Centre Europe The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium Phone:+32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: geert.uytterhoe...@sonycom.com Internet: http://www.sony-europe.com/ A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010 ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Re: Dynamic-ftrace not working in PlayStation3
Complementing what Arnd said, we got the dynamic ftrace to work. Just changed 4 things in the *menuconfig*. Before these were built as modules (this is the default I guess): CONFIG_USB=m CONFIG_SOUND=m CONFIG_SND=m CONFIG_SND_USB_AUDIO=m Then got them built into the kernel: CONFIG_USB=y CONFIG_SOUND=y CONFIG_SND=y CONFIG_SND_USB_AUDIO=y And everything worked just fine. In my config when I also tried to build the *CONFIG_SND_PS3* into the kernel, the kernel did not boot (probably another bug !? :-). Cheers, Remis On Sat, Jan 31, 2009 at 11:25 AM, Arnd Bergmann wrote: > On Saturday 31 January 2009, Geoff Levand wrote: >> > >> > So I take it that the above showed that the code worked for some? >> >> In my trials it blows up on the first load_module() call, and for my >> config that was usbcore: > > I looked into it some more with Remis yesterday, and we got ftrace > working by building usbcore and alsa into the kernel. Incidentally, > usbcore is the largest module in this installation (~3MB), which > indicates that it is in some way either related to the size of the > module. > >> This ptr of 5f666f725f726573 seems bogus. I think this code is not >> working correctly: > > This is an ascii string "_for_res", probably part of > usb_lock_device_for_reset. We saw the same behaviour, but > a different string. > >Arnd <>< ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Re: Dynamic-ftrace not working in PlayStation3
On Saturday 31 January 2009, Geoff Levand wrote: > > > > So I take it that the above showed that the code worked for some? > > In my trials it blows up on the first load_module() call, and for my > config that was usbcore: I looked into it some more with Remis yesterday, and we got ftrace working by building usbcore and alsa into the kernel. Incidentally, usbcore is the largest module in this installation (~3MB), which indicates that it is in some way either related to the size of the module. > This ptr of 5f666f725f726573 seems bogus. I think this code is not > working correctly: This is an ascii string "_for_res", probably part of usb_lock_device_for_reset. We saw the same behaviour, but a different string. Arnd <>< ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Re: Dynamic-ftrace not working in PlayStation3
[ added Ben and Paul to Cc, since they helped me write the code ] On Fri, 30 Jan 2009, Geoff Levand wrote: > Hi, > > I did a bit of work on this today. > > Steven Rostedt wrote: > >> > ip:d0045aec jumps to d0046340 r2: d0050c00 > >> > 3d82 398c5740 5740 toc: d0046360 c000 7cac > >> > ip:d00458d0 jumps to d0046340 r2: d0050c00 > >> > 3d82 398c5740 5740 toc: d0046360 c000 7cac > >> > ip:d0045838 jumps to d0046340 r2: d0050c00 > >> > 3d82 398c5740 5740 toc: d0046360 c000 7cac > >> > ip:d00456dc jumps to d0046340 r2: d0050c00 > >> > 3d82 398c5740 5740 toc: d0046360 c000 7cac > >> ... > > > > So I take it that the above showed that the code worked for some? > > In my trials it blows up on the first load_module() call, and for my > config that was usbcore: But does it blow up on the first instance? If it does not, then those instances are working. > > ip:d0305298 jumps to d0306ad8 r2: d0329bb8 3d82fffe > 398ccf20 fffecf20 toc: d0316af8 5f666f72 5f726573 > ptr 5f666f725f726573, addr c05d0e40, GET_ADDR c0007cac > <3>addr does not match 5f666f725f726573 > <0>[ cut here ] > <3>Badness at > /home/geoff/projects/cell/ps3-linux-dev/kernel/trace/ftrace.c:441 > NIP: c00b3038 LR: c00b46bc CTR: > REGS: c0001d143780 TRAP: 0700 Not tainted > (2.6.29-rc3-02162-gec97e82-dirty) > MSR: 80020032 CR: 4442 XER: > TASK = c6f64640[373] 'modprobe' THREAD: c0001d14 CPU: 1 > <6>GPR00: 0001 c0001d143a00 c0625988 ffea > <6>GPR04: d0305298 c00628c0 0002 > <6>GPR08: c0c6ce80 0002 c6f64640 > <6>GPR12: 4444 c0655500 d031e9d0 c0001d143c78 > <6>GPR16: d018cbe0 0549 d018c208 d031e998 > <6>GPR20: d031de00 d031e980 0001e48bc1f0 0001 > <6>GPR24: c05d0e40 c0001d47b2b8 c0576270 d0305298 > <6>GPR28: d0305298 c0001d47b2e0 c05c6960 c0001d143a00 > NIP [c00b3038] .ftrace_bug+0x98/0x210 > LR [c00b46bc] .ftrace_convert_nops+0x23c/0x344 > Call Trace: > [c0001d143a00] [c00628c0] .vprintk+0x394/0x42c (unreliable) > [c0001d143aa0] [c00b46bc] .ftrace_convert_nops+0x23c/0x344 > [c0001d143b70] [c00b47fc] .ftrace_init_module+0x38/0x50 > [c0001d143bf0] [c00a1c70] .load_module+0x12e0/0x189c > [c0001d143d80] [c00a23fc] .SyS_init_module+0x90/0x248 > [c0001d143e30] [c00074dc] syscall_exit+0x0/0x40 > Instruction dump: > 41fe0010 e93e8010 3801 9009002c e93e8000 e97e8008 e87e80b0 48000130 > 80090030 7c34 5400d97e 7820 <0b00> 2fa0 41fe0010 e93e8010 > <6>ftrace failed to modify [] 0xd0305298 > actual: 48:00:18:41 > <6>usbcore: registered new interface driver usbfs > > > This ptr of 5f666f725f726573 seems bogus. I think this code is not > working correctly: Well, obviously the code is not working correctly, if it is blowing up ;-) > > offset = (unsigned)((unsigned short)jmp[0]) << 16 | > (unsigned)((unsigned short)jmp[1]); > > tramp = mod->arch.toc + offset + 32; > > if (probe_kernel_read(jmp, (void *)tramp, 8)) { > printk(KERN_ERR "Failed to read %lx\n", tramp); > return -EFAULT; > } > > ptr = ((unsigned long)jmp[0] << 32) + jmp[1]; > > addr and GET_ADDR(addr) seem OK, System.map shows both to be _mcount. > If I force it to continue with this: > > if (ptr != GET_ADDR(addr)) { > printk(KERN_ERR "addr does not match %lx\n", ptr); > //return -EINVAL; Please do not do that! You risk writing random stuff over random memory. If the ptr does not equal what we want, then we can not guarantee that we are writing what we want to where we want. > } > > It loads a lot of modules, but eventually it fails with the > following, which seems to mean a bad tramp value. > > ip:d05f08cc jumps to d05f1920 r2: d06031c8 3d82 > 398ce758 e758 toc: d0601940<3>Failed to read d0601940 > <0>[ cut here ] > <3>Badness at > /home/geoff/projects/cell/ps3-linux-dev/kernel/trace/ftrace.c:436 > NIP: c00b3000 LR: c00b46dc CTR: 0001 > REGS: c0001d5b7780 TRAP: 0700 Not tainted > (2.6.29-rc3-02162-gec97e82-dirty) > MSR: 80020032 CR: 2442 XER: 2000 > TASK = c6fea7c0[712] 'modprobe' THREAD: c0001d5b4000 CPU: 0 > <6>GPR00: 0001 c0001d5b7a00 c0625998 fff2 > <6>GPR04: d05
Re: Dynamic-ftrace not working in PlayStation3
Hi, I did a bit of work on this today. Steven Rostedt wrote: >> > ip:d0045aec jumps to d0046340 r2: d0050c00 >> > 3d82 398c5740 5740 toc: d0046360 c000 7cac >> > ip:d00458d0 jumps to d0046340 r2: d0050c00 >> > 3d82 398c5740 5740 toc: d0046360 c000 7cac >> > ip:d0045838 jumps to d0046340 r2: d0050c00 >> > 3d82 398c5740 5740 toc: d0046360 c000 7cac >> > ip:d00456dc jumps to d0046340 r2: d0050c00 >> > 3d82 398c5740 5740 toc: d0046360 c000 7cac >> ... > > So I take it that the above showed that the code worked for some? In my trials it blows up on the first load_module() call, and for my config that was usbcore: ip:d0305298 jumps to d0306ad8 r2: d0329bb8 3d82fffe 398ccf20 fffecf20 toc: d0316af8 5f666f72 5f726573 ptr 5f666f725f726573, addr c05d0e40, GET_ADDR c0007cac <3>addr does not match 5f666f725f726573 <0>[ cut here ] <3>Badness at /home/geoff/projects/cell/ps3-linux-dev/kernel/trace/ftrace.c:441 NIP: c00b3038 LR: c00b46bc CTR: REGS: c0001d143780 TRAP: 0700 Not tainted (2.6.29-rc3-02162-gec97e82-dirty) MSR: 80020032 CR: 4442 XER: TASK = c6f64640[373] 'modprobe' THREAD: c0001d14 CPU: 1 <6>GPR00: 0001 c0001d143a00 c0625988 ffea <6>GPR04: d0305298 c00628c0 0002 <6>GPR08: c0c6ce80 0002 c6f64640 <6>GPR12: 4444 c0655500 d031e9d0 c0001d143c78 <6>GPR16: d018cbe0 0549 d018c208 d031e998 <6>GPR20: d031de00 d031e980 0001e48bc1f0 0001 <6>GPR24: c05d0e40 c0001d47b2b8 c0576270 d0305298 <6>GPR28: d0305298 c0001d47b2e0 c05c6960 c0001d143a00 NIP [c00b3038] .ftrace_bug+0x98/0x210 LR [c00b46bc] .ftrace_convert_nops+0x23c/0x344 Call Trace: [c0001d143a00] [c00628c0] .vprintk+0x394/0x42c (unreliable) [c0001d143aa0] [c00b46bc] .ftrace_convert_nops+0x23c/0x344 [c0001d143b70] [c00b47fc] .ftrace_init_module+0x38/0x50 [c0001d143bf0] [c00a1c70] .load_module+0x12e0/0x189c [c0001d143d80] [c00a23fc] .SyS_init_module+0x90/0x248 [c0001d143e30] [c00074dc] syscall_exit+0x0/0x40 Instruction dump: 41fe0010 e93e8010 3801 9009002c e93e8000 e97e8008 e87e80b0 48000130 80090030 7c34 5400d97e 7820 <0b00> 2fa0 41fe0010 e93e8010 <6>ftrace failed to modify [] 0xd0305298 actual: 48:00:18:41 <6>usbcore: registered new interface driver usbfs This ptr of 5f666f725f726573 seems bogus. I think this code is not working correctly: offset = (unsigned)((unsigned short)jmp[0]) << 16 | (unsigned)((unsigned short)jmp[1]); tramp = mod->arch.toc + offset + 32; if (probe_kernel_read(jmp, (void *)tramp, 8)) { printk(KERN_ERR "Failed to read %lx\n", tramp); return -EFAULT; } ptr = ((unsigned long)jmp[0] << 32) + jmp[1]; addr and GET_ADDR(addr) seem OK, System.map shows both to be _mcount. If I force it to continue with this: if (ptr != GET_ADDR(addr)) { printk(KERN_ERR "addr does not match %lx\n", ptr); //return -EINVAL; } It loads a lot of modules, but eventually it fails with the following, which seems to mean a bad tramp value. ip:d05f08cc jumps to d05f1920 r2: d06031c8 3d82 398ce758 e758 toc: d0601940<3>Failed to read d0601940 <0>[ cut here ] <3>Badness at /home/geoff/projects/cell/ps3-linux-dev/kernel/trace/ftrace.c:436 NIP: c00b3000 LR: c00b46dc CTR: 0001 REGS: c0001d5b7780 TRAP: 0700 Not tainted (2.6.29-rc3-02162-gec97e82-dirty) MSR: 80020032 CR: 2442 XER: 2000 TASK = c6fea7c0[712] 'modprobe' THREAD: c0001d5b4000 CPU: 0 <6>GPR00: 0001 c0001d5b7a00 c0625998 fff2 <6>GPR04: d05f08cc c0062054 0002 <6>GPR08: c0c6ce80 0001 c6fea7c0 <6>GPR12: 2444 c0655300 d05f9fd0 c0001d5b7c78 <6>GPR16: d0565c58 0215 d05653e5 d05f9f98 <6>GPR20: d05f9860 d05f9f80 0002cfe82a50 0001 <6>GPR24: c05d0e50 c0001d21e6f0 c0576270 d05f08cc <6>GPR28: d05f08cc c0001d21e718 c05c6960 c0001d5b7a00 NIP [c00b3000] .ftrace_bug+0x60/0x210 LR [c00b46dc] .ftrace_convert_nops+0x25c/0x364 Call Trace: [c0
Re: Dynamic-ftrace not working in PlayStation3
On Thu, 2009-01-29 at 22:38 -0500, Steven Rostedt wrote: > On Thu, 29 Jan 2009, Geoff Levand wrote: > > > > > > > //* > > > ip:d0045aec jumps to d0046340 r2: d0050c00 > > > 3d82 398c5740 5740 toc: d0046360 c000 7cac > > > ip:d00458d0 jumps to d0046340 r2: d0050c00 > > > 3d82 398c5740 5740 toc: d0046360 c000 7cac > > > ip:d0045838 jumps to d0046340 r2: d0050c00 > > > 3d82 398c5740 5740 toc: d0046360 c000 7cac > > > ip:d00456dc jumps to d0046340 r2: d0050c00 > > > 3d82 398c5740 5740 toc: d0046360 c000 7cac > > ... > > So I take it that the above showed that the code worked for some? > > > > ps3_system_bus_match:362: dev=11.0(lpm_01), drv=11.0(ps3-lpm): match > > > ps3_system_bus_match:362: dev=11.0(lpm_01), drv=11.0(ps3-lpm): match > > > ps3-lpm lpm_01: <- ps3_lpm_probe:1245: > > > ip:d03fe280 jumps to d03ffad8 r2: d0422c70 > > Could you find out what that function is? Perhaps do a: > > printk("ip:%pF\n", ip); > > As long as you have kallsyms on, that should point to the function that's > the problem. > > > > 3d82fffe 398cce68 fffece68 toc: d040faf8 6c656400 5f5f6b73 > > > addr does not match > > > ptr: 6c6564005f5f6b73 > > ptr is the same info as the last line above. 6c656400 5f5f6b73 = "led\0__ks" :) > > --- a/arch/powerpc/kernel/ftrace.c > > +++ b/arch/powerpc/kernel/ftrace.c > > @@ -23,7 +23,7 @@ > > #if 0 > > Heh, I would have just done: > > - #if 0 > + #if 1 > > ;-) Should do s/DEBUGP/pr_debug/g cheers -- Michael Ellerman OzLabs, IBM Australia Development Lab wwweb: http://michael.ellerman.id.au phone: +61 2 6212 1183 (tie line 70 21183) We do not inherit the earth from our ancestors, we borrow it from our children. - S.M.A.R.T Person signature.asc Description: This is a digitally signed message part ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Re: Dynamic-ftrace not working in PlayStation3
On Thu, 29 Jan 2009, Geoff Levand wrote: > > > > //* > > ip:d0045aec jumps to d0046340 r2: d0050c00 > > 3d82 398c5740 5740 toc: d0046360 c000 7cac > > ip:d00458d0 jumps to d0046340 r2: d0050c00 > > 3d82 398c5740 5740 toc: d0046360 c000 7cac > > ip:d0045838 jumps to d0046340 r2: d0050c00 > > 3d82 398c5740 5740 toc: d0046360 c000 7cac > > ip:d00456dc jumps to d0046340 r2: d0050c00 > > 3d82 398c5740 5740 toc: d0046360 c000 7cac > ... So I take it that the above showed that the code worked for some? > > ps3_system_bus_match:362: dev=11.0(lpm_01), drv=11.0(ps3-lpm): match > > ps3_system_bus_match:362: dev=11.0(lpm_01), drv=11.0(ps3-lpm): match > > ps3-lpm lpm_01: <- ps3_lpm_probe:1245: > > ip:d03fe280 jumps to d03ffad8 r2: d0422c70 Could you find out what that function is? Perhaps do a: printk("ip:%pF\n", ip); As long as you have kallsyms on, that should point to the function that's the problem. > > 3d82fffe 398cce68 fffece68 toc: d040faf8 6c656400 5f5f6b73 > > addr does not match > > ptr: 6c6564005f5f6b73 ptr is the same info as the last line above. > > addr: c04ff128 > > GET_ADDR(addr): c0007cac > > I don't know so much about ftrace, but to get a better idea of the > problem, could you try with the patch below? > > Is ps3-lpm built as a loadable module, and if so, is it the first one > that got loaded? > > Also, try to either build with CONFIG_PS3_LPM=n, or delete ps3-lpm.ko > so it does not get loaded to see what happens. > > -Geoff > > --- a/arch/powerpc/kernel/ftrace.c > +++ b/arch/powerpc/kernel/ftrace.c > @@ -23,7 +23,7 @@ > #if 0 Heh, I would have just done: - #if 0 + #if 1 ;-) -- Steve > #define DEBUGP printk > #else > -#define DEBUGP(fmt , ...)do { } while (0) > +#define DEBUGP printk > #endif > > static unsigned int ftrace_nop = PPC_NOP_INSTR; > @@ -213,6 +213,8 @@ __ftrace_make_nop(struct module *mod, > > ptr = ((unsigned long)jmp[0] << 32) + jmp[1]; > > + printk("ptr %lx, addr %lx, GET_ADDR %lx\n", ptr, addr, GET_ADDR(addr)); > + > /* This should match what was called */ > if (ptr != GET_ADDR(addr)) { > printk(KERN_ERR "addr does not match %lx\n", ptr); > > > > ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Re: Dynamic-ftrace not working in PlayStation3
Hi, Remis Lima Baima wrote: > Hi to all. I was tracing a bug in the snd_usb_audio driver > (PlayStation 3, Kernel 2.6.29-rc2, > git://git.kernel.org/pub/scm/linux/kernel/git/geoff/ps3-linux.git) and > for that I tried to use the "dynamic ftrace" to get less debug output. > But it did not work at all. Ftrace (without the "dynamic ftrace" > support) works normally. So I traced the "dynamic ftrace" bug and I > think that I found the bug location but I got stucked. Could anyone > help, please? Below is the link with the debug output: > > PS1: The problem seems to happen in the file > "./git/linux-2.6/arch/powerpc/kernel/ftrace.c" around the line 217: > "if (ptr != GET_ADDR(addr)) {". It returns true. > PS2: I enabled the "DEBUGP" macro to get the debug output below and > changed the line 218 to "printk(KERN_ERR "addr does not match \nptr: > %lx \naddr: %lx \nGET_ADDR(addr): %lx \n", ptr, addr, > GET_ADDR(addr));". > > //* > ip:d0045aec jumps to d0046340 r2: d0050c00 > 3d82 398c5740 5740 toc: d0046360 c000 7cac > ip:d00458d0 jumps to d0046340 r2: d0050c00 > 3d82 398c5740 5740 toc: d0046360 c000 7cac > ip:d0045838 jumps to d0046340 r2: d0050c00 > 3d82 398c5740 5740 toc: d0046360 c000 7cac > ip:d00456dc jumps to d0046340 r2: d0050c00 > 3d82 398c5740 5740 toc: d0046360 c000 7cac ... > ps3_system_bus_match:362: dev=11.0(lpm_01), drv=11.0(ps3-lpm): match > ps3_system_bus_match:362: dev=11.0(lpm_01), drv=11.0(ps3-lpm): match > ps3-lpm lpm_01: <- ps3_lpm_probe:1245: > ip:d03fe280 jumps to d03ffad8 r2: d0422c70 > 3d82fffe 398cce68 fffece68 toc: d040faf8 6c656400 5f5f6b73 > addr does not match > ptr: 6c6564005f5f6b73 > addr: c04ff128 > GET_ADDR(addr): c0007cac I don't know so much about ftrace, but to get a better idea of the problem, could you try with the patch below? Is ps3-lpm built as a loadable module, and if so, is it the first one that got loaded? Also, try to either build with CONFIG_PS3_LPM=n, or delete ps3-lpm.ko so it does not get loaded to see what happens. -Geoff --- a/arch/powerpc/kernel/ftrace.c +++ b/arch/powerpc/kernel/ftrace.c @@ -23,7 +23,7 @@ #if 0 #define DEBUGP printk #else -#define DEBUGP(fmt , ...) do { } while (0) +#define DEBUGP printk #endif static unsigned int ftrace_nop = PPC_NOP_INSTR; @@ -213,6 +213,8 @@ __ftrace_make_nop(struct module *mod, ptr = ((unsigned long)jmp[0] << 32) + jmp[1]; + printk("ptr %lx, addr %lx, GET_ADDR %lx\n", ptr, addr, GET_ADDR(addr)); + /* This should match what was called */ if (ptr != GET_ADDR(addr)) { printk(KERN_ERR "addr does not match %lx\n", ptr); ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Dynamic-ftrace not working in PlayStation3
Hi to all. I was tracing a bug in the snd_usb_audio driver (PlayStation 3, Kernel 2.6.29-rc2, git://git.kernel.org/pub/scm/linux/kernel/git/geoff/ps3-linux.git) and for that I tried to use the "dynamic ftrace" to get less debug output. But it did not work at all. Ftrace (without the "dynamic ftrace" support) works normally. So I traced the "dynamic ftrace" bug and I think that I found the bug location but I got stucked. Could anyone help, please? Below is the link with the debug output: PS1: The problem seems to happen in the file "./git/linux-2.6/arch/powerpc/kernel/ftrace.c" around the line 217: "if (ptr != GET_ADDR(addr)) {". It returns true. PS2: I enabled the "DEBUGP" macro to get the debug output below and changed the line 218 to "printk(KERN_ERR "addr does not match \nptr: %lx \naddr: %lx \nGET_ADDR(addr): %lx \n", ptr, addr, GET_ADDR(addr));". //* ip:d0045aec jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d00458d0 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d0045838 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d00456dc jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d0045624 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d00453b4 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d004527c jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d0045204 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d004519c jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d0044844 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d0044770 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d004465c jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d00c jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d0044390 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d004427c jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d00441f8 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d004418c jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d0044118 jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d004402c jumps to d0046340 r2: d0050c00 3d82 398c5740 5740 toc: d0046360 c000 7cac ip:d008cce0 jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008cb90 jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008ca10 jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008c924 jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008c808 jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008c57c jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008c4ac jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008c328 jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008c260 jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008c0d4 jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008d8d0 c000 7cac ip:d008c02c jumps to d008d8b0 r2: d0097cc0 3d82 398c5bf0 5bf0 toc: d008