Re: module init code placement offset ? - Was : Re: Changing path in kernel object & Debug of Kernel module with gdb
On Tue, Sep 1, 2009 at 6:37 AM, Microbit_Ubuntu wrote: > Hi Mulyadi, > > On Tue, 2009-09-01 at 01:44 +0700, Mulyadi Santosa wrote: >> Hi Kris... >> >> On Mon, Aug 31, 2009 at 9:38 PM, >> Microbit_Ubuntu wrote: >> > Hi Mulyadi/all, >> > This ended up being a valuable tip, thanks ! >> > I'd been looking at module.c too much, and missed the do_one_initcall()... >> >> I am glad you find it useful. That was actually a quick research, >> mixing a fast stack strace analysis and knowing the fact that symbol >> resolving/relinking is done in insertion stage. >> >> So just knowing the point where the point the module is inserted but >> not yet running is the key here, and stack trace provides us the clear >> picture instead of doing static code analysis. Again, another thanks >> to dump_stack(). :) >> >> > I just have a breakpoint at Line 709 ( fn = (initcall_t)...) where the >> > actual initial >> > module code is called. >> > This ended up working great for me, but I ran across something bizarre >> > about mem placement >> > of the module init code : >> > >> > In my particular embedded target, kernel modules always are placed (the >> > 1st time) at 0xbf00 by vmalloc(). >> > >> > As an example, consider a simple (arbitrary) module header inspection (as >> > per Robert's column, tnx !) : >> > >> > .init.text 0174 >> > .text 0058 >> > .bss 061c >> > .data 04ac >> > >> > Using add-symbol-file, gdb is informed of the sections. >> > >> > The strange thing is that the .init.text section gets placed at >> > 0xBF003000, NOT eg. >> > 0xBF000174 like in example above. >> > I've been trying to figure this out, but can't make ends of it... >> > >> > Does anyone know why .init.text address is placed by vmalloc() at the >> > 0x3000 offset ? >> > >> > Note : removing and re-inserting modules normally happens in increments of >> > 0x3000 (but I guess >> > this depends on module code size...) in an example such as above. >> > IOW, if I remove the module above and then insmod again, it will get >> > placed at 0xBF003000, >> > 0xBF006000,.. etc. - .init.text.address will always be the newly >> > allocated address + the offset >> > where the next insmod would place the module init code... >> > >> > Can anyone shed light on this >> >> very wild guess, in your embedded target, the "pad" between vmalloc >> virtual address range is given such amount so that you are always >> given starting address of 0x3000 and its multiplication. >> >> I forgot the official name of this "pad", but IIRC it works as >> "shield" between vmalloc area. This area is made non accessible, so >> every access beyond allocated area for module will cause "oops" or >> something like that. >> >> Perhaps somebody knows better than me, but that's all I can recall right now. >> > > Oh, perhaps I should have noted that the consecutive module placements are > 0x3000 > when my test module doesn't use __init... look into layout_sections() and load_module() in http://lxr.linux.no/#linux+v2.6.30.5/kernel/module.c#L1649 u can see the init sections always comes after other core sections. the virtual memory address are subsequently only derived after all other sections have taken the virtual address. and remember all the sections are always allocated and aligned along PAGE_SIZE alignment (for pagefault reason). > Else - of course - the consecutive placements are 0x6000, but I guess that > was implied :-) > > It's still a strange ranger scenario, the init code being placed where - > normally - the next insmod > would go. Perhaps the init code is padded as well for some sort of > protection... dunno. > > I'll keep the padding issue in the back of my head as I learn further with > Linux. I haven't enabled > debug VM in kernel config yet - that's for later.. :-) > > But thanks for the advice and review ! > > -- > Best regards, > Kris > > > > -- > To unsubscribe from this list: send an email with > "unsubscribe kernelnewbies" to ecar...@nl.linux.org > Please read the FAQ at http://kernelnewbies.org/FAQ > > -- Regards, Peter Teoh -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: module init code placement offset ? - Was : Re: Changing path in kernel object & Debug of Kernel module with gdb
On Tue, Sep 1, 2009 at 5:37 AM, Microbit_Ubuntu wrote: > Hi Mulyadi, > > Oh, perhaps I should have noted that the consecutive module placements are > 0x3000 > when my test module doesn't use __init... > Else - of course - the consecutive placements are 0x6000, but I guess that > was implied :-) > > It's still a strange ranger scenario, the init code being placed where - > normally - the next insmod > would go. Perhaps the init code is padded as well for some sort of > protection... dunno. > > I'll keep the padding issue in the back of my head as I learn further with > Linux. I haven't enabled > debug VM in kernel config yet - that's for later.. :-) > > But thanks for the advice and review ! Well, sorry if at this point I am unable to give further ideas. Hopefully it gives you enough fundamentals to go further by your own. Please keep us informed on what you've found -- regards, Mulyadi Santosa Freelance Linux trainer blog: the-hydra.blogspot.com -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: module init code placement offset ? - Was : Re: Changing path in kernel object & Debug of Kernel module with gdb
Hi Mulyadi, On Tue, 2009-09-01 at 01:44 +0700, Mulyadi Santosa wrote: > Hi Kris... > > On Mon, Aug 31, 2009 at 9:38 PM, > Microbit_Ubuntu wrote: > > Hi Mulyadi/all, > > This ended up being a valuable tip, thanks ! > > I'd been looking at module.c too much, and missed the do_one_initcall()... > > I am glad you find it useful. That was actually a quick research, > mixing a fast stack strace analysis and knowing the fact that symbol > resolving/relinking is done in insertion stage. > > So just knowing the point where the point the module is inserted but > not yet running is the key here, and stack trace provides us the clear > picture instead of doing static code analysis. Again, another thanks > to dump_stack(). :) > > > I just have a breakpoint at Line 709 ( fn = (initcall_t)...) where the > > actual initial > > module code is called. > > This ended up working great for me, but I ran across something bizarre > > about mem placement > > of the module init code : > > > > In my particular embedded target, kernel modules always are placed (the 1st > > time) at 0xbf00 by vmalloc(). > > > > As an example, consider a simple (arbitrary) module header inspection (as > > per Robert's column, tnx !) : > > > > .init.text 0174 > > .text 0058 > > .bss061c > > .data 04ac > > > > Using add-symbol-file, gdb is informed of the sections. > > > > The strange thing is that the .init.text section gets placed at 0xBF003000, > > NOT eg. > > 0xBF000174 like in example above. > > I've been trying to figure this out, but can't make ends of it... > > > > Does anyone know why .init.text address is placed by vmalloc() at the > > 0x3000 offset ? > > > > Note : removing and re-inserting modules normally happens in increments of > > 0x3000 (but I guess > > this depends on module code size...) in an example such as above. > > IOW, if I remove the module above and then insmod again, it will get placed > > at 0xBF003000, > > 0xBF006000,.. etc. - .init.text.address will always be the newly allocated > > address + the offset > > where the next insmod would place the module init code... > > > > Can anyone shed light on this > > very wild guess, in your embedded target, the "pad" between vmalloc > virtual address range is given such amount so that you are always > given starting address of 0x3000 and its multiplication. > > I forgot the official name of this "pad", but IIRC it works as > "shield" between vmalloc area. This area is made non accessible, so > every access beyond allocated area for module will cause "oops" or > something like that. > > Perhaps somebody knows better than me, but that's all I can recall right now. > Oh, perhaps I should have noted that the consecutive module placements are 0x3000 when my test module doesn't use __init... Else - of course - the consecutive placements are 0x6000, but I guess that was implied :-) It's still a strange ranger scenario, the init code being placed where - normally - the next insmod would go. Perhaps the init code is padded as well for some sort of protection... dunno. I'll keep the padding issue in the back of my head as I learn further with Linux. I haven't enabled debug VM in kernel config yet - that's for later.. :-) But thanks for the advice and review ! -- Best regards, Kris -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: module init code placement offset ? - Was : Re: Changing path in kernel object & Debug of Kernel module with gdb
Hi Kris... On Mon, Aug 31, 2009 at 9:38 PM, Microbit_Ubuntu wrote: > Hi Mulyadi/all, > This ended up being a valuable tip, thanks ! > I'd been looking at module.c too much, and missed the do_one_initcall()... I am glad you find it useful. That was actually a quick research, mixing a fast stack strace analysis and knowing the fact that symbol resolving/relinking is done in insertion stage. So just knowing the point where the point the module is inserted but not yet running is the key here, and stack trace provides us the clear picture instead of doing static code analysis. Again, another thanks to dump_stack(). :) > I just have a breakpoint at Line 709 ( fn = (initcall_t)...) where the actual > initial > module code is called. > This ended up working great for me, but I ran across something bizarre about > mem placement > of the module init code : > > In my particular embedded target, kernel modules always are placed (the 1st > time) at 0xbf00 by vmalloc(). > > As an example, consider a simple (arbitrary) module header inspection (as per > Robert's column, tnx !) : > > .init.text 0174 > .text 0058 > .bss 061c > .data 04ac > > Using add-symbol-file, gdb is informed of the sections. > > The strange thing is that the .init.text section gets placed at 0xBF003000, > NOT eg. > 0xBF000174 like in example above. > I've been trying to figure this out, but can't make ends of it... > > Does anyone know why .init.text address is placed by vmalloc() at the 0x3000 > offset ? > > Note : removing and re-inserting modules normally happens in increments of > 0x3000 (but I guess > this depends on module code size...) in an example such as above. > IOW, if I remove the module above and then insmod again, it will get placed > at 0xBF003000, > 0xBF006000,.. etc. - .init.text.address will always be the newly allocated > address + the offset > where the next insmod would place the module init code... > > Can anyone shed light on this very wild guess, in your embedded target, the "pad" between vmalloc virtual address range is given such amount so that you are always given starting address of 0x3000 and its multiplication. I forgot the official name of this "pad", but IIRC it works as "shield" between vmalloc area. This area is made non accessible, so every access beyond allocated area for module will cause "oops" or something like that. Perhaps somebody knows better than me, but that's all I can recall right now. -- regards, Mulyadi Santosa Freelance Linux trainer blog: the-hydra.blogspot.com -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
module init code placement offset ? - Was : Re: Changing path in kernel object & Debug of Kernel module with gdb
Hi Mulyadi/all, On Thu, 2009-08-06 at 01:51 +0700, Mulyadi Santosa wrote: > On 8/6/09, Microbit_Ubuntu wrote: > > > > 2. As referred to in the u32 previous post, I'm a bit stumped how to > >invoke GBD on debugging kernel modules. > >Do I need a separate "pre-command" from the debugger, launching > >insmod, or some such ? > > What do you want to debug against your kernel module? decoding oops > message? tracking the code flow? > > Maybe what you need to do is setup virtual environment first e.g using > Qemu. Put your kernel module inside the guest image and run Qemu plus > enable its gdb stub. > > After that, insert your module inside the guest using normal insmod > command. The trick here is, assuming you want to stop at module init > function, how to stop exactly when entering it while its symbol is > still not resolved 'til the moment it is loaded? > > Check the following stack trace: > [278057.822340] [] hi+0x8/0x44 [mymodule] > [278057.822353] [] do_one_initcall+0x65/0x172 > > "hi" is module_init function ( I steal that from Robert Day's article, > sorry Robert...). So as you can see, put breakpoint in > do_one_initcall(). Once it is hit, it means all the symbols of the > module are resolved already... and for the rest of the work...you know > what to do :) > > Please CMIIW people This ended up being a valuable tip, thanks ! I'd been looking at module.c too much, and missed the do_one_initcall()... I just have a breakpoint at Line 709 ( fn = (initcall_t)...) where the actual initial module code is called. This ended up working great for me, but I ran across something bizarre about mem placement of the module init code : In my particular embedded target, kernel modules always are placed (the 1st time) at 0xbf00 by vmalloc(). As an example, consider a simple (arbitrary) module header inspection (as per Robert's column, tnx !) : .init.text 0174 .text 0058 .bss061c .data 04ac Using add-symbol-file, gdb is informed of the sections. The strange thing is that the .init.text section gets placed at 0xBF003000, NOT eg. 0xBF000174 like in example above. I've been trying to figure this out, but can't make ends of it... Does anyone know why .init.text address is placed by vmalloc() at the 0x3000 offset ? Note : removing and re-inserting modules normally happens in increments of 0x3000 (but I guess this depends on module code size...) in an example such as above. IOW, if I remove the module above and then insmod again, it will get placed at 0xBF003000, 0xBF006000,.. etc. - .init.text.address will always be the newly allocated address + the offset where the next insmod would place the module init code... Can anyone shed light on this -- Best regards, Kris -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: Changing path in kernel object & Debug of Kernel module with gdb
Hi Mulyadi, On Thu, 2009-08-06 at 01:51 +0700, Mulyadi Santosa wrote: > On 8/6/09, Microbit_Ubuntu wrote: > > > > 2. As referred to in the u32 previous post, I'm a bit stumped how to > >invoke GBD on debugging kernel modules. > >Do I need a separate "pre-command" from the debugger, launching > >insmod, or some such ? > > What do you want to debug against your kernel module? decoding oops > message? tracking the code flow? > > Maybe what you need to do is setup virtual environment first e.g using > Qemu. Put your kernel module inside the guest image and run Qemu plus > enable its gdb stub. > > After that, insert your module inside the guest using normal insmod > command. The trick here is, assuming you want to stop at module init > function, how to stop exactly when entering it while its symbol is > still not resolved 'til the moment it is loaded? > > Check the following stack trace: > [278057.822340] [] hi+0x8/0x44 [mymodule] > [278057.822353] [] do_one_initcall+0x65/0x172 > > "hi" is module_init function ( I steal that from Robert Day's article, > sorry Robert...). So as you can see, put breakpoint in > do_one_initcall(). Once it is hit, it means all the symbols of the > module are resolved already... and for the rest of the work...you know > what to do :) > > Please CMIIW people I'm only attempting to track code flow. I've taken some liking to Eclipse's debugger - and its content indexer (giving a great and fast lookup of all symbols and calls involved) This is more an "academic" exercise, because I can see things happening "hands-on", rather than reading - or - just insmodding and seeing the module code execute. I've played with Qemu, but I like "stepping" through my actual target executables. Using SSH/SFTP, the "Remote System Explorer" in Eclipse is a great tool too - I find. I can expand "my processes", "all processes" & their related info and see all information on my target (Linux-wise I mean). I do debug after mounting my target through NFS, so I don't have to 'copy' any code to the target. My target already 'sees' all necessary info by mounting to my host's HDD workspace folder. This works so smooth for user space executables, it'd be great to get it breathing for modules. The big problem here - of course - is to figure out how to get gdb to already 'know' the module's entry address as you describe above (or Robert in his up and coming article) - but without it running yet ! You see, the remote debug starts gdbserver through the SSH connection (I use Ethernet), and then "remote gdb/mi" takes over in starting things properly, including tracking the entry break point (which in my case of course would be the module's init function - but normally 'main' in the remote C application). I've been reading about kgdb et al, but I can't see how that helps me. Also, the gdbstub is already there anyway. The GUI does have a means to get remote gdb/mi to first insmod the module, but that happens _before_ the actual "application" is started (IOW the module's debugger-context code isn't invoked yet when insmodding). So basically, I have no means to first tell remote gdb/mi what the entry address is for the module.. I'm considering trying the "attach to application" debugger option, to see if that yields results... Hmm, perhaps I'm attempting to run when I can't walk properly yet - dunno. The exercise seems sound to me though, it would tremendously speed up the learning process of kernel internals in a practical context. Not to mention its use when I start writing actual driver code. The reason I actually started this whole learning process is to write a driver for SDIO WiFi cards. I spent a lot of time reverse engineering the SDW-820 Spectec SDIO 802.11b card (IPN2128 Procomm MAC based). I wrote my own complete driver (OS-less) for STA, Ad-Hoc + AP operation and I'm contemplating writing an open source Linux driver for it (the MAC parts will have to be object code though :-( ). I figure that, by the time I've written a sound driver for that SDIO WiFi card, I'm surely well on my way to writing driver code on Linux and mastering the kernel a bit more.. :-) PS : As you probably can tell, I'm from a JTAG ARM background :-) My main tool is normally Rowley CrossWorks for ARM... -- Best regards, Kris -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: Changing path in kernel object & Debug of Kernel module with gdb
Hi Manish, On Wed, 2009-08-05 at 23:30 +0530, Manish Katiyar wrote: > On Wed, Aug 5, 2009 at 11:18 PM, > Microbit_Ubuntu wrote: > > Hello Lin/all, > > > > > > Following up on my reply to the u32 issue and Kernel compilation (in > > Eclipse/CDT), I am wondering about 2 issues someone here might have > > figured before : > > > > 1. How do you (re) change path to set the module output when building ? > > -> The hierarchy of my project (ideally) requires that my Kernel > > Module source resides in a sub-dir called 'src'. > > Thus, my (test) module is defined as such : > > > > obj-m := ../src/my_test.o > > > > OK, all is good, no problem. But now my module output ends up > > in the /src/ sub directory. > > However, I want the executable code to land in my /Debug > > folder > > > > Can anyone give a hint how to accomplish this ? > > A "make install" target ??? > > > I can't seem to > > readily find a solution to this... > > > > So, basically, the folders are like this : > > > > My Project : > > ... > > src/ > > Debug/ > > > > My Makefile is in the Debug folder - the module source is in /src. > > But the module output needs to go into Debug/ folder ? > > > > > > 2. As referred to in the u32 previous post, I'm a bit stumped how to > > invoke GBD on debugging kernel modules. > > The commands would depend on what environment are yoiu developing your > modules. A vmware based kernel can easily talk to gdb via some > predefined port as specified in vmx file. A UML kernel can be debugged > by just starting the kernel in gdb . > > Thanks - > Manish > > > Do I need a separate "pre-command" from the debugger, launching > > insmod, or some such ? > > > > > > I look forward to any help or suggestions I can get... > > > > -- > > Best regards, > > Kris > > > > > > > > -- > > To unsubscribe from this list: send an email with > > "unsubscribe kernelnewbies" to ecar...@nl.linux.org > > Please read the FAQ at http://kernelnewbies.org/FAQ > > > > > > > Thank you for your advice. I'll have to go Googling a bit further to figure out your "make install" answer. :-) To be more precise, I can't find any docs on telling the top kernel Makefile how to output module exes in a target different from the default. I don't understand how 'make install' relates to this ? (I'm building a module cross-compiled, on a Linux Host machine (Ubuntu 8.10), but to load/debug on an ARM9 EABI target) The gist of my Makefile under Eclipse was posted in my reply to Lin : "status of u32 type in 2.6.29.4" (IOW I'm using buildroot with Kernel 2.6.29.4) > The commands would depend on what environment are yoiu developing your > modules. A vmware based kernel can easily talk to gdb via some > predefined port as specified in vmx file. A UML kernel can be debugged > by just starting the kernel in gdb . OK. I'm writing module code from Eclipse/CDT 6.0. I'm using my own Makefile to build modules from within Eclipse with arm-linux-gcc 4.3.2. I usually debug using SSH through EClipse/CDT's remote debugger gdb/mi. I hope this clarifies better what I'm trying to do. Thanks for getting back ! -- Best regards, Kris -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: Changing path in kernel object & Debug of Kernel module with gdb
On 8/6/09, Microbit_Ubuntu wrote: > > 2. As referred to in the u32 previous post, I'm a bit stumped how to >invoke GBD on debugging kernel modules. >Do I need a separate "pre-command" from the debugger, launching >insmod, or some such ? What do you want to debug against your kernel module? decoding oops message? tracking the code flow? Maybe what you need to do is setup virtual environment first e.g using Qemu. Put your kernel module inside the guest image and run Qemu plus enable its gdb stub. After that, insert your module inside the guest using normal insmod command. The trick here is, assuming you want to stop at module init function, how to stop exactly when entering it while its symbol is still not resolved 'til the moment it is loaded? Check the following stack trace: [278057.822340] [] hi+0x8/0x44 [mymodule] [278057.822353] [] do_one_initcall+0x65/0x172 "hi" is module_init function ( I steal that from Robert Day's article, sorry Robert...). So as you can see, put breakpoint in do_one_initcall(). Once it is hit, it means all the symbols of the module are resolved already... and for the rest of the work...you know what to do :) Please CMIIW people -- regards, Mulyadi Santosa Freelance Linux trainer blog: the-hydra.blogspot.com -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: Changing path in kernel object & Debug of Kernel module with gdb
On Wed, Aug 5, 2009 at 11:18 PM, Microbit_Ubuntu wrote: > Hello Lin/all, > > > Following up on my reply to the u32 issue and Kernel compilation (in > Eclipse/CDT), I am wondering about 2 issues someone here might have > figured before : > > 1. How do you (re) change path to set the module output when building ? > -> The hierarchy of my project (ideally) requires that my Kernel > Module source resides in a sub-dir called 'src'. > Thus, my (test) module is defined as such : > > obj-m := ../src/my_test.o > > OK, all is good, no problem. But now my module output ends up > in the /src/ sub directory. > However, I want the executable code to land in my /Debug > folder > > Can anyone give a hint how to accomplish this ? A "make install" target ??? > I can't seem to > readily find a solution to this... > > So, basically, the folders are like this : > > My Project : > ... > src/ > Debug/ > > My Makefile is in the Debug folder - the module source is in /src. > But the module output needs to go into Debug/ folder ? > > > 2. As referred to in the u32 previous post, I'm a bit stumped how to > invoke GBD on debugging kernel modules. The commands would depend on what environment are yoiu developing your modules. A vmware based kernel can easily talk to gdb via some predefined port as specified in vmx file. A UML kernel can be debugged by just starting the kernel in gdb . Thanks - Manish > Do I need a separate "pre-command" from the debugger, launching > insmod, or some such ? > > > I look forward to any help or suggestions I can get... > > -- > Best regards, > Kris > > > > -- > To unsubscribe from this list: send an email with > "unsubscribe kernelnewbies" to ecar...@nl.linux.org > Please read the FAQ at http://kernelnewbies.org/FAQ > > -- Thanks - Manish -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Changing path in kernel object & Debug of Kernel module with gdb
Hello Lin/all, Following up on my reply to the u32 issue and Kernel compilation (in Eclipse/CDT), I am wondering about 2 issues someone here might have figured before : 1. How do you (re) change path to set the module output when building ? -> The hierarchy of my project (ideally) requires that my Kernel Module source resides in a sub-dir called 'src'. Thus, my (test) module is defined as such : obj-m := ../src/my_test.o OK, all is good, no problem. But now my module output ends up in the /src/ sub directory. However, I want the executable code to land in my /Debug folder Can anyone give a hint how to accomplish this ? I can't seem to readily find a solution to this... So, basically, the folders are like this : My Project : ... src/ Debug/ My Makefile is in the Debug folder - the module source is in /src. But the module output needs to go into Debug/ folder ? 2. As referred to in the u32 previous post, I'm a bit stumped how to invoke GBD on debugging kernel modules. Do I need a separate "pre-command" from the debugger, launching insmod, or some such ? I look forward to any help or suggestions I can get... -- Best regards, Kris -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
