gdb vmlinux scripting
so Im debugging a kernel, running inside virtme (I cant recommend it enough) using 2 terminals, each in build-dir 1st runs function dkrunk () { echo vm$KRUN_SHOW $KRUN_STDS $KDBG_OPTS $* $QM_OPTS --smp 3 -s -S virtme-run $KRUN_SHOW $KRUN_STDS $KDBG_OPTS $* $QM_OPTS --smp 3 -s -S # -qmp tcp:localhost:,server,nowait } 2nd runs: alias tui='gdb --tui -q -x ../../../gdb-2 vmlinux' that script is # seems I need a hardware breakpoint to get started, # is that the case for anyone else ? hbreak dynamic_debug_init c b strcmp if cs == ct c If you do this, youll find all the places where strcmp is called on identical args. Most of them I dont want to see, but what kind of syntax lets me look up the frame(s) to select only the contexts of interest ? it would be useful enough to test just the caller file, that would get me a long way ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: make vmlinux .bss PROGBITS
On Tue, 06 Oct 2020 13:56:11 -0500, William Tambe said: > We have a loader that loads vmlinux.bin (created from vmlinux using > objcopy -O binary), however if section .bss is not PROGBITS, > vmlinux.bin does not include that space, which the loader will not > reserve; by allocating that space in vmlinux.bin the loader also > reserves that space. Sounds like a buggy loader if it can't reserve a .bss segment. I mean, how hard is that to get right? Or are you saying that your linker is buggy, and won't output an entry sizing the.bss unless it's got bits set, at which point your loader never sees a .bss entry and things go pear shaped? > To solve the problem we are looking to make section .bss PROBGITS > using objcopy --set-section-flags .bss=alloc,load,contents , but it is > not trivial how to modify the Linux Makefile to achieve that. So now you have a buggy linker or loader and a custom kernel hack that you'll have to maintain and rework every time you upgrade the kernel. Probably more productive to figure out why you're hitting this issue when pretty much nobody else is... pgpeJdaxP1mr5.pgp Description: PGP signature ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: make vmlinux .bss PROGBITS
On Tue, Oct 6, 2020 at 1:11 PM Valdis Klētnieks wrote: > > On Tue, 06 Oct 2020 08:17:44 -0500, William Tambe said: > > How can I modify Makefile such that when vmlinux is created, following > > command is run on vmlinux to make section .bss PROGBITS: > > objcopy --set-section-flags .bss=alloc,load,contents > > Remember - vmlinux isn't going to be loaded by the userspace loader, > but rather by a bootstrap loader. So those flags probably don't actually > do what you think they do. We have a loader that loads vmlinux.bin (created from vmlinux using objcopy -O binary), however if section .bss is not PROGBITS, vmlinux.bin does not include that space, which the loader will not reserve; by allocating that space in vmlinux.bin the loader also reserves that space. To solve the problem we are looking to make section .bss PROBGITS using objcopy --set-section-flags .bss=alloc,load,contents , but it is not trivial how to modify the Linux Makefile to achieve that. > > For that matter, what *do* you think they do, and what problem are you > trying to solve with them? > ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: make vmlinux .bss PROGBITS
On Tue, 06 Oct 2020 08:17:44 -0500, William Tambe said: > How can I modify Makefile such that when vmlinux is created, following > command is run on vmlinux to make section .bss PROGBITS: > objcopy --set-section-flags .bss=alloc,load,contents Remember - vmlinux isn't going to be loaded by the userspace loader, but rather by a bootstrap loader. So those flags probably don't actually do what you think they do. For that matter, what *do* you think they do, and what problem are you trying to solve with them? pgpk8TabVH5kt.pgp Description: PGP signature ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
make vmlinux .bss PROGBITS
How can I modify Makefile such that when vmlinux is created, following command is run on vmlinux to make section .bss PROGBITS: objcopy --set-section-flags .bss=alloc,load,contents ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Virtualbox + kgdb: Can't see vmlinux symbols
Disabling CONFIG_RANDOMIZE_MEMORY did not help. Disabled CONFIG_RANDOMIZE_BASE as well and the symbols came up. The weird thing is that CONFIG_RANDOMIZE_BASE is enabled on Xenial (4.4) kernel as well, but somehow having it enabled on Zesty (4.10) kernel throws off kgdb. Thanks, Kamran. On Fri, Jun 16, 2017 at 3:02 PM, Kamran Khanwrote: > The debug symbols are on for all of them, and gdb *does* load the > symbols. I have a suspicion that KASLR that was introduced around 4.7 > might be the culprit here. CONFIG_RANDOMIZE_MEMORY is an option that's > not present in the Xenial config. > > Anyone here has attached kgdb to 4.8/4.10? > > Thanks, > Kamran. > > On Fri, Jun 16, 2017 at 1:41 PM, wrote: >> On Fri, 16 Jun 2017 13:24:58 -0700, Kamran Khan said: >>> This is happening only with kernel 4.8+. >>> >>> When I compile kernel 4.4 with a stock Ubuntu Xenial .config, symbols >>> appear just fine. >>> >>> When I compile kernel 4.8/4.10 with Yakkety/Zesty stock configs the >>> symbols disappear. >> >> What happens when you try to compile 4.8/r.10 with a Xenial .config? >> >> I'm willing to bet the problem is that the Xenial config turns on symbols >> and Yakkety/Zesty disable them by default. >> >> See these for further info: >> >> CONFIG_DEBUG_INFO=y >> CONFIG_DEBUG_INFO_REDUCED=y >> CONFIG_DEBUG_INFO_SPLIT=y >> >> It appears that you'll need DEBUG_INFO_REDUCED=n for kgdb to work properly. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Virtualbox + kgdb: Can't see vmlinux symbols
The debug symbols are on for all of them, and gdb *does* load the symbols. I have a suspicion that KASLR that was introduced around 4.7 might be the culprit here. CONFIG_RANDOMIZE_MEMORY is an option that's not present in the Xenial config. Anyone here has attached kgdb to 4.8/4.10? Thanks, Kamran. On Fri, Jun 16, 2017 at 1:41 PM,wrote: > On Fri, 16 Jun 2017 13:24:58 -0700, Kamran Khan said: >> This is happening only with kernel 4.8+. >> >> When I compile kernel 4.4 with a stock Ubuntu Xenial .config, symbols >> appear just fine. >> >> When I compile kernel 4.8/4.10 with Yakkety/Zesty stock configs the >> symbols disappear. > > What happens when you try to compile 4.8/r.10 with a Xenial .config? > > I'm willing to bet the problem is that the Xenial config turns on symbols > and Yakkety/Zesty disable them by default. > > See these for further info: > > CONFIG_DEBUG_INFO=y > CONFIG_DEBUG_INFO_REDUCED=y > CONFIG_DEBUG_INFO_SPLIT=y > > It appears that you'll need DEBUG_INFO_REDUCED=n for kgdb to work properly. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Virtualbox + kgdb: Can't see vmlinux symbols
On Fri, 16 Jun 2017 13:24:58 -0700, Kamran Khan said: > This is happening only with kernel 4.8+. > > When I compile kernel 4.4 with a stock Ubuntu Xenial .config, symbols > appear just fine. > > When I compile kernel 4.8/4.10 with Yakkety/Zesty stock configs the > symbols disappear. What happens when you try to compile 4.8/r.10 with a Xenial .config? I'm willing to bet the problem is that the Xenial config turns on symbols and Yakkety/Zesty disable them by default. See these for further info: CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_REDUCED=y CONFIG_DEBUG_INFO_SPLIT=y It appears that you'll need DEBUG_INFO_REDUCED=n for kgdb to work properly. pgp4jwrnxYEv6.pgp Description: PGP signature ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Virtualbox + kgdb: Can't see vmlinux symbols
This is happening only with kernel 4.8+. When I compile kernel 4.4 with a stock Ubuntu Xenial .config, symbols appear just fine. When I compile kernel 4.8/4.10 with Yakkety/Zesty stock configs the symbols disappear. On Wed, Jun 14, 2017 at 4:58 PM, Kamran Khan <krk...@inspirated.com> wrote: > I have an Ubuntu 16.04 VM running in Virtualbox. > > I'm building kernel from git using make deb-pkg. > > After I install all the deb packages (including debug symbols) on the > VM and restart it, I can see in uname -r output that the appropriate > kernel was booted from. > > I'm passing the following parameters to the kernel: > >> kgdboc=ttyS0,115200 kgdbwait > > Once the kernel boots, I am able to connect to it via socat + gdb from host. > > I'm using the full uncompressed vmlinux file to start the gdb which > contains all the debugging symbols. > > However, I can't see any of the symbols in backtrace etc. > >> $ du -sh vmlinux >> 425M vmlinux >> $ gdb vmlinux >> GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1 >> Copyright (C) 2016 Free Software Foundation, Inc. >> License GPLv3+: GNU GPL version 3 or later [gnu.org] >> [...] >> For help, type "help". >> Type "apropos word" to search for commands related to "word"... >> Reading symbols from vmlinux...done. >> (gdb) target remote /dev/pts/2 >> Remote debugging using /dev/pts/2 >> 0xb113c5b4 in ?? () >> (gdb) bt >> #0 0xb113c5b4 in ?? () >> #1 0xb035421c7dd0 in ?? () >> #2 0xb113c60c in ?? () >> #3 0xb035421c7e00 in ?? () >> #4 0xb153035a in ?? () >> #5 0x0002 in irq_stack_union () >> #6 0x013a7408 in ?? () >> #7 0x8fb8bd0d3b00 in ?? () >> #8 0xb035421c7e18 in ?? () >> #9 0xb15307df in ?? () >> #10 0x8fb8b8779900 in ?? () >> #11 0xb035421c7e38 in ?? () >> #12 0xb12ab412 in ?? () >> #13 0x8fb8bd0d3b00 in ?? () > > > Any ideas what am I doing wrong? > > Thanks, > Kamran. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmlinux
Hi All ! In my learning of linux there is the stupide questions i would like to ask you. 1- Why the name of the kernel is vmlinux ? What is the origine of this name ? Perhaps you know why ? Tell me please or give the link which can explan it. Please have a look here: http://www.linfo.org/vmlinuz.html 2 - When linux start the init process launch the default shell process and give the prompt. The others process as linux commands or users process are the forks of this first shell process ? Can you explan simply this point or give me the link which can help me please in the starting of linux. Have a look here too: http://unix.stackexchange.com/questions/5518/what-is-the-difference-between-the-following-kernel-makefile-terms-vmlinux-vml More reference material : https://en.wikipedia.org/wiki/Vmlinux Thanks - Aruna ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
vmlinux
Hi All ! In my learning of linux there is the stupide questions i would like to ask you. 1- Why the name of the kernel is vmlinux ? What is the origine of this name ? Perhaps you know why ? Tell me please or give the link which can explan it. 2 - When linux start the init process launch the default shell process and give the prompt. The others process as linux commands or users process are the forks of this first shell process ? Can you explan simply this point or give me the link which can help me please in the starting of linux. Do something as everybody is good but know why, is better to do it. Help please. Gnogbo ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmlinux
Hi, 1.I think that vmlinux refers to vmlinzx correctly .In linux booting,PC first check RAM information,after that,pc start loading OS'information.But,there is a problem:how to load it?In unix,firstly we must mount filesystem to execute program.Before loading os'information,we dont have any filesystem.In order to solve it,We make a boot image,which includes any configure information to initial/start a operation system.The image as if seem to vmlinxx.Additional we prepare to save space(RAM?),the image is compressed as one type of file,which refer to nz.By the way,after compiling kernel,we can find the vmlinzx in /boot in kernel source.It should be a binary file i rember. 2.In my view,the init is a root process.Aftet loading os susccsful,init start taking over the systm.For example,a user to login in,init invoke responding login process;a web request ,which can invoke (stocket'process?).but child process will fork parent process firstly,include all information :stack,heap,text,var.These information will change into its value according to child process once it is to be executed . Recently i read[linux programming interface].i think it should help you. Best regrads. Sent from Netease Mail On 2015-08-15 15:33 , Gnoleba GNOGBO Wrote: Hi All ! In my learning of linux there is the stupide questions i would like to ask you. 1- Why the name of the kernel is vmlinux ? What is the origine of this name ? Perhaps you know why ? Tell me please or give the link which can explan it. 2 - When linux start the init process launch the default shell process and give the prompt. The others process as linux commands or users process are the forks of this first shell process ? Can you explan simply this point or give me the link which can help me please in the starting of linux. Do something as everybody is good but know why, is better to do it. Help please. Gnogbo ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Need for .rel.text in vmlinux!
Hello, I was just verifying the sections in vmlinux file, I am just thinking why there is a .rel.text or more generally relocation sections in a vmlinux file. If it is relocatable , then does it not require a linker of some sort to fix the relocation symbols? Thanks in advance! Best regards, -sujeen ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Link an out-of-tree module into vmlinux
Hello, Does the build system support building built-in components of vmlinux out-of-tree? That is, I know that these two are possible: (1) build out-of-tree code as a loadable kernel module (.ko) (use the M=/path/to/out-of-tree/dir) (2) build an in-tree module as a statically built-in component of vmlinux instead of as a LKM (use the CONFIG_module=y, or otherwise add the object files to obj-y instead of obj-m) The question is can you build an an _out-of-tree_ module as a statically built-in component of vmlinux? It seems you can't do it without modifying the root Makefile. That is, the reason (2) works is because the directory that will have built-in.o in it is in-tree and explicitly referenced by the root Makefile. I would hope that the M=dir parameter would effectively add the dir to the list, but it seems that it does not, spoiling the whole party: the built-in.o is correctly created, but it is not picked up by the vmlinux link-line. The only workaround I have working is to add an ugly out-of-tree path into the list of directories in the root Makefile. I could not find this use-case in the neither in the Kbuild documentation nor online. Thank you in advance. -alexei ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Link an out-of-tree module into vmlinux
Hi Alexei, On Fri, Sep 28, 2012 at 9:01 AM, Alexei Colin ale...@alexeicolin.com wrote: Hello, Does the build system support building built-in components of vmlinux out-of-tree? That is, I know that these two are possible: (1) build out-of-tree code as a loadable kernel module (.ko) (use the M=/path/to/out-of-tree/dir) (2) build an in-tree module as a statically built-in component of vmlinux instead of as a LKM (use the CONFIG_module=y, or otherwise add the object files to obj-y instead of obj-m) The question is can you build an an _out-of-tree_ module as a statically built-in component of vmlinux? It seems you can't do it without modifying the root Makefile. That is, the reason (2) works is because the directory that will have built-in.o in it is in-tree and explicitly referenced by the root Makefile. I would hope that the M=dir parameter would effectively add the dir to the list, but it seems that it does not, spoiling the whole party: the built-in.o is correctly created, but it is not picked up by the vmlinux link-line. The only workaround I have working is to add an ugly out-of-tree path into the list of directories in the root Makefile. I could not find this use-case in the neither in the Kbuild documentation nor online. Thank you in advance. It depends on what you mean by out-of-tree You can build the entire kernel out-of-tree, and by that I mean have the object files go into a different tree than the source files. You can't build the static portions separate from the rest of the kernel, i.e. all of the objects are in the same tree, either the source tree, or some other tree. You use O= to specify the tree that the object files will go in. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Link an out-of-tree module into vmlinux
Thank you for your reply. I am pursuing a slightly unusual goal: On 09/28/2012 02:01 PM, Dave Hylands wrote: On Fri, Sep 28, 2012 at 9:01 AM, Alexei Colin ale...@alexeicolin.com wrote: Does the build system support building built-in components of vmlinux out-of-tree? That is, I know that these two are possible: (1) build out-of-tree code as a loadable kernel module (.ko) (use the M=/path/to/out-of-tree/dir) (2) build an in-tree module as a statically built-in component of vmlinux instead of as a LKM (use the CONFIG_module=y, or otherwise add the object files to obj-y instead of obj-m) The question is can you build an an _out-of-tree_ module as a statically built-in component of vmlinux? You can build the entire kernel out-of-tree, and by that I mean have the object files go into a different tree than the source files. Yes, the O= usage is clear, and I do use it. It depends on what you mean by out-of-tree Sorry, by out-of-tree, I meant to refer to source code. For example, in (1) above, the module's *code* can be out-of-tree: it can live outside of the kernel source directory ($KDIR). The modules object files would go into same place as its source. You can't build the static portions separate from the rest of the kernel, i.e. all of the objects are in the same tree, either the source tree, or some other tree. In the case of out-of-tree modules, though, the objects do not have to go to the same tree where the rest of the objects go. They end up in the location specified by 'M='. That's were my inspiration comes from. In fact 'M=' and 'obj-y' bring me half way there: I get a built-in.o in the 'M=' location. All I need the next step to be: link 'vmlinux' as usual, except include the built-in.o from the 'M='. To clarify: can I statically link some code into the kernel, but keep this code (and its object files) in a directory that's not a descendant of KDIR? I can already accomplish this by editting root makefile and adding value of an MBUILTIN variable to vmlinux-dirs list, e.g. via core-y, then, do 'make M=my-out-of-tree-dir make MBUILTIN=my-out-of-tree-dir vmlinux'. But I hoped there's some already supported MBUILTIN-like var that does this in one step and without hacking the root makefile. I realize that this is strange and has implications on prerequisite checks -- introduces a strange dependency of vmlinux on something outside of KDIR, which doesn't happen for modules, of course, because vmlinux doesn't depend on them. But, this seems solvable by persisting what went into the vmlinux link and invalidating if that list changes. Thanks again. -alexei ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: how to get create/compile time of vmlinux
Building time has already been statically compiled into the kernel image file, you can use ' strings vmlinux | grep Linux version ' to get it back. 2012/4/20 卜弋天 bu...@live.cn Hi All: i have a vmlinux file, how can i know when it is created/compiled? thanks. Best Regards ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
how to get create/compile time of vmlinux
Hi All:i have a vmlinux file, how can i know when it is created/compiled? thanks. Best Regards ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: how to get create/compile time of vmlinux
2012/4/20 卜弋天 bu...@live.cn: Hi All: i have a vmlinux file, how can i know when it is created/compiled? thanks. If it's the original image (not a copy) you can always do a ls -l or stats on it... ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: how to get create/compile time of vmlinux
2012/4/20 卜弋天 bu...@live.cn: Hi All: i have a vmlinux file, how can i know when it is created/compiled? thanks. If you manage to boot the image, /proc/version should tell you when the image was compiled. Vlad ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: how to get create/compile time of vmlinux
On Fri, Apr 20, 2012 at 08:09:27PM +0800, 卜弋天 wrote: Hi All:i have a vmlinux file, how can i know when it is created/compiled? thanks. Best Regards The file utility should give you the version, the compile time, and some other information about vmlinuz and probably also vmlinux files. HTH, Jonathan Neuschäfer ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: x86: Executing a raw vmlinux image (embedded environment)
Thanks syed, Ok, I did a little more digging... On 22/11/11 16:34, sk.syed2 wrote: /vmlinux2,629,659 bytes /vmlinux.o 2,889,050 bytes /arch/i386/boot/bzImage 1,104,864 bytes /arch/x86/boot/bzImage 1,104,864 bytes /arch/x86/boot/vmlinux.bin 1,092,060 bytes /arch/x86/boot/compressed/vmlinux 1,099,538 bytes /arch/x86/boot/compressed/vmlinux.bin 2,094,132 bytes /arch/x86/boot/compressed/vmlinux.bin.gz1,074,711 bytes I understand that /arch/x86/boot/compressed/vmlinux.bin.gz is a compressed version of /arch/x86/boot/compressed/vmlinux.bin, and /arch/i386/boot/bzImage and /arch/x86/boot/bzImage are the same file and that it is the 16-bit boot code + /arch/x86/boot/compressed/vmlinux.bin.gz This is correct. but I don't understand the rest... I traced it all out. I sent a separate message to the ML documenting the bzImage build chain - It's rather fascinating My guess is that /vmlinux.o is the ELF image generated by the compiler + linker stage and /vmlinux may be /vmlinux.o objdump'd into a raw binary and perhaps /arch/x86/boot/vmlinux.bin is a further stripped version of/vmlinux, but I'm at a loss with /arch/x86/boot/compressed/vmlinux vmlinux is ELF image with ELF header. So actual point of kernel entry would be at an offset, somewhere after the ELF header. vmlinux.bin is what you would get after doing #objcopy -O binary vmlinux vmlinux.bin. vmlinux.bin has only obj code and nothing else. To be more precise: arch/x86/boot/compressed/vmlinux.bin is the result of #objcopy -R .comment -S vmlinux so it is still an ELF image arch/x86/boot/vmlinux.bin is the result of: #objcopy -O binary -R .note -R .comment -S arch/x86/boot/compressed/vmlinux arch/x86/boot/compressed/vmlinux has the decompression stub + compressed version of arch/x86/boot/compressed/vmlinux.bin In any event, it looks like either /arch/x86/boot/compressed/vmlinux.bin or /vmlinux is what I need to copy into RAM @ 0x10 (1MiB) which is where my non-relocatable kernel is compiled to. copy vmlinux.bin. I don't think either is what I want - arch/x86/boot/vmlinux.bin still contains a compressed kernel. And the contents of the compressed section is an ELF image which requires more memcpys and memsets What I really want, I think, is: #objcopy -O binary -R .comment -S vmlinux Which is not generated during the build process - How to setup the memory map (keeping in mind I have 2GB of contiguous memory with no BIOS/ACPI etc to worry about clobbering - Any other tricks I need to be aware of.. Check if x86 kernel expects some parameters(like machineid, bootargs location etc) in some registers. Check if x86 has low level debug support(like DEBUG_LL). Also you might want to check how initial page tables are being setup in kernel. I need to have a good look at what goes into arch/x86/boot/setup.bin - That will really tell me what I need to do Thanks, Graeme ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
x86: Executing a raw vmlinux image (embedded environment)
Hi All, Firstly, a little introduction an background - I am currently the maintainer for the x86 port of Das U-Boot (or more simply U-Boot). While the x86 port is not as well known as the more major ports like ARM and PPC, it is starting to gain more attention and more developer input which has beed nice to see after my solo efforts over the last few years. One of my biggest annoyances with the x86 U-Boot port is that, while it is an embedded boot loader, it still tries to boot Linux as if it was a conventional PC - i.e. it has 'real mode' and 'BIOS' implementation and loads a bzImage. But this approach is completely unnecessary and only adds to the boot time (load bzImage from storage into RAM, decompress, then run). So I want to shortcut bzImage and migrate towards treating the loading of the Linux kernel from U-Boot like any other embedded environment: - Decompress vmlinux directly from storage into the appropriate memory location - Setup required data structures - Jump into vmlinux U-Boot has it's own compressed kernel image container (uImage) and support code which will allow me to decompress the vmlinux directly from storage into RAM. In my current situation, I have ~1.7MB of onboard (i.e. cached) flash memory where I plan to store the compressed kernel and an MMC where I plan to store the file system. So after building a very stripped down kernel (no TCP/IP for example) I get: /vmlinux2,629,659 bytes /vmlinux.o 2,889,050 bytes /arch/i386/boot/bzImage 1,104,864 bytes /arch/x86/boot/bzImage 1,104,864 bytes /arch/x86/boot/vmlinux.bin 1,092,060 bytes /arch/x86/boot/compressed/vmlinux 1,099,538 bytes /arch/x86/boot/compressed/vmlinux.bin 2,094,132 bytes /arch/x86/boot/compressed/vmlinux.bin.gz1,074,711 bytes I understand that /arch/x86/boot/compressed/vmlinux.bin.gz is a compressed version of /arch/x86/boot/compressed/vmlinux.bin, and /arch/i386/boot/bzImage and /arch/x86/boot/bzImage are the same file and that it is the 16-bit boot code + /arch/x86/boot/compressed/vmlinux.bin.gz but I don't understand the rest... My guess is that /vmlinux.o is the ELF image generated by the compiler + linker stage and /vmlinux may be /vmlinux.o objdump'd into a raw binary and perhaps /arch/x86/boot/vmlinux.bin is a further stripped version of/vmlinux, but I'm at a loss with /arch/x86/boot/compressed/vmlinux In any event, it looks like either /arch/x86/boot/compressed/vmlinux.bin or /vmlinux is what I need to copy into RAM @ 0x10 (1MiB) which is where my non-relocatable kernel is compiled to. I also have looked at the documentation for the x86 32-boot protocol found in linux/Documentation/x86/boot.txt So what I'm needing is: - Confirmation of exactly which vmlinux to use - Confirmation that I do load it @ 0x10 - How to setup the memory map (keeping in mind I have 2GB of contiguous memory with no BIOS/ACPI etc to worry about clobbering - Any other tricks I need to be aware of.. Any help will be greatly appreciated Thanks, Graeme ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: x86: Executing a raw vmlinux image (embedded environment)
/vmlinux 2,629,659 bytes /vmlinux.o 2,889,050 bytes /arch/i386/boot/bzImage 1,104,864 bytes /arch/x86/boot/bzImage 1,104,864 bytes /arch/x86/boot/vmlinux.bin 1,092,060 bytes /arch/x86/boot/compressed/vmlinux 1,099,538 bytes /arch/x86/boot/compressed/vmlinux.bin 2,094,132 bytes /arch/x86/boot/compressed/vmlinux.bin.gz 1,074,711 bytes I understand that /arch/x86/boot/compressed/vmlinux.bin.gz is a compressed version of /arch/x86/boot/compressed/vmlinux.bin, and /arch/i386/boot/bzImage and /arch/x86/boot/bzImage are the same file and that it is the 16-bit boot code + /arch/x86/boot/compressed/vmlinux.bin.gz This is correct. but I don't understand the rest... My guess is that /vmlinux.o is the ELF image generated by the compiler + linker stage and /vmlinux may be /vmlinux.o objdump'd into a raw binary and perhaps /arch/x86/boot/vmlinux.bin is a further stripped version of/vmlinux, but I'm at a loss with /arch/x86/boot/compressed/vmlinux vmlinux is ELF image with ELF header. So actual point of kernel entry would be at an offset, somewhere after the ELF header. vmlinux.bin is what you would get after doing #objcopy -O binary vmlinux vmlinux.bin. vmlinux.bin has only obj code and nothing else. In any event, it looks like either /arch/x86/boot/compressed/vmlinux.bin or /vmlinux is what I need to copy into RAM @ 0x10 (1MiB) which is where my non-relocatable kernel is compiled to. copy vmlinux.bin. - How to setup the memory map (keeping in mind I have 2GB of contiguous memory with no BIOS/ACPI etc to worry about clobbering - Any other tricks I need to be aware of.. Check if x86 kernel expects some parameters(like machineid, bootargs location etc) in some registers. Check if x86 has low level debug support(like DEBUG_LL). Also you might want to check how initial page tables are being setup in kernel. -syed ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
On Mon, May 16, 2011 at 9:14 AM, Peter Teoh htmldevelo...@gmail.com wrote: I loved this reply...can I annotate it with references to the linux kernel sources? On Fri, May 13, 2011 at 9:42 AM, Dave Hylands dhyla...@gmail.com wrote: Hi Vikram, ...snip... So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped Take luca's email and start at the bottom working towards the top. linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file. A binary is then extracted from this to create arch/x86/boot/compressed/vmlinux.bin yes: See ./arch/x86/boot/Makefile This binary is then compressed to produce arch/x86/boot/compressed/vmlinux.bin.gz See ./arch/x86/boot/compressed/Makefile This gzipped binary is then converted into an object file (which just contains the gzipped data) but now we're back to having an ELF file ./arch/x86/boot/compressed/mkpiggy.c is compiled into a commandline binary - mkpiggy which will generate the piggy.o. called arch/x86/boot/compressed/piggy.o The linker then compiles a decompressor (misc.o) and piggy.o together Yes, the routine is called decompress_kernel, residing inside ./arch/x86/boot/compressed/misc.c. And this routine is called from ./arch/x86/boot/compressed/head_32.S (or head_64.S) and at runtime, the gzipped data is decompressed and immediately jumped into (perhaps after some relocation if needed): /* * Do the decompression, and jump to the new kernel.. */ leal z_extract_offset_negative(%ebx), %ebp /* push arguments for decompress_kernel: */ pushl %ebp /* output address */ pushl $z_input_len /* input_len */ leal input_data(%ebx), %eax pushl %eax /* input_data */ leal boot_heap(%ebx), %eax pushl %eax /* heap area */ pushl %esi /* real mode pointer */ call decompress_kernel addl $20, %esp to produce arch/x86/boot/compressed/vmlinux (an ELF file). objcopy is used again to convert this ELF into a binary: arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin Finally, the binary is compressed to produce bzImage. Inside arch/x86/boot/Makefile: Creating the vmlinux.bin from vmlinux via objcopy (note that this operation will throw all relocation information): $(obj)/vmlinux.bin: $(obj)/compressed/vmlinux FORCE $(call if_changed,objcopy) And then packing together linearly to form the bzImage (output from make): make -f scripts/Makefile.build obj=arch/x86/boot arch/x86/boot/bzImage make -f scripts/Makefile.build obj=arch/x86/boot/compressed arch/x86/boot/compressed/vmlinux arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT arch/x86/boot/bzImage So what you get is a compressed binary which contains a decompressor and another compressed binary, this inner compressed binary being the kernel. GRUB loads bzImage into memory and decompresses it and then executes the resulting binary. To be more precise, grub will load bzImage and jump into the startup_32 function located in arch/x86/boot/compressed/head_32.S at the following fixed address (from source code): /* * head.S contains the 32-bit startup code. * * NOTE!!! Startup happens at absolute address 0x1000, which is also where * the page directory will exist. The startup code will be overwritten by * the page directory. [According to comments etc elsewhere on a compressed * kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC] * * Page 0 is deliberately kept safe, since System Management Mode code in * laptops may need to access the BIOS data stored there. This is also * useful for future device drivers that either access the BIOS via VM86 * mode. */ More info: http://books.google.com/books?id=e8BbHxVhzFACpg=PA1224lpg=PA1224dq=grub+head_32.Ssource=blots=0MSdKwBoM6sig=2RyEpprl25zueiqi332TQHLIj0Ehl=enei=y5vQTY7eBNDNrQeI3bTCCgsa=Xoi=book_resultct=resultresnum=3ved=0CCkQ6AEwAg#v=onepageq=grub%20head_32.Sf=false This binary starts with a decompressor which then decompresses the kernel, and executes the resulting binary. This binary may relocate itself (probably depends on the architecture) to a different spot in memory, and then runs. The kernel is now running. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ K -- Regards, Peter Teoh That was a great explanation. Thanks a lot. I think this will be very much useful for people who want to know how things
Re: How vmlinux is recognized?
On Fri, May 13, 2011 at 7:12 AM, Dave Hylands dhyla...@gmail.com wrote: Hi Vikram, ...snip... So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped Take luca's email and start at the bottom working towards the top. linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file. A binary is then extracted from this to create arch/x86/boot/compressed/vmlinux.bin This binary is then compressed to produce arch/x86/boot/compressed/vmlinux.bin.gz This gzipped binary is then converted into an object file (which just contains the gzipped data) but now we're back to having an ELF file called arch/x86/boot/compressed/piggy.o The linker then compiles a decompressor (misc.o) and piggy.o together to produce arch/x86/boot/compressed/vmlinux (an ELF file). objcopy is used again to convert this ELF into a binary: arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin Finally, the binary is compressed to produce bzImage. So what you get is a compressed binary which contains a decompressor and another compressed binary, this inner compressed binary being the kernel. GRUB loads bzImage into memory and decompresses it and then executes the resulting binary. This binary starts with a decompressor which then decompresses the kernel, and executes the resulting binary. This binary may relocate itself (probably depends on the architecture) to a different spot in memory, and then runs. The kernel is now running. Thanks for the detailed explanation. Clarified. :) - Thanks Vikram ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
I loved this reply...can I annotate it with references to the linux kernel sources? On Fri, May 13, 2011 at 9:42 AM, Dave Hylands dhyla...@gmail.com wrote: Hi Vikram, ...snip... So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped Take luca's email and start at the bottom working towards the top. linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file. A binary is then extracted from this to create arch/x86/boot/compressed/vmlinux.bin yes: See ./arch/x86/boot/Makefile This binary is then compressed to produce arch/x86/boot/compressed/vmlinux.bin.gz See ./arch/x86/boot/compressed/Makefile This gzipped binary is then converted into an object file (which just contains the gzipped data) but now we're back to having an ELF file ./arch/x86/boot/compressed/mkpiggy.c is compiled into a commandline binary - mkpiggy which will generate the piggy.o. called arch/x86/boot/compressed/piggy.o The linker then compiles a decompressor (misc.o) and piggy.o together Yes, the routine is called decompress_kernel, residing inside ./arch/x86/boot/compressed/misc.c. And this routine is called from ./arch/x86/boot/compressed/head_32.S (or head_64.S) and at runtime, the gzipped data is decompressed and immediately jumped into (perhaps after some relocation if needed): /* * Do the decompression, and jump to the new kernel.. */ lealz_extract_offset_negative(%ebx), %ebp /* push arguments for decompress_kernel: */ pushl %ebp/* output address */ pushl $z_input_len/* input_len */ lealinput_data(%ebx), %eax pushl %eax/* input_data */ lealboot_heap(%ebx), %eax pushl %eax/* heap area */ pushl %esi/* real mode pointer */ calldecompress_kernel addl$20, %esp to produce arch/x86/boot/compressed/vmlinux (an ELF file). objcopy is used again to convert this ELF into a binary: arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin Finally, the binary is compressed to produce bzImage. Inside arch/x86/boot/Makefile: Creating the vmlinux.bin from vmlinux via objcopy (note that this operation will throw all relocation information): $(obj)/vmlinux.bin: $(obj)/compressed/vmlinux FORCE $(call if_changed,objcopy) And then packing together linearly to form the bzImage (output from make): make -f scripts/Makefile.build obj=arch/x86/boot arch/x86/boot/bzImage make -f scripts/Makefile.build obj=arch/x86/boot/compressed arch/x86/boot/compressed/vmlinux arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT arch/x86/boot/bzImage So what you get is a compressed binary which contains a decompressor and another compressed binary, this inner compressed binary being the kernel. GRUB loads bzImage into memory and decompresses it and then executes the resulting binary. To be more precise, grub will load bzImage and jump into the startup_32 function located in arch/x86/boot/compressed/head_32.S at the following fixed address (from source code): /* * head.S contains the 32-bit startup code. * * NOTE!!! Startup happens at absolute address 0x1000, which is also where * the page directory will exist. The startup code will be overwritten by * the page directory. [According to comments etc elsewhere on a compressed * kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC] * * Page 0 is deliberately kept safe, since System Management Mode code in * laptops may need to access the BIOS data stored there. This is also * useful for future device drivers that either access the BIOS via VM86 * mode. */ More info: http://books.google.com/books?id=e8BbHxVhzFACpg=PA1224lpg=PA1224dq=grub+head_32.Ssource=blots=0MSdKwBoM6sig=2RyEpprl25zueiqi332TQHLIj0Ehl=enei=y5vQTY7eBNDNrQeI3bTCCgsa=Xoi=book_resultct=resultresnum=3ved=0CCkQ6AEwAg#v=onepageq=grub%20head_32.Sf=false This binary starts with a decompressor which then decompresses the kernel, and executes the resulting binary. This binary may relocate itself (probably depends on the architecture) to a different spot in memory, and then runs. The kernel is now running. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ K -- Regards, Peter Teoh ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
On Thu, May 12, 2011 at 10:02 AM, Sudheer Divakaran inbox1.sudh...@gmail.com wrote: Hi Vikram, On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan vikram...@gmail.com wrote: On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: On Thu, May 12, 2011 at 03:11, Vikram Narayanan vikram...@gmail.com wrote: Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done? OK I try my best to understand your question :) i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both. I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? 5) The vmlinux is in ELF format. Correct? 6) If the OS boots and if u try to run an ELF file, the loader knows how to load that in the RAM. (I mean it knows how to interpret the ELF format) 7) Coming back to the vmlinux image, Who takes care of the loading activity.? 8) Who recognizes that the image is ELF format and do the necessary things accordingly.? Hope I have my question clear now. If understand your question correctly, you believe that the uncompressed kernel is in elf format. correct?. it is in binary format, so elf interpretation is not required, #5 is wrong. You can see this by building the kernel using 'make V=1' and note the following line in the output, arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT arch/x86/boot/bzImage means bzImage is made out of two binary files extracted from the elf images. One more info I want to clarify is, vmlinux.bin mentioned in the above snippet contains the compressed binary image and some other routines. Just go through the 'make V=1' output, you can see that the build process is actually compressing binary file extracted from the vmlinux elf image, which is again combined with some object files, creates another elf and again extracts the binary and finally combined with the setup.bin to create the final bzImage. So, elf interpretation doesn't happen on the uncompressed code. -- Thanks Sudheer ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
On 12/05/2011 8.21, Sudheer Divakaran wrote: On Thu, May 12, 2011 at 10:02 AM, Sudheer Divakaran inbox1.sudh...@gmail.com wrote: Hi Vikram, On Thu, May 12, 2011 at 9:02 AM, Vikram Narayananvikram...@gmail.com wrote: On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: On Thu, May 12, 2011 at 03:11, Vikram Narayananvikram...@gmail.com wrote: Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done? OK I try my best to understand your question :) i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both. I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? 5) The vmlinux is in ELF format. Correct? 6) If the OS boots and if u try to run an ELF file, the loader knows how to load that in the RAM. (I mean it knows how to interpret the ELF format) 7) Coming back to the vmlinux image, Who takes care of the loading activity.? 8) Who recognizes that the image is ELF format and do the necessary things accordingly.? Hope I have my question clear now. If understand your question correctly, you believe that the uncompressed kernel is in elf format. correct?. it is in binary format, so elf interpretation is not required, #5 is wrong. You can see this by building the kernel using 'make V=1' and note the following line in the output, arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT arch/x86/boot/bzImage means bzImage is made out of two binary files extracted from the elf images. One more info I want to clarify is, vmlinux.bin mentioned in the above snippet contains the compressed binary image and some other routines. Just go through the 'make V=1' output, you can see that the build process is actually compressing binary file extracted from the vmlinux elf image, which is again combined with some object files, creates another elf and again extracts the binary and finally combined with the setup.bin to create the final bzImage. So, elf interpretation doesn't happen on the uncompressed code. Let's put some order here. The image that almost all bootloaders use is arch/x86/boot/bzImage which is made of a setup binary file (executable) joined with some compressed code (the real kernel) which is uncompressed in memory by the setup binary. The big suggestion I can give is to check the hidden files which end with .cmd. There is one of these for every object created by the compilation process. For example there is a file called .bzImage.cmd which tell you how bzImage was made: arch/x86/boot/tools/build -b arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT arch/x86/boot/bzImage NOTE: I refer to a quite old kernel here, it's likely that the compilation process has changed somehow. You can proceed now in reverse order to find how bzImage was made (if I understand correctly that is the one you are interested in). Here is how is made on my kernel tree: bzImage: arch/x86/boot/tools/build -b arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT arch/x86/boot/bzImage arch/x86/boot/vmlinux.bin: objcopy -O binary -R .note -R .comment -S arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin arch/x86/boot/compressed/vmlinux: ld -m elf_i386 -T arch/x86/boot/compressed/vmlinux_32.lds arch/x86/boot/compressed/head_32.o arch/x86/boot/compressed/misc.o arch/x86/boot/compressed/piggy.o -o arch/x86/boot/compressed/vmlinux arch/x86/boot/compressed/piggy.o: ld -m elf_i386 -r --format binary --oformat elf32-i386 -T arch/x86/boot/compressed/vmlinux.scr arch/x86/boot/compressed/vmlinux.bin.gz -o arch/x86/boot/compressed/piggy.o arch/x86/boot/compressed/vmlinux.bin.gz: gzip -f -9 arch/x86/boot/compressed/vmlinux.bin arch/x86/boot/compressed/vmlinux.bin.gz arch/x86/boot/compressed/vmlinux.bin: objcopy -O binary -R .note -R .comment -S vmlinux arch/x86/boot/compressed/vmlinux.bin vmlinux: ld -m elf_i386 --build-id -o vmlinux -T arch/x86/kernel/vmlinux.lds arch/x86/kernel/head_32.o arch/x86/kernel/init_task.o init/built-in.o --start-group usr/built-in.o arch/x86/kernel/built-in.o arch/x86/mm/built-in.o arch/x86/mach-default/built-in.o arch/x86/crypto/built-in.o arch/x86/vdso/built-in.o kernel/built-in.o mm/built-in.o fs/built-in.o ipc/built-in.o security/built-in.o crypto/built-in.o block/built-in.o lib/lib.a arch/x86/lib/lib.a lib/built-in.o arch/x86/lib/built-in.o drivers/built-in.o sound/built
Re: How vmlinux is recognized?
Hi... On Thu, May 12, 2011 at 10:32, Vikram Narayanan vikram...@gmail.com wrote: I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? nope... it's a binarybut not ELF...and that's not even named vmlinux or similar to vmlinux... 5) The vmlinux is in ELF format. Correct? yes but see above... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
Hi All On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan vikram...@gmail.comwrote: On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: On Thu, May 12, 2011 at 03:11, Vikram Narayanan vikram...@gmail.com wrote: Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done? OK I try my best to understand your question :) i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both. I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? 5) The vmlinux is in ELF format. Correct? I Guess Yes. 6) If the OS boots and if u try to run an ELF file, the loader knows how to load that in the RAM. (I mean it knows how to interpret the ELF format) See the multi-boot specification. GRUB is a multi-boot compliant boot loader 7) Coming back to the vmlinux image, Who takes care of the loading activity.? GRUB 8) Who recognizes that the image is ELF format and do the necessary things accordingly.? GRUB Hope I have my question clear now. - Thanks, Vikram ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies -- Anuj Aggarwal .''`. : :Ⓐ : # apt-get install hakuna-matata `. `'` `- ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
This way, GRUB doesn't need to know how to decode ELF files and the job is left to the kernel code. GRUB has a elf decoder, but it should have multiboot header. http://osdev.berlios.de/grub.html#multiboot Hope this answers your doubt. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
On Thu, May 12, 2011 at 1:47 PM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: Hi... On Thu, May 12, 2011 at 10:32, Vikram Narayanan vikram...@gmail.com wrote: I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? nope... it's a binarybut not ELF...and that's not even named vmlinux or similar to vmlinux... 5) The vmlinux is in ELF format. Correct? yes but see above... Thanks for all your explanations. So the uncompressed one is _NOT_ an ELF file, but a raw binary. So it doesn't need any interpretation. Hope this is right. So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped Thanks, Vikram ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
Hi Vikram, ...snip... So when compiling the kernel, what is the purpose of the other files(mentioned below) linux-2.6/vmlinux - ELF executable, not stripped linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the one which is inside the bzImage) linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped Take luca's email and start at the bottom working towards the top. linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file. A binary is then extracted from this to create arch/x86/boot/compressed/vmlinux.bin This binary is then compressed to produce arch/x86/boot/compressed/vmlinux.bin.gz This gzipped binary is then converted into an object file (which just contains the gzipped data) but now we're back to having an ELF file called arch/x86/boot/compressed/piggy.o The linker then compiles a decompressor (misc.o) and piggy.o together to produce arch/x86/boot/compressed/vmlinux (an ELF file). objcopy is used again to convert this ELF into a binary: arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin Finally, the binary is compressed to produce bzImage. So what you get is a compressed binary which contains a decompressor and another compressed binary, this inner compressed binary being the kernel. GRUB loads bzImage into memory and decompresses it and then executes the resulting binary. This binary starts with a decompressor which then decompresses the kernel, and executes the resulting binary. This binary may relocate itself (probably depends on the architecture) to a different spot in memory, and then runs. The kernel is now running. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
On Thu, May 12, 2011 at 12:47 AM, Dave Hylands dhyla...@gmail.com wrote: Hi Vikram, On Wed, May 11, 2011 at 11:06 AM, Vikram Narayanan vikram...@gmail.com wrote: Hi, Sorry if this question is stupid. How the vmlinux (an ELF executable) is recognized by the processor? What are the files that are responsible for this? Well the short answer is that it isn't. The ELF file is normally just one stage of the process. You still need to extract a binary from the ELF, and the binary contains the raw executable code that the processor uses. Normally the boot loader will extract a binary (perhaps from an ELF, or perhaps from a raw binary image) and this is what the processor sees. So in case of x86, say Grub will be taking care of this extraction. Right? If, so the grub code will have the mechanisms for extracting the raw binary from ELF. Am i right? - Thanks, Vikram ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
On Thu, May 12, 2011 at 1:15 AM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: On Thu, May 12, 2011 at 02:31, Vikram Narayanan vikram...@gmail.com wrote: So in case of x86, say Grub will be taking care of this extraction. Right? If, so the grub code will have the mechanisms for extracting the raw binary from ELF. Am i right? you mean, vmlinuz right? the bzImage right? not the vmlinux because that's the one GRUB handles...not vmlinux one... The vmlinux is an ELF binary. right? If so, Who does the unpacking of raw binary image from that ELF? well, in that case, see this first: $ file -k -z /boot/vmlinuz-2.6.32-31-generic This is the compressed one. The uncompression is done by the kernel and not by the grub, If I am not wrong. /boot/vmlinuz-2.6.32-31-generic: Linux kernel x86 boot executable bzImage, version 2.6.32-31-generic (buildd@rothe, RO-rootFS, root_dev 0x801, swap_dev 0x3, Normal VGA\012- x86 boot sector, code offset 0x5 i am sure you will get idea based upon the above file identification, on what vmlinuz is and how is it supposed to be treated by boot loader I am still confused :( Thanks, Vikram ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
On Thu, May 12, 2011 at 03:04, Vikram Narayanan vikram...@gmail.com wrote: The vmlinux is an ELF binary. right? If so, Who does the unpacking of raw binary image from that ELF? why do you put concern on vmlinux anyway? boot loader loads vmlinuz, not vmlinux -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
On Thu, May 12, 2011 at 03:11, Vikram Narayanan vikram...@gmail.com wrote: Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done? OK I try my best to understand your question :) i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both. After final phase of final kernel image creation, it will go into making bootable image first. in order to do that, first it will be compressed 1st. These days, gz is the choice. So, it is gzipped..and the boot loading code is appended in front of it... there, you get vmlinuz. And vmlinux? developers usually use vmlinux as symbol file... and the way it is created, back to the above phase, is by linking it according to the accompanying elf linker script. Finally, ELF that contains kernel is there. Another guess, maybe you wanna know how to extract the kernel code from ELF image? then why so? that is indeed the kernel image itself...it is just appended ELF headers, sections and so on just to represent ELF construction. But it is not behaving like standart ELF binary i.e the entry point is not main() but IIRC start_kernel or something like that. that helps you? -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
Hi Vikram, How the vmlinux (an ELF executable) is recognized by the processor? ELF is just a file format. That is, the machine instructions and data are stored in a specific format. The _processor_ simply recognizes machine instructions and this needs to be taken from the ELF file and loaded into memory (the instruction pointer is then pointed to the place the instructions were loaded). The format is simply a set of rules defined in the specification (a pretty nice introduction is available at www.skyfree.org/linux/references/*ELF* _Format.pdf http://www.skyfree.org/linux/references/ELF_Format.pdf). For example, when you ask a Linux kernel to execute an ELF file, it has code to know how to decode the information and place it into memory (see fs/binfmt_elf.c). As for the vmlinux file specifically, the Wikipedia page on vmlinux ( http://en.wikipedia.org/wiki/Vmlinux) seems like a good start. As shown above with Linux, GRUB needs to have a way to decode whatever format is passed to it (bzImage). The kernel however places the unzipping code into the bzImage itself so that it is loaded into memory by the bootloader and is then run. This code then unzips the kernel. This way, GRUB doesn't need to know how to decode ELF files and the job is left to the kernel code. You can see arch/x86/boot/Makefile and look for the bzImage target to see what files constitute the bzImage. I may be wrong about this with regard to newer kernels so I hope others correct me in this case. Another great explanation is by Alessandro Rubini at: http://www.ibiblio.org/oswg/oswg-nightly/oswg/en_US.ISO_8859-1/articles/alessandro-rubini/boot/boot/zimage.html Hope this helped! :-) -- /manohar ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
On Thu, May 12, 2011 at 2:03 AM, Manohar Vanga manohar.va...@gmail.com wrote: Hi Vikram, How the vmlinux (an ELF executable) is recognized by the processor? ELF is just a file format. That is, the machine instructions and data are stored in a specific format. The _processor_ simply recognizes machine instructions and this needs to be taken from the ELF file and loaded into memory (the instruction pointer is then pointed to the place the instructions were loaded). Hope everyone here got my question wrong. I am aware that ELF is a format and there will be specific loader for loading ELF files. Please refer to the previous reply. Thanks, Vikram ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How vmlinux is recognized?
Hi Vikram, On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan vikram...@gmail.com wrote: On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: On Thu, May 12, 2011 at 03:11, Vikram Narayanan vikram...@gmail.com wrote: Yes. I agree. But how who converts the ELF binary to raw binary so that the processor understands. Or how is it actually done? OK I try my best to understand your question :) i think I got it...you probably guessed that vmlinux created first, then vmlinuz... AFAIK, it's the other way around...or more precisely, not both. I think you got it wrong. I will try to put my question more elaborately. 1) The system is on and BIOS code runs. It gives the control to the boot loader, say GRUB. 2) Grub picks up the kernel from the specific partition. (i.e a vmlinuz image), which denotes that it is compressed. 3) There are uncompression routines in the kernel itself, If I am not wrong. So the kernel uncompresses itself. 4) Now the uncompressed thing is the vmlinux image, right? 5) The vmlinux is in ELF format. Correct? 6) If the OS boots and if u try to run an ELF file, the loader knows how to load that in the RAM. (I mean it knows how to interpret the ELF format) 7) Coming back to the vmlinux image, Who takes care of the loading activity.? 8) Who recognizes that the image is ELF format and do the necessary things accordingly.? Hope I have my question clear now. If understand your question correctly, you believe that the uncompressed kernel is in elf format. correct?. it is in binary format, so elf interpretation is not required, #5 is wrong. You can see this by building the kernel using 'make V=1' and note the following line in the output, arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin CURRENT arch/x86/boot/bzImage means bzImage is made out of two binary files extracted from the elf images. -- Thanks Sudheer ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies