gdb vmlinux scripting

[jim . cromie]

so Im debugging a kernel,
running inside virtme  (I cant recommend it enough)

using 2 terminals, each in build-dir

1st runs
function dkrunk () {
echo vm$KRUN_SHOW $KRUN_STDS $KDBG_OPTS $* $QM_OPTS --smp 3 -s -S
virtme-run $KRUN_SHOW $KRUN_STDS $KDBG_OPTS $* $QM_OPTS --smp 3 -s -S
# -qmp tcp:localhost:,server,nowait
}

2nd runs:
alias tui='gdb --tui -q -x ../../../gdb-2 vmlinux'

that script is

# seems I need a hardware breakpoint to get started,
# is that the case for anyone else ?
hbreak dynamic_debug_init
c
b strcmp if cs == ct
c


If you do this, youll find all the places where
strcmp is called on identical args.

Most of them I dont want to see,
but what kind of syntax lets me look up the frame(s)
to select only the contexts of interest ?

it would be useful enough to test just the caller file,
that would get me a long way

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: make vmlinux .bss PROGBITS

[Valdis Klētnieks]

On Tue, 06 Oct 2020 13:56:11 -0500, William Tambe said:
> We have a loader that loads vmlinux.bin (created from vmlinux using
> objcopy -O binary), however if section .bss is not PROGBITS,
> vmlinux.bin does not include that space, which the loader will not
> reserve; by allocating that space in vmlinux.bin the loader also
> reserves that space.

Sounds like a buggy loader if it can't reserve a .bss segment. I mean,
how hard is that to get right?

Or are you saying that your linker is buggy, and won't output an entry
sizing the.bss unless it's got bits set, at which point your loader never
sees a .bss entry and things go pear shaped?

> To solve the problem we are looking to make section .bss PROBGITS
> using objcopy --set-section-flags .bss=alloc,load,contents , but it is
> not trivial how to modify the Linux Makefile to achieve that.

So now you have a buggy linker or loader and a custom kernel hack that you'll
have to maintain and rework every time you upgrade the kernel.

Probably more productive to figure out why you're hitting this issue when
pretty much nobody else is...




pgpeJdaxP1mr5.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: make vmlinux .bss PROGBITS

[William Tambe]

On Tue, Oct 6, 2020 at 1:11 PM Valdis Klētnieks  wrote:
>
> On Tue, 06 Oct 2020 08:17:44 -0500, William Tambe said:
> > How can I modify Makefile such that when vmlinux is created, following
> > command is run on vmlinux to make section .bss PROGBITS:
> > objcopy --set-section-flags .bss=alloc,load,contents
>
> Remember - vmlinux isn't going to be loaded by the userspace loader,
> but rather by a bootstrap loader.  So those flags probably don't actually
> do what you think they do.

We have a loader that loads vmlinux.bin (created from vmlinux using
objcopy -O binary), however if section .bss is not PROGBITS,
vmlinux.bin does not include that space, which the loader will not
reserve; by allocating that space in vmlinux.bin the loader also
reserves that space.
To solve the problem we are looking to make section .bss PROBGITS
using objcopy --set-section-flags .bss=alloc,load,contents , but it is
not trivial how to modify the Linux Makefile to achieve that.

>
> For that matter, what *do* you think they do, and what problem are you
> trying to solve with them?
>

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: make vmlinux .bss PROGBITS

[Valdis Klētnieks]

On Tue, 06 Oct 2020 08:17:44 -0500, William Tambe said:
> How can I modify Makefile such that when vmlinux is created, following
> command is run on vmlinux to make section .bss PROGBITS:
> objcopy --set-section-flags .bss=alloc,load,contents

Remember - vmlinux isn't going to be loaded by the userspace loader,
but rather by a bootstrap loader.  So those flags probably don't actually
do what you think they do.

For that matter, what *do* you think they do, and what problem are you
trying to solve with them?



pgpk8TabVH5kt.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

make vmlinux .bss PROGBITS

[William Tambe]

How can I modify Makefile such that when vmlinux is created, following
command is run on vmlinux to make section .bss PROGBITS:
objcopy --set-section-flags .bss=alloc,load,contents

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Virtualbox + kgdb: Can't see vmlinux symbols

[Kamran Khan]

Disabling CONFIG_RANDOMIZE_MEMORY did not help. Disabled
CONFIG_RANDOMIZE_BASE as well and the symbols came up.

The weird thing is that CONFIG_RANDOMIZE_BASE is enabled on Xenial
(4.4) kernel as well, but somehow having it enabled on Zesty (4.10)
kernel throws off kgdb.

Thanks,
Kamran.

On Fri, Jun 16, 2017 at 3:02 PM, Kamran Khan  wrote:
> The debug symbols are on for all of them, and gdb *does* load the
> symbols. I have a suspicion that KASLR that was introduced around 4.7
> might be the culprit here. CONFIG_RANDOMIZE_MEMORY is an option that's
> not present in the Xenial config.
>
> Anyone here has attached kgdb to 4.8/4.10?
>
> Thanks,
> Kamran.
>
> On Fri, Jun 16, 2017 at 1:41 PM,   wrote:
>> On Fri, 16 Jun 2017 13:24:58 -0700, Kamran Khan said:
>>> This is happening only with kernel 4.8+.
>>>
>>> When I compile kernel 4.4 with a stock Ubuntu Xenial .config, symbols
>>> appear just fine.
>>>
>>> When I compile kernel 4.8/4.10 with Yakkety/Zesty stock configs the
>>> symbols disappear.
>>
>> What happens when you try to compile 4.8/r.10 with a Xenial .config?
>>
>> I'm willing to bet the problem is that the Xenial config turns on symbols
>> and Yakkety/Zesty disable them by default.
>>
>> See these for further info:
>>
>> CONFIG_DEBUG_INFO=y
>> CONFIG_DEBUG_INFO_REDUCED=y
>> CONFIG_DEBUG_INFO_SPLIT=y
>>
>> It appears that you'll need DEBUG_INFO_REDUCED=n for kgdb to work properly.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Virtualbox + kgdb: Can't see vmlinux symbols

[Kamran Khan]

The debug symbols are on for all of them, and gdb *does* load the
symbols. I have a suspicion that KASLR that was introduced around 4.7
might be the culprit here. CONFIG_RANDOMIZE_MEMORY is an option that's
not present in the Xenial config.

Anyone here has attached kgdb to 4.8/4.10?

Thanks,
Kamran.

On Fri, Jun 16, 2017 at 1:41 PM,   wrote:
> On Fri, 16 Jun 2017 13:24:58 -0700, Kamran Khan said:
>> This is happening only with kernel 4.8+.
>>
>> When I compile kernel 4.4 with a stock Ubuntu Xenial .config, symbols
>> appear just fine.
>>
>> When I compile kernel 4.8/4.10 with Yakkety/Zesty stock configs the
>> symbols disappear.
>
> What happens when you try to compile 4.8/r.10 with a Xenial .config?
>
> I'm willing to bet the problem is that the Xenial config turns on symbols
> and Yakkety/Zesty disable them by default.
>
> See these for further info:
>
> CONFIG_DEBUG_INFO=y
> CONFIG_DEBUG_INFO_REDUCED=y
> CONFIG_DEBUG_INFO_SPLIT=y
>
> It appears that you'll need DEBUG_INFO_REDUCED=n for kgdb to work properly.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Virtualbox + kgdb: Can't see vmlinux symbols

[valdis . kletnieks]

On Fri, 16 Jun 2017 13:24:58 -0700, Kamran Khan said:
> This is happening only with kernel 4.8+.
>
> When I compile kernel 4.4 with a stock Ubuntu Xenial .config, symbols
> appear just fine.
>
> When I compile kernel 4.8/4.10 with Yakkety/Zesty stock configs the
> symbols disappear.

What happens when you try to compile 4.8/r.10 with a Xenial .config?

I'm willing to bet the problem is that the Xenial config turns on symbols
and Yakkety/Zesty disable them by default.

See these for further info:

CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_INFO_REDUCED=y
CONFIG_DEBUG_INFO_SPLIT=y

It appears that you'll need DEBUG_INFO_REDUCED=n for kgdb to work properly.


pgp4jwrnxYEv6.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Virtualbox + kgdb: Can't see vmlinux symbols

[Kamran Khan]

This is happening only with kernel 4.8+.

When I compile kernel 4.4 with a stock Ubuntu Xenial .config, symbols
appear just fine.

When I compile kernel 4.8/4.10 with Yakkety/Zesty stock configs the
symbols disappear.

On Wed, Jun 14, 2017 at 4:58 PM, Kamran Khan <krk...@inspirated.com> wrote:
> I have an Ubuntu 16.04 VM running in Virtualbox.
>
> I'm building kernel from git using make deb-pkg.
>
> After I install all the deb packages (including debug symbols) on the
> VM and restart it, I can see in uname -r output that the appropriate
> kernel was booted from.
>
> I'm passing the following parameters to the kernel:
>
>> kgdboc=ttyS0,115200 kgdbwait
>
> Once the kernel boots, I am able to connect to it via socat + gdb from host.
>
> I'm using the full uncompressed vmlinux file to start the gdb which
> contains all the debugging symbols.
>
> However, I can't see any of the symbols in backtrace etc.
>
>> $ du -sh vmlinux
>> 425M vmlinux
>> $ gdb vmlinux
>> GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
>> Copyright (C) 2016 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later [gnu.org]
>> [...]
>> For help, type "help".
>> Type "apropos word" to search for commands related to "word"...
>> Reading symbols from vmlinux...done.
>> (gdb) target remote /dev/pts/2
>> Remote debugging using /dev/pts/2
>> 0xb113c5b4 in ?? ()
>> (gdb) bt
>> #0  0xb113c5b4 in ?? ()
>> #1  0xb035421c7dd0 in ?? ()
>> #2  0xb113c60c in ?? ()
>> #3  0xb035421c7e00 in ?? ()
>> #4  0xb153035a in ?? ()
>> #5  0x0002 in irq_stack_union ()
>> #6  0x013a7408 in ?? ()
>> #7  0x8fb8bd0d3b00 in ?? ()
>> #8  0xb035421c7e18 in ?? ()
>> #9  0xb15307df in ?? ()
>> #10 0x8fb8b8779900 in ?? ()
>> #11 0xb035421c7e38 in ?? ()
>> #12 0xb12ab412 in ?? ()
>> #13 0x8fb8bd0d3b00 in ?? ()
>
>
> Any ideas what am I doing wrong?
>
> Thanks,
> Kamran.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: vmlinux

[Aruna Hewapathirane]

 Hi All !

 In my learning of linux there is the stupide questions i would like to ask
 you.

 1- Why the name of the  kernel is vmlinux ?
 What is the origine of this name ?
 Perhaps you know why ?
 Tell me please or give the link which can explan it.

Please have a look here: http://www.linfo.org/vmlinuz.html

 2 - When linux start the init process launch the default shell process and
 give the prompt. The others process as linux commands or users process are
 the forks of this first shell process ?
 Can you explan simply  this point or  give me the link which can help me
 please in the starting of linux.

Have a look here too:
http://unix.stackexchange.com/questions/5518/what-is-the-difference-between-the-following-kernel-makefile-terms-vmlinux-vml

More reference material : https://en.wikipedia.org/wiki/Vmlinux

Thanks - Aruna
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

vmlinux

[Gnoleba GNOGBO]

Hi All !

In my learning of linux there is the stupide questions i would like to ask
you.

1- Why the name of the  kernel is vmlinux ?
What is the origine of this name ?
Perhaps you know why ?
Tell me please or give the link which can explan it.
2 - When linux start the init process launch the default shell process and
give the prompt. The others process as linux commands or users process are
the forks of this first shell process ?
Can you explan simply  this point or  give me the link which can help me
please in the starting of linux.

Do something as everybody is good but know why,  is better to do it.
Help please.

Gnogbo
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: vmlinux

[YU Bo]

Hi,
1.I think that vmlinux refers to vmlinzx correctly .In linux booting,PC first 
check RAM information,after that,pc start loading OS'information.But,there is a 
problem:how to load it?In unix,firstly we must mount filesystem to execute 
program.Before loading os'information,we dont have any filesystem.In order to 
solve it,We make a boot image,which includes any configure information to 
initial/start a operation system.The image as if seem to vmlinxx.Additional we 
prepare to save space(RAM?),the image is compressed as one type of file,which 
refer to nz.By the way,after compiling kernel,we can find the vmlinzx in /boot 
in kernel source.It should be  a binary file i rember.
2.In my view,the init is a root process.Aftet  loading os susccsful,init start 
taking over the systm.For example,a user to login in,init invoke responding 
login process;a web request ,which can invoke (stocket'process?).but child 
process will fork parent process firstly,include all information 
:stack,heap,text,var.These information will change  into its value according to 
child process once it is to be executed .
Recently i read[linux programming interface].i think it should help you.
Best regrads.

Sent from Netease Mail



On 2015-08-15 15:33 , Gnoleba GNOGBO Wrote:



Hi All !

In my learning of linux there is the stupide questions i would like to ask you.

1- Why the name of the  kernel is vmlinux ?
What is the origine of this name ?
Perhaps you know why ?
Tell me please or give the link which can explan it.
2 - When linux start the init process launch the default shell process and give 
the prompt. The others process as linux commands or users process are the forks 
of this first shell process ?
Can you explan simply  this point or  give me the link which can help me please 
in the starting of linux.

Do something as everybody is good but know why,  is better to do it.
Help please.

Gnogbo
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Need for .rel.text in vmlinux!

[sebas sujeen]

Hello,
  I was just verifying the sections in vmlinux file, I am just thinking why
there is a .rel.text or more generally relocation sections in a vmlinux
file. If it is relocatable , then does it not require a linker of some sort
to fix the relocation symbols? Thanks in advance!

Best regards,
-sujeen
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Link an out-of-tree module into vmlinux

[Alexei Colin]

Hello,

Does the build system support building built-in components of vmlinux
out-of-tree?

That is, I know that these two are possible:
(1) build out-of-tree code as a loadable kernel module (.ko) (use the
M=/path/to/out-of-tree/dir)
(2) build an in-tree module as a statically built-in component of
vmlinux instead of as a LKM (use the CONFIG_module=y, or otherwise add
the object files to obj-y instead of obj-m)

The question is can you build an an _out-of-tree_ module as a statically
built-in component of vmlinux?

It seems you can't do it without modifying the root Makefile. That is,
the reason (2) works is because the directory that will have built-in.o
in it is in-tree and explicitly referenced by the root Makefile. I would
hope that the M=dir parameter would effectively add the dir to the list,
but it seems that it does not, spoiling the whole party: the built-in.o
is correctly created, but it is not picked up by the vmlinux link-line.
The only workaround I have working is to add an ugly out-of-tree path
into the list of directories in the root Makefile.

I could not find this use-case in the neither in the Kbuild
documentation nor online. Thank you in advance.

-alexei

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Link an out-of-tree module into vmlinux

[Dave Hylands]

Hi Alexei,

On Fri, Sep 28, 2012 at 9:01 AM, Alexei Colin ale...@alexeicolin.com wrote:
 Hello,

 Does the build system support building built-in components of vmlinux
 out-of-tree?

 That is, I know that these two are possible:
 (1) build out-of-tree code as a loadable kernel module (.ko) (use the
 M=/path/to/out-of-tree/dir)
 (2) build an in-tree module as a statically built-in component of
 vmlinux instead of as a LKM (use the CONFIG_module=y, or otherwise add
 the object files to obj-y instead of obj-m)

 The question is can you build an an _out-of-tree_ module as a statically
 built-in component of vmlinux?

 It seems you can't do it without modifying the root Makefile. That is,
 the reason (2) works is because the directory that will have built-in.o
 in it is in-tree and explicitly referenced by the root Makefile. I would
 hope that the M=dir parameter would effectively add the dir to the list,
 but it seems that it does not, spoiling the whole party: the built-in.o
 is correctly created, but it is not picked up by the vmlinux link-line.
 The only workaround I have working is to add an ugly out-of-tree path
 into the list of directories in the root Makefile.

 I could not find this use-case in the neither in the Kbuild
 documentation nor online. Thank you in advance.

It depends on what you mean by out-of-tree

You can build the entire kernel out-of-tree, and by that I mean have
the object files go into a different tree than the source files.

You can't build the static portions separate from the rest of the
kernel, i.e. all of the objects are in the same tree, either the
source tree, or some other tree.

You use O= to specify the tree that the object files will go in.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Link an out-of-tree module into vmlinux

[Alexei Colin]

Thank you for your reply. I am pursuing a slightly unusual goal:

On 09/28/2012 02:01 PM, Dave Hylands wrote:
 On Fri, Sep 28, 2012 at 9:01 AM, Alexei Colin ale...@alexeicolin.com
wrote:
 Does the build system support building built-in components of vmlinux
 out-of-tree?

 That is, I know that these two are possible:
 (1) build out-of-tree code as a loadable kernel module (.ko) (use the
 M=/path/to/out-of-tree/dir)
 (2) build an in-tree module as a statically built-in component of
 vmlinux instead of as a LKM (use the CONFIG_module=y, or otherwise add
 the object files to obj-y instead of obj-m)

 The question is can you build an an _out-of-tree_ module as a statically
 built-in component of vmlinux?


 You can build the entire kernel out-of-tree, and by that I mean have
 the object files go into a different tree than the source files.
Yes, the O= usage is clear, and I do use it.

 It depends on what you mean by out-of-tree
Sorry, by out-of-tree, I meant to refer to source code. For example, in
(1) above, the module's *code* can be out-of-tree: it can live outside
of the kernel source directory ($KDIR). The modules object files would
go into same place as its source.

 You can't build the static portions separate from the rest of the
 kernel, i.e. all of the objects are in the same tree, either the
 source tree, or some other tree.
In the case of out-of-tree modules, though, the objects do not have to
go to the same tree where the rest of the objects go. They end up in the
location specified by 'M='. That's were my inspiration comes from. In
fact 'M=' and 'obj-y' bring me half way there: I get a built-in.o in the
'M=' location. All I need the next step to be: link 'vmlinux' as usual,
except include the built-in.o from the 'M='.

To clarify: can I statically link some code into the kernel, but keep
this code (and its object files) in a directory that's not a descendant
of KDIR?

I can already accomplish this by editting root makefile and adding value
of an MBUILTIN variable to vmlinux-dirs list, e.g. via core-y, then, do
'make M=my-out-of-tree-dir  make MBUILTIN=my-out-of-tree-dir vmlinux'.
But I hoped there's some already supported MBUILTIN-like var that does
this in one step and without hacking the root makefile.

I realize that this is strange and has implications on prerequisite
checks -- introduces a strange dependency of vmlinux on something
outside of KDIR, which doesn't happen for modules, of course, because
vmlinux doesn't depend on them. But, this seems solvable by persisting
what went into the vmlinux link and invalidating if that list changes.

Thanks again.

-alexei


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: how to get create/compile time of vmlinux

[brooks]

Building time has already been statically compiled into the kernel image
file, you can use ' strings vmlinux | grep Linux version ' to get it back.

2012/4/20 卜弋天 bu...@live.cn

  Hi All:

i have a vmlinux file, how can i know when it is created/compiled?
thanks.


 Best Regards

 ___
 Kernelnewbies mailing list
 Kernelnewbies@kernelnewbies.org
 http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies


___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

how to get create/compile time of vmlinux

[卜弋天]

Hi All:i have a vmlinux file, how can i know when it is created/compiled?   
thanks.  Best Regards ___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: how to get create/compile time of vmlinux

[Alexandru Juncu]

2012/4/20 卜弋天 bu...@live.cn:
 Hi All:

    i have a vmlinux file, how can i know when it is created/compiled?
    thanks.

If it's the original image (not a copy) you can always do a ls -l or
stats on it...

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: how to get create/compile time of vmlinux

[Vlad Dogaru]

2012/4/20 卜弋天 bu...@live.cn:
 Hi All:

    i have a vmlinux file, how can i know when it is created/compiled?
    thanks.

If you manage to boot the image, /proc/version should tell you when
the image was compiled.

Vlad

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: how to get create/compile time of vmlinux

[Jonathan Neuschäfer]

On Fri, Apr 20, 2012 at 08:09:27PM +0800, 卜弋天 wrote:
 Hi All:i have a vmlinux file, how can i know when it is created/compiled? 
   thanks.  Best Regards

The file utility should give you the version, the compile time, and
some other information about vmlinuz and probably also vmlinux files.

HTH,
Jonathan Neuschäfer

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: x86: Executing a raw vmlinux image (embedded environment)

[Graeme Russ]

Thanks syed,

Ok, I did a little more digging...

On 22/11/11 16:34, sk.syed2 wrote:
   /vmlinux2,629,659  bytes
   /vmlinux.o  2,889,050  bytes
   /arch/i386/boot/bzImage 1,104,864  bytes
   /arch/x86/boot/bzImage  1,104,864  bytes
   /arch/x86/boot/vmlinux.bin  1,092,060  bytes
   /arch/x86/boot/compressed/vmlinux   1,099,538  bytes
   /arch/x86/boot/compressed/vmlinux.bin   2,094,132  bytes
   /arch/x86/boot/compressed/vmlinux.bin.gz1,074,711  bytes

 I understand that /arch/x86/boot/compressed/vmlinux.bin.gz is a compressed
 version of /arch/x86/boot/compressed/vmlinux.bin, and
 /arch/i386/boot/bzImage and /arch/x86/boot/bzImage are the same file and
 that it is the 16-bit boot code + /arch/x86/boot/compressed/vmlinux.bin.gz
 This is correct.
 but I don't understand the rest...

I traced it all out. I sent a separate message to the ML documenting the
bzImage build chain - It's rather fascinating

 

 My guess is that /vmlinux.o is the ELF image generated by the compiler +
 linker stage and /vmlinux may be /vmlinux.o objdump'd into a raw binary and
 perhaps /arch/x86/boot/vmlinux.bin is a further stripped version
 of/vmlinux, but I'm at a loss with /arch/x86/boot/compressed/vmlinux
 
 vmlinux is ELF image with ELF header. So actual point of kernel entry
 would be at an offset, somewhere after the ELF header.
 vmlinux.bin is what you would get after doing
 #objcopy -O binary vmlinux vmlinux.bin.
 vmlinux.bin has only obj code and nothing else.

To be more precise:

arch/x86/boot/compressed/vmlinux.bin is the result of
#objcopy -R .comment -S vmlinux
so it is still an ELF image

arch/x86/boot/vmlinux.bin is the result of:
#objcopy -O binary -R .note -R .comment -S arch/x86/boot/compressed/vmlinux

arch/x86/boot/compressed/vmlinux has the decompression stub + compressed
version of arch/x86/boot/compressed/vmlinux.bin


 In any event, it looks like either /arch/x86/boot/compressed/vmlinux.bin or
 /vmlinux is what I need to copy into RAM @ 0x10 (1MiB) which is where
 my non-relocatable kernel is compiled to.
 
 copy vmlinux.bin.

I don't think either is what I want - arch/x86/boot/vmlinux.bin still
contains a compressed kernel. And the contents of the compressed section is
an ELF image which requires more memcpys and memsets

What I really want, I think, is:

#objcopy -O binary -R .comment -S vmlinux

Which is not generated during the build process

  - How to setup the memory map (keeping in mind I have 2GB of contiguous
   memory with no BIOS/ACPI etc to worry about clobbering
  - Any other tricks I need to be aware of..
 Check if x86 kernel expects some parameters(like machineid, bootargs
 location etc) in some registers.
 Check if x86 has low level debug support(like DEBUG_LL).
 Also you might want to check how initial page tables are being setup in 
 kernel.

I need to have a good look at what goes into arch/x86/boot/setup.bin - That
will really tell me what I need to do

Thanks,

Graeme

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

x86: Executing a raw vmlinux image (embedded environment)

[Graeme Russ]

Hi All,

Firstly, a little introduction an background - I am currently the maintainer
for the x86 port of Das U-Boot (or more simply U-Boot). While the x86 port
is not as well known as the more major ports like ARM and PPC, it is
starting to gain more attention and more developer input which has beed
nice to see after my solo efforts over the last few years.

One of my biggest annoyances with the x86 U-Boot port is that, while it is
an embedded boot loader, it still tries to boot Linux as if it was a
conventional PC - i.e. it has 'real mode' and 'BIOS' implementation and
loads a bzImage. But this approach is completely unnecessary and only
adds to the boot time (load bzImage from storage into RAM, decompress,
then run).

So I want to shortcut bzImage and migrate towards treating the loading
of the Linux kernel from U-Boot like any other embedded environment:
 - Decompress vmlinux directly from storage into the appropriate memory
   location
 - Setup required data structures
 - Jump into vmlinux

U-Boot has it's own compressed kernel image container (uImage) and support
code which will allow me to decompress the vmlinux directly from storage
into RAM. In my current situation, I have ~1.7MB of onboard (i.e. cached)
flash memory where I plan to store the compressed kernel and an MMC where
I plan to store the file system.

So after building a very stripped down kernel (no TCP/IP for example) I
get:
   /vmlinux2,629,659  bytes
   /vmlinux.o  2,889,050  bytes
   /arch/i386/boot/bzImage 1,104,864  bytes
   /arch/x86/boot/bzImage  1,104,864  bytes
   /arch/x86/boot/vmlinux.bin  1,092,060  bytes
   /arch/x86/boot/compressed/vmlinux   1,099,538  bytes
   /arch/x86/boot/compressed/vmlinux.bin   2,094,132  bytes
   /arch/x86/boot/compressed/vmlinux.bin.gz1,074,711  bytes

I understand that /arch/x86/boot/compressed/vmlinux.bin.gz is a compressed
version of /arch/x86/boot/compressed/vmlinux.bin, and
/arch/i386/boot/bzImage and /arch/x86/boot/bzImage are the same file and
that it is the 16-bit boot code + /arch/x86/boot/compressed/vmlinux.bin.gz
but I don't understand the rest...

My guess is that /vmlinux.o is the ELF image generated by the compiler +
linker stage and /vmlinux may be /vmlinux.o objdump'd into a raw binary and
perhaps /arch/x86/boot/vmlinux.bin is a further stripped version
of/vmlinux, but I'm at a loss with /arch/x86/boot/compressed/vmlinux

In any event, it looks like either /arch/x86/boot/compressed/vmlinux.bin or
/vmlinux is what I need to copy into RAM @ 0x10 (1MiB) which is where
my non-relocatable kernel is compiled to.

I also have looked at the documentation for the x86 32-boot protocol found
in linux/Documentation/x86/boot.txt

So what I'm needing is:
 - Confirmation of exactly which vmlinux to use
 - Confirmation that I do load it @ 0x10
 - How to setup the memory map (keeping in mind I have 2GB of contiguous
   memory with no BIOS/ACPI etc to worry about clobbering
 - Any other tricks I need to be aware of..

Any help will be greatly appreciated

Thanks,

Graeme

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: x86: Executing a raw vmlinux image (embedded environment)

[sk.syed2]

       /vmlinux                                        2,629,659  bytes
       /vmlinux.o                                      2,889,050  bytes
       /arch/i386/boot/bzImage                         1,104,864  bytes
       /arch/x86/boot/bzImage                          1,104,864  bytes
       /arch/x86/boot/vmlinux.bin                      1,092,060  bytes
       /arch/x86/boot/compressed/vmlinux               1,099,538  bytes
       /arch/x86/boot/compressed/vmlinux.bin           2,094,132  bytes
       /arch/x86/boot/compressed/vmlinux.bin.gz        1,074,711  bytes

 I understand that /arch/x86/boot/compressed/vmlinux.bin.gz is a compressed
 version of /arch/x86/boot/compressed/vmlinux.bin, and
 /arch/i386/boot/bzImage and /arch/x86/boot/bzImage are the same file and
 that it is the 16-bit boot code + /arch/x86/boot/compressed/vmlinux.bin.gz
This is correct.
 but I don't understand the rest...


 My guess is that /vmlinux.o is the ELF image generated by the compiler +
 linker stage and /vmlinux may be /vmlinux.o objdump'd into a raw binary and
 perhaps /arch/x86/boot/vmlinux.bin is a further stripped version
 of/vmlinux, but I'm at a loss with /arch/x86/boot/compressed/vmlinux

vmlinux is ELF image with ELF header. So actual point of kernel entry
would be at an offset, somewhere after the ELF header.
vmlinux.bin is what you would get after doing
#objcopy -O binary vmlinux vmlinux.bin.
vmlinux.bin has only obj code and nothing else.

 In any event, it looks like either /arch/x86/boot/compressed/vmlinux.bin or
 /vmlinux is what I need to copy into RAM @ 0x10 (1MiB) which is where
 my non-relocatable kernel is compiled to.

copy vmlinux.bin.

  - How to setup the memory map (keeping in mind I have 2GB of contiguous
   memory with no BIOS/ACPI etc to worry about clobbering
  - Any other tricks I need to be aware of..
Check if x86 kernel expects some parameters(like machineid, bootargs
location etc) in some registers.
Check if x86 has low level debug support(like DEBUG_LL).
Also you might want to check how initial page tables are being setup in kernel.

-syed

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Vikram Narayanan]

On Mon, May 16, 2011 at 9:14 AM, Peter Teoh htmldevelo...@gmail.com wrote:
 I loved this reply...can I annotate it with references to the linux
 kernel sources?

 On Fri, May 13, 2011 at 9:42 AM, Dave Hylands dhyla...@gmail.com wrote:

 Hi Vikram,

 ...snip...
  So when compiling the kernel, what is the purpose of the other
  files(mentioned below)
  linux-2.6/vmlinux - ELF executable, not stripped
  linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
  one which is inside the bzImage)
  linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable,
  stripped
  linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not
  stripped

 Take luca's email and start at the bottom working towards the top.

 linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file.
 A binary is then extracted from this to create
 arch/x86/boot/compressed/vmlinux.bin

 yes:
 See ./arch/x86/boot/Makefile


 This binary is then compressed to produce
 arch/x86/boot/compressed/vmlinux.bin.gz

 See ./arch/x86/boot/compressed/Makefile


 This gzipped binary is then converted into an object file (which just
 contains the gzipped data) but now we're back to having an ELF file

 ./arch/x86/boot/compressed/mkpiggy.c is compiled into a commandline binary -
 mkpiggy which will generate the piggy.o.


 called arch/x86/boot/compressed/piggy.o
 The linker then compiles a decompressor (misc.o) and piggy.o together

 Yes, the routine is called decompress_kernel, residing inside
 ./arch/x86/boot/compressed/misc.c.   And this routine is called
 from ./arch/x86/boot/compressed/head_32.S (or head_64.S) and at runtime, the
 gzipped data is decompressed and immediately jumped into (perhaps after some
 relocation if needed):
 /*
  * Do the decompression, and jump to the new kernel..
  */
         leal    z_extract_offset_negative(%ebx), %ebp
                                 /* push arguments for decompress_kernel: */
         pushl   %ebp            /* output address */
         pushl   $z_input_len    /* input_len */
         leal    input_data(%ebx), %eax
         pushl   %eax            /* input_data */
         leal    boot_heap(%ebx), %eax
         pushl   %eax            /* heap area */
         pushl   %esi            /* real mode pointer */
         call    decompress_kernel
         addl    $20, %esp

 to produce arch/x86/boot/compressed/vmlinux (an ELF file).
 objcopy is used again to convert this ELF into a binary:
 arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin
 Finally, the binary is compressed to produce bzImage.

 Inside arch/x86/boot/Makefile:
 Creating the vmlinux.bin from vmlinux via objcopy (note that this operation
 will throw all relocation information):
 $(obj)/vmlinux.bin: $(obj)/compressed/vmlinux FORCE
         $(call if_changed,objcopy)
 And then packing together linearly to form the bzImage (output from make):
 make -f scripts/Makefile.build obj=arch/x86/boot arch/x86/boot/bzImage
 make -f scripts/Makefile.build obj=arch/x86/boot/compressed
 arch/x86/boot/compressed/vmlinux
 arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin
 CURRENT  arch/x86/boot/bzImage

 So what you get is a compressed binary which contains a decompressor
 and another compressed binary, this inner compressed binary being the
 kernel.

 GRUB loads bzImage into memory and decompresses it and then executes
 the resulting binary.

 To be more precise, grub will load bzImage and jump into the startup_32
 function located in arch/x86/boot/compressed/head_32.S at the following
 fixed address (from source code):
 /*
  *  head.S contains the 32-bit startup code.
  *
  * NOTE!!! Startup happens at absolute address 0x1000, which is also
 where
  * the page directory will exist. The startup code will be overwritten by
  * the page directory. [According to comments etc elsewhere on a compressed
  * kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC]
  *
  * Page 0 is deliberately kept safe, since System Management Mode code in
  * laptops may need to access the BIOS data stored there.  This is also
  * useful for future device drivers that either access the BIOS via VM86
  * mode.
  */
 More info:
 http://books.google.com/books?id=e8BbHxVhzFACpg=PA1224lpg=PA1224dq=grub+head_32.Ssource=blots=0MSdKwBoM6sig=2RyEpprl25zueiqi332TQHLIj0Ehl=enei=y5vQTY7eBNDNrQeI3bTCCgsa=Xoi=book_resultct=resultresnum=3ved=0CCkQ6AEwAg#v=onepageq=grub%20head_32.Sf=false


 This binary starts with a decompressor which then decompresses the
 kernel, and executes the resulting binary.
 This binary may relocate itself (probably depends on the architecture)
 to a different spot in memory, and then runs.
 The kernel is now running.

 --
 Dave Hylands
 Shuswap, BC, Canada
 http://www.davehylands.com

 ___
 K

 --
 Regards,
 Peter Teoh

That was a great explanation. Thanks a lot. I think this will be very
much useful for people who want to know how things

Re: How vmlinux is recognized?

[Vikram Narayanan]

On Fri, May 13, 2011 at 7:12 AM, Dave Hylands dhyla...@gmail.com wrote:

 Hi Vikram,

 ...snip...
  So when compiling the kernel, what is the purpose of the other
  files(mentioned below)
  linux-2.6/vmlinux - ELF executable, not stripped
  linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
  one which is inside the bzImage)
  linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped
  linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped

 Take luca's email and start at the bottom working towards the top.

 linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file.
 A binary is then extracted from this to create
 arch/x86/boot/compressed/vmlinux.bin
 This binary is then compressed to produce
 arch/x86/boot/compressed/vmlinux.bin.gz
 This gzipped binary is then converted into an object file (which just
 contains the gzipped data) but now we're back to having an ELF file
 called arch/x86/boot/compressed/piggy.o
 The linker then compiles a decompressor (misc.o) and piggy.o together
 to produce arch/x86/boot/compressed/vmlinux (an ELF file).
 objcopy is used again to convert this ELF into a binary:
 arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin
 Finally, the binary is compressed to produce bzImage.

 So what you get is a compressed binary which contains a decompressor
 and another compressed binary, this inner compressed binary being the
 kernel.

 GRUB loads bzImage into memory and decompresses it and then executes
 the resulting binary.
 This binary starts with a decompressor which then decompresses the
 kernel, and executes the resulting binary.
 This binary may relocate itself (probably depends on the architecture)
 to a different spot in memory, and then runs.
 The kernel is now running.
Thanks for the detailed explanation. Clarified. :)

-
Thanks
Vikram

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Peter Teoh]

I loved this reply...can I annotate it with references to the linux
kernel sources?

On Fri, May 13, 2011 at 9:42 AM, Dave Hylands dhyla...@gmail.com wrote:

 Hi Vikram,

 ...snip...
  So when compiling the kernel, what is the purpose of the other
  files(mentioned below)
  linux-2.6/vmlinux - ELF executable, not stripped
  linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
  one which is inside the bzImage)
  linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped
  linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped

 Take luca's email and start at the bottom working towards the top.

 linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file.
 A binary is then extracted from this to create
 arch/x86/boot/compressed/vmlinux.bin


yes:

See ./arch/x86/boot/Makefile


  This binary is then compressed to produce
 arch/x86/boot/compressed/vmlinux.bin.gz


See ./arch/x86/boot/compressed/Makefile


 This gzipped binary is then converted into an object file (which just
 contains the gzipped data) but now we're back to having an ELF file


./arch/x86/boot/compressed/mkpiggy.c is compiled into a commandline binary -
mkpiggy which will generate the piggy.o.


 called arch/x86/boot/compressed/piggy.o
 The linker then compiles a decompressor (misc.o) and piggy.o together


Yes, the routine is called decompress_kernel, residing inside
./arch/x86/boot/compressed/misc.c.   And this routine is called
from ./arch/x86/boot/compressed/head_32.S (or head_64.S) and at runtime, the
gzipped data is decompressed and immediately jumped into (perhaps after some
relocation if needed):

/*
 * Do the decompression, and jump to the new kernel..
 */
lealz_extract_offset_negative(%ebx), %ebp
/* push arguments for decompress_kernel: */
pushl   %ebp/* output address */
pushl   $z_input_len/* input_len */
lealinput_data(%ebx), %eax
pushl   %eax/* input_data */
lealboot_heap(%ebx), %eax
pushl   %eax/* heap area */
pushl   %esi/* real mode pointer */
calldecompress_kernel
addl$20, %esp


 to produce arch/x86/boot/compressed/vmlinux (an ELF file).
 objcopy is used again to convert this ELF into a binary:
 arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin
 Finally, the binary is compressed to produce bzImage.


Inside arch/x86/boot/Makefile:

Creating the vmlinux.bin from vmlinux via objcopy (note that this operation
will throw all relocation information):

$(obj)/vmlinux.bin: $(obj)/compressed/vmlinux FORCE
$(call if_changed,objcopy)

And then packing together linearly to form the bzImage (output from make):

make -f scripts/Makefile.build obj=arch/x86/boot arch/x86/boot/bzImage

make -f scripts/Makefile.build obj=arch/x86/boot/compressed
arch/x86/boot/compressed/vmlinux

arch/x86/boot/tools/build arch/x86/boot/setup.bin arch/x86/boot/vmlinux.bin
CURRENT  arch/x86/boot/bzImage


 So what you get is a compressed binary which contains a decompressor
 and another compressed binary, this inner compressed binary being the
 kernel.

 GRUB loads bzImage into memory and decompresses it and then executes
 the resulting binary.


To be more precise, grub will load bzImage and jump into the startup_32
function located in arch/x86/boot/compressed/head_32.S at the following
fixed address (from source code):

/*
 *  head.S contains the 32-bit startup code.
 *
 * NOTE!!! Startup happens at absolute address 0x1000, which is also
where
 * the page directory will exist. The startup code will be overwritten by
 * the page directory. [According to comments etc elsewhere on a compressed
 * kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC]
 *
 * Page 0 is deliberately kept safe, since System Management Mode code in
 * laptops may need to access the BIOS data stored there.  This is also
 * useful for future device drivers that either access the BIOS via VM86
 * mode.
 */

More info:

http://books.google.com/books?id=e8BbHxVhzFACpg=PA1224lpg=PA1224dq=grub+head_32.Ssource=blots=0MSdKwBoM6sig=2RyEpprl25zueiqi332TQHLIj0Ehl=enei=y5vQTY7eBNDNrQeI3bTCCgsa=Xoi=book_resultct=resultresnum=3ved=0CCkQ6AEwAg#v=onepageq=grub%20head_32.Sf=false


 This binary starts with a decompressor which then decompresses the
 kernel, and executes the resulting binary.
 This binary may relocate itself (probably depends on the architecture)
 to a different spot in memory, and then runs.
 The kernel is now running.

 --
 Dave Hylands
 Shuswap, BC, Canada
 http://www.davehylands.com

 ___
 K


-- 
Regards,
Peter Teoh
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Sudheer Divakaran]

On Thu, May 12, 2011 at 10:02 AM, Sudheer Divakaran
inbox1.sudh...@gmail.com wrote:
 Hi Vikram,

 On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan vikram...@gmail.com wrote:
 On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa
 mulyadi.sant...@gmail.com wrote:
 On Thu, May 12, 2011 at 03:11, Vikram Narayanan vikram...@gmail.com wrote:
 Yes. I agree. But how who converts the ELF binary to raw binary so
 that the processor understands. Or how is it actually done?

 OK I try my best to understand your question :)

 i think I got it...you probably guessed that vmlinux created first,
 then vmlinuz... AFAIK, it's the other way around...or more precisely,
 not both.

 I think you got it wrong. I will try to put my question more elaborately.
 1) The system is on and BIOS code runs. It gives the control to the
 boot loader, say GRUB.
 2) Grub picks up the kernel from the specific partition. (i.e a
 vmlinuz image), which denotes that it is compressed.
 3) There are uncompression routines in the kernel itself, If I am not
 wrong. So the kernel uncompresses itself.
 4) Now the uncompressed thing is the vmlinux image, right?
 5) The vmlinux is in ELF format. Correct?
 6) If the OS boots and if u try to run an ELF file, the loader knows
 how to load that in the RAM. (I mean it knows how to interpret the ELF
 format)
 7) Coming back to the vmlinux image, Who takes care of the loading activity.?
 8) Who recognizes that the image is ELF format and do the necessary
 things accordingly.?

 Hope I have my question clear now.




 If understand your question correctly, you believe that the
 uncompressed kernel is in elf format. correct?. it is in binary
 format, so elf interpretation is not required, #5 is wrong.

 You can see this by building the kernel using 'make V=1'  and note the
 following line in the output,

 arch/x86/boot/tools/build arch/x86/boot/setup.bin
 arch/x86/boot/vmlinux.bin CURRENT  arch/x86/boot/bzImage

 means bzImage is made out of two binary files extracted from the elf images.

One more info I want to clarify is,  vmlinux.bin mentioned in the
above snippet contains the compressed binary image and some other
routines. Just go through the 'make V=1' output, you can see that the
build process is actually compressing binary file extracted from the
vmlinux elf image, which is again combined with some object files,
creates another elf and again extracts the binary and finally combined
with the setup.bin to create the final bzImage. So, elf interpretation
 doesn't happen on the uncompressed code.

-- 
Thanks
Sudheer

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[luca ellero]

On 12/05/2011 8.21, Sudheer Divakaran wrote:
 On Thu, May 12, 2011 at 10:02 AM, Sudheer Divakaran
 inbox1.sudh...@gmail.com  wrote:
 Hi Vikram,

 On Thu, May 12, 2011 at 9:02 AM, Vikram Narayananvikram...@gmail.com  
 wrote:
 On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa
 mulyadi.sant...@gmail.com  wrote:
 On Thu, May 12, 2011 at 03:11, Vikram Narayananvikram...@gmail.com  
 wrote:
 Yes. I agree. But how who converts the ELF binary to raw binary so
 that the processor understands. Or how is it actually done?

 OK I try my best to understand your question :)

 i think I got it...you probably guessed that vmlinux created first,
 then vmlinuz... AFAIK, it's the other way around...or more precisely,
 not both.

 I think you got it wrong. I will try to put my question more elaborately.
 1) The system is on and BIOS code runs. It gives the control to the
 boot loader, say GRUB.
 2) Grub picks up the kernel from the specific partition. (i.e a
 vmlinuz image), which denotes that it is compressed.
 3) There are uncompression routines in the kernel itself, If I am not
 wrong. So the kernel uncompresses itself.
 4) Now the uncompressed thing is the vmlinux image, right?
 5) The vmlinux is in ELF format. Correct?
 6) If the OS boots and if u try to run an ELF file, the loader knows
 how to load that in the RAM. (I mean it knows how to interpret the ELF
 format)
 7) Coming back to the vmlinux image, Who takes care of the loading 
 activity.?
 8) Who recognizes that the image is ELF format and do the necessary
 things accordingly.?

 Hope I have my question clear now.




 If understand your question correctly, you believe that the
 uncompressed kernel is in elf format. correct?. it is in binary
 format, so elf interpretation is not required, #5 is wrong.

 You can see this by building the kernel using 'make V=1'  and note the
 following line in the output,

 arch/x86/boot/tools/build arch/x86/boot/setup.bin
 arch/x86/boot/vmlinux.bin CURRENT  arch/x86/boot/bzImage

 means bzImage is made out of two binary files extracted from the elf images.

 One more info I want to clarify is,  vmlinux.bin mentioned in the
 above snippet contains the compressed binary image and some other
 routines. Just go through the 'make V=1' output, you can see that the
 build process is actually compressing binary file extracted from the
 vmlinux elf image, which is again combined with some object files,
 creates another elf and again extracts the binary and finally combined
 with the setup.bin to create the final bzImage. So, elf interpretation
   doesn't happen on the uncompressed code.



Let's put some order here. The image that almost all bootloaders use is 
arch/x86/boot/bzImage which is made of a setup binary file (executable) 
joined with some compressed code (the real kernel) which is uncompressed 
in memory by the setup binary.

The big suggestion I can give is to check the hidden files which end 
with .cmd. There is one of these for every object created by the 
compilation process. For example there is a file called .bzImage.cmd 
which tell you how bzImage was made:

arch/x86/boot/tools/build -b arch/x86/boot/setup.bin 
arch/x86/boot/vmlinux.bin CURRENT  arch/x86/boot/bzImage

NOTE: I refer to a quite old kernel here, it's likely that the 
compilation process has changed somehow.

You can proceed now in reverse order to find how bzImage was made (if I 
understand correctly that is the one you are interested in).

Here is how is made on my kernel tree:

bzImage:
arch/x86/boot/tools/build -b arch/x86/boot/setup.bin 
arch/x86/boot/vmlinux.bin CURRENT  arch/x86/boot/bzImage

arch/x86/boot/vmlinux.bin:
objcopy  -O binary -R .note -R .comment -S 
arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin

arch/x86/boot/compressed/vmlinux:
ld -m elf_i386   -T arch/x86/boot/compressed/vmlinux_32.lds 
arch/x86/boot/compressed/head_32.o arch/x86/boot/compressed/misc.o 
arch/x86/boot/compressed/piggy.o -o arch/x86/boot/compressed/vmlinux

arch/x86/boot/compressed/piggy.o:
ld -m elf_i386   -r --format binary --oformat elf32-i386 -T 
arch/x86/boot/compressed/vmlinux.scr 
arch/x86/boot/compressed/vmlinux.bin.gz -o arch/x86/boot/compressed/piggy.o

arch/x86/boot/compressed/vmlinux.bin.gz:
gzip -f -9  arch/x86/boot/compressed/vmlinux.bin  
arch/x86/boot/compressed/vmlinux.bin.gz

arch/x86/boot/compressed/vmlinux.bin:
objcopy  -O binary -R .note -R .comment -S vmlinux 
arch/x86/boot/compressed/vmlinux.bin

vmlinux:
ld -m elf_i386 --build-id -o vmlinux -T arch/x86/kernel/vmlinux.lds 
arch/x86/kernel/head_32.o arch/x86/kernel/init_task.o  init/built-in.o 
--start-group  usr/built-in.o  arch/x86/kernel/built-in.o 
arch/x86/mm/built-in.o  arch/x86/mach-default/built-in.o 
arch/x86/crypto/built-in.o  arch/x86/vdso/built-in.o  kernel/built-in.o 
  mm/built-in.o  fs/built-in.o  ipc/built-in.o  security/built-in.o 
crypto/built-in.o  block/built-in.o  lib/lib.a  arch/x86/lib/lib.a 
lib/built-in.o  arch/x86/lib/built-in.o  drivers/built-in.o 
sound/built

Re: How vmlinux is recognized?

[Mulyadi Santosa]

Hi...

On Thu, May 12, 2011 at 10:32, Vikram Narayanan vikram...@gmail.com wrote:
 I think you got it wrong. I will try to put my question more elaborately.
 1) The system is on and BIOS code runs. It gives the control to the
 boot loader, say GRUB.
 2) Grub picks up the kernel from the specific partition. (i.e a
 vmlinuz image), which denotes that it is compressed.
 3) There are uncompression routines in the kernel itself, If I am not
 wrong. So the kernel uncompresses itself.
 4) Now the uncompressed thing is the vmlinux image, right?

nope... it's a binarybut not ELF...and that's not even named
vmlinux or similar to vmlinux...

 5) The vmlinux is in ELF format. Correct?
yes but see above...


-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[अनुज]

Hi All

On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan vikram...@gmail.comwrote:

 On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa
 mulyadi.sant...@gmail.com wrote:
  On Thu, May 12, 2011 at 03:11, Vikram Narayanan vikram...@gmail.com
 wrote:
  Yes. I agree. But how who converts the ELF binary to raw binary so
  that the processor understands. Or how is it actually done?
 
  OK I try my best to understand your question :)
 
  i think I got it...you probably guessed that vmlinux created first,
  then vmlinuz... AFAIK, it's the other way around...or more precisely,
  not both.

 I think you got it wrong. I will try to put my question more elaborately.
 1) The system is on and BIOS code runs. It gives the control to the
 boot loader, say GRUB.
 2) Grub picks up the kernel from the specific partition. (i.e a
 vmlinuz image), which denotes that it is compressed.
 3) There are uncompression routines in the kernel itself, If I am not
 wrong. So the kernel uncompresses itself.
 4) Now the uncompressed thing is the vmlinux image, right?
 5) The vmlinux is in ELF format. Correct?


I Guess Yes.

6) If the OS boots and if u try to run an ELF file, the loader knows
 how to load that in the RAM. (I mean it knows how to interpret the ELF
 format)


See the multi-boot specification. GRUB is a multi-boot compliant boot loader


 7) Coming back to the vmlinux image, Who takes care of the loading
 activity.?


GRUB

8) Who recognizes that the image is ELF format and do the necessary
 things accordingly.?


GRUB


 Hope I have my question clear now.

 -
 Thanks,
 Vikram

 ___
 Kernelnewbies mailing list
 Kernelnewbies@kernelnewbies.org
 http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies




-- 
Anuj Aggarwal

 .''`.
: :Ⓐ :   # apt-get install hakuna-matata
`. `'`
   `-
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[mindentropy]

 This way, GRUB doesn't need to know how to decode ELF
 files and the job is left to the kernel code. 

GRUB has a elf decoder, but it should have multiboot header.
http://osdev.berlios.de/grub.html#multiboot

Hope this answers your doubt.

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 1:47 PM, Mulyadi Santosa
mulyadi.sant...@gmail.com wrote:
 Hi...

 On Thu, May 12, 2011 at 10:32, Vikram Narayanan vikram...@gmail.com wrote:
 I think you got it wrong. I will try to put my question more elaborately.
 1) The system is on and BIOS code runs. It gives the control to the
 boot loader, say GRUB.
 2) Grub picks up the kernel from the specific partition. (i.e a
 vmlinuz image), which denotes that it is compressed.
 3) There are uncompression routines in the kernel itself, If I am not
 wrong. So the kernel uncompresses itself.
 4) Now the uncompressed thing is the vmlinux image, right?

 nope... it's a binarybut not ELF...and that's not even named
 vmlinux or similar to vmlinux...

 5) The vmlinux is in ELF format. Correct?
 yes but see above...

Thanks for all your explanations. So the uncompressed one is _NOT_ an
ELF file, but a raw binary. So it doesn't need any interpretation.
Hope this is right.

So when compiling the kernel, what is the purpose of the other
files(mentioned below)
linux-2.6/vmlinux - ELF executable, not stripped
linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
one which is inside the bzImage)
linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped
linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped

Thanks,
Vikram

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Dave Hylands]

Hi Vikram,

...snip...
 So when compiling the kernel, what is the purpose of the other
 files(mentioned below)
 linux-2.6/vmlinux - ELF executable, not stripped
 linux-2.6/arch/x86/boot/vmlinux.bin - Raw binary (Guess this is the
 one which is inside the bzImage)
 linux-2.6/arch/x86/boot/compressed/vmlinux.bin - ELF executable, stripped
 linux-2.6/arch/x86/boot/compressed/vmlinux - ELF executable, not stripped

Take luca's email and start at the bottom working towards the top.

linux-2.6/vmlinux is the output of the linker. As such, it is an ELF file.
A binary is then extracted from this to create
arch/x86/boot/compressed/vmlinux.bin
This binary is then compressed to produce
arch/x86/boot/compressed/vmlinux.bin.gz
This gzipped binary is then converted into an object file (which just
contains the gzipped data) but now we're back to having an ELF file
called arch/x86/boot/compressed/piggy.o
The linker then compiles a decompressor (misc.o) and piggy.o together
to produce arch/x86/boot/compressed/vmlinux (an ELF file).
objcopy is used again to convert this ELF into a binary:
arch/x86/boot/compressed/vmlinux arch/x86/boot/vmlinux.bin
Finally, the binary is compressed to produce bzImage.

So what you get is a compressed binary which contains a decompressor
and another compressed binary, this inner compressed binary being the
kernel.

GRUB loads bzImage into memory and decompresses it and then executes
the resulting binary.
This binary starts with a decompressor which then decompresses the
kernel, and executes the resulting binary.
This binary may relocate itself (probably depends on the architecture)
to a different spot in memory, and then runs.
The kernel is now running.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 12:47 AM, Dave Hylands dhyla...@gmail.com wrote:
 Hi Vikram,

 On Wed, May 11, 2011 at 11:06 AM, Vikram Narayanan vikram...@gmail.com 
 wrote:
 Hi,

 Sorry if this question is stupid.
 How the vmlinux (an ELF executable) is recognized by the processor?
 What are the files that are responsible for this?

 Well the short answer is that it isn't.

 The ELF file is normally just one stage of the process. You still need
 to extract a binary from the ELF, and the binary contains the raw
 executable code that the processor uses.

 Normally the boot loader will extract a binary (perhaps from an ELF,
 or perhaps from a raw binary image) and this is what the processor
 sees.
So in case of x86, say Grub will be taking care of this extraction. Right?
If, so the grub code will have the mechanisms for extracting the raw
binary from ELF.
Am i right?

-
Thanks,
Vikram

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 1:15 AM, Mulyadi Santosa
mulyadi.sant...@gmail.com wrote:
 On Thu, May 12, 2011 at 02:31, Vikram Narayanan vikram...@gmail.com wrote:
 So in case of x86, say Grub will be taking care of this extraction. Right?
 If, so the grub code will have the mechanisms for extracting the raw
 binary from ELF.
 Am i right?

 you mean, vmlinuz right? the bzImage right? not the vmlinux
 because that's the one GRUB handles...not vmlinux one...

The vmlinux is an ELF binary. right? If so, Who does the unpacking of
raw binary image from that ELF?

 well, in that case, see this first:
 $ file -k -z  /boot/vmlinuz-2.6.32-31-generic

This is the compressed one. The uncompression is done by the kernel
and not by the grub, If I am not wrong.

 /boot/vmlinuz-2.6.32-31-generic: Linux kernel x86 boot executable
 bzImage, version 2.6.32-31-generic (buildd@rothe, RO-rootFS, root_dev
 0x801, swap_dev 0x3, Normal VGA\012- x86 boot sector, code offset 0x5

 i am sure you will get idea based upon the above file identification,
 on what vmlinuz is and how is it supposed to be treated by boot
 loader
I am still confused :(

Thanks,
Vikram

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Mulyadi Santosa]

On Thu, May 12, 2011 at 03:04, Vikram Narayanan vikram...@gmail.com wrote:
 The vmlinux is an ELF binary. right? If so, Who does the unpacking of
 raw binary image from that ELF?

why do you put concern on vmlinux anyway? boot loader loads vmlinuz,
not vmlinux
-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Mulyadi Santosa]

On Thu, May 12, 2011 at 03:11, Vikram Narayanan vikram...@gmail.com wrote:
 Yes. I agree. But how who converts the ELF binary to raw binary so
 that the processor understands. Or how is it actually done?

OK I try my best to understand your question :)

i think I got it...you probably guessed that vmlinux created first,
then vmlinuz... AFAIK, it's the other way around...or more precisely,
not both.

After final phase of final kernel image creation, it will go into
making bootable image first. in order to do that, first it will be
compressed 1st. These days, gz is the choice.

So, it is gzipped..and the boot loading code is appended in front of
it... there, you get vmlinuz.

And vmlinux? developers usually use vmlinux as symbol file... and the
way it is created, back to the above phase, is by linking it according
to the accompanying elf linker script. Finally, ELF that contains
kernel is there.

Another guess, maybe you wanna know how to extract the kernel code
from ELF image? then why so? that is indeed the kernel image
itself...it is just appended ELF headers, sections and so on just to
represent ELF construction. But it is not behaving like standart ELF
binary i.e the entry point is not main() but IIRC start_kernel or
something like that.

that helps you?
-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Manohar Vanga]

Hi Vikram,

How the vmlinux (an ELF executable) is recognized by the processor?


ELF is just a file format. That is, the machine instructions and data are
stored in a specific format. The _processor_ simply recognizes machine
instructions and this needs to be taken from the ELF file and loaded into
memory (the instruction pointer is then pointed to the place the
instructions were loaded).

The format is simply a set of rules defined in the specification (a pretty
nice introduction is available at www.skyfree.org/linux/references/*ELF*
_Format.pdf http://www.skyfree.org/linux/references/ELF_Format.pdf). For
example, when you ask a Linux kernel to execute an ELF file, it has code to
know how to decode the information and place it into memory (see
fs/binfmt_elf.c).

As for the vmlinux file specifically, the Wikipedia page on vmlinux (
http://en.wikipedia.org/wiki/Vmlinux) seems like a good start. As shown
above with Linux, GRUB needs to have a way to decode whatever format is
passed to it (bzImage).

The kernel however places the unzipping code into the bzImage itself so that
it is loaded into memory by the bootloader and is then run. This code then
unzips the kernel. This way, GRUB doesn't need to know how to decode ELF
files and the job is left to the kernel code. You can see
arch/x86/boot/Makefile and look for the bzImage target to see what files
constitute the bzImage. I may be wrong about this with regard to newer
kernels so I hope others correct me in this case. Another great explanation
is by Alessandro Rubini at:

http://www.ibiblio.org/oswg/oswg-nightly/oswg/en_US.ISO_8859-1/articles/alessandro-rubini/boot/boot/zimage.html

Hope this helped! :-)

-- 
/manohar
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Vikram Narayanan]

On Thu, May 12, 2011 at 2:03 AM, Manohar Vanga manohar.va...@gmail.com wrote:
 Hi Vikram,

 How the vmlinux (an ELF executable) is recognized by the processor?

 ELF is just a file format. That is, the machine instructions and data are
 stored in a specific format. The _processor_ simply recognizes machine
 instructions and this needs to be taken from the ELF file and loaded into
 memory (the instruction pointer is then pointed to the place the
 instructions were loaded).

Hope everyone here got my question wrong. I am aware that ELF is a
format and there will be specific loader for loading ELF files. Please
refer to the previous reply.

Thanks,
Vikram

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How vmlinux is recognized?

[Sudheer Divakaran]

Hi Vikram,

On Thu, May 12, 2011 at 9:02 AM, Vikram Narayanan vikram...@gmail.com wrote:
 On Thu, May 12, 2011 at 1:51 AM, Mulyadi Santosa
 mulyadi.sant...@gmail.com wrote:
 On Thu, May 12, 2011 at 03:11, Vikram Narayanan vikram...@gmail.com wrote:
 Yes. I agree. But how who converts the ELF binary to raw binary so
 that the processor understands. Or how is it actually done?

 OK I try my best to understand your question :)

 i think I got it...you probably guessed that vmlinux created first,
 then vmlinuz... AFAIK, it's the other way around...or more precisely,
 not both.

 I think you got it wrong. I will try to put my question more elaborately.
 1) The system is on and BIOS code runs. It gives the control to the
 boot loader, say GRUB.
 2) Grub picks up the kernel from the specific partition. (i.e a
 vmlinuz image), which denotes that it is compressed.
 3) There are uncompression routines in the kernel itself, If I am not
 wrong. So the kernel uncompresses itself.
 4) Now the uncompressed thing is the vmlinux image, right?
 5) The vmlinux is in ELF format. Correct?
 6) If the OS boots and if u try to run an ELF file, the loader knows
 how to load that in the RAM. (I mean it knows how to interpret the ELF
 format)
 7) Coming back to the vmlinux image, Who takes care of the loading activity.?
 8) Who recognizes that the image is ELF format and do the necessary
 things accordingly.?

 Hope I have my question clear now.




If understand your question correctly, you believe that the
uncompressed kernel is in elf format. correct?. it is in binary
format, so elf interpretation is not required, #5 is wrong.

You can see this by building the kernel using 'make V=1'  and note the
following line in the output,

arch/x86/boot/tools/build arch/x86/boot/setup.bin
arch/x86/boot/vmlinux.bin CURRENT  arch/x86/boot/bzImage

means bzImage is made out of two binary files extracted from the elf images.
-- 
Thanks
Sudheer

___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies