Re: How to understand the macro __init?
On Thu, Aug 16, 2012 at 12:39 PM, Amarnath Revanna amarnath.reva...@gmail.com wrote: Just want to add a little more for better understanding: When I spoke about .init section of the final kernel image, please note that this section is going to contain all the __init data (and functions) coming from _All_ the drivers and modules that were included as part of the kernel image. Hence, after initialization when we look at the print: [1.011596] Freeing unused kernel memory: 664k freed we see 664k bytes being freed. This is a significant amount of contiguous physical memory that we can see being released by the kernel. The same cannot be held true for a single loadable module which may be releasing just a few, virtually contiguous memory. FWIW, theres no reason that the loadable module's __init section must be in the same allocation as the module code itself. The lifetime is certainly different, so pre-fragmenting it could leave fewer holes overall. I dont know if this is actually done (this way), but it seems reasonably likely, given that the per-module sections exist. $ readelf -a /lib/modules/3.6.0-rc3-cha-dyndbg-8-g6e433ac/kernel/drivers/usb/serial/pl2303.ko | grep init [ 4] .init.textPROGBITS 00197c 19 00 AX 0 0 1 [ 5] .rel.init.textREL 02a58c 20 08 44 4 4 1754 b702 R_386_PC32 __init_waitqueue_head Relocation section '.rel.init.text' at offset 0x2a58c contains 4 entries: ... Regards, -Amar ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
To be more precise, all the content of the .init section will be freed at the end of the boot. (see vmlinux.lds.S) This is done by the function free_initmem() which is an architecture specific function defined in linux-*/arch/arch/mm/init.c. This function frees the memory between the symbols __init_begin and __init_end (which need to be page-aligned). During compilation, all symbols defined with __init macro are put in the .init section. As explained before, these a ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
sorry for the wrong manipulation (resume of the previous mail) As explained before, the symbols and functions defined with __init are only used during boot initialization. Thez will never be used again. So The entire .init section is freed, and this freed memory will become available memory pages for the kernel. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
Hi, On Tue, Aug 14, 2012 at 9:34 AM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: Hi.. :) On Tue, Aug 14, 2012 at 9:14 AM, 王哲 wangzhe5...@gmail.com wrote: i use the __init for function print_k. in my opinion after the fisrt invoking the print_k in the hello_init. the memory of print_k will be freed,and the second invoking will not be executed.but the result of second invoking is executing . why? because you're still in module_init :) right after modul init stage is done, _init marked function is thrown away... Even if we call print_k() function inside hello_exit, it still works. At that point __init hello_init execution is over. How its still working? Regards, Vijay -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
Hi Ezequiel, On Thu, Aug 16, 2012 at 10:53 PM, Ezequiel Garcia elezegar...@gmail.comwrote: Hi Amar, On Thu, Aug 16, 2012 at 1:08 PM, Amarnath Revanna amarnath.reva...@gmail.com wrote: On the other hand, any other kernel module that you load using insmod or modprobe comes after this stage, wherein the kernel was already booted, and hence, no memory area of __init will ever be freed. Modules are loaded with vmalloc, right? Could you explain why the kernel can't free those __init symbols from memory also in this case? Thanks, Ezequiel. When we look at the definition of __init __initdata in http://lxr.free-electrons.com/source/include/linux/init.h#L44, 44 http://lxr.free-electrons.com/source/include/linux/init.h#L44 #define __init http://lxr.free-electrons.com/ident?i=__init __section http://lxr.free-electrons.com/ident?i=__section(.init http://lxr.free-electrons.com/ident?i=init.text http://lxr.free-electrons.com/ident?i=text) __cold http://lxr.free-electrons.com/ident?i=__cold notrace http://lxr.free-electrons.com/ident?i=notrace 45 http://lxr.free-electrons.com/source/include/linux/init.h#L45 #define __initdata http://lxr.free-electrons.com/ident?i=__initdata __section http://lxr.free-electrons.com/ident?i=__section(.init http://lxr.free-electrons.com/ident?i=init.data http://lxr.free-electrons.com/ident?i=data) 46 http://lxr.free-electrons.com/source/include/linux/init.h#L46 #define __initconst http://lxr.free-electrons.com/ident?i=__initconst __section http://lxr.free-electrons.com/ident?i=__section(.init http://lxr.free-electrons.com/ident?i=init.rodata http://lxr.free-electrons.com/ident?i=rodata) 47 http://lxr.free-electrons.com/source/include/linux/init.h#L47 #define __exitdata http://lxr.free-electrons.com/ident?i=__exitdata __section http://lxr.free-electrons.com/ident?i=__section(.exit http://lxr.free-electrons.com/ident?i=exit.data http://lxr.free-electrons.com/ident?i=data) 48 http://lxr.free-electrons.com/source/include/linux/init.h#L48 #define __exit_call http://lxr.free-electrons.com/ident?i=__exit_call __used http://lxr.free-electrons.com/ident?i=__used __section http://lxr.free-electrons.com/ident?i=__section(.exitcall.exit http://lxr.free-electrons.com/ident?i=exit) we can notice that the functions represented by __init and any data represented by __initdata are going to be placed in a separate section of the final kernel binary image (zImage/uImage/vmlinux) by the linker. This section is going to be called the .init section. The idea behind forming this separate .init section in the final kernel image is to hold all those functions and data structures that are going to be required only once during initialization, together. By doing so, the kernel, once it boots up, would have already utilized all these resources once during bootup sequence and hence, can now be released from the memory. As a result, the kernel would simply discard this entire .init section from the RAM in one go, there by freeing the memory. The amount of memory being freed by removing this section is thus printed in the line: [1.011596] Freeing unused kernel memory: 664k freed Now, when we think about loadable modules, as you rightly said, are loaded into the kernel memory by obtaining the memory area from the heap using vmalloc. The interesting thing about this is that, since we are going to load only one module within this vmalloc'd area, we can normally expect the size of __initdata and __init function to be pretty small, in few bytes. Now, it becomes too difficult for the kernel to manage (keep track of and free) these smaller memory areas coming up from every individual loaded module. Another thing to add is that, in case of freeing up an entire .init section from the RAM, we are recovering the entire .init section size'd _contiguous_ physical memory area back to the kernel. However, in case of Loaded Kernel Module (LKM) if we try to free up the __init memory of an LKM that was allocated using vmalloc, we may only be freeing up few bytes of memory that was virtually contiguous. This may not be of much significance for the kernel operation as compared to its overhead involved with managing a process to keep track of and freeing up all these __init memory in vmalloc area. In short, its kind of a nice trade off done to leave this __init data cleanup for LKM while keeping its significance for all built in drivers/modules. Regards, -Amar ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
Just want to add a little more for better understanding: When I spoke about .init section of the final kernel image, please note that this section is going to contain all the __init data (and functions) coming from _All_ the drivers and modules that were included as part of the kernel image. Hence, after initialization when we look at the print: [1.011596] Freeing unused kernel memory: 664k freed we see 664k bytes being freed. This is a significant amount of contiguous physical memory that we can see being released by the kernel. The same cannot be held true for a single loadable module which may be releasing just a few, virtually contiguous memory. Regards, -Amar On Thu, Aug 16, 2012 at 11:57 PM, Amarnath Revanna amarnath.reva...@gmail.com wrote: Hi Ezequiel, On Thu, Aug 16, 2012 at 10:53 PM, Ezequiel Garcia elezegar...@gmail.comwrote: Hi Amar, On Thu, Aug 16, 2012 at 1:08 PM, Amarnath Revanna amarnath.reva...@gmail.com wrote: On the other hand, any other kernel module that you load using insmod or modprobe comes after this stage, wherein the kernel was already booted, and hence, no memory area of __init will ever be freed. Modules are loaded with vmalloc, right? Could you explain why the kernel can't free those __init symbols from memory also in this case? Thanks, Ezequiel. When we look at the definition of __init __initdata in http://lxr.free-electrons.com/source/include/linux/init.h#L44, 44 http://lxr.free-electrons.com/source/include/linux/init.h#L44 #define __init http://lxr.free-electrons.com/ident?i=__init __section http://lxr.free-electrons.com/ident?i=__section(.init http://lxr.free-electrons.com/ident?i=init.text http://lxr.free-electrons.com/ident?i=text) __cold http://lxr.free-electrons.com/ident?i=__cold notrace http://lxr.free-electrons.com/ident?i=notrace 45 http://lxr.free-electrons.com/source/include/linux/init.h#L45 #define __initdata http://lxr.free-electrons.com/ident?i=__initdata __section http://lxr.free-electrons.com/ident?i=__section(.init http://lxr.free-electrons.com/ident?i=init.data http://lxr.free-electrons.com/ident?i=data) 46 http://lxr.free-electrons.com/source/include/linux/init.h#L46 #define __initconst http://lxr.free-electrons.com/ident?i=__initconst __section http://lxr.free-electrons.com/ident?i=__section(.init http://lxr.free-electrons.com/ident?i=init.rodata http://lxr.free-electrons.com/ident?i=rodata) 47 http://lxr.free-electrons.com/source/include/linux/init.h#L47 #define __exitdata http://lxr.free-electrons.com/ident?i=__exitdata __section http://lxr.free-electrons.com/ident?i=__section(.exit http://lxr.free-electrons.com/ident?i=exit.data http://lxr.free-electrons.com/ident?i=data) 48 http://lxr.free-electrons.com/source/include/linux/init.h#L48 #define __exit_call http://lxr.free-electrons.com/ident?i=__exit_call __used http://lxr.free-electrons.com/ident?i=__used __section http://lxr.free-electrons.com/ident?i=__section(.exitcall.exit http://lxr.free-electrons.com/ident?i=exit) we can notice that the functions represented by __init and any data represented by __initdata are going to be placed in a separate section of the final kernel binary image (zImage/uImage/vmlinux) by the linker. This section is going to be called the .init section. The idea behind forming this separate .init section in the final kernel image is to hold all those functions and data structures that are going to be required only once during initialization, together. By doing so, the kernel, once it boots up, would have already utilized all these resources once during bootup sequence and hence, can now be released from the memory. As a result, the kernel would simply discard this entire .init section from the RAM in one go, there by freeing the memory. The amount of memory being freed by removing this section is thus printed in the line: [1.011596] Freeing unused kernel memory: 664k freed Now, when we think about loadable modules, as you rightly said, are loaded into the kernel memory by obtaining the memory area from the heap using vmalloc. The interesting thing about this is that, since we are going to load only one module within this vmalloc'd area, we can normally expect the size of __initdata and __init function to be pretty small, in few bytes. Now, it becomes too difficult for the kernel to manage (keep track of and free) these smaller memory areas coming up from every individual loaded module. Another thing to add is that, in case of freeing up an entire .init section from the RAM, we are recovering the entire .init section size'd _contiguous_ physical memory area back to the kernel. However, in case of Loaded Kernel Module (LKM) if we try to free up the __init memory of an LKM that was allocated using vmalloc, we may only be freeing up few bytes of memory that was virtually contiguous. This may not be of much
Re: How to understand the macro __init?
Hey Amar, On Thu, Aug 16, 2012 at 3:39 PM, Amarnath Revanna amarnath.reva...@gmail.com wrote: Just want to add a little more for better understanding: When I spoke about .init section of the final kernel image, please note that this section is going to contain all the __init data (and functions) coming from _All_ the drivers and modules that were included as part of the kernel image. Hence, after initialization when we look at the print: [1.011596] Freeing unused kernel memory: 664k freed we see 664k bytes being freed. This is a significant amount of contiguous physical memory that we can see being released by the kernel. The same cannot be held true for a single loadable module which may be releasing just a few, virtually contiguous memory. It's crystal clear ;-) Nice explanation. It's important to add something to clearify a bit your explanation (please correct me if I'm wrong): When Amar is talking about virtually contiguous kernel memory he implies that this memory is physically *dis*contiguous, i.e. based on page-entries. This is the kind of memory used for loadable modules, for instance, modules that get loaded with modprobe. On the other hand, built-in modules are compiled *inside* the kernel image (bzImage). The memory used for this image is physically contiguous: it's a big contiguous block of memory pages. Contiguous memory is important for kernel, and therefore is *very* important to spend some effort minimizing it. Regards, Ezequiel. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
On Fri, Aug 17, 2012 at 12:19 AM, Ezequiel Garcia elezegar...@gmail.comwrote: Hey Amar, On Thu, Aug 16, 2012 at 3:39 PM, Amarnath Revanna amarnath.reva...@gmail.com wrote: Just want to add a little more for better understanding: When I spoke about .init section of the final kernel image, please note that this section is going to contain all the __init data (and functions) coming from _All_ the drivers and modules that were included as part of the kernel image. Hence, after initialization when we look at the print: [1.011596] Freeing unused kernel memory: 664k freed we see 664k bytes being freed. This is a significant amount of contiguous physical memory that we can see being released by the kernel. The same cannot be held true for a single loadable module which may be releasing just a few, virtually contiguous memory. It's crystal clear ;-) Nice explanation. It's important to add something to clearify a bit your explanation (please correct me if I'm wrong): When Amar is talking about virtually contiguous kernel memory he implies that this memory is physically *dis*contiguous, i.e. based on page-entries. This is the kind of memory used for loadable modules, for instance, modules that get loaded with modprobe. On the other hand, built-in modules are compiled *inside* the kernel image (bzImage). The memory used for this image is physically contiguous: it's a big contiguous block of memory pages. Contiguous memory is important for kernel, and therefore is *very* important to spend some effort minimizing it. Ezequiel, Thanks for adding the clarification here :-) Regards, -Amar Regards, Ezequiel. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
Hey wanny, On Tue, Aug 14, 2012 at 1:04 AM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: Hi.. :) On Tue, Aug 14, 2012 at 9:14 AM, 王哲 wangzhe5...@gmail.com wrote: i use the __init for function print_k. in my opinion after the fisrt invoking the print_k in the hello_init. the memory of print_k will be freed,and the second invoking will not be executed.but the result of second invoking is executing . why? because you're still in module_init :) Yes, and the function is still in the stack! On the other hand.. think what would happen if things would work like you say. We would have a *very* strange behavior, and pretty counter-intuitive, don't you think so? ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
2012/8/14 Ezequiel Garcia elezegar...@gmail.com Hey wanny, On Tue, Aug 14, 2012 at 1:04 AM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: Hi.. :) On Tue, Aug 14, 2012 at 9:14 AM, 王哲 wangzhe5...@gmail.com wrote: i use the __init for function print_k. in my opinion after the fisrt invoking the print_k in the hello_init. the memory of print_k will be freed,and the second invoking will not be executed.but the result of second invoking is executing . why? because you're still in module_init :) Yes, and the function is still in the stack! On the other hand.. think what would happen if things would work like you say. We would have a *very* strange behavior, and pretty counter-intuitive, don't you think so? Thank you very much for reply. as you say,and function is still in the stack,don't be freed in the memory,what is __init had done? who can give a sample example to explain the difference existing __init or not? ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
On Tue, Aug 14, 2012 at 8:46 AM, 王哲 wangzhe5...@gmail.com wrote: 2012/8/14 Ezequiel Garcia elezegar...@gmail.com Hey wanny, On Tue, Aug 14, 2012 at 1:04 AM, Mulyadi Santosa mulyadi.sant...@gmail.com wrote: Hi.. :) On Tue, Aug 14, 2012 at 9:14 AM, 王哲 wangzhe5...@gmail.com wrote: i use the __init for function print_k. in my opinion after the fisrt invoking the print_k in the hello_init. the memory of print_k will be freed,and the second invoking will not be executed.but the result of second invoking is executing . why? because you're still in module_init :) Yes, and the function is still in the stack! On the other hand.. think what would happen if things would work like you say. We would have a *very* strange behavior, and pretty counter-intuitive, don't you think so? Thank you very much for reply. as you say,and function is still in the stack,don't be freed in the memory,what is __init had done? who can give a sample example to explain the difference existing __init or not? This is documented in Documentation/DocBook/ After boot, the kernel frees up a special section; functions marked with __init and data structures marked with __initdata are dropped after boot is complete: similarly modules discard this memory after initialization. __exit is used to declare a function which is only required on exit: the function will be dropped if this file is not compiled as a module. See the header file for use. Note that it makes no sense for a function marked with __init to be exported to modules with EXPORT_SYMBOL() - this will break. I'll try to put it in plain english, I'm sure someone will correct me if I mess up: When the kernel starts (machine boots), kernel code and data are loaded onto RAM*. But some code and some data is only needed for initialization, and there's no point on keeping them around after this stage is done. So it gets dropped, i.e. it's removed from RAM. But of course, this is done when the kernel knows it won't need it anymore. Since a module is like a little piece of kernel that gets loaded dynamically, I guess you can think it work more or less the same way. Hope it's clear now (and hope it's correct :-) Ezequiel. * This is pretty much like any executable binary, except kernel won't page out so it has to be completely loaded in RAM. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: How to understand the macro __init?
Hi.. :) On Tue, Aug 14, 2012 at 9:14 AM, 王哲 wangzhe5...@gmail.com wrote: i use the __init for function print_k. in my opinion after the fisrt invoking the print_k in the hello_init. the memory of print_k will be freed,and the second invoking will not be executed.but the result of second invoking is executing . why? because you're still in module_init :) right after modul init stage is done, _init marked function is thrown away... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
