Re: Learning Linux Kernel Development

[o]

  10.10.2018, 19:36, "Carter Cheng" : -- Forwarded message -From: Carter Cheng Date: Wed, Oct 10, 2018 at 10:52 PMSubject: Learning Linux Kernel DevelopmentTo:  Hello,  Hello,  I am actually kind of interested in learning linux kernel development and have been over the last few weeks going through Love's Linux Kernel Development book which details the situation in the kernel around linux 2.6. Obviously for a book of such limited length he can probably only go into some of the details and give a taste of what is actually is going on. I have a couple questions however and I was wondering if this was the correct place to ask-   The way to go deeper is to understand the operating system basis and this requires good knowledge of C and hardware and kernel experience. 1. After finishing the book and perhaps Understanding the Linux Kernel and Linux Device Drivers. What is the best way to dig deeper. This driver is just one of the things that Linux developers work. 2. Is there some good way to figure out how to update knowledge gained from this book to what is in the 4.x series of kernels?  I think all C code-based drivers will work on all Linux versions after downloading kernel please checkout the staging/ folder. Thanks,  Regards Carter___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Learning Linux Kernel Development

[Carter Cheng]

Thanks a lot.

On Thu, Oct 11, 2018 at 6:42 PM  wrote:

>
>
> 10.10.2018, 19:36, "Carter Cheng" :
>
>
> -- Forwarded message -
> From: *Carter Cheng* 
> Date: Wed, Oct 10, 2018 at 10:52 PM
> Subject: Learning Linux Kernel Development
> To: 
>
> Hello,
>
>
>
> Hello,
>
>
>
> I am actually kind of interested in learning linux kernel development and
> have been over the last few weeks going through Love's Linux Kernel
> Development book which details the situation in the kernel around linux
> 2.6. Obviously for a book of such limited length he can probably only go
> into some of the details and give a taste of what is actually is going on.
> I have a couple questions however and I was wondering if this was the
> correct place to ask-
>
>
>
>
>
> The way to go deeper is to understand the operating system basis and this
> requires good knowledge of C and hardware and kernel experience.
>
>
> 1. After finishing the book and perhaps Understanding the Linux Kernel and
> Linux Device Drivers. What is the best way to dig deeper.
>
>
> This driver is just one of the things that Linux developers work.
>
>
> 2. Is there some good way to figure out how to update knowledge gained
> from this book to what is in the 4.x series of kernels?
>
>
>
> I think all C code-based drivers will work on all Linux versions after
> downloading kernel please checkout the staging/ folder.
>
>
> Thanks,
>
>
>
> Regards
>
>
> Carter
>
>
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

link time analysis for the kernel.

[Carter Cheng]

Hi,

There are some detaills about the current procedures for linking the kernel
that I am unfamiliar with. My understanding is that GCC and Clang both have
the ability to do link time analysis and transforms on code but is it
possible to write link time passes that will run on the kernel since the
linking phase is a bit different (i.e. doesnt produce an ELF file)?

Regards,

Carter.
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: link time analysis for the kernel.

[o]

  11.10.2018, 17:48, "Carter Cheng" :Hi,  Hello, I want to ask pardon me and have you ever compiled a linux kernel?In my opinion you should first examine gcc ld and make process :) The elf format executable format and  the process after compiling the c code.Please read: http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html keep calm and go step by step and continue to learn c, gcc, ld, make, c code compile to machine code. Ozgur   There are some detaills about the current procedures for linking the kernel that I am unfamiliar with. My understanding is that GCC and Clang both have the ability to do link time analysis and transforms on code but is it possible to write link time passes that will run on the kernel since the linking phase is a bit different (i.e. doesnt produce an ELF file)? Regards, Carter.,___Kernelnewbies mailing listKernelnewbies@kernelnewbies.orghttps://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: link time analysis for the kernel.

[Carter Cheng]

Actually I have compiled and installed kernels before. I am wondering
however if LTO still works for compiling kernel images on clang or gcc
since my understanding is the kernel code includes a kernel loader which
loads the ELF format but the image of an OS kernel is loaded either
directly or via a bootloader which my understanding is cannot read ELF(is
this correct?).



On Fri, Oct 12, 2018 at 2:02 AM  wrote:

>
>
> 11.10.2018, 17:48, "Carter Cheng" :
>
> Hi,
>
>
>
> Hello,
>
> I want to ask pardon me and have you ever compiled a linux kernel?
> In my opinion you should first examine gcc ld and make process :)
>
> The elf format executable format and  the process after compiling the c
> code.
> Please read:
>
> http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html
>
> keep calm and go step by step and continue to learn c, gcc, ld, make, c
> code compile to machine code.
>
> Ozgur
>
>
>
>
> There are some detaills about the current procedures for linking the
> kernel that I am unfamiliar with. My understanding is that GCC and Clang
> both have the ability to do link time analysis and transforms on code but
> is it possible to write link time passes that will run on the kernel since
> the linking phase is a bit different (i.e. doesnt produce an ELF file)?
>
> Regards,
>
> Carter.
> ,
>
> ___
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: link time analysis for the kernel.

[Ozgur]

  11.10.2018, 22:55, "Carter Cheng" :Actually I have compiled and installed kernels before. I am wondering however if LTO still works for compiling kernel images on clang or gcc since my understanding is the kernel code includes a kernel loader which loads the ELF format but the image of an OS kernel is loaded either directly or via a bootloader which my understanding is cannot read ELF(is this correct?).  Hello,you are right my friend about this but you want can read through a tool such as readelf after executable.hexdump and readelf will help with read and analysis elf file :) OzgurOn Fri, Oct 12, 2018 at 2:02 AM  wrote:  11.10.2018, 17:48, "Carter Cheng" :Hi,  Hello, I want to ask pardon me and have you ever compiled a linux kernel?In my opinion you should first examine gcc ld and make process :) The elf format executable format and  the process after compiling the c code.Please read: http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html keep calm and go step by step and continue to learn c, gcc, ld, make, c code compile to machine code. Ozgur   There are some detaills about the current procedures for linking the kernel that I am unfamiliar with. My understanding is that GCC and Clang both have the ability to do link time analysis and transforms on code but is it possible to write link time passes that will run on the kernel since the linking phase is a bit different (i.e. doesnt produce an ELF file)? Regards, Carter.,___Kernelnewbies mailing listKernelnewbies@kernelnewbies.orghttps://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: link time analysis for the kernel.

[Ozgur]

  11.10.2018, 22:55, "Carter Cheng" :Actually I have compiled and installed kernels before. I am wondering however if LTO still works for compiling kernel images on clang or gcc since my understanding is the kernel code includes a kernel loader which loads the ELF format but the image of an OS kernel is loaded either directly or via a bootloader which my understanding is cannot read ELF(is this correct?).   Also if examine the phase-by phase source codecompilationobject filelinkingexecutable <- elf file creating I think you can analyze to the compilation and linking process so kernel will send it to the screen while it is already compile mesage. please read web site: http://www.cirosantilli.com/elf-hello-world/ Ozgur On Fri, Oct 12, 2018 at 2:02 AM  wrote:  11.10.2018, 17:48, "Carter Cheng" :Hi,  Hello, I want to ask pardon me and have you ever compiled a linux kernel?In my opinion you should first examine gcc ld and make process :) The elf format executable format and  the process after compiling the c code.Please read: http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html keep calm and go step by step and continue to learn c, gcc, ld, make, c code compile to machine code. Ozgur   There are some detaills about the current procedures for linking the kernel that I am unfamiliar with. My understanding is that GCC and Clang both have the ability to do link time analysis and transforms on code but is it possible to write link time passes that will run on the kernel since the linking phase is a bit different (i.e. doesnt produce an ELF file)? Regards, Carter.,___Kernelnewbies mailing listKernelnewbies@kernelnewbies.orghttps://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: link time analysis for the kernel.

[Carter Cheng]

sorry not a kernel loader but an ELF loader.

On Fri, Oct 12, 2018 at 2:55 AM Carter Cheng  wrote:

> Actually I have compiled and installed kernels before. I am wondering
> however if LTO still works for compiling kernel images on clang or gcc
> since my understanding is the kernel code includes a kernel loader which
> loads the ELF format but the image of an OS kernel is loaded either
> directly or via a bootloader which my understanding is cannot read ELF(is
> this correct?).
>
>
>
> On Fri, Oct 12, 2018 at 2:02 AM  wrote:
>
>>
>>
>> 11.10.2018, 17:48, "Carter Cheng" :
>>
>> Hi,
>>
>>
>>
>> Hello,
>>
>> I want to ask pardon me and have you ever compiled a linux kernel?
>> In my opinion you should first examine gcc ld and make process :)
>>
>> The elf format executable format and  the process after compiling the c
>> code.
>> Please read:
>>
>> http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html
>>
>> keep calm and go step by step and continue to learn c, gcc, ld, make, c
>> code compile to machine code.
>>
>> Ozgur
>>
>>
>>
>>
>> There are some detaills about the current procedures for linking the
>> kernel that I am unfamiliar with. My understanding is that GCC and Clang
>> both have the ability to do link time analysis and transforms on code but
>> is it possible to write link time passes that will run on the kernel since
>> the linking phase is a bit different (i.e. doesnt produce an ELF file)?
>>
>> Regards,
>>
>> Carter.
>> ,
>>
>> ___
>> Kernelnewbies mailing list
>> Kernelnewbies@kernelnewbies.org
>> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>>
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Learning Linux Kernel Development

[valdis . kletnieks]

On Thu, 11 Oct 2018 14:42:56 +0400, o...@goosey.org said:
> 10.10.2018, 19:36, "Carter Cheng" :
> >2. Is there some good way to figure out how to update knowledge gained 
> > from
> >this book to what is in the 4.x series of kernels?

> I think all C code-based drivers will work on all Linux versions after
> downloading kernel please checkout the staging/ folder.

Nowhere close.  Working on "all Linux versions" would imply a stable API.

The internal kernel APIs change all the time - a non-toy driver written for a
2.6.10 kernel has little to no chance of working on 4.19 without updating to
use the APIs presented by the 4.19 kernel.

See Documentation/process/stable-api-nonsense.rst for the details.


pgp57mAUy8HMr.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: link time analysis for the kernel.

[valdis . kletnieks]

On Thu, 11 Oct 2018 21:45:16 +0800, Carter Cheng said:

> There are some detaills about the current procedures for linking the kernel
> that I am unfamiliar with. My understanding is that GCC and Clang both have
> the ability to do link time analysis and transforms on code but is it
> possible to write link time passes that will run on the kernel since the
> linking phase is a bit different (i.e. doesnt produce an ELF file)?

The fact that the kernel gets linked is an existence proof that it is possible
to do link time processing on the kernel.

There's no LTO support in the stock 4.19 tree, but Andi Kleen did a patchset
for 4.15, and there's another patchset to enable LTO when using Clang rather
than gcc. (I haven't tried either one, don't use on a production machine, as
the resulting kernel may crash, eat filesystems, and/or turn your dog green...)

Note that 'vmlinux' is a statically linked ELF binary. That  plus a bootstrap
code gets merged to create a bzImage or similar thing that can be loaded by
Grub2 or whatever boot loader.



pgpDhe8KBlmOu.pgp
Description: PGP signature
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: link time analysis for the kernel.

[Carter Cheng]

I managed to find some information on this from Prof John Criswell who did
something similar for his dissertation but I do wonder how complicated the
make files for building the kernel are since the method he used would
require using llvm-link to stitch together all the different object
files(bitcode) into a single file and then convert them into machine code
at one go.

On Fri, Oct 12, 2018 at 10:20 AM  wrote:

> On Thu, 11 Oct 2018 21:45:16 +0800, Carter Cheng said:
>
> > There are some detaills about the current procedures for linking the
> kernel
> > that I am unfamiliar with. My understanding is that GCC and Clang both
> have
> > the ability to do link time analysis and transforms on code but is it
> > possible to write link time passes that will run on the kernel since the
> > linking phase is a bit different (i.e. doesnt produce an ELF file)?
>
> The fact that the kernel gets linked is an existence proof that it is
> possible
> to do link time processing on the kernel.
>
> There's no LTO support in the stock 4.19 tree, but Andi Kleen did a
> patchset
> for 4.15, and there's another patchset to enable LTO when using Clang
> rather
> than gcc. (I haven't tried either one, don't use on a production machine,
> as
> the resulting kernel may crash, eat filesystems, and/or turn your dog
> green...)
>
> Note that 'vmlinux' is a statically linked ELF binary. That  plus a
> bootstrap
> code gets merged to create a bzImage or similar thing that can be loaded by
> Grub2 or whatever boot loader.
>
>
___
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies