Re: link time analysis for the kernel.
12.10.2018, 10:50, "Carter Cheng" :I managed to find some information on this from Prof John Criswell who did something similar for his dissertation but I do wonder how complicated the make files for building the kernel are since the method he used would require using llvm-link to stitch together all the different object files(bitcode) into a single file and then convert them into machine code at one go. Hello, I don't think it turns into machine code at once and at the bottom Dear Valdis say could be analyzed with a patchset and LTO support last time.All object files return step by step to the machine code and I think we need to understand Andi Kleen's patchset logic. https://github.com/SveSop/Kernel_LTO I'll look at the evening as well. Regards Ozgur On Fri, Oct 12, 2018 at 10:20 AMwrote:On Thu, 11 Oct 2018 21:45:16 +0800, Carter Cheng said:> There are some detaills about the current procedures for linking the kernel> that I am unfamiliar with. My understanding is that GCC and Clang both have> the ability to do link time analysis and transforms on code but is it> possible to write link time passes that will run on the kernel since the> linking phase is a bit different (i.e. doesnt produce an ELF file)?The fact that the kernel gets linked is an existence proof that it is possibleto do link time processing on the kernel.There's no LTO support in the stock 4.19 tree, but Andi Kleen did a patchsetfor 4.15, and there's another patchset to enable LTO when using Clang ratherthan gcc. (I haven't tried either one, don't use on a production machine, asthe resulting kernel may crash, eat filesystems, and/or turn your dog green...)Note that 'vmlinux' is a statically linked ELF binary. That plus a bootstrapcode gets merged to create a bzImage or similar thing that can be loaded byGrub2 or whatever boot loader.___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: link time analysis for the kernel.
I managed to find some information on this from Prof John Criswell who did something similar for his dissertation but I do wonder how complicated the make files for building the kernel are since the method he used would require using llvm-link to stitch together all the different object files(bitcode) into a single file and then convert them into machine code at one go. On Fri, Oct 12, 2018 at 10:20 AM wrote: > On Thu, 11 Oct 2018 21:45:16 +0800, Carter Cheng said: > > > There are some detaills about the current procedures for linking the > kernel > > that I am unfamiliar with. My understanding is that GCC and Clang both > have > > the ability to do link time analysis and transforms on code but is it > > possible to write link time passes that will run on the kernel since the > > linking phase is a bit different (i.e. doesnt produce an ELF file)? > > The fact that the kernel gets linked is an existence proof that it is > possible > to do link time processing on the kernel. > > There's no LTO support in the stock 4.19 tree, but Andi Kleen did a > patchset > for 4.15, and there's another patchset to enable LTO when using Clang > rather > than gcc. (I haven't tried either one, don't use on a production machine, > as > the resulting kernel may crash, eat filesystems, and/or turn your dog > green...) > > Note that 'vmlinux' is a statically linked ELF binary. That plus a > bootstrap > code gets merged to create a bzImage or similar thing that can be loaded by > Grub2 or whatever boot loader. > > ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: link time analysis for the kernel.
On Thu, 11 Oct 2018 21:45:16 +0800, Carter Cheng said: > There are some detaills about the current procedures for linking the kernel > that I am unfamiliar with. My understanding is that GCC and Clang both have > the ability to do link time analysis and transforms on code but is it > possible to write link time passes that will run on the kernel since the > linking phase is a bit different (i.e. doesnt produce an ELF file)? The fact that the kernel gets linked is an existence proof that it is possible to do link time processing on the kernel. There's no LTO support in the stock 4.19 tree, but Andi Kleen did a patchset for 4.15, and there's another patchset to enable LTO when using Clang rather than gcc. (I haven't tried either one, don't use on a production machine, as the resulting kernel may crash, eat filesystems, and/or turn your dog green...) Note that 'vmlinux' is a statically linked ELF binary. That plus a bootstrap code gets merged to create a bzImage or similar thing that can be loaded by Grub2 or whatever boot loader. pgpDhe8KBlmOu.pgp Description: PGP signature ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: link time analysis for the kernel.
sorry not a kernel loader but an ELF loader. On Fri, Oct 12, 2018 at 2:55 AM Carter Cheng wrote: > Actually I have compiled and installed kernels before. I am wondering > however if LTO still works for compiling kernel images on clang or gcc > since my understanding is the kernel code includes a kernel loader which > loads the ELF format but the image of an OS kernel is loaded either > directly or via a bootloader which my understanding is cannot read ELF(is > this correct?). > > > > On Fri, Oct 12, 2018 at 2:02 AM wrote: > >> >> >> 11.10.2018, 17:48, "Carter Cheng" : >> >> Hi, >> >> >> >> Hello, >> >> I want to ask pardon me and have you ever compiled a linux kernel? >> In my opinion you should first examine gcc ld and make process :) >> >> The elf format executable format and the process after compiling the c >> code. >> Please read: >> >> http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html >> >> keep calm and go step by step and continue to learn c, gcc, ld, make, c >> code compile to machine code. >> >> Ozgur >> >> >> >> >> There are some detaills about the current procedures for linking the >> kernel that I am unfamiliar with. My understanding is that GCC and Clang >> both have the ability to do link time analysis and transforms on code but >> is it possible to write link time passes that will run on the kernel since >> the linking phase is a bit different (i.e. doesnt produce an ELF file)? >> >> Regards, >> >> Carter. >> , >> >> ___ >> Kernelnewbies mailing list >> Kernelnewbies@kernelnewbies.org >> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies >> >> ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: link time analysis for the kernel.
11.10.2018, 22:55, "Carter Cheng" :Actually I have compiled and installed kernels before. I am wondering however if LTO still works for compiling kernel images on clang or gcc since my understanding is the kernel code includes a kernel loader which loads the ELF format but the image of an OS kernel is loaded either directly or via a bootloader which my understanding is cannot read ELF(is this correct?). Also if examine the phase-by phase source codecompilationobject filelinkingexecutable <- elf file creating I think you can analyze to the compilation and linking process so kernel will send it to the screen while it is already compile mesage. please read web site: http://www.cirosantilli.com/elf-hello-world/ Ozgur On Fri, Oct 12, 2018 at 2:02 AMwrote: 11.10.2018, 17:48, "Carter Cheng" :Hi, Hello, I want to ask pardon me and have you ever compiled a linux kernel?In my opinion you should first examine gcc ld and make process :) The elf format executable format and the process after compiling the c code.Please read: http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html keep calm and go step by step and continue to learn c, gcc, ld, make, c code compile to machine code. Ozgur There are some detaills about the current procedures for linking the kernel that I am unfamiliar with. My understanding is that GCC and Clang both have the ability to do link time analysis and transforms on code but is it possible to write link time passes that will run on the kernel since the linking phase is a bit different (i.e. doesnt produce an ELF file)? Regards, Carter.,___Kernelnewbies mailing listKernelnewbies@kernelnewbies.orghttps://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: link time analysis for the kernel.
11.10.2018, 22:55, "Carter Cheng" :Actually I have compiled and installed kernels before. I am wondering however if LTO still works for compiling kernel images on clang or gcc since my understanding is the kernel code includes a kernel loader which loads the ELF format but the image of an OS kernel is loaded either directly or via a bootloader which my understanding is cannot read ELF(is this correct?). Hello,you are right my friend about this but you want can read through a tool such as readelf after executable.hexdump and readelf will help with read and analysis elf file :) OzgurOn Fri, Oct 12, 2018 at 2:02 AMwrote: 11.10.2018, 17:48, "Carter Cheng" :Hi, Hello, I want to ask pardon me and have you ever compiled a linux kernel?In my opinion you should first examine gcc ld and make process :) The elf format executable format and the process after compiling the c code.Please read: http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html keep calm and go step by step and continue to learn c, gcc, ld, make, c code compile to machine code. Ozgur There are some detaills about the current procedures for linking the kernel that I am unfamiliar with. My understanding is that GCC and Clang both have the ability to do link time analysis and transforms on code but is it possible to write link time passes that will run on the kernel since the linking phase is a bit different (i.e. doesnt produce an ELF file)? Regards, Carter.,___Kernelnewbies mailing listKernelnewbies@kernelnewbies.orghttps://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: link time analysis for the kernel.
Actually I have compiled and installed kernels before. I am wondering however if LTO still works for compiling kernel images on clang or gcc since my understanding is the kernel code includes a kernel loader which loads the ELF format but the image of an OS kernel is loaded either directly or via a bootloader which my understanding is cannot read ELF(is this correct?). On Fri, Oct 12, 2018 at 2:02 AM wrote: > > > 11.10.2018, 17:48, "Carter Cheng" : > > Hi, > > > > Hello, > > I want to ask pardon me and have you ever compiled a linux kernel? > In my opinion you should first examine gcc ld and make process :) > > The elf format executable format and the process after compiling the c > code. > Please read: > > http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html > > keep calm and go step by step and continue to learn c, gcc, ld, make, c > code compile to machine code. > > Ozgur > > > > > There are some detaills about the current procedures for linking the > kernel that I am unfamiliar with. My understanding is that GCC and Clang > both have the ability to do link time analysis and transforms on code but > is it possible to write link time passes that will run on the kernel since > the linking phase is a bit different (i.e. doesnt produce an ELF file)? > > Regards, > > Carter. > , > > ___ > Kernelnewbies mailing list > Kernelnewbies@kernelnewbies.org > https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies > > ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: link time analysis for the kernel.
11.10.2018, 17:48, "Carter Cheng" :Hi, Hello, I want to ask pardon me and have you ever compiled a linux kernel?In my opinion you should first examine gcc ld and make process :) The elf format executable format and the process after compiling the c code.Please read: http://www.ntu.edu.sg/home/ehchua/programming/cpp/gcc_make.html keep calm and go step by step and continue to learn c, gcc, ld, make, c code compile to machine code. Ozgur There are some detaills about the current procedures for linking the kernel that I am unfamiliar with. My understanding is that GCC and Clang both have the ability to do link time analysis and transforms on code but is it possible to write link time passes that will run on the kernel since the linking phase is a bit different (i.e. doesnt produce an ELF file)? Regards, Carter.,___Kernelnewbies mailing listKernelnewbies@kernelnewbies.orghttps://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies