[PATCH] lto: Bypass assembler when generating LTO object files. & libiberty: lto: Addition of .symtab in elf file.
This series of two patches enables the output of the LTO object file
without an assembler. As of now, .symtab is emitted with __gnu_lto_slim
symbol. To test, follow the instructions in the commit
message of patch 1. Also, as suggested by Honza, I am putting these patches
on devel/bypass-asm branch.
>From 46c766d242fd248abc49201cf6419735c8415a6f Mon Sep 17 00:00:00 2001
From: Rishi Raj
Date: Sat, 1 Jul 2023 10:28:11 +0530
Subject: [PATCH 1/2] lto: Bypass assembler when generating LTO object files.
This patch applies Jan Hubicka previous patch on current sources.
Now the compiler is able to produce object files without assembler,
although
a lot of things are missing, such as __lto_slim symbol, debug symbols
etc. They will be added in future patches. To test this current patch,
use these commands below.
1) ./xgcc -B ./ -O3 a.c -flto -S -fbypass-asm=crtbegin.o -o a.o
2) ./xgcc -B ./ -O2 a.o -flto
3) ./a.out
We are currently working with elf-only support (mach-o, coff, xcoff etc
will be dealt later) so this will only work on a linux machine. I have
tested
this on my machine ( Arch linux, Machine: Advanced Micro Devices
X86-64) and
all LTO test cases passed as expected.
gcc/ChangeLog:
* Makefile.in:
* common.opt:
* langhooks.cc (lhd_begin_section):
(lhd_append_data):
(lhd_end_section):
* lto/lto-object.cc: Moved to...
* lto-object.cc: ...here.
* lto-streamer.h (struct lto_section_slot):
(struct lto_section_list):
(struct lto_file):
(lto_obj_file_open):
(lto_obj_file_close):
(lto_obj_build_section_table):
(lto_obj_create_section_hash_table):
(lto_obj_begin_section):
(lto_obj_append_data):
(lto_obj_end_section):
(lto_set_current_out_file):
(lto_get_current_out_file):
* toplev.cc (compile_file):
(lang_dependent_init):
gcc/lto/ChangeLog:
* Make-lang.in:
* lto-common.cc (lto_file_read):
* lto-lang.cc:
* lto.h (struct lto_file):
(lto_obj_file_open):
(lto_obj_file_close):
(struct lto_section_list):
(lto_obj_build_section_table):
(lto_obj_create_section_hash_table):
(lto_obj_begin_section):
(lto_obj_append_data):
(lto_obj_end_section):
(lto_set_current_out_file):
(lto_get_current_out_file):
(struct lto_section_slot):
Signed-off-by: Rishi Raj
---
gcc/Makefile.in | 1 +
gcc/common.opt | 3 +++
gcc/langhooks.cc| 29 +++-
gcc/{lto => }/lto-object.cc | 29 +---
gcc/lto-streamer.h | 35 ++
gcc/lto/Make-lang.in| 4 ++--
gcc/lto/lto-common.cc | 3 ++-
gcc/lto/lto-lang.cc | 1 +
gcc/lto/lto.h | 38 -
gcc/toplev.cc | 19 ---
10 files changed, 110 insertions(+), 52 deletions(-)
rename gcc/{lto => }/lto-object.cc (94%)
diff --git a/gcc/Makefile.in b/gcc/Makefile.in
index c478ec85201..c9ae222fb59 100644
--- a/gcc/Makefile.in
+++ b/gcc/Makefile.in
@@ -1560,6 +1560,7 @@ OBJS = \
lto-section-out.o \
lto-opts.o \
lto-compress.o \
+ lto-object.o \
mcf.o \
mode-switching.o \
modulo-sched.o \
diff --git a/gcc/common.opt b/gcc/common.opt
index 25f650e2dae..ba7a18ece8c 100644
--- a/gcc/common.opt
+++ b/gcc/common.opt
@@ -1169,6 +1169,9 @@ fbtr-bb-exclusive
Common Ignore
Does nothing. Preserved for backward compatibility.
+fbypass-asm=
+Common Joined Var(flag_bypass_asm)
+
fcallgraph-info
Common RejectNegative Var(flag_callgraph_info) Init(NO_CALLGRAPH_INFO);
Output callgraph information on a per-file basis.
diff --git a/gcc/langhooks.cc b/gcc/langhooks.cc
index 9a1a9eccca9..a76ed974d58 100644
--- a/gcc/langhooks.cc
+++ b/gcc/langhooks.cc
@@ -38,6 +38,10 @@ along with GCC; see the file COPYING3. If not see
#include "stor-layout.h"
#include "cgraph.h"
#include "debug.h"
+#include "function.h"
+#include "basic-block.h"
+#include "gimple.h"
+#include "lto-streamer.h"
/* Do nothing; in many cases the default hook. */
@@ -817,6 +821,19 @@ lhd_begin_section (const char *name)
{
section *section;
+ if (flag_bypass_asm)
+{
+ static int initialized = false;
+ if (!initialized)
+ {
+ gcc_assert (asm_out_file == NULL);
+ lto_set_current_out_file (lto_obj_file_open (asm_file_name,
true));
+ initialized = true;
+ }
+ lto_obj_begin_section (name);
+ return;
+}
+
/* Save the old section so we can restore it in lto_end_asm_section. */
gcc_assert (!saved_section);
saved
Re: [GSOC] few question about Bypass assembler when generating LTO object files
stead of > > > > > assembler files (the current approach involves passing the -S and -o > > > > > .o options) and also skip the assembler option while > > using > > > > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed > > some > > > > > light on this? > > > > Yes, compiler drivers decides what to do and it needs to know that with > > > > -flto it does not need to produce assembly file and then invoke gas. > > If > > > > we go the way of reading in crtbegin.o it will also need to pass > > correct > > > > crtbegin to *1 binary. This is generally not that hard to do, just > > > > needs to be done :) > > > > > > > Honza > > > > > > > > > > Thanks & Regards > > > > > > > > > > Rishi Raj > > > > > > > > > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj > > wrote: > > > > > > > > > > > Hii Everyone, > > > > > > I had already expressed my interest in the " Bypass assembler when > > > > > > generating LTO object files" project and making a proposal for the > > > > same. I > > > > > > know I should have done it earlier but I was admitted to the > > hospital > > > > for > > > > > > past few days :(. > > > > > > I have a few doubts. > > > > > > 1) > > > > > > > > > > > > "One problem is that the object files produced by > > > > libiberty/simple-object.c > > > > > > (which is the low-level API used by the LTO code) > > > > > > are missing some information (such as the architecture info and > > symbol > > > > > > table) and API of the simple object will need to be extended to > > handle > > > > > > that" I found this in the previous mailing list discussion. So who > > > > output this information currently in the object file, is it assembler? > > > > > > > > > > > > Also in the current patch for this project by Jan Hubica, from > > where > > > > are we getting these information from? Is it from crtbegin.o? > > > > > > > > > > > > 2) > > > > > > "Support in driver to properly execute *1 binary." I found this on > > Jan > > > > original patch's email. what does it mean > > > > > > > > > > > > exactly? > > > > > > > > > > > > Regards > > > > > > > > > > > > Rishi Raj > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [GSOC] few question about Bypass assembler when generating LTO object files
Thanks to Martin, Honza, and Théo for your feedback. I have incorporated almost all of it, updated my proposal accordingly, and submitted it. Regarding grammar errors, I have fixed many, but there may still be some left (I could be better at grammar, to be honest :( ). On Tue, 4 Apr 2023 at 15:55, Jan Hubicka wrote: > Hello, > > Thanks, Jan for the Reply! I have completed a draft proposal for this > > project. I will appreciate your's, Martin's, or anybody else feedback on > > the same. > > Here is the link to my proposal > > > https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing > > Here are few comments on the proposal: > > > The current Implementation of GCC first write the IL representation > along with other section in an assembly file and then the assembler is used > to convert it into LTO object files. Sections containing different IL > representation is created and data is appended in lto-streamer-out.cc.I > > The .o generated withh -flto file contains the IL (in different > sections), debug info, symbol table, etc. > "along with other section" sounds odd to me. Perhaps sections. > > Second sentence seems bit odd too. Perhaps "Streaming of individual > sections is implemented in lto-streamer-out.cc which can either be used > to produce assembly code containing the section data (dumped > hexadecimally) or simple-object API provided by libiberty to produce > object files directly" > > > In the slim object file (Default when using -flto, fat lto can be > obtained using -ffat-lto-object) some section contains the IL and other > contains the info related to architecture, command line options, symbol > table, etc. > > Technically the architecture is part of ELF header and not section > itself (I think). > > There are some other grammar errors, but I am not too good on fixing > these, so perhaps Martin can help. > > The timeline looks reasonable. It certianly makes sense to look into > non-ELF object files to understand what API we need, but implementation > wise I would suggest implementing ELF path first to get a working > implementation. Adding support for other object formats can be done > incrementally. > > Honza > > > > On Tue, 4 Apr 2023 at 04:35, Jan Hubicka wrote: > > > > > Hello, > > > > While going through the patch and simple-object.c I understood that > the > > > > file simple-object.c is used to handle the object file format. > However, > > > > this file does not contain all the architecture information required > for > > > > LTO object files, so the workaround used in the patch is to read the > > > > crtbegin.o file and merge the missing attributes. While this > workaround > > > is > > > > functional, it is not optimal, and the ideal solution would be to > extend > > > > simple-object.c to include the missing information. > > > > > > Yes, simple-object.c simply uses architecture settings it read earlier > > > which is problem since at compile time we do not read any object files, > > > just parse sources). In my original patch the architecture flags were > > > simply left blank. I am not sure if there is a version reading > > > crtbeing.o which would probably not a be that bad workaround, at least > > > for the start. Having a way to specify this from the machine > descriptions > > > would be better. > > > > > > > > > > > > > Besides the architecture bits, for simple-object files to work we need > > > to add the symbol table. For practically useful information we also > need > > > to stream the debug info. > > > > > > > > > > Regarding the phrase "Support in the driver to properly execute *1 > > > binary", > > > > it is not entirely clear what it refers to. My interpretation is > that the > > > > compiler driver (the program that coordinates the compilation > process) > > > > needs to be modified to correctly output LTO object files instead of > > > > assembler files (the current approach involves passing the -S and -o > > > > .o options) and also skip the assembler option while > using > > > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed > some > > > > light on this? > > > Yes, compiler drivers decides what to do and it needs to know that with > > > -flto it does not need to produce assembly file and then invoke gas. > If > > > we go the way of reading in crtbegin.o it will also need to pa
Re: [GSOC] few question about Bypass assembler when generating LTO object files
Hello, > Thanks, Jan for the Reply! I have completed a draft proposal for this > project. I will appreciate your's, Martin's, or anybody else feedback on > the same. > Here is the link to my proposal > https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing Here are few comments on the proposal: > The current Implementation of GCC first write the IL representation along > with other section in an assembly file and then the assembler is used to > convert it into LTO object files. Sections containing different IL > representation is created and data is appended in lto-streamer-out.cc.I The .o generated withh -flto file contains the IL (in different sections), debug info, symbol table, etc. "along with other section" sounds odd to me. Perhaps sections. Second sentence seems bit odd too. Perhaps "Streaming of individual sections is implemented in lto-streamer-out.cc which can either be used to produce assembly code containing the section data (dumped hexadecimally) or simple-object API provided by libiberty to produce object files directly" > In the slim object file (Default when using -flto, fat lto can be obtained > using -ffat-lto-object) some section contains the IL and other contains the > info related to architecture, command line options, symbol table, etc. Technically the architecture is part of ELF header and not section itself (I think). There are some other grammar errors, but I am not too good on fixing these, so perhaps Martin can help. The timeline looks reasonable. It certianly makes sense to look into non-ELF object files to understand what API we need, but implementation wise I would suggest implementing ELF path first to get a working implementation. Adding support for other object formats can be done incrementally. Honza > > On Tue, 4 Apr 2023 at 04:35, Jan Hubicka wrote: > > > Hello, > > > While going through the patch and simple-object.c I understood that the > > > file simple-object.c is used to handle the object file format. However, > > > this file does not contain all the architecture information required for > > > LTO object files, so the workaround used in the patch is to read the > > > crtbegin.o file and merge the missing attributes. While this workaround > > is > > > functional, it is not optimal, and the ideal solution would be to extend > > > simple-object.c to include the missing information. > > > > Yes, simple-object.c simply uses architecture settings it read earlier > > which is problem since at compile time we do not read any object files, > > just parse sources). In my original patch the architecture flags were > > simply left blank. I am not sure if there is a version reading > > crtbeing.o which would probably not a be that bad workaround, at least > > for the start. Having a way to specify this from the machine descriptions > > would be better. > > > > > > > > Besides the architecture bits, for simple-object files to work we need > > to add the symbol table. For practically useful information we also need > > to stream the debug info. > > > > > > > Regarding the phrase "Support in the driver to properly execute *1 > > binary", > > > it is not entirely clear what it refers to. My interpretation is that the > > > compiler driver (the program that coordinates the compilation process) > > > needs to be modified to correctly output LTO object files instead of > > > assembler files (the current approach involves passing the -S and -o > > > .o options) and also skip the assembler option while using > > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some > > > light on this? > > Yes, compiler drivers decides what to do and it needs to know that with > > -flto it does not need to produce assembly file and then invoke gas. If > > we go the way of reading in crtbegin.o it will also need to pass correct > > crtbegin to *1 binary. This is generally not that hard to do, just > > needs to be done :) > > > Honza > > > > > > Thanks & Regards > > > > > > Rishi Raj > > > > > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj wrote: > > > > > > > Hii Everyone, > > > > I had already expressed my interest in the " Bypass assembler when > > > > generating LTO object files" project and making a proposal for the > > same. I > > > > know I should have done it earlier but I was admitted to the hospital > > for > > > > past few days :(. > > > > I have a few doubts. > > > > 1) > >
Re: [GSOC] few question about Bypass assembler when generating LTO object files
Hi, On Tue, Apr 04 2023, Rishi Raj wrote: > Thanks, Jan for the Reply! I have completed a draft proposal for this > project. I will appreciate your's, Martin's, or anybody else feedback on > the same. > Here is the link to my proposal > https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing > in my opinion, the proposal looks rather good. I don't know how to significantly improve it, not in the remaining nine hours before the deadline (just maybe fix the spelling of Honza's surname, it is Hubicka :-). So please go ahead and submit it. (How the selection goes will then depend on how many slots we get from Google). Thanks for putting in the effort, Martin > On Tue, 4 Apr 2023 at 04:35, Jan Hubicka wrote: > >> Hello, >> > While going through the patch and simple-object.c I understood that the >> > file simple-object.c is used to handle the object file format. However, >> > this file does not contain all the architecture information required for >> > LTO object files, so the workaround used in the patch is to read the >> > crtbegin.o file and merge the missing attributes. While this workaround >> is >> > functional, it is not optimal, and the ideal solution would be to extend >> > simple-object.c to include the missing information. >> >> Yes, simple-object.c simply uses architecture settings it read earlier >> which is problem since at compile time we do not read any object files, >> just parse sources). In my original patch the architecture flags were >> simply left blank. I am not sure if there is a version reading >> crtbeing.o which would probably not a be that bad workaround, at least >> for the start. Having a way to specify this from the machine descriptions >> would be better. >> > > >> >> Besides the architecture bits, for simple-object files to work we need >> to add the symbol table. For practically useful information we also need >> to stream the debug info. >> >> >> > Regarding the phrase "Support in the driver to properly execute *1 >> binary", >> > it is not entirely clear what it refers to. My interpretation is that the >> > compiler driver (the program that coordinates the compilation process) >> > needs to be modified to correctly output LTO object files instead of >> > assembler files (the current approach involves passing the -S and -o >> > .o options) and also skip the assembler option while using >> > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some >> > light on this? >> Yes, compiler drivers decides what to do and it needs to know that with >> -flto it does not need to produce assembly file and then invoke gas. If >> we go the way of reading in crtbegin.o it will also need to pass correct >> crtbegin to *1 binary. This is generally not that hard to do, just >> needs to be done :) >> > Honza >> > >> > Thanks & Regards >> > >> > Rishi Raj >> > >> > On Sun, 2 Apr 2023 at 03:05, Rishi Raj wrote: >> > >> > > Hii Everyone, >> > > I had already expressed my interest in the " Bypass assembler when >> > > generating LTO object files" project and making a proposal for the >> same. I >> > > know I should have done it earlier but I was admitted to the hospital >> for >> > > past few days :(. >> > > I have a few doubts. >> > > 1) >> > > >> > > "One problem is that the object files produced by >> libiberty/simple-object.c >> > > (which is the low-level API used by the LTO code) >> > > are missing some information (such as the architecture info and symbol >> > > table) and API of the simple object will need to be extended to handle >> > > that" I found this in the previous mailing list discussion. So who >> output this information currently in the object file, is it assembler? >> > > >> > > Also in the current patch for this project by Jan Hubica, from where >> are we getting these information from? Is it from crtbegin.o? >> > > >> > > 2) >> > > "Support in driver to properly execute *1 binary." I found this on Jan >> original patch's email. what does it mean >> > > >> > > exactly? >> > > >> > > Regards >> > > >> > > Rishi Raj >> > > >> > > >> > > >> > > >>
[GSOC] Submission of draft proposal for Bypass assembler when generating LTO object files
Sorry, I messed subject in my previous two emails :( so I am sending it again. I have completed a draft proposal for this project. I will appreciate Jan, Martin, or anybody else feedback on the same. Here is the link to my proposal https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing
Fwd: [GSOC] few question about Bypass assembler when generating LTO object files
-- Forwarded message - From: Rishi Raj Date: Tue, 4 Apr 2023 at 05:57 Subject: Re: [GSOC] Submission of draft proposal. To: Jan Hubicka Cc: , oops, I forgot to change the subject in previous email :( Thanks, Jan for the Reply! I have completed a draft proposal for this project. I will appreciate your's, Martin's, or anybody else feedback on the same. Here is the link to my proposal https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing On Tue, 4 Apr 2023 at 04:35, Jan Hubicka wrote: > Hello, > > While going through the patch and simple-object.c I understood that the > > file simple-object.c is used to handle the object file format. However, > > this file does not contain all the architecture information required for > > LTO object files, so the workaround used in the patch is to read the > > crtbegin.o file and merge the missing attributes. While this workaround > is > > functional, it is not optimal, and the ideal solution would be to extend > > simple-object.c to include the missing information. > > Yes, simple-object.c simply uses architecture settings it read earlier > which is problem since at compile time we do not read any object files, > just parse sources). In my original patch the architecture flags were > simply left blank. I am not sure if there is a version reading > crtbeing.o which would probably not a be that bad workaround, at least > for the start. Having a way to specify this from the machine descriptions > would be better. > > > Besides the architecture bits, for simple-object files to work we need > to add the symbol table. For practically useful information we also need > to stream the debug info. > > > > Regarding the phrase "Support in the driver to properly execute *1 > binary", > > it is not entirely clear what it refers to. My interpretation is that the > > compiler driver (the program that coordinates the compilation process) > > needs to be modified to correctly output LTO object files instead of > > assembler files (the current approach involves passing the -S and -o > > .o options) and also skip the assembler option while using > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some > > light on this? > Yes, compiler drivers decides what to do and it needs to know that with > -flto it does not need to produce assembly file and then invoke gas. If > we go the way of reading in crtbegin.o it will also need to pass correct > crtbegin to *1 binary. This is generally not that hard to do, just > needs to be done :) > Honza > > > > Thanks & Regards > > > > Rishi Raj > > > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj wrote: > > > > > Hii Everyone, > > > I had already expressed my interest in the " Bypass assembler when > > > generating LTO object files" project and making a proposal for the > same. I > > > know I should have done it earlier but I was admitted to the hospital > for > > > past few days :(. > > > I have a few doubts. > > > 1) > > > > > > "One problem is that the object files produced by > libiberty/simple-object.c > > > (which is the low-level API used by the LTO code) > > > are missing some information (such as the architecture info and symbol > > > table) and API of the simple object will need to be extended to handle > > > that" I found this in the previous mailing list discussion. So who > output this information currently in the object file, is it assembler? > > > > > > Also in the current patch for this project by Jan Hubica, from where > are we getting these information from? Is it from crtbegin.o? > > > > > > 2) > > > "Support in driver to properly execute *1 binary." I found this on Jan > original patch's email. what does it mean > > > > > > exactly? > > > > > > Regards > > > > > > Rishi Raj > > > > > > > > > > > > >
Re: [GSOC] few question about Bypass assembler when generating LTO object files
Thanks, Jan for the Reply! I have completed a draft proposal for this project. I will appreciate your's, Martin's, or anybody else feedback on the same. Here is the link to my proposal https://docs.google.com/document/d/1r9kzsU96kOYfIhWZx62jx4ALG-J_aJs5U0sDpwFUtts/edit?usp=sharing On Tue, 4 Apr 2023 at 04:35, Jan Hubicka wrote: > Hello, > > While going through the patch and simple-object.c I understood that the > > file simple-object.c is used to handle the object file format. However, > > this file does not contain all the architecture information required for > > LTO object files, so the workaround used in the patch is to read the > > crtbegin.o file and merge the missing attributes. While this workaround > is > > functional, it is not optimal, and the ideal solution would be to extend > > simple-object.c to include the missing information. > > Yes, simple-object.c simply uses architecture settings it read earlier > which is problem since at compile time we do not read any object files, > just parse sources). In my original patch the architecture flags were > simply left blank. I am not sure if there is a version reading > crtbeing.o which would probably not a be that bad workaround, at least > for the start. Having a way to specify this from the machine descriptions > would be better. > > > Besides the architecture bits, for simple-object files to work we need > to add the symbol table. For practically useful information we also need > to stream the debug info. > > > > Regarding the phrase "Support in the driver to properly execute *1 > binary", > > it is not entirely clear what it refers to. My interpretation is that the > > compiler driver (the program that coordinates the compilation process) > > needs to be modified to correctly output LTO object files instead of > > assembler files (the current approach involves passing the -S and -o > > .o options) and also skip the assembler option while using > > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some > > light on this? > Yes, compiler drivers decides what to do and it needs to know that with > -flto it does not need to produce assembly file and then invoke gas. If > we go the way of reading in crtbegin.o it will also need to pass correct > crtbegin to *1 binary. This is generally not that hard to do, just > needs to be done :) > Honza > > > > Thanks & Regards > > > > Rishi Raj > > > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj wrote: > > > > > Hii Everyone, > > > I had already expressed my interest in the " Bypass assembler when > > > generating LTO object files" project and making a proposal for the > same. I > > > know I should have done it earlier but I was admitted to the hospital > for > > > past few days :(. > > > I have a few doubts. > > > 1) > > > > > > "One problem is that the object files produced by > libiberty/simple-object.c > > > (which is the low-level API used by the LTO code) > > > are missing some information (such as the architecture info and symbol > > > table) and API of the simple object will need to be extended to handle > > > that" I found this in the previous mailing list discussion. So who > output this information currently in the object file, is it assembler? > > > > > > Also in the current patch for this project by Jan Hubica, from where > are we getting these information from? Is it from crtbegin.o? > > > > > > 2) > > > "Support in driver to properly execute *1 binary." I found this on Jan > original patch's email. what does it mean > > > > > > exactly? > > > > > > Regards > > > > > > Rishi Raj > > > > > > > > > > > > >
Re: [GSOC] few question about Bypass assembler when generating LTO object files
Hello, > While going through the patch and simple-object.c I understood that the > file simple-object.c is used to handle the object file format. However, > this file does not contain all the architecture information required for > LTO object files, so the workaround used in the patch is to read the > crtbegin.o file and merge the missing attributes. While this workaround is > functional, it is not optimal, and the ideal solution would be to extend > simple-object.c to include the missing information. Yes, simple-object.c simply uses architecture settings it read earlier which is problem since at compile time we do not read any object files, just parse sources). In my original patch the architecture flags were simply left blank. I am not sure if there is a version reading crtbeing.o which would probably not a be that bad workaround, at least for the start. Having a way to specify this from the machine descriptions would be better. Besides the architecture bits, for simple-object files to work we need to add the symbol table. For practically useful information we also need to stream the debug info. > > Regarding the phrase "Support in the driver to properly execute *1 binary", > it is not entirely clear what it refers to. My interpretation is that the > compiler driver (the program that coordinates the compilation process) > needs to be modified to correctly output LTO object files instead of > assembler files (the current approach involves passing the -S and -o > .o options) and also skip the assembler option while using > -fbypass-asm option but I am not sure. Can Jan or Martin please shed some > light on this? Yes, compiler drivers decides what to do and it needs to know that with -flto it does not need to produce assembly file and then invoke gas. If we go the way of reading in crtbegin.o it will also need to pass correct crtbegin to *1 binary. This is generally not that hard to do, just needs to be done :) Honza > > Thanks & Regards > > Rishi Raj > > On Sun, 2 Apr 2023 at 03:05, Rishi Raj wrote: > > > Hii Everyone, > > I had already expressed my interest in the " Bypass assembler when > > generating LTO object files" project and making a proposal for the same. I > > know I should have done it earlier but I was admitted to the hospital for > > past few days :(. > > I have a few doubts. > > 1) > > > > "One problem is that the object files produced by libiberty/simple-object.c > > (which is the low-level API used by the LTO code) > > are missing some information (such as the architecture info and symbol > > table) and API of the simple object will need to be extended to handle > > that" I found this in the previous mailing list discussion. So who output > > this information currently in the object file, is it assembler? > > > > Also in the current patch for this project by Jan Hubica, from where are we > > getting these information from? Is it from crtbegin.o? > > > > 2) > > "Support in driver to properly execute *1 binary." I found this on Jan > > original patch's email. what does it mean > > > > exactly? > > > > Regards > > > > Rishi Raj > > > > > > > >
Re: [GSOC] few question about Bypass assembler when generating LTO object files
While going through the patch and simple-object.c I understood that the file simple-object.c is used to handle the object file format. However, this file does not contain all the architecture information required for LTO object files, so the workaround used in the patch is to read the crtbegin.o file and merge the missing attributes. While this workaround is functional, it is not optimal, and the ideal solution would be to extend simple-object.c to include the missing information. Regarding the phrase "Support in the driver to properly execute *1 binary", it is not entirely clear what it refers to. My interpretation is that the compiler driver (the program that coordinates the compilation process) needs to be modified to correctly output LTO object files instead of assembler files (the current approach involves passing the -S and -o .o options) and also skip the assembler option while using -fbypass-asm option but I am not sure. Can Jan or Martin please shed some light on this? Thanks & Regards Rishi Raj On Sun, 2 Apr 2023 at 03:05, Rishi Raj wrote: > Hii Everyone, > I had already expressed my interest in the " Bypass assembler when > generating LTO object files" project and making a proposal for the same. I > know I should have done it earlier but I was admitted to the hospital for > past few days :(. > I have a few doubts. > 1) > > "One problem is that the object files produced by libiberty/simple-object.c > (which is the low-level API used by the LTO code) > are missing some information (such as the architecture info and symbol > table) and API of the simple object will need to be extended to handle > that" I found this in the previous mailing list discussion. So who output > this information currently in the object file, is it assembler? > > Also in the current patch for this project by Jan Hubica, from where are we > getting these information from? Is it from crtbegin.o? > > 2) > "Support in driver to properly execute *1 binary." I found this on Jan > original patch's email. what does it mean > > exactly? > > Regards > > Rishi Raj > > > >
[GSOC] few question about Bypass assembler when generating LTO object files
Hii Everyone, I had already expressed my interest in the " Bypass assembler when generating LTO object files" project and making a proposal for the same. I know I should have done it earlier but I was admitted to the hospital for past few days :(. I have a few doubts. 1) "One problem is that the object files produced by libiberty/simple-object.c (which is the low-level API used by the LTO code) are missing some information (such as the architecture info and symbol table) and API of the simple object will need to be extended to handle that" I found this in the previous mailing list discussion. So who output this information currently in the object file, is it assembler? Also in the current patch for this project by Jan Hubica, from where are we getting these information from? Is it from crtbegin.o? 2) "Support in driver to properly execute *1 binary." I found this on Jan original patch's email. what does it mean exactly? Regards Rishi Raj
Re: GSoC'2023: Bypass assembler when generating LTO object files: GCC
Hello, please make sure you CC the mailing list in these communications. On Thu, Mar 16 2023, Madhu patel wrote: > Hi, > >>That is an impressive list. On a more specific note, do you have any - >> rudimentary is fine - background in the theory of compilers? Are you >> familiar with concepts like intermediate representation (intermediate >> language)? > > Yes! I have Built the GCC from source and Yes, I do have good knowledge of > compilers. We had this subject in the college curriculum where we studied > intermediate representation. I have good knowledge of C++, c, Java, and > Bash. etc. > >>Great. What are the issues you have encountered? I assume you have >>seen David's guide for newcomers too? If not, have a look at it at >>https://gcc-newbies-guide.readthedocs.io/en/latest/index.html > > Yes, I have built the GCC from scratch on my Linux Machine. > I have written the documentation for the same. And working on understanding > the details of offLoading. > https://www.dropbox.com/scl/fi/ffx7c29f0yfhvbiuvndku/GCC.paper?dl=0=76x89xkzz3h41uemor9ksw6sl Just a note, using images to capture terminal (instead of the text that is there) is a very bad practice. (The configure step is also quite conspicuously missing in your text, why?) > And gone through the > https://gcc-newbies-guide.readthedocs.io/en/latest/index.html > <https://gcc-newbies-guide.readthedocs.io/en/latest/index.html> and did a > basic setup for GCC on my Linux machine. > > Can we schedule a meeting to discuss the project's timeline and specific > tasks that I can take on? > Please let me know your availability. Sorry but no, I am not able to and do not intend to schedule calls with GSoC applicants. I'll be happy to discuss stuff on the mailing-list. Estimating a timeline and coming up with (at least some) milestones of the project is part of the exercise of writing a proposal. GSoC contributors need to demonstrate a certain level of independent thought and initiative - though of course the community is always there to help with specific questions and difficulties. The project has been discussed already in https://gcc.gnu.org/pipermail/gcc/2023-March/240833.html and this message should be enough to get you started. Good luck, Martin > > > On Tue, Mar 14, 2023 at 10:48 PM Martin Jambor wrote: > >> Hello, >> >> We are delighted you found looking into GCC interesting. >> >> On Thu, Mar 09 2023, Madhu patel via Gcc wrote: >> > Hi Jan, >> > >> > I'm interested in working on the project `Bypass assembler when >> generating >> > LTO object files` in the GCC organization through GSoC'2023. >> >> Great, please note that the task as already been discussed on the >> mailing list and some information is available in the archives at >> >>https://gcc.gnu.org/pipermail/gcc/2023-March/240833.html >> >> > >> > I am Madhu Patel, a fourth-year B.Tech. student in Computer Science at >> > IGDTUW, with a CGPA of 8.7/10. I have previously interned at Adobe India, >> > Rabvik Innovations, and FM solutions, and I am currently a research >> intern >> > at IIT Roorkee. I am also working on a research paper on Linux Kernel >> > Evolution for the USENIX publication. Moreover, my research paper on >> Stock >> > Price Prediction was recently accepted at the IEEE Conference. You can >> find >> > more information about my work on my LinkedIn and GitHub profiles. >> >> That is an impressive list. On a more specific note, do you have any - >> rudimentary is fine - background in the theory of compilers? Are you >> familiar with concepts like intermediate representation (intermediate >> language)? >> >> > >> > Please suggest a few initial tasks I can work on during the application >> > period and attach them to my application. I have already prepared a >> > timeline/planner, great if you could have a look at it and suggest any >> > enhancements. Additionally, I have signed in to the mailing lists, and >> IRCs >> > and done the initial tasks as described on the project page [1] >> > <https://gcc.gnu.org/wiki/SummerOfCode>. >> >> Great. What are the issues you have encountered? I assume you have >> seen David's guide for newcomers too? If not, have a look at it at >> https://gcc-newbies-guide.readthedocs.io/en/latest/index.html >> >> I am afraid there are not very many small issues or problems that can be >> dealt with in a few weeks. Instead, I would encourage you to keep >> investigating the code, particularly around the areas described in the >> mailing list post I linked above, and start thinking about how you'd >> achieve the overall goal. Feel free to ask any specific questions you >> might have about the project and GCC development in general here. >> >> Good luck! >> >> Martin >> >>
Re: GSoC'2023: Bypass assembler when generating LTO object files: GCC
Hello, We are delighted you found looking into GCC interesting. On Thu, Mar 09 2023, Madhu patel via Gcc wrote: > Hi Jan, > > I'm interested in working on the project `Bypass assembler when generating > LTO object files` in the GCC organization through GSoC'2023. Great, please note that the task as already been discussed on the mailing list and some information is available in the archives at https://gcc.gnu.org/pipermail/gcc/2023-March/240833.html > > I am Madhu Patel, a fourth-year B.Tech. student in Computer Science at > IGDTUW, with a CGPA of 8.7/10. I have previously interned at Adobe India, > Rabvik Innovations, and FM solutions, and I am currently a research intern > at IIT Roorkee. I am also working on a research paper on Linux Kernel > Evolution for the USENIX publication. Moreover, my research paper on Stock > Price Prediction was recently accepted at the IEEE Conference. You can find > more information about my work on my LinkedIn and GitHub profiles. That is an impressive list. On a more specific note, do you have any - rudimentary is fine - background in the theory of compilers? Are you familiar with concepts like intermediate representation (intermediate language)? > > Please suggest a few initial tasks I can work on during the application > period and attach them to my application. I have already prepared a > timeline/planner, great if you could have a look at it and suggest any > enhancements. Additionally, I have signed in to the mailing lists, and IRCs > and done the initial tasks as described on the project page [1] > <https://gcc.gnu.org/wiki/SummerOfCode>. Great. What are the issues you have encountered? I assume you have seen David's guide for newcomers too? If not, have a look at it at https://gcc-newbies-guide.readthedocs.io/en/latest/index.html I am afraid there are not very many small issues or problems that can be dealt with in a few weeks. Instead, I would encourage you to keep investigating the code, particularly around the areas described in the mailing list post I linked above, and start thinking about how you'd achieve the overall goal. Feel free to ask any specific questions you might have about the project and GCC development in general here. Good luck! Martin
GSoC'2023: Bypass assembler when generating LTO object files: GCC
Hi Jan, I'm interested in working on the project `Bypass assembler when generating LTO object files` in the GCC organization through GSoC'2023. I am Madhu Patel, a fourth-year B.Tech. student in Computer Science at IGDTUW, with a CGPA of 8.7/10. I have previously interned at Adobe India, Rabvik Innovations, and FM solutions, and I am currently a research intern at IIT Roorkee. I am also working on a research paper on Linux Kernel Evolution for the USENIX publication. Moreover, my research paper on Stock Price Prediction was recently accepted at the IEEE Conference. You can find more information about my work on my LinkedIn and GitHub profiles. Please suggest a few initial tasks I can work on during the application period and attach them to my application. I have already prepared a timeline/planner, great if you could have a look at it and suggest any enhancements. Additionally, I have signed in to the mailing lists, and IRCs and done the initial tasks as described on the project page [1] <https://gcc.gnu.org/wiki/SummerOfCode>. Thanks, Madhu Patel LinkedIn <https://www.linkedin.com/in/madhu-patel/> | Github <https://github.com/Madhupatel08>
Re: Bypass assembler when generating LTO object files
Hello, On Sun, Dec 18 2022, Mohamed Atef wrote: > Hello, >I am interested in working in this project during my free time, is > understanding this https://gcc.gnu.org/wiki/LinkTimeOptimization > A good starting point That section of the Wiki is very old. You may find bits there that are still valid and relevant, but I would be actually a bit careful with that content. If you're looking for high-level overview of LTO, unfortunately I can only recommend videos: - Honza's "Building openSUSE with GCC's link time optimization" https://events.opensuse.org/conferences/oSC18/program/proposals/1846#2 - my "Interprodecural optimizations in GCC" https://www.youtube.com/watch?v=oQ71ZbOuSW4 (the first 12 minutes or so, the rest is then about optimizations) For the task specifically, the patch from 2014 https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html is still a good starting point, even if not a very clear one. The crux of the matter is to enhance libiberty/simple-object*.[ch] to be able to create elf from scratch (as opposed to modifying an existing one). So look there too. If you have any questions, feel free to ask. Good luck, Martin
Bypass assembler when generating LTO object files
Hello, I am interested in working in this project during my free time, is understanding this https://gcc.gnu.org/wiki/LinkTimeOptimization A good starting point Mohamed
Re: [GSoC]Bypass assembler when generating LTO object files
On Tue, Apr 12, 2022 at 2:53 PM Iain Sandoe wrote: > > > > > On 12 Apr 2022, at 13:31, Martin Liška wrote: > > > > On 4/12/22 11:58, Richard Biener wrote: > >> On Tue, Apr 12, 2022 at 11:20 AM Jan Hubicka via Gcc > >> wrote: > >>> > >>> Hi, > > > > On 08-Apr-2022, at 6:32 PM, Jan Hubicka wrote: > > > > Ankur, > >> I was browsing the list of submitted GSoC projects this year and the > >> project regarding bypassing assembler when generating LTO object files > >> caught my eye. > > I apologize for late reply. I would be very happy to mentor this > > project. > > Thanks for the reply, but unfortunately, due to some reasons, I would not > be able to take part in GSoC this year. > But the project seems interesting and would be amazing opportunity to > learn a lot more things for me, so would it be okay if I try to give it a > go outside GSoC if no-one else picks it as their GSoC project this year ? > >>> > >>> I would be still very happy to help with that! However it would be also > >>> pity to not take part of GSoC, so if there is something I can help with > >>> on that let me know. > > >> > >> I already have a gcc built from source (sync-ed with trunk/master) and > >> launched the test-suite on it. > >> > >> I am currently in process of understanding the primilary patch > >> (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and > >> experimenting with it. > >> > >> are there any other things I should be aware of (useful Doc/blog or a > >> bug tracking the project) before proceeding further ? > > > > I think it is pretty much all that exists. Basically we will need to > > implement everything that is necessary to stream out valid object file > > directly from GCC rather than going through gas. The experimental > > prototype sort of worked but it was lacking few things. > > When I try to apply that patch on my local branch ( branched from trunk > ), > it seem to be incompatible with the current working tree. Is there a > specific branch that I have to apply it to ? or is it due to the recent > file rename patch ( changing extensions from .c to .cc ) ? > > ``` > $ git apply --check bypass_asm_patch > > error: patch failed: Makefile.in:1300 > error: Makefile.in: patch does not apply > error: common.opt: No such file or directory > error: langhooks.c: No such file or directory > error: lto/Make-lang.in: No such file or directory > error: lto/lto-object.c: No such file or directory > error: lto/lto.c: No such file or directory > error: lto/lto.h: No such file or directory > error: lto-streamer.h: No such file or directory > error: toplev.c: No such file or directory > ``` > >>> > >>> I can try to update the patch, or it probably should apply to trunk > >>> checked out around the date I sent the patch. Indeed we need to change > >>> c to cc but there are likely more changes since then - most importnatly > >>> the early debug info. > >>> At I will see how easy/hard is to make the patch build with current > >>> trunk. > >> We do have ideas for the early debug with the asm-out abstraction to > >> also solve a different issue (missing simple-object support for > >> mingw/darwin). > > > > Note the debug info will be stored to a different .s file, then the file > > will be converted .o by GAS and then the bytecode will be stored to an ELF > > section of a compiled object. I've got also binutils patch when we'll > > be able to directly use the embedded section with compile.o@offset. > > Which will not work, as written, for Darwin since that is neither ELF nor > does it > use GAS - but hopefully, we can find some equivalent mechanism (there is > already > some support in the Darwin backend for switching asm context for LTO output). I think using compile.o@offset will be optional and the fallback is to extract the "file" back to a regular object file just containing debug info like we do currently but with the difference that we do not need to understand the object format since it is created "correctly" by the assembler during compile-time (and we just use the compile-object as a container to not confuse build systems). > >> Namely assemble the early debug in a different file and include the > >> resulting > >> native object in binary form in the compile output - not needing to write > >> assembly .data for that would be a good way to make this more viable. > > > > Btw. do you have any estimation how slow is GAS when we speak about debug > > info? > > How much time can we save since the latest speed-up achieved by GAS? > > > >> You might want to talk to Martin Liska for this who I think had some > >> prototype on this? > > > > I can provide a prototype if needed. > > I’d like to be in to loop from the Darwin PoV.. > thanks > Iain > > > > > Cheers, > > Martin >
Re: [GSoC]Bypass assembler when generating LTO object files
> On 12 Apr 2022, at 13:31, Martin Liška wrote: > > On 4/12/22 11:58, Richard Biener wrote: >> On Tue, Apr 12, 2022 at 11:20 AM Jan Hubicka via Gcc wrote: >>> >>> Hi, > On 08-Apr-2022, at 6:32 PM, Jan Hubicka wrote: > > Ankur, >> I was browsing the list of submitted GSoC projects this year and the >> project regarding bypassing assembler when generating LTO object files >> caught my eye. > I apologize for late reply. I would be very happy to mentor this > project. Thanks for the reply, but unfortunately, due to some reasons, I would not be able to take part in GSoC this year. But the project seems interesting and would be amazing opportunity to learn a lot more things for me, so would it be okay if I try to give it a go outside GSoC if no-one else picks it as their GSoC project this year ? >>> >>> I would be still very happy to help with that! However it would be also >>> pity to not take part of GSoC, so if there is something I can help with >>> on that let me know. >> >> I already have a gcc built from source (sync-ed with trunk/master) and >> launched the test-suite on it. >> >> I am currently in process of understanding the primilary patch >> (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and >> experimenting with it. >> >> are there any other things I should be aware of (useful Doc/blog or a >> bug tracking the project) before proceeding further ? > > I think it is pretty much all that exists. Basically we will need to > implement everything that is necessary to stream out valid object file > directly from GCC rather than going through gas. The experimental > prototype sort of worked but it was lacking few things. When I try to apply that patch on my local branch ( branched from trunk ), it seem to be incompatible with the current working tree. Is there a specific branch that I have to apply it to ? or is it due to the recent file rename patch ( changing extensions from .c to .cc ) ? ``` $ git apply --check bypass_asm_patch error: patch failed: Makefile.in:1300 error: Makefile.in: patch does not apply error: common.opt: No such file or directory error: langhooks.c: No such file or directory error: lto/Make-lang.in: No such file or directory error: lto/lto-object.c: No such file or directory error: lto/lto.c: No such file or directory error: lto/lto.h: No such file or directory error: lto-streamer.h: No such file or directory error: toplev.c: No such file or directory ``` >>> >>> I can try to update the patch, or it probably should apply to trunk >>> checked out around the date I sent the patch. Indeed we need to change >>> c to cc but there are likely more changes since then - most importnatly >>> the early debug info. >>> At I will see how easy/hard is to make the patch build with current >>> trunk. >> We do have ideas for the early debug with the asm-out abstraction to >> also solve a different issue (missing simple-object support for >> mingw/darwin). > > Note the debug info will be stored to a different .s file, then the file > will be converted .o by GAS and then the bytecode will be stored to an ELF > section of a compiled object. I've got also binutils patch when we'll > be able to directly use the embedded section with compile.o@offset. Which will not work, as written, for Darwin since that is neither ELF nor does it use GAS - but hopefully, we can find some equivalent mechanism (there is already some support in the Darwin backend for switching asm context for LTO output). >> Namely assemble the early debug in a different file and include the resulting >> native object in binary form in the compile output - not needing to write >> assembly .data for that would be a good way to make this more viable. > > Btw. do you have any estimation how slow is GAS when we speak about debug > info? > How much time can we save since the latest speed-up achieved by GAS? > >> You might want to talk to Martin Liska for this who I think had some >> prototype on this? > > I can provide a prototype if needed. I’d like to be in to loop from the Darwin PoV.. thanks Iain > > Cheers, > Martin > >> Richard. >>> Honza Thanks - Ankur
Re: [GSoC]Bypass assembler when generating LTO object files
On 4/12/22 11:58, Richard Biener wrote: On Tue, Apr 12, 2022 at 11:20 AM Jan Hubicka via Gcc wrote: Hi, On 08-Apr-2022, at 6:32 PM, Jan Hubicka wrote: Ankur, I was browsing the list of submitted GSoC projects this year and the project regarding bypassing assembler when generating LTO object files caught my eye. I apologize for late reply. I would be very happy to mentor this project. Thanks for the reply, but unfortunately, due to some reasons, I would not be able to take part in GSoC this year. But the project seems interesting and would be amazing opportunity to learn a lot more things for me, so would it be okay if I try to give it a go outside GSoC if no-one else picks it as their GSoC project this year ? I would be still very happy to help with that! However it would be also pity to not take part of GSoC, so if there is something I can help with on that let me know. I already have a gcc built from source (sync-ed with trunk/master) and launched the test-suite on it. I am currently in process of understanding the primilary patch (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and experimenting with it. are there any other things I should be aware of (useful Doc/blog or a bug tracking the project) before proceeding further ? I think it is pretty much all that exists. Basically we will need to implement everything that is necessary to stream out valid object file directly from GCC rather than going through gas. The experimental prototype sort of worked but it was lacking few things. When I try to apply that patch on my local branch ( branched from trunk ), it seem to be incompatible with the current working tree. Is there a specific branch that I have to apply it to ? or is it due to the recent file rename patch ( changing extensions from .c to .cc ) ? ``` $ git apply --check bypass_asm_patch error: patch failed: Makefile.in:1300 error: Makefile.in: patch does not apply error: common.opt: No such file or directory error: langhooks.c: No such file or directory error: lto/Make-lang.in: No such file or directory error: lto/lto-object.c: No such file or directory error: lto/lto.c: No such file or directory error: lto/lto.h: No such file or directory error: lto-streamer.h: No such file or directory error: toplev.c: No such file or directory ``` I can try to update the patch, or it probably should apply to trunk checked out around the date I sent the patch. Indeed we need to change c to cc but there are likely more changes since then - most importnatly the early debug info. At I will see how easy/hard is to make the patch build with current trunk. We do have ideas for the early debug with the asm-out abstraction to also solve a different issue (missing simple-object support for mingw/darwin). Note the debug info will be stored to a different .s file, then the file will be converted .o by GAS and then the bytecode will be stored to an ELF section of a compiled object. I've got also binutils patch when we'll be able to directly use the embedded section with compile.o@offset. Namely assemble the early debug in a different file and include the resulting native object in binary form in the compile output - not needing to write assembly .data for that would be a good way to make this more viable. Btw. do you have any estimation how slow is GAS when we speak about debug info? How much time can we save since the latest speed-up achieved by GAS? You might want to talk to Martin Liska for this who I think had some prototype on this? I can provide a prototype if needed. Cheers, Martin Richard. Honza Thanks - Ankur
Re: [GSoC]Bypass assembler when generating LTO object files
On Tue, Apr 12, 2022 at 11:20 AM Jan Hubicka via Gcc wrote: > > Hi, > > > > > > > On 08-Apr-2022, at 6:32 PM, Jan Hubicka wrote: > > > > > > Ankur, > > >> I was browsing the list of submitted GSoC projects this year and the > > >> project regarding bypassing assembler when generating LTO object files > > >> caught my eye. > > > I apologize for late reply. I would be very happy to mentor this > > > project. > > > > Thanks for the reply, but unfortunately, due to some reasons, I would not > > be able to take part in GSoC this year. > > But the project seems interesting and would be amazing opportunity to > > learn a lot more things for me, so would it be okay if I try to give it a > > go outside GSoC if no-one else picks it as their GSoC project this year ? > > I would be still very happy to help with that! However it would be also > pity to not take part of GSoC, so if there is something I can help with > on that let me know. > > > > >> > > >> I already have a gcc built from source (sync-ed with trunk/master) and > > >> launched the test-suite on it. > > >> > > >> I am currently in process of understanding the primilary patch > > >> (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and > > >> experimenting with it. > > >> > > >> are there any other things I should be aware of (useful Doc/blog or a > > >> bug tracking the project) before proceeding further ? > > > > > > I think it is pretty much all that exists. Basically we will need to > > > implement everything that is necessary to stream out valid object file > > > directly from GCC rather than going through gas. The experimental > > > prototype sort of worked but it was lacking few things. > > > > When I try to apply that patch on my local branch ( branched from trunk ), > > it seem to be incompatible with the current working tree. Is there a > > specific branch that I have to apply it to ? or is it due to the recent > > file rename patch ( changing extensions from .c to .cc ) ? > > > > ``` > > $ git apply --check bypass_asm_patch > > > > error: patch failed: Makefile.in:1300 > > error: Makefile.in: patch does not apply > > error: common.opt: No such file or directory > > error: langhooks.c: No such file or directory > > error: lto/Make-lang.in: No such file or directory > > error: lto/lto-object.c: No such file or directory > > error: lto/lto.c: No such file or directory > > error: lto/lto.h: No such file or directory > > error: lto-streamer.h: No such file or directory > > error: toplev.c: No such file or directory > > ``` > > I can try to update the patch, or it probably should apply to trunk > checked out around the date I sent the patch. Indeed we need to change > c to cc but there are likely more changes since then - most importnatly > the early debug info. > At I will see how easy/hard is to make the patch build with current > trunk. We do have ideas for the early debug with the asm-out abstraction to also solve a different issue (missing simple-object support for mingw/darwin). Namely assemble the early debug in a different file and include the resulting native object in binary form in the compile output - not needing to write assembly .data for that would be a good way to make this more viable. You might want to talk to Martin Liska for this who I think had some prototype on this? Richard. > Honza > > > > Thanks > > - Ankur > >
Re: [GSoC]Bypass assembler when generating LTO object files
Hi, > > > > On 08-Apr-2022, at 6:32 PM, Jan Hubicka wrote: > > > > Ankur, > >> I was browsing the list of submitted GSoC projects this year and the > >> project regarding bypassing assembler when generating LTO object files > >> caught my eye. > > I apologize for late reply. I would be very happy to mentor this > > project. > > Thanks for the reply, but unfortunately, due to some reasons, I would not > be able to take part in GSoC this year. > But the project seems interesting and would be amazing opportunity to > learn a lot more things for me, so would it be okay if I try to give it a > go outside GSoC if no-one else picks it as their GSoC project this year ? I would be still very happy to help with that! However it would be also pity to not take part of GSoC, so if there is something I can help with on that let me know. > > >> > >> I already have a gcc built from source (sync-ed with trunk/master) and > >> launched the test-suite on it. > >> > >> I am currently in process of understanding the primilary patch > >> (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and > >> experimenting with it. > >> > >> are there any other things I should be aware of (useful Doc/blog or a > >> bug tracking the project) before proceeding further ? > > > > I think it is pretty much all that exists. Basically we will need to > > implement everything that is necessary to stream out valid object file > > directly from GCC rather than going through gas. The experimental > > prototype sort of worked but it was lacking few things. > > When I try to apply that patch on my local branch ( branched from trunk ), > it seem to be incompatible with the current working tree. Is there a > specific branch that I have to apply it to ? or is it due to the recent > file rename patch ( changing extensions from .c to .cc ) ? > > ``` > $ git apply --check bypass_asm_patch > > error: patch failed: Makefile.in:1300 > error: Makefile.in: patch does not apply > error: common.opt: No such file or directory > error: langhooks.c: No such file or directory > error: lto/Make-lang.in: No such file or directory > error: lto/lto-object.c: No such file or directory > error: lto/lto.c: No such file or directory > error: lto/lto.h: No such file or directory > error: lto-streamer.h: No such file or directory > error: toplev.c: No such file or directory > ``` I can try to update the patch, or it probably should apply to trunk checked out around the date I sent the patch. Indeed we need to change c to cc but there are likely more changes since then - most importnatly the early debug info. At I will see how easy/hard is to make the patch build with current trunk. Honza > > Thanks > - Ankur >
Re: [GSoC]Bypass assembler when generating LTO object files
> On 08-Apr-2022, at 6:32 PM, Jan Hubicka wrote: > > Ankur, >> I was browsing the list of submitted GSoC projects this year and the >> project regarding bypassing assembler when generating LTO object files >> caught my eye. > I apologize for late reply. I would be very happy to mentor this > project. Thanks for the reply, but unfortunately, due to some reasons, I would not be able to take part in GSoC this year. But the project seems interesting and would be amazing opportunity to learn a lot more things for me, so would it be okay if I try to give it a go outside GSoC if no-one else picks it as their GSoC project this year ? >> >> I already have a gcc built from source (sync-ed with trunk/master) and >> launched the test-suite on it. >> >> I am currently in process of understanding the primilary patch >> (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and >> experimenting with it. >> >> are there any other things I should be aware of (useful Doc/blog or a >> bug tracking the project) before proceeding further ? > > I think it is pretty much all that exists. Basically we will need to > implement everything that is necessary to stream out valid object file > directly from GCC rather than going through gas. The experimental > prototype sort of worked but it was lacking few things. When I try to apply that patch on my local branch ( branched from trunk ), it seem to be incompatible with the current working tree. Is there a specific branch that I have to apply it to ? or is it due to the recent file rename patch ( changing extensions from .c to .cc ) ? ``` $ git apply --check bypass_asm_patch error: patch failed: Makefile.in:1300 error: Makefile.in: patch does not apply error: common.opt: No such file or directory error: langhooks.c: No such file or directory error: lto/Make-lang.in: No such file or directory error: lto/lto-object.c: No such file or directory error: lto/lto.c: No such file or directory error: lto/lto.h: No such file or directory error: lto-streamer.h: No such file or directory error: toplev.c: No such file or directory ``` Thanks - Ankur
Re: [GSoC]Bypass assembler when generating LTO object files
Ankur, > I was browsing the list of submitted GSoC projects this year and the > project regarding bypassing assembler when generating LTO object files > caught my eye. I apologize for late reply. I would be very happy to mentor this project. > > I already have a gcc built from source (sync-ed with trunk/master) and > launched the test-suite on it. > > I am currently in process of understanding the primilary patch > (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and > experimenting with it. > > are there any other things I should be aware of (useful Doc/blog or a > bug tracking the project) before proceeding further ? I think it is pretty much all that exists. Basically we will need to implement everything that is necessary to stream out valid object file directly from GCC rather than going through gas. The experimental prototype sort of worked but it was lacking few things. First is the production of proper object file hearder (it encodes things like architeture ELF is produced for), production of symbol table that is necessary to mark the LTO object file and also we now need a way to stream debug info (DWARF) directly to the object from dwarf2out. So I think first step would be to produce object files w/o debug info which can be consumed by unmodified linkers and then look into DWARF bytecode streaming. Honza > > I am Ankur Saini, a B.Tech CSE 3rd year student at USICT, GGSIPU india > and a former GSoC contributor at gcc ( worked on expanding gcc static > analyzer's C++ support in GSoC 2021 > [https://gist.github.com/Arsenic-ATG/8f4ac194f460dd9b2c78cf51af39afef]) > > Thanks > - Ankur
[GSoC]Bypass assembler when generating LTO object files
Hi, I was browsing the list of submitted GSoC projects this year and the project regarding bypassing assembler when generating LTO object files caught my eye. I already have a gcc built from source (sync-ed with trunk/master) and launched the test-suite on it. I am currently in process of understanding the primilary patch (https://gcc.gnu.org/legacy-ml/gcc/2014-09/msg00340.html), and experimenting with it. are there any other things I should be aware of (useful Doc/blog or a bug tracking the project) before proceeding further ? I am Ankur Saini, a B.Tech CSE 3rd year student at USICT, GGSIPU india and a former GSoC contributor at gcc ( worked on expanding gcc static analyzer's C++ support in GSoC 2021 [https://gist.github.com/Arsenic-ATG/8f4ac194f460dd9b2c78cf51af39afef]) Thanks - Ankur
