Re: GSOC
On Thu, 28 Mar 2019, Giuliano Belinassi wrote: > Hi, Richard > > On 03/28, Richard Biener wrote: > > On Wed, Mar 27, 2019 at 2:55 PM Giuliano Belinassi > > wrote: > > > > > > Hi, > > > > > > On 03/26, Richard Biener wrote: > > > > On Tue, 26 Mar 2019, David Malcolm wrote: > > > > > > > > > On Mon, 2019-03-25 at 19:51 -0400, nick wrote: > > > > > > Greetings All, > > > > > > > > > > > > I would like to take up parallelize compilation using threads or > > > > > > make > > > > > > c++/c > > > > > > memory issues not automatically promote. I did ask about this before > > > > > > but > > > > > > not get a reply. When someone replies I'm just a little concerned as > > > > > > my writing for proposals has never been great so if someone just > > > > > > reviews > > > > > > and doubt checks that's fine. > > > > > > > > > > > > As for the other things building gcc and running the testsuite is > > > > > > fine. Plus > > > > > > I already working on gcc so I've pretty aware of most things and > > > > > > this > > > > > > would > > > > > > be a great steeping stone into more serious gcc development work. > > > > > > > > > > > > If sample code is required that's in mainline gcc I sent out a trial > > > > > > patch > > > > > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395 > > > > > > > > > > > > Cheers, > > > > > > > > > > > > Nick > > > > > > > > > > It's good to see that you've gotten as far as attaching a patch to BZ > > > > > [1] > > > > > > > > > > I think someone was going to attempt the "parallelize compilation > > > > > using > > > > > threads" idea last year, but then pulled out before the summer; you > > > > > may > > > > > want to check the archives (or was that you?) > > > > > > > > There's also Giuliano Belinassi who is interested in the same project > > > > (CCed). > > > > > > Yes, I will apply for this project, and I will submit the final version > > > of my proposal by the end of the week. > > > > > > Currently, my target is the `expand_all_functions` routine, as most of > > > the time is spent on it according to the experiments that I performed as > > > part of my Master's research on compiler parallelization. > > > (-O2, --disable-checking) > > > > Yes, more specifically I think the realistic target is the GIMPLE part > > of execute_pass_list (cfun, g->get_passes ()->all_passes); done in > > cgraph_node::expand. If you look at passes.def you'll see all_passes > > also contains RTL expansion (pass_expand) and the RTL optimization > > queue (pass_rest_of_compilation). The RTL part isn't a realistic target. > > Without changing the pass hierarchy the obvious part that can be > > handled would be the pass_all_optimizations pass sub-queue of > > all_passes since those are all passes that perform transforms on the > > GIMPLE IL where we have all functions in this state at the same time > > and where no interactions between the functions happen anymore > > and thus functions can be processed in parallel (as much as make > > processes individual translation units in parallel). > > > > Great. So if I understood correctly, I will need to split > cgraph_node::expand() into three parts: IPA, GIMPLE and RTL, and then > refactor `expand_all_functions` so that the loop > > for (i = new_order_pos - 1; i >= 0; i--) > > use these three functions, then partition > > g->get_passes()->all_passes > > into get_passes()->gimple_passes and get_passes()->rtl_passes, so I > can run RTL after GIMPLE is finished, to finally start the > paralellization of per function GIMPLE passes. Yes, it involves refactoring of the loop - you may notice that parts of the compilation pipeline are under control of the pass manager (passes.c) but some is still manually driven by symbol_table::compile. Whether it's more convenient to get more control stuffed to the pass manager and perform the threading under its control (I'd say that would be the cleaner design) or to try do this in the current ad-hoc parts remains to be seen. You can see symbol_table::compile hands over control to the pass manager multiple times, first ipa_passes () then all_late_ipa_passes and finally the expand_all_functions code. I guess it would simplify things if you'd split pass_all_passes in passes.def at pass_expand like so: diff --git a/gcc/passes.def b/gcc/passes.def index 2fcd80e53a3..bb0453b36a7 100644 --- a/gcc/passes.def +++ b/gcc/passes.def @@ -403,11 +403,10 @@ along with GCC; see the file COPYING3. If not see NEXT_PASS (pass_spectrev1); NEXT_PASS (pass_warn_function_noreturn); NEXT_PASS (pass_gen_hsail); + TERMINATE_PASS_LIST (all_passes) - NEXT_PASS (pass_expand); - - NEXT_PASS (pass_rest_of_compilation); - PUSH_INSERT_PASSES_WITHIN (pass_rest_of_compilation) + INSERT_PASSES_AFTER (pass_rest_of_compilation) + NEXT_PASS (pass_expand); NEXT_PASS (pass_instantiate_virtual_regs); NEXT_PASS (pass_into_cfg_layout_mode); NEXT_PASS (pass_jump); @@ -505,6 +504,5 @@ along with
Re: GSOC Proposal
On Thu, 28 Mar 2019, nick wrote: > > > On 2019-03-28 4:59 a.m., Richard Biener wrote: > > On Wed, Mar 27, 2019 at 6:31 PM nick wrote: > >> > >> Greetings All, > >> > >> I've already done most of the work required for signing up for GSoC > >> as of last year i.e. reading getting started, being signed up legally > >> for contributions. > >> > >> My only real concern would be the proposal which I started writing here: > >> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing > >> > >> The biography and success section I'm fine with my bigger concern would be > >> the project and roadmap > >> section. The roadmap is there and I will go into more detail about it in > >> the projects section as > >> need be. Just wanted to known if the roadmap is detailed enough or can I > >> just write out a few > >> paragraphs discussing it in the Projects Section. > > > > I'm not sure I understand either the problem analysis nor the project > > goal parts. What > > shared state with respect to garbage collection are you talking about? > > > > Richard. > > > I just fixed it. Seems we were discussing RTL itself. I edited it to > reflect those changes. Let me know if it's unclear or you would actually > like me to discuss some changes that may occur in the RTL layer itself. > > > I'm glad to be more exact if that's better but seems your confusion was > just what layer we were touching. Let me just throw in some knowledge here. The issue with RTL is that we currently can only have a single function in this intermediate language state since a function in RTL has some state in global variables that would differ if it were another function. We can have multiple functions in GIMPLE intermediate language state since all such state is in a function-specific data structure (struct function). The hard thing about moving all this "global" state of RTL into the same place is that there's global state in the various backends (and there's already a struct funtion 'machine' part for such state, so there's hope the issue isn't as big as it could be) and that some of the global state is big and only changes very rarely. That said, I'm not sure if anybody knows the full details here. So as far as I understand you'd like to tackle this as project with the goal to be able to have multiple functions in RTL state. That's laudable but IMHO also quite ambitious for a GSoC project. It's also an area I am not very familiar with so I opt out of being a mentor for this project. Richard. > Nick > >> Any other comments are welcome as well as I write it there, > >> Nick > -- Richard Biener SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany; GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)
GCC 9 Status Report (2019-03-29)
Status
==
We should be at the end of the stabilization phase (Stage 4) having
made some good progress in the long march towards zero P1 regressions.
Please have a look at those that you assigned yourself to.
There's still 12 P1 left at the moment though at some point bugs
not severe (wrong-code, rejects-valid) might be downgraded and
defered for fixing in a subsequent release. We would like to follow
the historical precedent of releasing at the end of April or the
beginning of May and not slip further.
As usual this is a good time to test your non-{primary,secondary}
target making sure it builds and works correctly.
Quality Data
Priority # Change from last report
--- ---
P1 12 - 12
P2 158 - 27
P3 25 - 7
P4 138 - 31
P5 24
--- ---
Total P1-P3 195 - 46
Total 357 - 77
Previous Report
===
https://gcc.gnu.org/ml/gcc/2019-02/msg00028.html
Re: GSOC Proposal
On 2019-03-29 5:08 a.m., Richard Biener wrote: > On Thu, 28 Mar 2019, nick wrote: > >> >> >> On 2019-03-28 4:59 a.m., Richard Biener wrote: >>> On Wed, Mar 27, 2019 at 6:31 PM nick wrote: Greetings All, I've already done most of the work required for signing up for GSoC as of last year i.e. reading getting started, being signed up legally for contributions. My only real concern would be the proposal which I started writing here: https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing The biography and success section I'm fine with my bigger concern would be the project and roadmap section. The roadmap is there and I will go into more detail about it in the projects section as need be. Just wanted to known if the roadmap is detailed enough or can I just write out a few paragraphs discussing it in the Projects Section. >>> >>> I'm not sure I understand either the problem analysis nor the project >>> goal parts. What >>> shared state with respect to garbage collection are you talking about? >>> >>> Richard. >>> >> I just fixed it. Seems we were discussing RTL itself. I edited it to >> reflect those changes. Let me know if it's unclear or you would actually >> like me to discuss some changes that may occur in the RTL layer itself. >> >> >> I'm glad to be more exact if that's better but seems your confusion was >> just what layer we were touching. > > Let me just throw in some knowledge here. The issue with RTL > is that we currently can only have a single function in this > intermediate language state since a function in RTL has some > state in global variables that would differ if it were another > function. We can have multiple functions in GIMPLE intermediate > language state since all such state is in a function-specific > data structure (struct function). The hard thing about moving > all this "global" state of RTL into the same place is that > there's global state in the various backends (and there's > already a struct funtion 'machine' part for such state, so there's > hope the issue isn't as big as it could be) and that some of > the global state is big and only changes very rarely. > That said, I'm not sure if anybody knows the full details here. > > So as far as I understand you'd like to tackle this as project > with the goal to be able to have multiple functions in RTL > state. > > That's laudable but IMHO also quite ambitious for a GSoC > project. It's also an area I am not very familiar with so > I opt out of being a mentor for this project. > While I'm aware of three areas where the shared state is an issue currently: 1, Compiler's Proper 2. The expand_functions 3. RTL 4.Garbage Collector Or maybe a project to be more explicit about regions of the code that assume that the garbage- collector can't run within them?[3] (since the GC is state that would be shared by the threads). This is what we were discussing previously and I wrote my proposal for that. You however seem confused about what parts of the garbage collector would be touched. That's fine with me, however seems you want be to be more exact about which part is touched. My questions would be as it's changed back to the garbage collector project: https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit 1. Your confusion about which part of the garbage collector is touched doesn't really make sense s it's for the whole garbage collector as related to shared state? 2. Injection was my code here in phase 3 for the callers of the new functions or macros, perhaps this is not needed as the work with the garbage collector is enough? 3. Am I not understanding this project as I thought I was in the proposal I wrote? Seems your more confusing my wording probably so I'm going to suggest one of two things here: a) I'm going to allow you to make comments with what's confusing you and it needs that's the issue here more than anything else so I sent you a link and please comment where you are having issues with this not be clear for you: Or maybe a project to be more explicit about regions of the code that assume that the garbage- collector can't run within them?[3] (since the GC is state that would be shared by the threads). as that's the actual project b) Just comment here about the wording that's an issue for you or where you want more exact wording Sorry and hopefully this is helps you understand where I'm going, Nick > Richard. > >> Nick Any other comments are welcome as well as I write it there, Nick >> >
Re: GSOC Proposal
On 2019-03-29 10:28 a.m., nick wrote: > > > On 2019-03-29 5:08 a.m., Richard Biener wrote: >> On Thu, 28 Mar 2019, nick wrote: >> >>> >>> >>> On 2019-03-28 4:59 a.m., Richard Biener wrote: On Wed, Mar 27, 2019 at 6:31 PM nick wrote: > > Greetings All, > > I've already done most of the work required for signing up for GSoC > as of last year i.e. reading getting started, being signed up legally > for contributions. > > My only real concern would be the proposal which I started writing here: > https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing > > The biography and success section I'm fine with my bigger concern would > be the project and roadmap > section. The roadmap is there and I will go into more detail about it in > the projects section as > need be. Just wanted to known if the roadmap is detailed enough or can I > just write out a few > paragraphs discussing it in the Projects Section. I'm not sure I understand either the problem analysis nor the project goal parts. What shared state with respect to garbage collection are you talking about? Richard. >>> I just fixed it. Seems we were discussing RTL itself. I edited it to >>> reflect those changes. Let me know if it's unclear or you would actually >>> like me to discuss some changes that may occur in the RTL layer itself. >>> >>> >>> I'm glad to be more exact if that's better but seems your confusion was >>> just what layer we were touching. >> >> Let me just throw in some knowledge here. The issue with RTL >> is that we currently can only have a single function in this >> intermediate language state since a function in RTL has some >> state in global variables that would differ if it were another >> function. We can have multiple functions in GIMPLE intermediate >> language state since all such state is in a function-specific >> data structure (struct function). The hard thing about moving >> all this "global" state of RTL into the same place is that >> there's global state in the various backends (and there's >> already a struct funtion 'machine' part for such state, so there's >> hope the issue isn't as big as it could be) and that some of >> the global state is big and only changes very rarely. >> That said, I'm not sure if anybody knows the full details here. >> >> So as far as I understand you'd like to tackle this as project >> with the goal to be able to have multiple functions in RTL >> state. >> >> That's laudable but IMHO also quite ambitious for a GSoC >> project. It's also an area I am not very familiar with so >> I opt out of being a mentor for this project. >> > While I'm aware of three areas where the shared state is an issue > currently: > 1, Compiler's Proper > 2. The expand_functions > 3. RTL > 4.Garbage Collector > > Or maybe a project to be more > explicit about regions of the code that assume that the garbage- > collector can't run within them?[3] (since the GC is state that would > be shared by the threads). > > This is what we were discussing previously and I wrote my proposal for > that. You however seem confused about what parts of the garbage collector > would be touched. That's fine with me, however seems you want be to > be more exact about which part is touched. > > My questions would be as it's changed back to the garbage collector project: > https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit > > 1. Your confusion about which part of the garbage collector is touched doesn't > really make sense s it's for the whole garbage collector as related to shared > state? > 2. Injection was my code here in phase 3 for the callers of the new functions > or > macros, perhaps this is not needed as the work with the garbage collector is > enough? > 3. Am I not understanding this project as I thought I was in the proposal I > wrote? > > Seems your more confusing my wording probably so I'm going to suggest one of > two things here: > a) I'm going to allow you to make comments with what's confusing you and > it needs that's the issue here more than anything else so I sent you > a link and please comment where you are having issues with this not > be clear for you: > Or maybe a project to be more > explicit about regions of the code that assume that the garbage- > collector can't run within them?[3] (since the GC is state that would > be shared by the threads). > as that's the actual project > > b) Just comment here about the wording that's an issue for you or > where you want more exact wording > > Sorry and hopefully this is helps you understand where I'm going, > Nick > >> Richard. >> >>> Nick > Any other comments are welcome as well as I write it there, > Nick >>> Richard, Seems your right touching the complete garbage collector is too much. I'm just looking at the users of the garbage collector and it seems one
FIXME in gcc/gimplify.c
Greetings all, Not sure why this exists still as tree-eh.h is including in tree-eh.c which defines this header as used for this FIXME: #include "tree-pass.h" /* FIXME: only for PROP_gimple_any */ Unless there is something in the build ordering that would cause issues it's indirectly including that way so this header inclusion should now be removed. Unless I'm missing something else which is fine. If not just let me known and I will just send a patch for it, Nick
gcc-8-20190329 is now available
Snapshot gcc-8-20190329 is now available on ftp://gcc.gnu.org/pub/gcc/snapshots/8-20190329/ and on various mirrors, see http://gcc.gnu.org/mirrors.html for details. This snapshot has been generated from the GCC 8 SVN branch with the following options: svn://gcc.gnu.org/svn/gcc/branches/gcc-8-branch revision 270026 You'll find: gcc-8-20190329.tar.xzComplete GCC SHA256=29e85ff7e5f29f139e30669b1b3192348f72f81b1ba7de4e4686733b79c96e5d SHA1=ab2c078d86de88423758f471a7078fbbcc0263fb Diffs from 8-20190322 are available in the diffs/ subdirectory. When a particular snapshot is ready for public consumption the LATEST-8 link is updated and a message is sent to the gcc list. Please do not use a snapshot before it has been announced that way.
