Re: Information about LTO
Sjodin, Jan wrote: I agree. Also, the LTO requirements and high-level design document states that the external format should be compiler-independent and it should be possible for other tools to read and write that format. Is this still a goal? It was a goal for me, but I don't think other participants were ever as excited about it. It's certainly not paramount, and Kenny has shown that it's probably more practical just to read/write the IL directly. -- Mark Mitchell CodeSourcery [EMAIL PROTECTED] (650) 331-3385 x713
Re: Information about LTO
On 5/2/07, Sjodin, Jan [EMAIL PROTECTED] wrote: Thanks for all the responses. It seems like LTO will have to wait for the tuples or there will be a lot of throw-away code. If you really only can think of LTO as the reader/writer, then perhaps yes. But if you read back this thread, you would have seen a bunch of suggestions for how you can help with LTO prerequisites that do not depend on tuples at all. You should look at the bigger picture. Gr. Steven
RE: Information about LTO
I agree. Also, the LTO requirements and high-level design document states that the external format should be compiler-independent and it should be possible for other tools to read and write that format. Is this still a goal? It would require a separate design with a distinct API to read and write code. This work could be done in parallel. The document mentions a GNU Virtual Machine. Would that be abandoned for a tuple-like external representation? Jan -Original Message- From: Steven Bosscher [mailto:[EMAIL PROTECTED] Sent: Thursday, May 03, 2007 2:39 AM To: Sjodin, Jan Cc: Diego Novillo; Joseph S. Myers; Ian Lance Taylor; gcc@gcc.gnu.org Subject: Re: Information about LTO On 5/2/07, Sjodin, Jan [EMAIL PROTECTED] wrote: Thanks for all the responses. It seems like LTO will have to wait for the tuples or there will be a lot of throw-away code. If you really only can think of LTO as the reader/writer, then perhaps yes. But if you read back this thread, you would have seen a bunch of suggestions for how you can help with LTO prerequisites that do not depend on tuples at all. You should look at the bigger picture. Gr. Steven
Re: Information about LTO
On 5/2/07, Sjodin, Jan [EMAIL PROTECTED] wrote: Thanks for all the responses. It seems like LTO will have to wait for the tuples or there will be a lot of throw-away code. If you really only can think of LTO as the reader/writer, then perhaps yes. But if you read back this thread, you would have seen a bunch of suggestions for how you can help with LTO prerequisites that do not depend on tuples at all. You should look at the bigger picture. Even concerning the reader/writer there is alot of parallelizm - writting nested gimple is just slightly different from tuples and we still need to learn how to deal with the rest of infrastructure (types, declarations, add on datastructures, APIs). We also do have LTO for Java, getting our memory consumption down to level so -funit-at-a-time is practically useful for large Java applications would make C/C++ LTO work too once writer/reader is at the place. The memory consumption is quite evenly distributed across varioius datastructures - it is definitly not like tuples will solve the problem alone - it is going to make about 13-18% of overall footprint of parsed program. Cleanups and simplifications of other datastructures are easilly done in parallel and bring immediate benefits. Honza Gr. Steven
RE: Information about LTO
-Original Message- From: Steven Bosscher [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 01, 2007 4:09 PM To: Sjodin, Jan Cc: Diego Novillo; Joseph S. Myers; Ian Lance Taylor; gcc@gcc.gnu.org Subject: Re: Information about LTO On 5/1/07, Sjodin, Jan [EMAIL PROTECTED] wrote: Can someone give similar information about LTO? How many people (full/part time) and how long time it will take? How much work is LTO compared to the tuple representation? A vast amount more if we're going to work on LTO with the current GIMPLE representation, because we'll end up rewriting almost all of it when the tuples representation is merged. ;-) More seriously: LTO is, by my uneducated guess, about the same amount of work as the tuples work. I'm assuming single-language LTO here, because cross-language LTO is far harder (by the time we get there, we'll be talking about fixing the Fortran front end to play nice with the call graph, defining some kind of type system, etc.). Personally, I think the most useful thing you could do at this point, is helping with the tuples repesentation, or helping with the preparations for LTO such as removing lang_hooks and perhaps some intermediate representation oddities like early uses of DECL_RTL (for e.g. register variables). I think Kenny has shown that an IL reader/writer is not an awful lot of work, but IMHO it will be corner cases like the ones I just mention that will take a little while to resolve. So, better start early, to make work easier later on. But as I said, that's just my uneducated guess. I haven't done any LTO work myself yet ;-) Gr. Stevem Thanks for all the responses. It seems like LTO will have to wait for the tuples or there will be a lot of throw-away code. - Jan
Re: Information about LTO
Sjodin, Jan [EMAIL PROTECTED] writes: Hi I am new to GCC development and I have a few questions about LTO. What has been done since the last status report in January? I would also like to know what is most important to work on right now to make progress on LTO (e.g. type system, intermediate representation, reader/writer). What remaining work needs to be done to make LTO work for C? In the old status report a few things were mentioned: the front end was not complete, the type system was not being fully encoded and there are still some things that need to be explicitly represented in the IR. My take on it. Others may feel differently. Some langhooks have been removed. Removing langhooks one way or another is an important step in permitting the middle-end to work independently of any frontend. The gimple-tuple proposal has been made and people have started to work on it. The goal here is to modify the IR to reduce memory requirements. No work has been done on the reader/writer; it has been delayed by the dataflow work. It is not clear to what extent we need to fully define a middle-end type system. We may be able to get away with simple structural equivalence for now rather than a more formal approach. Ian
Re: Information about LTO
On May 1, 2007, at 9:42 AM, Ian Lance Taylor wrote: Sjodin, Jan [EMAIL PROTECTED] writes: Hi I am new to GCC development and I have a few questions about LTO. What has been done since the last status report in January? I would also like to know what is most important to work on right now to make progress on LTO (e.g. type system, intermediate representation, reader/writer). What remaining work needs to be done to make LTO work for C? In the old status report a few things were mentioned: the front end was not complete, the type system was not being fully encoded and there are still some things that need to be explicitly represented in the IR. My take on it. Others may feel differently. Some langhooks have been removed. Removing langhooks one way or another is an important step in permitting the middle-end to work independently of any frontend. In addition, other flags that affect codegen and can be mixed and matched between translation units need to be adjusted eventually. For example, semantic options like -fstrict-aliasing, -ffast-math, - fforce-mem, -falign-loops, lots of optimization options (e.g. -fgcse, etc), and even target options like -malign-double, -march, etc. -Chris
RE: Information about LTO
Sjodin, Jan [EMAIL PROTECTED] writes: Hi I am new to GCC development and I have a few questions about LTO. What has been done since the last status report in January? I would also like to know what is most important to work on right now to make progress on LTO (e.g. type system, intermediate representation, reader/writer). What remaining work needs to be done to make LTO work for C? In the old status report a few things were mentioned: the front end was not complete, the type system was not being fully encoded and there are still some things that need to be explicitly represented in the IR. -Original Message- From: Ian Lance Taylor [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 01, 2007 11:43 AM To: Sjodin, Jan Cc: gcc@gcc.gnu.org Subject: Re: Information about LTO My take on it. Others may feel differently. Some langhooks have been removed. Removing langhooks one way or another is an important step in permitting the middle-end to work independently of any frontend. The gimple-tuple proposal has been made and people have started to work on it. The goal here is to modify the IR to reduce memory requirements. No work has been done on the reader/writer; it has been delayed by the dataflow work. It is not clear to what extent we need to fully define a middle-end type system. We may be able to get away with simple structural equivalence for now rather than a more formal approach. Ian Does LTO have any hard dependencies on the gimple-tuples? I imagine the on-disk representation could be separate from any internal representation. I am curious if the two efforts can be worked on in parallel and how well they can be separated, since the gimple-tuple project seems like a big effort also. Jan
Re: Information about LTO
Sjodin, Jan wrote on 05/01/07 13:54: Does LTO have any hard dependencies on the gimple-tuples? I imagine the on-disk representation could be separate from any internal representation. I am curious if the two efforts can be worked on in parallel and how well they can be separated, since the gimple-tuple project seems like a big effort also. Some amount of parallelism may occur, but we will have to adapt many LTO chunks when the tuple work goes in mainline. The data structures for the IL will be completely different, after all.
Re: Information about LTO
On Tue, 1 May 2007, Diego Novillo wrote: Some amount of parallelism may occur, but we will have to adapt many LTO chunks when the tuple work goes in mainline. The data structures for the IL will be completely different, after all. Also, LTO hasn't been merged from mainline in a long time; there will be a big piece of work for whoever does the next merge from mainline to LTO. One big piece of the tree representation changes on LTO branch (CALL_EXPR) has been merged to trunk. Another big piece (TYPE_ARG_TYPES) hasn't, but is still scheduled for Stage 1. The remaining pieces are smaller - and many were preapproved for 4.3 - but I don't know what plans there are for merging them to trunk. Once all the tree representation changes are merged to trunk, that might make a merge into LTO branch easier (in that all non-LTO-related conflicts can be resolved by taking the trunk version). -- Joseph S. Myers [EMAIL PROTECTED]
RE: Information about LTO
-Original Message- From: Joseph S. Myers [mailto:[EMAIL PROTECTED] Sent: Tuesday, May 01, 2007 2:07 PM To: Diego Novillo Cc: Sjodin, Jan; Ian Lance Taylor; gcc@gcc.gnu.org Subject: Re: Information about LTO On Tue, 1 May 2007, Diego Novillo wrote: Also, LTO hasn't been merged from mainline in a long time; there will be a big piece of work for whoever does the next merge from mainline to LTO. One big piece of the tree representation changes on LTO branch (CALL_EXPR) has been merged to trunk. Another big piece (TYPE_ARG_TYPES) hasn't, but is still scheduled for Stage 1. The remaining pieces are smaller - and many were preapproved for 4.3 - but I don't know what plans there are for merging them to trunk. Once all the tree representation changes are merged to trunk, that might make a merge into LTO branch easier (in that all non-LTO-related conflicts can be resolved by taking the trunk version). -- Joseph S. Myers [EMAIL PROTECTED] Can anything be done in LTO to reduce the amount of work needed to transition to tuples in the future? One idea could be to use the tuple spec to define the external format for LTO. This could potentially uncover something that is needed by LTO but is not in the spec today. Does anyone know how many people that are currently working on the tuple representation and can perhaps guess how many months it would take to get into mainline? - Jan
Re: Information about LTO
On 5/1/07, Sjodin, Jan [EMAIL PROTECTED] wrote: Does anyone know how many people that are currently working on the tuple representation and can perhaps guess how many months it would take to get into mainline? Aldy is working full time on it, atm. Richard, Andrew and I may start working on it in the near future. As with anything in GCC, it is very hard to predict how long it will take. It depends greatly on how much time we can dedicate to it. I would guess that it won't take less than 4-5 months.
RE: Information about LTO
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Diego Novillo Sent: Tuesday, May 01, 2007 3:26 PM To: Sjodin, Jan Cc: Joseph S. Myers; Ian Lance Taylor; gcc@gcc.gnu.org Subject: Re: Information about LTO On 5/1/07, Sjodin, Jan [EMAIL PROTECTED] wrote: Does anyone know how many people that are currently working on the tuple representation and can perhaps guess how many months it would take to get into mainline? Aldy is working full time on it, atm. Richard, Andrew and I may start working on it in the near future. As with anything in GCC, it is very hard to predict how long it will take. It depends greatly on how much time we can dedicate to it. I would guess that it won't take less than 4-5 months. Can someone give similar information about LTO? How many people (full/part time) and how long time it will take? How much work is LTO compared to the tuple representation? Thanks, Jan
Re: Information about LTO
On 5/1/07, Sjodin, Jan [EMAIL PROTECTED] wrote: Can someone give similar information about LTO? How many people (full/part time) and how long time it will take? How much work is LTO compared to the tuple representation? A vast amount more if we're going to work on LTO with the current GIMPLE representation, because we'll end up rewriting almost all of it when the tuples representation is merged. ;-) More seriously: LTO is, by my uneducated guess, about the same amount of work as the tuples work. I'm assuming single-language LTO here, because cross-language LTO is far harder (by the time we get there, we'll be talking about fixing the Fortran front end to play nice with the call graph, defining some kind of type system, etc.). Personally, I think the most useful thing you could do at this point, is helping with the tuples repesentation, or helping with the preparations for LTO such as removing lang_hooks and perhaps some intermediate representation oddities like early uses of DECL_RTL (for e.g. register variables). I think Kenny has shown that an IL reader/writer is not an awful lot of work, but IMHO it will be corner cases like the ones I just mention that will take a little while to resolve. So, better start early, to make work easier later on. But as I said, that's just my uneducated guess. I haven't done any LTO work myself yet ;-) Gr. Stevem
