[Bug lto/41376] collect2 does not handle static libraries
--- Comment #8 from bmei at broadcom dot com 2010-05-24 09:31 --- I integrated Dave's patch into LD with some modification (only emit those with LTO sections) and hacked collect2 to support that. The size gain of LTO, our main concern, is quite limited for our application. Large amount of functions called only once cannot be inlined across files because compiler doesn't know whether they are referred in non-LTO compiled code (mostly hand-code assembly in our cases). We really need full resolution file, especially LDPR_PREVAILING_DEF_IRONLY type. I will try next to make LD emit full resolution file. Since GNU LD doesn't have plugin support like GOLD. Won't any changes here be too invasive/specific for LTO purpose to be accepted by LD? We are fine to live with that in our private port. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
[Bug lto/41376] collect2 does not handle static libraries
--- Comment #9 from rguenth at gcc dot gnu dot org 2010-05-24 12:15 --- (In reply to comment #8) I integrated Dave's patch into LD with some modification (only emit those with LTO sections) and hacked collect2 to support that. The size gain of LTO, our main concern, is quite limited for our application. Large amount of functions called only once cannot be inlined across files because compiler doesn't know whether they are referred in non-LTO compiled code (mostly hand-code assembly in our cases). We really need full resolution file, especially LDPR_PREVAILING_DEF_IRONLY type. I will try next to make LD emit full resolution file. Note that for a notable size-gain you likely need to build with -fwhole-program (which means GCC needs to see all references to functions or you need to mark functions which are accessed from assembly code with the externally_visible attribute). Since GNU LD doesn't have plugin support like GOLD. Won't any changes here be too invasive/specific for LTO purpose to be accepted by LD? We are fine to live with that in our private port. I guess that outputting a full resolution file might be ok with the LD folks as it can be a useful thing for debugging as well. But of course it would be their call, including details of the format. I would also expect that a very simple approach for a collect2-only patch would work most of the times (but when it wouldn't work it could result in wrong-code or final linker failures). Simply decompose lib*.a linker arguments to their respective lib...@offset members for the lto1 invocation (and make lto1 ignore, not barf on non-LTO objects). I will also try to see what incrementally doing the symbol resolution would take. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
[Bug lto/41376] collect2 does not handle static libraries
--- Comment #10 from bmei at broadcom dot com 2010-05-24 13:29 --- annotating functions with externally_visible sounds a bit difficult to maintain. Programmer needs to know whether a function is used outside of LTO objects. This can change over time and extra efforts are needed to keep it correct. It would be better if GCC can derive that info with -fwhole-program, whether it is deal with LTO-object file only or LTO/Regular object files, since it should have all the symbol reference information by then. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
[Bug lto/41376] collect2 does not handle static libraries
--- Comment #11 from rguenther at suse dot de 2010-05-24 17:27 --- Subject: Re: collect2 does not handle static libraries On Mon, 24 May 2010, bmei at broadcom dot com wrote: --- Comment #10 from bmei at broadcom dot com 2010-05-24 13:29 --- annotating functions with externally_visible sounds a bit difficult to maintain. Programmer needs to know whether a function is used outside of LTO objects. This can change over time and extra efforts are needed to keep it correct. It would be better if GCC can derive that info with -fwhole-program, whether it is deal with LTO-object file only or LTO/Regular object files, since it should have all the symbol reference information by then. Indeed - at least if you are not dealing with building a shared library or a program which needs to expose symbols to plugins that are loaded at runtime. Of course at the moment - if at all - the symbol reference information is only available when using something like the linker-plugin. Richard. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
[Bug lto/41376] collect2 does not handle static libraries
--- Comment #7 from rguenther at suse dot de 2010-05-05 08:55 --- Subject: Re: collect2 does not handle static libraries On Tue, 4 May 2010, bmei at broadcom dot com wrote: --- Comment #6 from bmei at broadcom dot com 2010-05-04 16:54 --- So this is a rough first draft of the-kind-of-thing-i-was-thinking-of. We get collect2 to run a dummy link early, and extract the output from the --lto-assist flag to get a list of archive members that we need lto to recompile for us. Well I spent some time to read into collect2/lto code and understand pro/cons of different approaches. So far, adding --lto-assist to ld/hacking collect2 approach looks reasonable to me, though it does require gnu ld. What extra info should be in a complete symbol resolution file? Basically what lto-plugin puts there. A list of symbols for each object files that is linked (thus, all required ar archive members) together with the symbol resolution the linker chose (whether the symbol was overridden, resolved to sth external, or used as the prevailing definition). Richard. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
[Bug lto/41376] collect2 does not handle static libraries
--- Comment #6 from bmei at broadcom dot com 2010-05-04 16:54 --- So this is a rough first draft of the-kind-of-thing-i-was-thinking-of. We get collect2 to run a dummy link early, and extract the output from the --lto-assist flag to get a list of archive members that we need lto to recompile for us. Well I spent some time to read into collect2/lto code and understand pro/cons of different approaches. So far, adding --lto-assist to ld/hacking collect2 approach looks reasonable to me, though it does require gnu ld. What extra info should be in a complete symbol resolution file? -- bmei at broadcom dot com changed: What|Removed |Added CC||bmei at broadcom dot com http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
[Bug lto/41376] collect2 does not handle static libraries
--- Comment #5 from rguenther at suse dot de 2010-04-29 08:29 --- Subject: Re: collect2 does not handle static libraries On Thu, 29 Apr 2010, davek at gcc dot gnu dot org wrote: --- Comment #4 from davek at gcc dot gnu dot org 2010-04-29 05:28 --- Created an attachment (id=20512) -- (http://gcc.gnu.org/bugzilla/attachment.cgi?id=20512action=view) -- (http://gcc.gnu.org/bugzilla/attachment.cgi?id=20512action=view) Extends GNU LD to parse archives for LTO. (In reply to comment #3) Ow. I think we need to get LD to help us out here. So this is a rough first draft of the-kind-of-thing-i-was-thinking-of. We get collect2 to run a dummy link early, and extract the output from the --lto-assist flag to get a list of archive members that we need lto to recompile for us. If we go that way (extending LD), then it might be worthwhile to go the full way and have it dump a complete symbol resolution file like we get from the gold linker plugin. Note that collect2 was intended to work with any linker (and it does already query symbols of everything via nm), so I think we have to emulate static archive linking properly. Or alternatively move static archive support completely into lto1 (though that would require to make symbol resolution follow incremental linking semantics). Richard. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
[Bug lto/41376] collect2 does not handle static libraries
--- Comment #2 from davek at gcc dot gnu dot org 2010-04-28 13:38 --- Quoting RG from the gcc list: [ ... ] Or you fix collect2 to do processing of archives and hand lto1 the required information (it expects archive components with LTO bytecode like archiv...@offset with offset being the offset of the .o file with LTO bytecode inside the archive). See lto/lto-elf.c:lto_obj_file_open for details. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
[Bug lto/41376] collect2 does not handle static libraries
--- Comment #3 from davek at gcc dot gnu dot org 2010-04-28 23:49 --- (In reply to comment #2) Quoting RG from the gcc list: [ ... ] Or you fix collect2 to do processing of archives and hand lto1 the required information (it expects archive components with LTO bytecode like archiv...@offset with offset being the offset of the .o file with LTO bytecode inside the archive). See lto/lto-elf.c:lto_obj_file_open for details. Of course, without unresolved symbol info to guide us, as the linker has, we won't know which archive members would be required in a link. So if we do try this in collect2, it would need to effectively recompile the entire library at link time. Ow. I think we need to get LD to help us out here. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
[Bug lto/41376] collect2 does not handle static libraries
--- Comment #4 from davek at gcc dot gnu dot org 2010-04-29 05:28 --- Created an attachment (id=20512) -- (http://gcc.gnu.org/bugzilla/attachment.cgi?id=20512action=view) Extends GNU LD to parse archives for LTO. (In reply to comment #3) Ow. I think we need to get LD to help us out here. So this is a rough first draft of the-kind-of-thing-i-was-thinking-of. We get collect2 to run a dummy link early, and extract the output from the --lto-assist flag to get a list of archive members that we need lto to recompile for us. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41376
