On Tuesday 11 February 2014 09:30 PM, Jan Hubicka wrote:
On Tuesday 11 February 2014 03:25 PM, Renato Golin wrote:
Hi Jan,
I think this is a very good example where we could all collaborate
(including binutils).
I'll leave your reply intact, so that Chandler (CC'd) can get a bit
more context. I'm copying him because he (and I believe Diego) had
more contact with LTO than I had.
If I got it right, LTO today:
- needs the drivers to explicitly declare the plugin
Yes.
- needs the library available somewhere
- may have to change the library loading semantics (via LD_PRELOAD)
Not in binutils implementation (I believe it is the case for darwin's libLTO).
With binutils you only need to pass explicit --plugin argument into all
tools that care (ld/ar/nm/ranlib)
There is another need that I have felt in LTO for quite some time.
Currently, it has a non-partitioned mode or a partitioned mode but
this decision is taken before the compilation begins. It would be
nice to have a mode that allows dynamic loading of function bodies
so that a flow and context sensitive IPA could load functions bodies
on demand, and unload them when they are not needed.
I implemented on-demand loading of function bodies into GCC-4.8 if I recall
correctly. Currently I thinko only Martin Liska's code unification pass uses it
to verify that two function bdoes it thinks are equivalent are actually
equivalent. Hopefully it will be merged into 4.10.
Great. We will experiment with it.
Uday.
Uday.
Since both toolchains do the magic, binutils has no incentive to
create any automatic detection of objects.
The part that I didn't get is when you say about backward
compatibility. Would LTO work on a newer binutils with the liblto but
on an older compiler that knew nothing about LTO?
Your proposal is, then, to get binutils:
- recognizing LTO logic in the objects
- automatically loading liblto if recognized
- warning if not
I basically think that binutils should have a way for installed compiler to
register a plugin and load all plugins by default (or perhaps for performance
or upon detecking an compatible LTO object file in some way, perhaps also by
information given in the config file) and let them claim the LTO objects they
understand to.
With the backward compatibility I mean that if we release a new version of
compiler that can no longer read the LTO objects of older compiler, one can
just install both versions and have their plugins to claim only LTO objects
they understand. Just if they were two different compilers.
Finally I think we can make binutils to recognize GCC/LLVM LTO objects
as a special case and produce friendly message when user try to handle
them witout plugin as oposed to today strange errors about file formats
or missing symbols.
Honza
I'm assuming the extra symbols would be discarded if no library is
found, together with the warning, right? Maybe an error if -Wall or
whatever.
Can we get someone from the binutils community to opine on that?
cheers,
--renato
On 11 February 2014 02:29, Jan Hubicka <hubi...@ucw.cz> wrote:
One practical experience I have with LLVM developers is sharing experiences
about getting Firefox to work with LTO with Rafael Espindola and I think it was
useful for both of us. I am definitly open to more discussion.
Lets try a specific topic that is on my TODO list for some time.
I would like to make it possible for mutliple compilers to be used to LTO a
single binary. As we are all making LTO more useful, I think it is matter of
time until people will start shipping LTO object files by default and users
will end up feeding them into different compilers or incompatible version of
the same compiler. We probably want to make this work, even thought the
cross-module optimization will not happen in this case.
The plugin interface in binutils seems to do its job well both for GCC and LLVM
and I hope that open64 and ICC will eventually join, too.
The trouble however is that one needs to pass explicit --plugin argument
specifying the particular plugin to load and so GCC ships with its own wrappers
(gcc-nm/gcc-ld/gcc-ar and the gcc driver itself) while LLVM does similar thing.
It may be smoother if binutils was able to load multiple plugins at once and
grab plugins from system and user installed compilers without explicit --plugin
argument.
Binutils probably should also have a way to detect LTO object files and produce
more useful diagnostic than they do now, when there is no plugin claiming them.
There are some PRs filled on the topic
http://cygwin.com/frysk/bugzilla/show_bug.cgi?id=15300
http://cygwin.com/frysk/bugzilla/show_bug.cgi?id=13227
but not much progress on them.
I wonder if we can get this designed and implemented.
On the other hand, GCC current maintains non-plugin path for LTO that is now
only used by darwin port due to lack of plugin enabled LD there. It seems
that liblto used by darwin is losely compatible with the plugin API, but it
makes
it harder to have different compilers share it (one has to LD_PRELOAD liblto
to different one prior executing the linker?)
I wonder, is there chance to implement linker plugin API to libLTO glue or add
plugin support to native Darwin tools?
Honza
