On 6/1/21 3:19 PM, Richard Biener wrote:
On Tue, Jun 1, 2021 at 1:25 PM Martin Liška <mli...@suse.cz> wrote:
On 6/1/21 9:42 AM, Richard Biener wrote:
On Tue, Jun 1, 2021 at 9:33 AM Martin Liška <mli...@suse.cz> wrote:
@Richi: Can you please reply to this email?
Not sure what I should add here? Honza suggested to mangle the
promoted symbol names.
Sure and I sent a patch for that.
I don't
really like the idea to compile multiple TUs into one object. Also
What's problematic is that we'll have to wait for one another release to make
it useful
(if you don't want to build the current master with a snapshot compiler).
IMHO it's a bugfix. Note that I'm not sure what the intent of the change is.
If it is to speedup bootstrap then using LTO bootstrap would do the trick
as well (and better) if we'd simply process all of libbackend.a this way
(and thus avoid re-linking that once for each frontend).
When building a GCC package, we intentionally re-link them with all FEs.
If it is to speedup
dev (re-)builds then dragging in more files will make it build longer since
for example insn-recog.c may be unchanged but gimple-match.c not.
Yes, the original motivation was a speed up of a dev. build and yes, the shown
example is problematic. Right now, I'm leaving that as I'm not interested enough
in the parallel build of a simple source file.
Martin
+LTO_LINKER_FLAGS = -flto=auto --param=lto-partitions=16
-flinker-output=nolto-rel -r
why hard-code to 16 partitions? You're side-stepping the driver
diagnostic by doing
compile & link separately, but in the end we're going to want sth like Giulianos
-fparallel-compile that works transparently from within the driver, so
the "manual"
operation should try to follow that or alternatively a driver-only
wrapper around the
"manual" processing could be added whose implementation can be optimized later.
All right. Do you want me refreshing his -fparallel-compile option introduction?
I'm not sure if we've arrived at mergeable state - but if it's
reasonably possible
to hide s/-fparallel-compile/-flto -r -flinker-output=nolto-rel/ split
into compile & link
parts (avoiding the diagnostic on -flinker-output) in the driver then
I think that's
a very reasonable first step (after fixing the symbol privatization issue). The
GSOC project then was to elide the IL streaming from the high-level operation.
Richard,
Why do you use -flto=auto? There should be a jobserver active.
Yes, that should not be needed.
Martin
On 5/21/21 10:43 AM, Martin Liška wrote:
On 5/20/21 2:54 PM, Richard Biener wrote:
On Thu, May 20, 2021 at 2:34 PM Martin Liška <mli...@suse.cz> wrote:
Hello.
I've got a patch candidate that leverages partial linking for a couple of
selected object files.
I'm sending make all-host- jX results for my machine:
before: 3m18s (user 32m52s)
https://gist.githubusercontent.com/marxin/223890df4d8d8e490b6b2918b77dacad/raw/1dd5eae5001295ba0230a689f7edc67284c9b742/gcc-all-host.svg
after: 2m57m (user 35m)
https://gist.githubusercontent.com/marxin/223890df4d8d8e490b6b2918b77dacad/raw/d659b2187cf622167841efbbe6bc93cb33855fa9/gcc-all-host-partial-lto.svg
One can utilize it with:
make -j16 all-host PARTIAL_LTO=1
@Segher, Andrew: Can you please measure time improvement for your slow
bootstrap?
One can also tweak --param=lto-partitions=16 param value.
Thoughts?
You're LTO linking multiple objects here - that's almost as if you
were doing this
for the whole of libbackend.a ... so $(OBJS)_CLFAGS += -flto and in the
libbackend.a rule do a similar partial link trick.
Yeah, apart from that one can't likely do partial linking for an archive:
$ g++ -no-pie -flto=auto --param=lto-partitions=16 -flinker-output=nolto-rel -r
libbackend.a
collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core
dumped
compilation terminated.
while ld.bfd immediately finishes.
That gets you half of a LTO bootstrap then.
So why did you go from applying this per-file to multiple files? Does $(LINKER)
have a proper rule to pick up a jobserver?
When upstreaming in any form you probably have to gate it on bootstrap-lto
being not active.
Sure, that's reasonable, we can likely detect a -flto option in $(COMPILE),
right?
One more thing I face is broken dependency:
$ make clean && make -j32 PARTIAL_LTO=1
g++ -fcf-protection -fno-PIE -c -g -DIN_GCC -fPIC -fno-exceptions
-fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings
-Wcast-qual -Wno-error=format-diag -Wmissing-format-attribute
-Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros
-Wno-overlength-strings -fno-common -Wno-unused -DHAVE_CONFIG_H -I. -I.
-I/home/marxin/Programming/gcc/gcc -I/home/marxin/Programming/gcc/gcc/.
-I/home/marxin/Programming/gcc/gcc/../include
-I/home/marxin/Programming/gcc/gcc/../libcpp/include
-I/home/marxin/Programming/gcc/gcc/../libcody
-I/home/marxin/Programming/gcc/gcc/../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libdecnumber/bid -I../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libbacktrace -o gimple-match-lto.o -MT
gimple-match-lto.o -MMD -MP -MF ./.deps/gimple-match-lto.TPo gimple-match.c
-flto
g++ -fcf-protection -fno-PIE -c -g -DIN_GCC -fPIC -fno-exceptions
-fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings
-Wcast-qual -Wno-error=format-diag -Wmissing-format-attribute
-Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros
-Wno-overlength-strings -fno-common -Wno-unused -DHAVE_CONFIG_H -I. -I.
-I/home/marxin/Programming/gcc/gcc -I/home/marxin/Programming/gcc/gcc/.
-I/home/marxin/Programming/gcc/gcc/../include
-I/home/marxin/Programming/gcc/gcc/../libcpp/include
-I/home/marxin/Programming/gcc/gcc/../libcody
-I/home/marxin/Programming/gcc/gcc/../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libdecnumber/bid -I../libdecnumber
-I/home/marxin/Programming/gcc/gcc/../libbacktrace -o generic-match-lto.o -MT
generic-match-lto.o -MMD -MP -MF ./.deps/generic-match-lto.TPo generic-match.c
-flto
In file included from ./tm.h:26,
from /home/marxin/Programming/gcc/gcc/backend.h:28,
from
/home/marxin/Programming/gcc/gcc/generic-match-head.c:23,
from generic-match.c:4:
/home/marxin/Programming/gcc/gcc/config/i386/i386.h:2286:10: fatal error:
insn-attr-common.h: No such file or directory
2286 | #include "insn-attr-common.h"
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:2678: generic-match-lto.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from ./tm.h:26,
from /home/marxin/Programming/gcc/gcc/backend.h:28,
from
/home/marxin/Programming/gcc/gcc/gimple-match-head.c:23,
from gimple-match.c:4:
/home/marxin/Programming/gcc/gcc/config/i386/i386.h:2286:10: fatal error:
insn-attr-common.h: No such file or directory
2286 | #include "insn-attr-common.h"
| ^~~~~~~~~~~~~~~~~~~~
I explicitly added:
gimple-match.o: gimple-match.c $(generated_files)
generic-match.o: generic-match.c $(generated_files)
But it's not obeyed.
Martin
Richard.
Thanks,
Martin
