Re: link-time optimization
On 04/07/2010 08:36 AM, Ralf Wildenhues wrote: * Török Edwin wrote on Sun, Apr 04, 2010 at 06:03:35PM CEST: On 04/04/2010 11:54 AM, Ralf Wildenhues wrote: I have a preliminary(!) patch set here for LTO support in libtool. Thanks a lot for the patches, they worked fine so far! Thanks for testing. I'd have one more request: please add -use-gold-plugin to the list of flags passed through to support LLVM's LTO described here: http://llvm.org/docs/GoldPlugin.html Yes. I'm not sure whether I mentioned it before, but I didn't manage to build a working llvm/clang from its svn last weekend (I tried about three revisions and all of them failed to build with syntax errors, so I gave up for the moment). If you tell me the error message, and how you configured LLVM I can probably fix it (I have commit access). Note that llvm-gcc should be built with objdir != srcdir, since it has an Apple-specific GNUmakefile in the root causes srcdir == objdir builds to fail, you should pass --enable-languages since not all languages that are supported by gcc are supported by LLVM. Do they not have a checkin policy that code needs to compile at least? There are some buildbots that test whether code compiles, and build/test failures are usually quickly fixed/reverted. All looks green (except some selfhosting and valgrind failures): http://google1.osuosl.org:8011/waterfall You can try the 2.7 prerelease branch (branches/release_27), or the tarballs here: http://llvm.org/pre-releases/2.7/pre-release1/ Anyway, it's good that you did some testing here. Can you please also test the Libtool package with this: ./configure CC=/home/edwin/llvm2.7/llvm-gcc-4.2-2.7-x86_64-linux/bin/llvm-gcc CXX=/home/edwin/llvm2.7/llvm-gcc-4.2-2.7-x86_64-linux/bin/llvm-g++ CFLAGS=-O4 -use-gold-plugin CXXFLAGS=-O4 -use-gold-plugin and run its testsuites and report results (see README for how)? Thanks. After adding the -use-gold-plugin to libltdl/config/ltmain.m4sh I 1 failure: FAIL: tests/pdemo-make.test 1 of 113 tests failed (11 tests were not run) 18: duplicate convenience archive names FAILED (duplicate_conv.at:79) 102: Run tests with low max_cmd_len FAILED (cmdline_wrap.at:43) ## - ## ## Test results. ## ## - ## ERROR: 99 tests were run, 6 failed (4 expected failures). 7 tests were skipped. I think these are the the same as gcc+gold (since LLVM uses gold for LTO too). The duplicate_conv failure is because apparently libtool doesn't pass -use-gold-plugin here (hence LD doesn't know that it needs to use the plugin to link, and the .o files are not ELF objects, as with gcc): ./duplicate_conv.at:79: $LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS -o cee.$OBJEXT c.lo a/liba.la b/liba.la stderr: /usr/bin/ld: error: .libs/c.o:1:3: invalid character /usr/bin/ld: error: .libs/c.o:1:3: syntax error, unexpected $end /usr/bin/ld: error: .libs/c.o: not an object or archive /usr/bin/ld: error: a/.libs/liba.a: member at 78 is not an ELF object /usr/bin/ld: error: b/.libs/liba.a: member at 86 is not an ELF object /usr/bin/ld: error: b/.libs/liba.a: member at 626 is not an ELF object ac_cv_c_fpu_bigendian=no How come you need to use that, and which Autoconf version do you use? This sounds like it needs to be addressed in Autoconf. Thanks. No this is a test implement in ClamAV's acinclude.m4 (yes I know it uses the wrong macro namespace). Problem is it uses 'grep' to find out if the compiled code has big-endian or little-endian FPU word ordering. That works with gcc, but doesn't work with LLVM's bitcode files (since the actualy constant is VBR-encoded). Everything worked fine, and make check passed too! (Now of course it didn't LTO the entire program because -fuse-linker-plugin triggers PR41584) AFAIK -fuse-linker-plugin does not trigger PR 41584, only -fwhopr does. Did you mean one of the other ones? -fuse-linker-plugin hits #42453 which is similar to 41584 (same assertion), either the optimizer creates empty TU, or it drops some symbols and later tries to write out the non-existent symbol. Best regards, --Edwin
Re: link-time optimization
On 04/07/2010 08:36 AM, Ralf Wildenhues wrote: * Török Edwin wrote on Sun, Apr 04, 2010 at 06:03:35PM CEST: On 04/04/2010 11:54 AM, Ralf Wildenhues wrote: I have a preliminary(!) patch set here for LTO support in libtool. Thanks a lot for the patches, they worked fine so far! Thanks for testing. I'd have one more request: please add -use-gold-plugin to the list of flags passed through to support LLVM's LTO described here: http://llvm.org/docs/GoldPlugin.html Yes. I'm not sure whether I mentioned it before, but I didn't manage to build a working llvm/clang from its svn last weekend (I tried about three revisions and all of them failed to build with syntax errors, so I gave up for the moment). If you tell me the error message, and how you configured LLVM I can probably fix it (I have commit access). Note that llvm-gcc should be built with objdir != srcdir, since it has an Apple-specific GNUmakefile in the root causes srcdir == objdir builds to fail, you should pass --enable-languages since not all languages that are supported by gcc are supported by LLVM. Do they not have a checkin policy that code needs to compile at least? There are some buildbots that test whether code compiles, and build/test failures are usually quickly fixed/reverted. All looks green (except some selfhosting and valgrind failures): http://google1.osuosl.org:8011/waterfall You can try the 2.7 prerelease branch (branches/release_27), or the tarballs here: http://llvm.org/pre-releases/2.7/pre-release1/ Anyway, it's good that you did some testing here. Can you please also test the Libtool package with this: ./configure CC=/home/edwin/llvm2.7/llvm-gcc-4.2-2.7-x86_64-linux/bin/llvm-gcc CXX=/home/edwin/llvm2.7/llvm-gcc-4.2-2.7-x86_64-linux/bin/llvm-g++ CFLAGS=-O4 -use-gold-plugin CXXFLAGS=-O4 -use-gold-plugin and run its testsuites and report results (see README for how)? Thanks. After adding the -use-gold-plugin to libltdl/config/ltmain.m4sh I 1 failure: FAIL: tests/pdemo-make.test 1 of 113 tests failed (11 tests were not run) 18: duplicate convenience archive names FAILED (duplicate_conv.at:79) 102: Run tests with low max_cmd_len FAILED (cmdline_wrap.at:43) ## - ## ## Test results. ## ## - ## ERROR: 99 tests were run, 6 failed (4 expected failures). 7 tests were skipped. I think these are the the same as gcc+gold (since LLVM uses gold for LTO too). The duplicate_conv failure is because apparently libtool doesn't pass -use-gold-plugin here (hence LD doesn't know that it needs to use the plugin to link, and the .o files are not ELF objects, as with gcc): ./duplicate_conv.at:79: $LIBTOOL --mode=link --tag=CC $CC $CFLAGS $LDFLAGS -o cee.$OBJEXT c.lo a/liba.la b/liba.la stderr: /usr/bin/ld: error: .libs/c.o:1:3: invalid character /usr/bin/ld: error: .libs/c.o:1:3: syntax error, unexpected $end /usr/bin/ld: error: .libs/c.o: not an object or archive /usr/bin/ld: error: a/.libs/liba.a: member at 78 is not an ELF object /usr/bin/ld: error: b/.libs/liba.a: member at 86 is not an ELF object /usr/bin/ld: error: b/.libs/liba.a: member at 626 is not an ELF object ac_cv_c_fpu_bigendian=no How come you need to use that, and which Autoconf version do you use? This sounds like it needs to be addressed in Autoconf. Thanks. No this is a test implement in ClamAV's acinclude.m4 (yes I know it uses the wrong macro namespace). Problem is it uses 'grep' to find out if the compiled code has big-endian or little-endian FPU word ordering. That works with gcc, but doesn't work with LLVM's bitcode files (since the actualy constant is VBR-encoded). Everything worked fine, and make check passed too! (Now of course it didn't LTO the entire program because -fuse-linker-plugin triggers PR41584) AFAIK -fuse-linker-plugin does not trigger PR 41584, only -fwhopr does. Did you mean one of the other ones? -fuse-linker-plugin hits #42453 which is similar to 41584 (same assertion), either the optimizer creates empty TU, or it drops some symbols and later tries to write out the non-existent symbol. Best regards, --Edwin ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: link-time optimization
On 04/04/2010 11:54 AM, Ralf Wildenhues wrote: * Ralf Wildenhues wrote on Sun, Mar 28, 2010 at 05:33:44PM CEST: I'm still working on a patch to support -flto in libtool, it needs changes in several places. I have a preliminary(!) patch set here for LTO support in libtool. Hi, Thanks a lot for the patches, they worked fine so far! I'd have one more request: please add -use-gold-plugin to the list of flags passed through to support LLVM's LTO described here: http://llvm.org/docs/GoldPlugin.html I've applied your patches, re-autoconfed ClamAV, and tested a GCC build with -flto: ./configure CFLAGS=-flto -O2 CXXFLAGS=-flto -O2 CC=gcc-4.5 CXX=g++-4.5 Everything worked fine, and make check passed too! (Now of course it didn't LTO the entire program because -fuse-linker-plugin triggers PR41584) If I edit libtool and add -use-gold-plugin (after -fuse-linker-plugin), then I can build ClamAV with llvm2.7[*] without using the workarounds described on that page, by simply doing this: ./configure CC=/home/edwin/llvm2.7/llvm-gcc-4.2-2.7-x86_64-linux/bin/llvm-gcc CXX=/home/edwin/llvm2.7/llvm-gcc-4.2-2.7-x86_64-linux/bin/llvm-g++ CFLAGS=-O4 -use-gold-plugin CXXFLAGS=-O4 -use-gold-plugin ac_cv_c_fpu_bigendian=no This worked too (with less optimizations): ./configure CC=/home/edwin/llvm2.7/llvm-gcc-4.2-2.7-x86_64-linux/bin/llvm-gcc CXX=/home/edwin/llvm2.7/llvm-gcc-4.2-2.7-x86_64-linux/bin/llvm-g++ CFLAGS=-flto -use-gold-plugin CXXFLAGS=-flto -use-gold-plugin ac_cv_c_fpu_bigendian=no I also had to add a symlink to system's libstdc++.so (the llvm-gcc binary ships with a libstdc++.la, and a libstdc++.a which break linking LTO programs, because they are not PIC). [*] After copying libLLVMgold.so as described in http://llvm.org/docs/GoldPlugin.html#lto_autotools DO NOT submit this to your favorite distribution, it is not ready for general consumption yet. Several upstream issues to fix here: - gold bug http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42757 (common symbols not emitted with -fuse-linker-plugin) - GCC bug http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43576 (ICE w/ partial linking) - GCC bug http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43542 (segfault if __gnu_lto_v1 is referenced) - gold bug http://sourceware.org/bugzilla/show_bug.cgi?id=11182 (names in linker scripts in subdirs are relative to subdir) - GCC bug http://gcc.gnu.org/bugzilla/show_bug.cgi?id=41584 (-fwhopr ICE with TU that contains no function) Yep, I'm hitting this one, so I can't test -fuse-linker-plugin. Best regards, --Edwin
Re: Bug#517501: libtool: uninitialized shell variable causes link failure when run under zsh
On 2009-02-28 14:29, Ralf Wildenhues wrote: tags fixed-upstream thanks [ adding libtool-patches; see http://bugs.debian.org/517501 ] OK, here's the deal: 'emulate sh' turns off the special properties of the $path array, but keeps its first entry: $ zsh -c 'echo $path.; emulate sh; echo $path.' /usr/local/bin /usr/bin /bin /usr/bin/X11 /usr/games /usr/X11R6/bin. /usr/local/bin. BTW, this doesn't happen if zsh is invoked as sh: $ ln -s /usr/bin/zsh sh; ./sh -c 'echo $path.; emulate sh; echo $path.' . . Not sure whether that may be considered a bug in zsh or not, but anyway libtool should not assume that the ordinary variable $path is unset. Your proposed patch does not set $path in all possible cases, so I'm applying this one to upstream GNU Libtool to fix it. Thanks. Is there a release date planned for next libtool release? I would like to know if I should apply this patch to ClamAV's config/ltmain.sh, or wait until a new version of libtool is released (ClamAV's release is planned for March 16th). Best regards, --Edwin
