Libtool and ASAN
Hi friends, I'm trying to use -fsanitize=address on OS X using MacPorts' Clang++ 3.5. The project consists of C++ libraries, on top of which is built a Python module (with a thin C++ layer which needs to be compiled). Libtool (2.4.2 - the name of a fine belgian band of electronic music btw) is used in all layers. To enable ASAN, I add '-fsanitize=address' to my CXXFLAGS. It works well except for the C++ part of the Python module: akim@erebus ~/src/lrde/vaucanson2 $ make -C _build/35d python/vcsn_cxx.la V=1 /bin/sh ./libtool --tag=CXX --mode=link ccache clang++-mp-3.5 -Wall -Wextra -Wcast-align -Wcast-qual -Wdocumentation -Wformat -Wmissing-declarations -Wno-parentheses -Woverloaded-virtual -Wpointer-arith -Wwrite-strings -Qunused-arguments -ggdb -fsanitize=address -std=c++11 -avoid-version -module -L/opt/local/lib -Wl,-rpath,/opt/local/lib -L/opt/local/lib -o python/vcsn_cxx.la -rpath /opt/gostai/lib/python2.7/site-packages python/python_vcsn_cxx_la-vcsn_cxx.lo -lboost_python-mt lib/liblal_char_b.la lib/liblal_char_br.la lib/liblal_char_q.la lib/liblal_char_r.la lib/liblal_char_z.la lib/liblal_char_zr.la lib/liblal_char_zrr.la lib/liblan_char_b.la lib/liblan_char_r.la lib/liblan_char_z.la lib/liblan_char_zr.la lib/liblao_br.la lib/liblao_z.la lib/liblaw_char_b.la lib/liblaw_char_br.la lib/liblaw_char_r.la lib/liblaw_char_z.la lib/liblaw_char_zr.la lib/liblaw_char_zrr.la lib/libvcsn.la libtool: link: rm -fr python/.libs/vcsn_cxx.la python/.libs/vcsn_cxx.lai python/.libs/vcsn_cxx.so libtool: link: ccache clang++-mp-3.5 -Wl,-undefined -Wl,dynamic_lookup -o python/.libs/vcsn_cxx.so -bundle python/.libs/python_vcsn_cxx_la-vcsn_cxx.o -L/opt/local/lib -lboost_python-mt lib/.libs/liblal_char_b.dylib lib/.libs/liblal_char_br.dylib lib/.libs/liblal_char_q.dylib lib/.libs/liblal_char_r.dylib lib/.libs/liblal_char_z.dylib lib/.libs/liblal_char_zr.dylib lib/.libs/liblal_char_zrr.dylib lib/.libs/liblan_char_b.dylib lib/.libs/liblan_char_r.dylib lib/.libs/liblan_char_z.dylib lib/.libs/liblan_char_zr.dylib lib/.libs/liblao_br.dylib lib/.libs/liblao_z.dylib lib/.libs/liblaw_char_b.dylib lib/.libs/liblaw_char_br.dylib lib/.libs/liblaw_char_r.dylib lib/.libs/liblaw_char_z.dylib lib/.libs/liblaw_char_zr.dylib lib/.libs/liblaw_char_zrr.dylib lib/.libs/libvcsn.dylib -lboost_filesystem-mt -lboost_system-mt -lltdl -Wl,-rpath -Wl,/opt/local/lib libtool: link: ( cd python/.libs rm -f vcsn_cxx.la ln -s ../vcsn_cxx.la vcsn_cxx.la ) What matters here is that -fsanitize=address is dropped, it is not passed to the linker. And I really need it: akim@erebus ~/src/lrde/vaucanson2 $ ./_build/35d/tests/bin/vcsn -e python -c 'import vcsn' Traceback (most recent call last): File string, line 1, in module File /Users/akim/src/lrde/vaucanson2/python/vcsn/__init__.py, line 4, in module from vcsn_cxx import * ImportError: dlopen(/Users/akim/src/lrde/vaucanson2/_build/35d/python/.libs/vcsn_cxx.so, 2): Symbol not found: ___asan_option_detect_stack_use_after_return Referenced from: /Users/akim/src/lrde/vaucanson2/_build/35d/lib/.libs/liblal_char_b.0.dylib Expected in: flat namespace in /Users/akim/src/lrde/vaucanson2/_build/35d/lib/.libs/liblal_char_b.0.dylib I have to use -Wc to force Libtool to pass it to the compiler used to link (I must not use -Wl, because then it is passed to the linker invoked by the compiler, and the linker rejects -fsanitize). akim@erebus ~/src/lrde/vaucanson2/_build/35d $ /bin/sh ./libtool --tag=CXX --mode=link ccache clang++-mp-3.5 -Wall -Wextra -Wcast-align -Wcast-qual -Wdocumentation -Wformat -Wmissing-declarations -Wno-parentheses -Woverloaded-virtual -Wpointer-arith -Wwrite-strings -Qunused-arguments -ggdb -Wc,-fsanitize=address -std=c++11 -avoid-version -module -L/opt/local/lib -Wl,-rpath,/opt/local/lib -L/opt/local/lib -o python/vcsn_cxx.la -rpath /opt/gostai/lib/python2.7/site-packages python/python_vcsn_cxx_la-vcsn_cxx.lo -lboost_python-mt lib/liblal_char_b.la lib/liblal_char_br.la lib/liblal_char_q.la lib/liblal_char_r.la lib/liblal_char_z.la lib/liblal_char_zr.la lib/liblal_char_zrr.la lib/liblan_char_b.la lib/liblan_char_r.la lib/liblan_char_z.la lib/liblan_char_zr.la lib/liblao_br.la lib/liblao_z.la lib/liblaw_char_b.la lib/liblaw_char_br.la lib/liblaw_char_r.la lib/liblaw_char_z.la lib/liblaw_char_zr.la lib/liblaw_char_zrr.la lib/libvcsn.la libtool: link: rm -fr python/.libs/vcsn_cxx.so.ld_QwmwqO libtool: link: ccache clang++-mp-3.5 -Wl,-undefined -Wl,dynamic_lookup -o python/.libs/vcsn_cxx.so -bundle python/.libs/python_vcsn_cxx_la-vcsn_cxx.o -L/opt/local/lib -lboost_python-mt lib/.libs/liblal_char_b.dylib lib/.libs/liblal_char_br.dylib lib/.libs/liblal_char_q.dylib lib/.libs/liblal_char_r.dylib lib/.libs/liblal_char_z.dylib lib/.libs/liblal_char_zr.dylib lib/.libs/liblal_char_zrr.dylib lib/.libs/liblan_char_b.dylib lib/.libs/liblan_char_r.dylib
Re: -no-undefined on Win32
20.04.2014, 05:15, Bob Friesenhahn bfrie...@simple.dallas.tx.us: On Fri, 18 Apr 2014, Evgeny Grin wrote: Libtool always defaults to successful compilation and link, to the maximum extent possible. That's nice, leave it to compiler and linker. If something can be compiled and linked, it will be compiled and linked. If it can't be, then compiler or linker will fail. Why giving up before even try? The GNU/Autoconf philosophy has always been that if software configures successfully that there should be a very high probability that the sofware will compile and work. Success at compiling on several platforms should indicate that it is highly likely to also compile on other platforms (including platforms that the package authors don't have access to). If the software fails to compile, or fails to work, there is substantial possibility that the user won't know how to solve the problem. Good. But requiring -no-undefined for Win32 flag lower probability of successful compile. It's absolutely normal that something can be compiled on one system and can't be on other system. According autoconf paradigm, code should be compiled if it use platform supported features. So configure checks host and target platforms, compiler, linker and other things and do dirty work to create correct configuration in makefiles, libtool, compiler wrapper etc. But if code designed to use some platform dependent features like epoll and epoll is not available on target platform then compile will fail. So let's add to libtool -disable-epoll flag and let libtool fail if target platform don't support epoll and this flag is not specified. It's far more logical to use flags like other GNU tools (GCC, binutils) use it. For example, this fictional flag -disable-epoll must prevent epoll usage on platforms that support epoll so code with epoll functions will fail to compile. On platforms without epoll support, code with epoll function will fail to comple in any case. Undefined symbols is just one of platform features. I can repeat again, let's simplify developer work instead of complicating. libtool must use -no-undefined for win32 targets automatically. Best Wishes, Evgeny ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: Forced static lib if any depend lib is static on win32
21.04.2014, 02:50, JonY 10wa...@gmail.com: On 4/19/2014 09:22, Evgeny Grin wrote: 19.04.2014, 04:45, JonY: On 4/19/2014 03:31, Evgeny Grin wrote: For XBMC we have 41 depend precompiled lib, 4 of them depend on zlib dll, all of 4 depend on zlib1.dll, but each one on specific zlib version. And with some zlib versions some of depend dlls crash. But it's just an example. Sometimes only a small fraction of lib functions is used, so it's better and wiser to statically link only those functions for shared lib and not redistribute heavy additional dll with your dll. There are far more possibile good uses for static libs in shared libs. MS dev tools allow any combinations of shared/static link, why libtool give worse possible options? I think you should be cleaning house, instead of allowing random bits and bobs to connect together. The real problem is that you have 4 incompatible versions of zlib1.dll in the first place. Of course cleaning is required, but it's not always possible to simple rebuild some particular lib on Win32. But I was talking about ability to link static in shared. This is relevant, it is what kicked off the discussion. How about a tighter reign over what gets contributed instead of randomly accepting executable code? Thanks for idea. Do you always have enough volunteers to get all tasks done? :) Libtool is good at preventing multiple exports on win32, I have seen disasters where zlib is exported *multiple* times in different DLLs of the same project because the author does not know what is going on. At least it was a compatible and same version of zlib. Right, it's nice idea to prevent static link for shared lib that blindly use --export-all-symbols. But for well-designed libs that use dllexport attribute or def-file disallowing static libs is just obstacle. You have to remember, using static libs with shared DLLs in itself is a bad idea, you still end up with *multiple* instance of it at runtime as well, even if it is not exported, moving the problem from link time to run time. Win32 dll processing is good designed. You will never have any problem at runtime with multiple functions with same name from different modules. They are isolated, just like local static variables. Above all, --export-all-symbols is default behavior in the absents of any dllexport attributes or def files, so yes, blocking static libraries is justified without introducing PE specific scanning rules. Unfortunately ld currently do not support --no-export-all-symbols (but dlltool does support). So I need to patch ld first. Don't think that special case dll-building with dlltool will be accepted for libtool code. :) Ok, I got the advantage of libtool, but still don't understand why this should prevent linking static libs in well-designed shared lib? Your best option is either go entirely static or DLLs only. Going somewhere in between is just plain trouble on Windows. What's the trouble? If you don't blindly use --export-all-symbols (either implicitly or explicitly) you will never have any trouble on win32. ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: -no-undefined on Win32
On Mon, 28 Apr 2014, Evgeny Grin wrote: Good. But requiring -no-undefined for Win32 flag lower probability of successful compile. In what way does it lower the probability of a successful compile? Static linkage is much more portable than dynamic. The situation you outlined is due to a defective package preparation/build environment. A proper build has just one version of a given library in a link. Regardless, it is very unlikely that libtool will react to your plea (if it does at all) in a timely fashion and so you are best advised to fix your build without relying on significant changes in libtool. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ https://lists.gnu.org/mailman/listinfo/libtool
Re: -no-undefined on Win32
29.04.2014, 05:59, Bob Friesenhahn bfrie...@simple.dallas.tx.us: On Mon, 28 Apr 2014, Evgeny Grin wrote: Good. But requiring -no-undefined for Win32 flag lower probability of successful compile. In what way does it lower the probability of a successful compile? Static linkage is much more portable than dynamic. That was another asked question: why libtool didn't fail if requested dynamic lib was not build. Anyway, if I'm building dynamic lib and it was not built, that's called unsuccessful build. The situation you outlined is due to a defective package preparation/build environment. A proper build has just one version of a given library in a link. Could you explain this a little bit? In the topic -no-undefined on Win32 I was talking about only one version of lib. Regardless, it is very unlikely that libtool will react to your plea (if it does at all) in a timely fashion and so you are best advised to fix your build without relying on significant changes in libtool. All my builds were fixed already. I'd like to improve libtool. If *uncommenting* one line and deleting other line in libtool are significant changes, then I'd like to significantly change libtool. :) Could you answer my main question: why libtool don't follow logic of other GNU tools? Instead of acting as a tool and passing required flags to compiler/linker, libtool is acting as mentor and does not do it work until you signal that you aware of something? -- Best Wishes, Evgeny Grin ___ https://lists.gnu.org/mailman/listinfo/libtool
