What happened is that the Util library was linked to the error-utils-test (which it is by default anyways because of the way how ADD_BE_TEST works). Now, what happened is that if you dynamically link the order of the libraries plays a role. The additional target-link-library would move the libUtil.so to the end of the library list and if the linker picks this up some of the symbols declared earlier where already dropped.
On Wed, Aug 24, 2016 at 11:00 AM Henry Robinson <he...@cloudera.com> wrote: > Seems great to me - but do we know why this fixes the issue? Not that that > should prevent us from merging the patch. > > On 24 August 2016 at 08:54, Tim Armstrong <tarmstr...@cloudera.com> wrote: > > > The fix works for me too, thanks Martin! > > > > I gave the patch a +1 but will upgrade to +2 if it seems everyone is > happy > > with it. > > > > Cheers, > > Tim > > > > On Wed, Aug 24, 2016 at 8:15 AM, Yonghyun Hwang <yongh...@cloudera.com> > > wrote: > > > > > Hello Martin, > > > > > > Your fix works great! :) As for the build failure, what I've found is > > that > > > cmake 3.2.3 causes a build failure w/o your fix while cmake 2.8.12 > works > > > good. > > > > > > After the commit for "IMPALA-3924: Ubuntu16 support", cmake 3.2.3 got > > > deployed under "toolchain", which reveals this build issue. > > > > > > So, any one using the newer cmake would require your fix to properly > > build > > > impala w/ -so. > > > > > > Thank you, > > > Yonghyun > > > > > > On Tue, Aug 23, 2016 at 11:40 PM, Daniel Hecht <dhe...@cloudera.com> > > > wrote: > > > > > > > Hi Martin, > > > > > > > > Nice to hear from you. > https://issues.cloudera.org/browse/IMPALA-3979 > > > > > > > > Thanks, > > > > Dan > > > > > > > > On Tuesday, August 23, 2016, Martin Grund (Das Grundprinzip.de) < > > > > grundprin...@gmail.com> wrote: > > > > > > > > > I've seen this in the past and propose a patch to fix it: > > > > > https://gerrit.cloudera.org/#/c/4108/ > > > > > > > > > > Let me know if you have JIRA to track this to update the commit > > > message. > > > > > > > > > > Martin > > > > > > > > > > On Mon, Aug 22, 2016 at 10:15 PM Laszlo Gaal < > > laszlo.g...@cloudera.com > > > > > <javascript:;>> > > > > > wrote: > > > > > > > > > > > link.txt is generated by cmake based on the contents of > > > CMakeLists.txt > > > > > and > > > > > > the project/environment settings. Most of the link-time > > dependencies > > > > seem > > > > > > to be set up in $(IMPALA_HOME)/be/CMakeLists.txt, based on > > settings > > > > > > calculated one level up in $(IMPALA_HOME)/CMakeLists.txt. > > > > > > $(IMPALA_HOME)/be/CMakeLists.txt looks complex; I'll look more > > into > > > it > > > > > > tomorrow. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > - LaszloG > > > > > > > > > > > > On Mon, Aug 22, 2016 at 6:07 PM, Yonghyun Hwang < > > > yongh...@cloudera.com > > > > > <javascript:;>> > > > > > > wrote: > > > > > > > > > > > > > I observed the same linking error and digged this into further. > > The > > > > > below > > > > > > > is what I've found. > > > > > > > > > > > > > > *1. how to reproduce* > > > > > > > > > > > > > > # assuming that impala-config.sh is sourced > > > > > > > $ cd ${IMPALA_HOME} > > > > > > > *$ make -j8 impalad* > > > > > > > ... > > > > > > > [100%] Built target CodeGen > > > > > > > Linking CXX executable ../../build/debug/service/impalad > > > > > > > ... > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > `google_breakpad::ExceptionHandler::WriteMinidump(std::string > > > > const&, > > > > > > bool > > > > > > > (*)(google_breakpad::MinidumpDescriptor const&, void*, bool), > > > > void*)' > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > `snappy::MaxCompressedLength(unsigned long)' > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > `my_strlen' > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > `LZ4_compress' > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > `BZ2_bzDecompress' > > > > > > > ... > > > > > > > > > > > > > > > > > > > > > *$ cd ${IMPALA_HOME}/be/src/service* > > > > > > > *$ ${IMPALA_HOME}/toolchain/cmake-3.2.3-p1/bin/cmake -E * > > > > > > > *cmake_link_script CMakeFiles/impalad.dir/link.txt --verbose=* > > > > > > > ... > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > ... > > > > > > > *../../build/debug/util/libUtil.so: undefined reference to > > > > > > `LZ4_compress'* > > > > > > > *../../build/debug/util/libUtil.so: undefined reference to > > > > > > > `BZ2_bzDecompress'* > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > > > > *`snappy::RawCompress*(char const*, > > > > > > > unsigned long, char*, unsigned long*)' > > > > > > > ... > > > > > > > boost::regex_traits<char, boost::cpp_regex_traits<char> > > > > > > > >::do_assign(char > > > > > > > const*, char > > > > > > > const*, unsigned int)' > > > > > > > ... > > > > > > > ../../build/debug/util/libUtil.so: undefined reference to > > > > *`snappy::* > > > > > > > RawUncompress(char > > > > > > > const*, unsigned long, char*)' > > > > > > > > > > > > > > *2. cause of the failure* > > > > > > > > > > > > > > $ cd ${IMPALA_HOME}/be/src/service > > > > > > > *$ cat CMakeFiles/impalad.dir/link.txt* > > > > > > > ${IMPALA_HOME}/toolchain/gcc-4.9.2/bin/g++ -Wall > > > > -Wno-sign-compare > > > > > > > -Wno-unknown-pragmas -pthread -fno-strict-aliasing -std=c++14 > > > > > > > -Wno-deprecated ... > > > > > > > *../../../toolchain/snappy-1.1.3/lib/libsnappy.a > > > > > > > ../../../toolchain/lz4-svn/lib/liblz4.a* > > > > > > > ... > > > > > > > *../../../toolchain/boost-1.57.0/lib/libboost_regex.a* > > > > > > > ... > > > > > > > *../../build/debug/util/libUtil.so* > > > > > > > -lpthread -ldl -lssl -lcrypto > > > > > > > > > > > > > > Apparently, libUtil.so __should__ show before libsnappy.a and > > > > *regex.a. > > > > > > > > > > > > > > > > > > > > > Next step is to find out who creates > CMakeFiles/impalad.dir/link. > > > txt > > > > > and > > > > > > > why libUtil.so is placed after libsnappy.a, liblz4.a, and > > > > > > libboost_regex.a > > > > > > > > > > > > > > Thank you, > > > > > > > Yonghyun > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Aug 17, 2016 at 10:13 AM, Henry Robinson < > > > he...@cloudera.com > > > > > <javascript:;>> > > > > > > > wrote: > > > > > > > > > > > > > > > Only that it's IMPALA-3979. Does anyone have any theories > yet? > > > > > > > > > > > > > > > > On 17 August 2016 at 10:10, Dimitris Tsirogiannis < > > > > > > > > dtsirogian...@cloudera.com <javascript:;>> wrote: > > > > > > > > > > > > > > > > > Hey guys, > > > > > > > > > > > > > > > > > > When I build from the latest master using the > > > -build_shared_libs > > > > > > > option I > > > > > > > > > get linking errors (see [1]). Building with static works > > fine. > > > > Any > > > > > > > ideas? > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > Dimitris > > > > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > Linking CXX executable ../../build/debug/service/impalad > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'BZ2_bzBuffToBuffCompress' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'snappy::MaxCompressedLength(unsigned long)' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'snappy::RawCompress(char const*, unsigned long, char*, > > > unsigned > > > > > > > long*)' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'LZ4_compress' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'BZ2_bzDecompressEnd' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'BZ2_bzDecompressInit' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'BZ2_bzBuffToBuffDecompress' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'BZ2_bzDecompress' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'snappy::GetUncompressedLength(char const*, unsigned long, > > > > > unsigned > > > > > > > > > long*)' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'snappy::RawUncompress(char const*, unsigned long, char*)' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'LZ4_uncompress' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'my_strlen' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'google_breakpad::ExceptionHandler:: > > WriteMinidump(std::string > > > > > const&, > > > > > > > > bool > > > > > > > > > (*)(google_breakpad::MinidumpDescriptor const&, void*, > > bool), > > > > > void*)' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > > 'google_breakpad::ExceptionHandler::ExceptionHandler(google_ > > > > > > > > > breakpad::MinidumpDescriptor > > > > > > > > > const&, bool (*)(void*), bool (*)(google_breakpad:: > > > > > MinidumpDescriptor > > > > > > > > > const&, void*, bool), void*, bool, int)' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'ProfilerStart' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'ProfilerStop' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'boost::basic_regex<char, boost::regex_traits<char, > > > > > > > > > boost::cpp_regex_traits<char> > >::do_assign(char const*, > > char > > > > > > const*, > > > > > > > > > unsigned int)' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'boost::re_detail::perl_matcher<__gnu_cxx::__normal_ > > > > iterator<char > > > > > > > > const*, > > > > > > > > > std::string>, > > > > > > > > > std::allocator<boost::sub_match<__gnu_cxx::__normal_ > > > > iterator<char > > > > > > > > const*, > > > > > > > > > std::string> > >, boost::regex_traits<char, > > > > > > > > boost::cpp_regex_traits<char> > > > > > > > > > > >::find()' > > > > > > > > > ../../build/debug/util/libUtil.so: error: undefined > > reference > > > to > > > > > > > > > 'boost::re_detail::perl_matcher<__gnu_cxx::__normal_ > > > > iterator<char > > > > > > > > const*, > > > > > > > > > std::string>, > > > > > > > > > std::allocator<boost::sub_match<__gnu_cxx::__normal_ > > > > iterator<char > > > > > > > > const*, > > > > > > > > > std::string> > >, boost::regex_traits<char, > > > > > > > > boost::cpp_regex_traits<char> > > > > > > > > > > >::construct_init(boost::basic_regex<char, > > > > > boost::regex_traits<char, > > > > > > > > > boost::cpp_regex_traits<char> > > const&, > > > > > > > > > boost::regex_constants::_match_flags)' > > > > > > > > > /opt/Impala-Toolchain/openldap-2.4.25/lib/libldap.a( > > > > > > > > > os-ip.o):os-ip.c:function > > > > > > > > > ldap_int_poll: warning: `sys_nerr' is deprecated; use > > > `strerror' > > > > or > > > > > > > > > `strerror_r' instead > > > > > > > > > /opt/Impala-Toolchain/openldap-2.4.25/lib/libldap.a( > > > > > > > > > os-ip.o):os-ip.c:function > > > > > > > > > ldap_int_poll: warning: `sys_errlist' is deprecated; use > > > > `strerror' > > > > > > or > > > > > > > > > `strerror_r' instead > > > > > > > > > collect2: error: ld returned 1 exit status > > > > > > > > > be/src/service/CMakeFiles/impalad.dir/build.make:166: > recipe > > > for > > > > > > > target > > > > > > > > > 'be/build/debug/service/impalad' failed > > > > > > > > > make[3]: *** [be/build/debug/service/impalad] Error 1 > > > > > > > > > CMakeFiles/Makefile2:3682: recipe for target > > > > > > > > > 'be/src/service/CMakeFiles/impalad.dir/all' failed > > > > > > > > > make[2]: *** [be/src/service/CMakeFiles/impalad.dir/all] > > > Error 2 > > > > > > > > > CMakeFiles/Makefile2:3694: recipe for target > > > > > > > > > 'be/src/service/CMakeFiles/impalad.dir/rule' failed > > > > > > > > > make[1]: *** [be/src/service/CMakeFiles/impalad.dir/rule] > > > Error > > > > 2 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Henry Robinson > > > > > > > > Software Engineer > > > > > > > > Cloudera > > > > > > > > 415-994-6679 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > Henry Robinson > Software Engineer > Cloudera > 415-994-6679 >
