Hi,
Ugh sorry something was weird with bjam. I've put a kludge in that
forces static linkage unless link=shared appears on the command line.
Kenneth
On 08/26/13 21:59, Lane Schwartz wrote:
> I'm attempting to compile Moses in such a way that at least the boost
> libraries are statically compiled. I'm fine if other libraries are shared.
>
> My interpretation of this thread (and BUILD-INSTRUCTIONS.txt) is that if
> I compile boost (following instructions in BUILD-INSTRUCTIONS.txt) so
> that it makes available static versions of its libraries, then compile
> moses using link=static, boost should be statically linked with moses.
> Unfortunately, that doesn't appear to be happening for me:
>
> ./bjam link=static -j8 --with-cmph=/tools/moses/cmph-2.0
> --with-srilm=/tools/SRILM/SRILM-1.7.0
> --with-boost=/tools/moses/boost_1_53_0 -q --debug-configuration -d2
>
> The log for the above is attached. If I try running moses after this
> compile (without adding /tools/moses/boost_1_53_0 to LIBRARY_PATH and
> LD_LIBRARY_PATH), I get the following error:
>
> ./bin/moses: error while loading shared libraries:
> libboost_system-mt.so.1.53.0: cannot open shared object file: No such
> file or directory
>
> Running ldd on bin/moses confirms that there is a dynamic boost library
> linking:
>
> linux-vdso.so.1 => (0x00007fff8adff000)
>
> libz.so.1 => /lib64/libz.so.1 (0x0000003949200000)
>
> librt.so.1 => /lib64/librt.so.1 (0x0000003949600000)
>
> libbz2.so.1 => /lib64/libbz2.so.1 (0x0000003955a00000)
>
> libboost_system-mt.so.1.53.0 => not found
>
> libSegFault.so => /lib64/libSegFault.so (0x00007f1671726000)
>
> libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x000000394fa00000)
>
> libm.so.6 => /lib64/libm.so.6 (0x0000003948600000)
>
> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x000000394d600000)
>
> libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003948e00000)
>
> libc.so.6 => /lib64/libc.so.6 (0x0000003948200000)
>
> /lib64/ld-linux-x86-64.so.2 (0x0000003947e00000)
>
>
> Looking in /tools/moses/boost_1_53_0/lib I can verify that the static .a
> file exists:
>
>
> libboost_system-mt.a
>
> libboost_system-mt.so
>
> libboost_system-mt.so.1.53.0
>
>
> Any help would be appreciated. I'm attaching the compile log and a
> listing of the boost files installed in /tools/moses/boost_1_53_0/lib.
>
>
> Thanks,
>
> Lane
>
>
>
>
> On Mon, Aug 6, 2012 at 12:39 PM, Kenneth Heafield <mo...@kheafield.com
> <mailto:mo...@kheafield.com>> wrote:
>
> Ok, committed. Here's how the build system now behaves:
>
> link=shared: Everything linked dynamically.
>
> Default: internal libraries are statically linked. Boost and zlib
> statically linked if possible. libSegFault dynamically linked.
> Dynamically linked executable.
>
> --without-libsegfault: Same as default but no libSegFault. Still a
> dynamically linked executable, even if you have static boost and zlib.
> It's complicated to do detect then be automatic.
>
> --static: No libSegFault. Print warning messages if you're missing
> static libraries, but keep building anyway. Static executable.
>
> Kenneth
>
> On 08/06/2012 12:00 PM, Kenneth Heafield wrote:
> > D'oh, it's a feature, not a bug. Add runtime-link=static and
> you'll get
> > a fully-static executable.
> >
> > Plan to add this to the Moses build system. Testing now.
> >
> > Kenneth
> >
> > On 08/06/2012 11:31 AM, Tom Hoar wrote:
> >> FYI, this build is on U-10.04.4 using the bjam shipped with
> Moses. I can
> >> send you the complete log if it helps. Let me know if you need
> anything
> >> else.
> >>
> >> Tom
> >>
> >>
> >> On Mon, 06 Aug 2012 11:27:27 -0400, Kenneth Heafield
> >> <mo...@kheafield.com <mailto:mo...@kheafield.com>> wrote:
> >>> Hi,
> >>>
> >>> There appears to be a bug in boost-build that prevents it from
> making
> >>> fully-static binaries with g++. This might take a while to sort out.
> >>>
> >>> Kenneth
> >>>
> >>> On 08/06/2012 10:48 AM, Tom Hoar wrote:
> >>>> Thanks, Ken. I think this is easier than adding another option
> to Moses
> >>>> if we can understand the output. The ldd manpage says "ldd
> prints the
> >>>> shared libraries required by each program or shared library
> specified on
> >>>> the command line." I assume "shared libraries" means not statically
> >>>> linked?
> >>>>
> >>>> Here's the output on the moses I just built with today's github
> updates.
> >>>> The log output reported exit code 0 for all lines with "g++
> -static".
> >>>>
> >>>> tahoar@library1:~$ ldd `which moses`
> >>>> linux-vdso.so.1 => (0x00007fffa2954000)
> >>>> librt.so.1 => /lib/librt.so.1 (0x00007f818aa7b000)
> >>>> libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f818a767000)
> >>>> libm.so.6 => /lib/libm.so.6 (0x00007f818a4e3000)
> >>>> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f818a2cc000)
> >>>> libpthread.so.0 => /lib/libpthread.so.0 (0x00007f818a0af000)
> >>>> libc.so.6 => /lib/libc.so.6 (0x00007f8189d2b000)
> >>>> /lib64/ld-linux-x86-64.so.2 (0x00007f818aca7000)
> >>>>
> >>>> Is this what I should expect if all of the libboost libraries are
> >>>> statically linked? I think so because there are no references
> to the
> >>>> lboost_* or lz libraries in the log file.
> >>>>
> >>>> Tom
> >>>>
> >>>>
> >>>> On Mon, 06 Aug 2012 10:14:35 -0400, Kenneth Heafield
> >>>> <mo...@kheafield.com <mailto:mo...@kheafield.com>> wrote:
> >>>>> On linux,
> >>>>>
> >>>>> ldd bin/moses
> >>>>>
> >>>>> On 08/06/2012 10:03 AM, Tom Hoar wrote:
> >>>>>> Thanks Ken. I downloaded/compiled with the latest changes up to
> >>>>>> BOOST_CHECK_CLOSE. The exit code 1 disappeared and everything
> seems
> >>>>>> okay.
> >>>>>>
> >>>>>> One more question. Other than the log output at compile time,
> is there
> >>>>>> any way to query the Moses binary to see which libraries are
> >>>>>> statically
> >>>>>> vs dynamically linked?
> >>>>>>
> >>>>>> For example, a while back, someone on the list gave this
> command to
> >>>>>> test
> >>>>>> if the binary was compiled --with-srilm, and there are others for
> >>>>>> IRSTLM, RANDLM and KENLM.
> >>>>>>
> >>>>>> nm -C "/path/to/moses" | grep "vtable for
> Moses::LanguageModelSRI"`
> >>>>>>
> >>>>>> I tried the obvious two grep searches "static" and "dynamic"
> (below),
> >>>>>> but they don't seem to relate to the libraries. Does anyone
> know a way
> >>>>>> to find/test if libraries are dynamically or statically linked?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Tom
> >>>>>>
> >>>>>>
> >>>>>> tahoar@library1:~/domy-2.5$ nm -C `which moses` | grep "static"
> >>>>>> 00000000005ddbc0 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005de7c1 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005ded37 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005e0067 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005e2fc6 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005ecae3 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005ef140 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f475f t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f4cd9 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f5e53 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f6f8e t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f7183 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 0000000000883cc0 d static_bl_desc
> >>>>>> 0000000000883ca0 d static_d_desc
> >>>>>> 0000000000610680 r static_dtree
> >>>>>> 0000000000883c80 d static_l_desc
> >>>>>> 0000000000610200 r static_ltree
> >>>>>>
> >>>>>> tahoar@library1:~/domy-2.5$ nm -C `which moses` | grep "dynamic"
> >>>>>> 0000000000570c50 W
> >>>>>>
> boost::unordered_detail::hash_table_unique_keys<std::pair<float const,
> >>>>>> boost::dynamic_bitset<unsigned long, std::allocator<unsigned
> long>
> >>>>>>>> ,
> >>>>>> float, boost::hash<float>, std::equal_to<float>,
> >>>>>> std::allocator<std::pair<float const,
> boost::dynamic_bitset<unsigned
> >>>>>> long, std::allocator<unsigned long> > > >
> >::operator[](float const&)
> >>>>>> 000000000057baf0 W
> >>>>>>
> >>>>>>
> >>>>>>
>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<std::pair<unsigned
> >>>>>>
> >>>>>>
> >>>>>> char, unsigned char> const, boost::dynamic_bitset<unsigned long,
> >>>>>> std::allocator<unsigned long> > > >
> >::~hash_table_data_unique_keys()
> >>>>>> 0000000000570af0 W
> >>>>>>
> >>>>>>
> >>>>>>
>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<float
> >>>>>>
> >>>>>>
> >>>>>> const, boost::dynamic_bitset<unsigned long,
> std::allocator<unsigned
> >>>>>> long> > > > >::create_buckets()
> >>>>>> 0000000000570c00 W
> >>>>>>
> >>>>>>
> >>>>>>
>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<float
> >>>>>>
> >>>>>>
> >>>>>> const, boost::dynamic_bitset<unsigned long,
> std::allocator<unsigned
> >>>>>> long> > > > >::node_constructor::~node_constructor()
> >>>>>> 0000000000570b70 W
> >>>>>>
> >>>>>>
> >>>>>>
>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<float
> >>>>>>
> >>>>>>
> >>>>>> const, boost::dynamic_bitset<unsigned long,
> std::allocator<unsigned
> >>>>>> long> > > > >::~hash_table_data_unique_keys()
> >>>>>> 000000000057ba60 W
> >>>>>>
> >>>>>>
> >>>>>>
>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<unsigned
> >>>>>>
> >>>>>>
> >>>>>> int const, boost::dynamic_bitset<unsigned long,
> >>>>>> std::allocator<unsigned
> >>>>>> long> > > > >::~hash_table_data_unique_keys()
> >>>>>> U __dynamic_cast@@CXXABI_1.3
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Mon, 06 Aug 2012 08:20:00 -0400, Kenneth Heafield
> >>>>>> <mo...@kheafield.com <mailto:mo...@kheafield.com>> wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> You're correct. There doesn't seem to be a static version of
> this
> >>>>>>> library. I've added the --nosegfault option (which isn't as
> cool as
> >>>>>>> it sounds) to skip this library.
> >>>>>>>
> >>>>>>> Kenneth
> >>>>>>>
> >>>>>>> On 08/06/2012 02:08 AM, Tom Hoar wrote:
> >>>>>>>> I read the comment "In order to obtain a fully static Moses,
> >>>>>>>> every g++
> >>>>>>>> command line that includes "-static" must pass with exit
> code 0."
> >>>>>>>> with
> >>>>>>>> interest.
> >>>>>>>>
> >>>>>>>> When we compile moses, this log output line shows an exit
> code 1.
> >>>>>>>> All
> >>>>>>>> the others are 0.
> >>>>>>>>
> >>>>>>>> bash -c "g++ -static -lSegFault -x c++ -<<<'int main() {}' -o
> >>>>>>>> /dev/null
> >>>>>>>>> /dev/null 2>/dev/null"
> >>>>>>>> 1
> >>>>>>>>
> >>>>>>>> Any suggestions as to what's missing/how to correct so we
> have a
> >>>>>>>> fully-static Moses?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Tom
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, 03 Aug 2012 18:13:47 -0400, Kenneth Heafield
> >>>>>>>> <mo...@kheafield.com <mailto:mo...@kheafield.com>> wrote:
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Moses attempts to link statically by default but falls back to
> >>>>>>>>> dynamic
> >>>>>>>>> links. You must have static versions of all the dependencies
> >>>>>>>>> installed
> >>>>>>>>> as well. Run
> >>>>>>>>>
> >>>>>>>>> bjam --debug-configuration
> >>>>>>>>>
> >>>>>>>>> and, near the top, it will show you some command lines
> followed by
> >>>>>>>>> their
> >>>>>>>>> exit code. In order to obtain a fully static Moses, every g++
> >>>>>>>>> command
> >>>>>>>>> line that includes "-static" must pass with exit code 0.
> >>>>>>>>>
> >>>>>>>>> Kenneth
> >>>>>>
> >>>>
> >>
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> When a place gets crowded enough to require ID's, social collapse is not
> far away. It is time to go elsewhere. The best thing about space travel
> is that it made it possible to go elsewhere.
> -- R.A. Heinlein, "Time Enough For Love"
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
