[
https://issues.apache.org/jira/browse/HADOOP-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456291#comment-13456291
]
Andy Isaacson commented on HADOOP-8806:
---------------------------------------
I ran a standalone test of {{DT_RPATH $ORIGIN}}. There's good news and bad
news which is probably ok.
On the plus side, setting DT_RPATH=$ORIGIN on libhadoop.so does allow it to
find libsnappy.so in the same directory. This is good.
On the downside, setting DT_RPATH=$ORIGIN on libhadoop.so pollutes the search
path for later dlopens in the main executable. In my test,
{noformat}
main.c
-> dlopens /path/to/libbar.so with DT_RPATH=$ORIGIN
-> dlopens libfoo.so
-> dlopens libfoo.so
{noformat}
Since main.c is supposed to be unaffected by the behavior of libbar, the final
dlopen should fail since main's search path does not include libfoo.
Unfortunately the final dlopen succeeds. This means that the libfoo opened by
libbar, while loaded, is available for open by main.
Modifying the test so that libbar.so dlclose()s libfoo before returning fixes
the problem; the final open of libfoo fails again. So it's not a problem of
libbar's DT_RPATH polluting the main executable's search path, but rather, a
single table of currently open objects.
What does this mean? Suppose we have a libsnappy.1.0.14 in $ORIGIN, and the
system has a libsnappy.1.0.20 in /usr/lib. A program which uses libhadoop will
get 1.0.14 if libhadoop is opened before libsnappy, and 1.0.20 if libsnappy is
opened before libhadoop. This kinda sucks. But, it's definitely a corner case,
so maybe it's OK.
If we used RUNPATH rather than RPATH, the user could work around the above
problem by setting LD_LIBRARY_PATH. Since it's probably safer from a build
perspective to use RPATH -- I couldn't even figure out how to get ld-2.22 to
set RUNPATH -- then LD_LIBRARY_PATH will not work.
> libhadoop.so: dlopen should be better at locating libsnappy.so, etc.
> --------------------------------------------------------------------
>
> Key: HADOOP-8806
> URL: https://issues.apache.org/jira/browse/HADOOP-8806
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8806.003.patch
>
>
> libhadoop calls {{dlopen}} to load {{libsnappy.so}} and {{libz.so}}. These
> libraries can be bundled in the {{$HADOOP_ROOT/lib/native}} directory. For
> example, the {{-Dbundle.snappy}} build option copies {{libsnappy.so}} to this
> directory. However, snappy can't be loaded from this directory unless
> {{LD_LIBRARY_PATH}} is set to include this directory.
> Can we make this configuration "just work" without needing to rely on
> {{LD_LIBRARY_PATH}}?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
