Add Jar "lib" directory to TaskRunner's library.path setting to allow JNI
libraries to be deployed via JAR file
-----------------------------------------------------------------------------------------------------------------
Key: HADOOP-4577
URL: https://issues.apache.org/jira/browse/HADOOP-4577
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Affects Versions: 0.18.1
Environment: Hadoop 18.1 Cluster with custom JNI shared libraries
deployed in lib directory of deployment JAR.
Reporter: Ahad Rana
Assignee: Ahad Rana
Fix For: 0.18.3
It is extremely convenient to be able to deploy JNI libraries utilized in a
custom map-reduce job via the job's JAR file. The TaskRunner already
establishes a precedent by automatically adding any jar files contained in the
"lib" directory of the job jar to the child map/reduce process's classpath.
Following this convention, it should also be possible to deploy custom JNI
libraries in the same lib directory. This involves adding the path to the job
jar's lib directory to the VM's library.path setting (after the jar has been
expanded in the job cache directory). This does not elimintate the need add
dependent shared libraries that may be referenced by the JNI libraries to the
system's LD_LIBRARY_PATH variable. In our deployment configuration, we usually
pre-install third party shared libraries across the cluster and only deploy our
custom JNI libraries via the job jar.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
