Setting extra classpath and library path for Yarn M/R Application

[Henning Blohm]

Hi,

I am having a hard time figuring out how to add command line options, an 
extra class path and library path to a map reduce application running on 
Hadoop 2.6.0 (CDH 5.6).

Unfortunately this application is using a native library with a 
pre-installed wrapper JAR that is installed on all data nodes. So what I 
am trying to achieve is to pass an LD_LIBRARY_PATH and an extra 
classpath to the application runner.

Previously, I used

    <property>
        <name>mapred.child.java.opts</name>
        <value>-XX:-UseSplitVerifier -Xmx500M -XX:MaxPermSize=256M 
-Dnosql.master=${nosql.master}</value>
    </property>

in the client config file for command line options and set

export HADOOP_CLASSPATH=/pathToWrapperJar
export JAVA_LIBRARY_PATH=/nativepath/

in my Hadoop environment.  None of this seems to work with Hadoop 2.6.0 
(Yarn).

Here is what I have now:

I managed to add to LD_LIBRARY_PATH by specifying

    <property>
        <name>yarn.app.mapreduce.am.env</name>
        <value>LD_LIBRARY_PATH=/nativepath</value>
    </property>

    <property>
        <name>yarn.app.mapreduce.am.command-opts</name>
        <value>-Da=1 -Db=2</value>
    </property>

in the client config file. I can tell that by cat'ing the 
launch_container.sh while it is present.

I do however fail to add anything to the application's class path. Plus 
I would rather not specify the native library path and wrapper jar in 
the client config but rather globally.

It would be great, if somebody can give me a hint first and foremost on 
the class path question and secondly some orientation on whether using 
the new configuration properties is the way to go.

Thanks,
Henning