Sometimes compiled native libraries included in tar ball doesn't work correctly - how about recompiling library on your environment?
Thanks, - Tsuyoshi On Tue, Mar 24, 2015 at 6:18 PM, 王鹏飞 <wpf5...@gmail.com> wrote: > I noticed a map-reduce job encountered an > Exception:java.lang.UnsatisfiedLinkError:org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z > I googled it and released that it was lack of snappy tool.I > Installed the snappy and copied libsnappy.so* to $HADOOP_HOME/lib/native > In .bashrc added: > export LD_LIBRARY_PATH="$HADOOP_HOME/lib/native/" > export JAVA_LIBRARY_PATH="$HADOOP_HOME/lib/native/" > export > HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native" > In core-site.xml added: > <property> > <name>io.compression.codecs</name> > <value> > > org.apache.hadoop.io.compress.GzipCodec, > > org.apache.hadoop.io.compress.DefaultCodec, > > org.apache.hadoop.io.compress.BZip2Codec, > > org.apache.hadoop.io.compress.SnappyCodec > </value> > </property> > In mapred-site.xml added: > <property> > <name>mapreduce.map.output.compress</name> > <value>true</value> > </property> > > <property> > <name>mapred.map.output.compress.codec</name> > > <value>org.apache.hadoop.io.compress.SnappyCodec</value> > </property> > > <property> > <name>mapreduce.admin.user.env</name> > > <value>LD_LIBRARY_PATH=/home/hadoop/software/hadoop-2.4.1/lib/native</value> > </property> > In yarn-site.xml added: > <property> > <name>yarn.app.mapreduce.am.env</name> > > <value>LD_LIBRARY_PATH=/home/hadoop/software/hadoop-2.4.1/lib/native</value> > </property> > Finally make the same setting in the datanode ,but it still didn't > work 。But in a Pseudo-Distributed cluster, I only copy the libsnappy.so* to > $HADOOP_HOME/lib/native without changing the configuration ,it works well > .Why in this cluster it appears so difficult? > > > > > >
