Your natives should be in LD_LIBRARY_PATH or java.library.path for hadoop to pick them up. You can try adding export HADOOP_OPTS=$HADOOP_OPTS -Djava.library.path=<Path to your natives lib> to hadoop-env.sh in TTs and clients/gateways and restart TTs and give it another try. The reason its working for Hbase is you are manually pointing HBASE_LIBRARY_PATH to the natives. My guess is they are in a wrong location.
On Thu, Jan 2, 2014 at 5:07 PM, Amit Sela <am...@infolinks.com> wrote: > I did everything mentioned in the link Ted mentioned, and the test > actually works, but using Snappy for MapReduce map output compression still > fails with "native snappy library not available". > > > On Wed, Jan 1, 2014 at 6:37 PM, bharath vissapragada < > bharathvissapragada1...@gmail.com> wrote: > >> Did you build it for your platform? You can do an "ldd" on the .so file >> to check if the dependent libs are present. Also make sure you placed it in >> the right directory for your platform (Linux-amd64-64 or Linux-i386-32) >> >> >> On Wed, Jan 1, 2014 at 10:02 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> Please take a look at >>> http://hbase.apache.org/book.html#snappy.compression >>> >>> Cheers >>> >>> >>> On Wed, Jan 1, 2014 at 8:05 AM, Amit Sela <am...@infolinks.com> wrote: >>> >>>> Hi all, >>>> >>>> I'm running on Hadoop 1.0.4 and I'd like to use Snappy for map output >>>> compression. >>>> I'm adding the configurations: >>>> >>>> configuration.setBoolean("mapred.compress.map.output", true); >>>> configuration.set("mapred.map.output.compression.codec", >>>> "org.apache.hadoop.io.compress.SnappyCodec"); >>>> >>>> And I've added libsnappy.so.1 to $HADOOP_HOME/lib/native/Linux-amd64-64/ >>>> >>>> Still, all map tasks fail with "native snappy library not available". >>>> >>>> Could anyone elaborate on how to install Snappy for Hadoop ? >>>> >>>> Thanks, >>>> >>>> Amit. >>>> >>> >>> >> >
