When you say you pointed LD_LIBRARY_PATH to the location of libsnappy.so, do
you mean just the setting of the “mapreduce.admin.user.env” property in
mapred-site.xml, or the actual environment variable before starting NiFi? The
mapred-site settings won’t be used as PutHDFS does not use MapReduce
Hi Jeremy,
Even after copying libsnappy.so to java_home/jre/lib it did not help much. I
also pointed LD_LIBRARY_PATH to the location of libsnappy.so. Even went to
the extent of modyfying bootstrap.conf with jvm params
-Djava.library.path=//.
But received the same error again. I have configured
Hi Joe,
Please find the contents of core-site.xml which contains following
property.
io.compression.codecs
org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec
mapred-site.xml
mapreduce.map.output.compress
To add to Jeremy's last point, even after the library is present, the files
must be greater than the HDFS block size (default is 64 MB I think?) or
Hadoop-snappy will also not compress them.
Sent from my iPhone
> On Feb 6, 2016, at 5:41 PM, Jeremy Dyer wrote:
>
> Shweta,
>
> Looks like your
Shweta,
Looks like your missing the snappy native library. I have seen this several
times before. Assuming your on a linux machine you have 2 options. You can
copy the libsnappy.so native library to your JAVA_HOME/jre/lib native
directory. Or you can set LD_LIBRARY_PATH to point to where your
libs
Can you show what is in your core-site.xml and the proc properties.
Also can you show the full log output?
Thanks
Joe
On Sat, Feb 6, 2016 at 9:11 AM, shweta wrote:
> Hi All,
>
> I'm getting a java.lang.UnsatisfiedLinkError while adding data into PutHDFS
> processor with compression codec as snap
Hi All,
I'm getting a java.lang.UnsatisfiedLinkError while adding data into PutHDFS
processor with compression codec as snappy. The error message says "Failed
to write to HDFS due to
org.apache.hadoop.util.NativeCodeloader.build.SupportsSnappy()Z.
Inspite of this error, .snappy files are being wr
