You can run
hadoop checknative -a
and see if bzip2 is detected correctly.
--
Ruslan Dautkhanov
On Fri, Jun 26, 2015 at 10:18 AM, Marcelo Vanzin van...@cloudera.com
wrote:
What master are you using? If this is not a local master, you'll need to
set LD_LIBRARY_PATH on the executors also
Hi,
I'm having trouble reading Bzip2 compressed sequence files after I
enabled hadoop native libraries in spark.
Running
LD_LIBRARY_PATH=$HADOOP_HOME/lib/native/ $SPARK_HOME/bin/spark-submit
--class gives the following error
5/06/26 00:48:02 INFO CodecPool: Got brand-new decompressor
What master are you using? If this is not a local master, you'll need to
set LD_LIBRARY_PATH on the executors also (using
spark.executor.extraLibraryPath).
If you are using local, then I don't know what's going on.
On Fri, Jun 26, 2015 at 1:39 AM, Arunabha Ghosh arunabha...@gmail.com
wrote:
