I used to use SPARK_LIBRARY_PATH to specify the location of native libs
for lzo compression when using spark 0.9.0.
The references to that environment variable have disappeared from the docs
for
spark 1.0.1 and it's not clear how to specify the location for lzo.
Any guidance?
One way is to set this in your conf/spark-defaults.conf:
spark.executor.extraLibraryPath /path/to/native/lib
The key is documented here:
http://spark.apache.org/docs/latest/configuration.html
On Thu, Jul 17, 2014 at 1:25 PM, Eric Friedman
eric.d.fried...@gmail.com wrote:
I used to use
but be aware that spark-defaults.conf is only used if you use spark-submit
On Jul 17, 2014 4:29 PM, Zongheng Yang zonghen...@gmail.com wrote:
One way is to set this in your conf/spark-defaults.conf:
spark.executor.extraLibraryPath /path/to/native/lib
The key is documented here:
