Thanks, Marcelo. Will give it a shot tomorrow.
-Alex
On 8/9/17, 5:59 PM, "Marcelo Vanzin" wrote:
Jars distributed using --jars are not added to the system classpath,
so log4j cannot see them.
To work around that, you need to manually add the *name* jar to
Jars distributed using --jars are not added to the system classpath,
so log4j cannot see them.
To work around that, you need to manually add the *name* jar to the
driver executor classpaths:
spark.driver.extraClassPath=some.jar
spark.executor.extraClassPath=some.jar
In client mode you should
I have log4j json layout jars added via spark-submit on EMR
/usr/lib/spark/bin/spark-submit --deploy-mode cluster --master yarn --jars
/home/hadoop/lib/jsonevent-layout-1.7.jar,/home/hadoop/lib/json-smart-1.1.1.jar
--driver-java-options "-XX:+AlwaysPreTouch -XX:MaxPermSize=6G" --class