Steve, I tried that, but didn't work. Any other ideas? On Mon, Feb 27, 2017 at 1:42 AM, Steve Loughran <ste...@hortonworks.com> wrote:
> try giving a resource of a file in the JAR, e.g add a file > "log4j-debugging.properties into the jar, and give a config option of > -Dlog4j.configuration=/log4j-debugging.properties (maybe also try > without the "/") > > > On 26 Feb 2017, at 16:31, Prithish <prith...@gmail.com> wrote: > > Hoping someone can answer this. > > I am unable to override and use a Custom log4j.properties on Amazon EMR. I > am running Spark on EMR (Yarn) and have tried all the below combinations in > the Spark-Submit to try and use the custom log4j. > > In Client mode > --driver-java-options "-Dlog4j.configuration=hdfs:// > host:port/user/hadoop/log4j.properties" > > In Cluster mode > --conf "spark.driver.extraJavaOptions=-Dlog4j.configuration=hdfs://host: > port/user/hadoop/log4j.properties" > > I have also tried picking from local filesystem using file://// instead > of hdfs. None of this seem to work. However, I can get this working when > running on my local Yarn setup. > > Any ideas? > > I have also posted on Stackoverflow (link below) > http://stackoverflow.com/questions/42452622/custom- > log4j-properties-on-aws-emr > > >
