You just have to tell Spark which log4j properties file to use. I think --driver-java-options="-Dlog4j.configuration=log4j.properties" should work but it didn't for me. set SPARK_JAVA_OPTS=-Dlog4j.configuration=log4j.properties did work though (this was on Windows, in local mode, assuming you put a file called log4j.properties in the bin directory where you run the shell from). In either case, like Sean said -Dlog4j.configuration=... is the magic incantation, you just have to figure out how to pass it depending on what version of SPARK you're using ( I personally find setting PRINT_SPARK_LAUNCH_COMMAND=1 very very useful when I'm trying to track a property)
-- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/disable-log4j-for-spark-shell-tp11278p12942.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
